Method, system and medium for extracting compressed package sub-files according to screening rules
By creating multiple threads in the main process and file extraction subprocess, adopting parallel working mode and using preset resource quota and heartbeat notification mechanisms, the CPU utilization rate and temporary file occupation in the existing technology is solved. Many other problems have achieved efficient processing of compressed bun file.
Patent Information
- Application Number
- CN202411399241.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-11
- Estimated Expiration
- 2044-10-09
AI Technical Summary
The existing technology has a serial working mode when processing compressed bun files, resulting in low CPU utilization, temporary files occupying too much disk space, multi-level subdirectory creation under temporary directories, and the possibility of missing some target files in the scenario of super-large compressed bun files. .
By creating business main thread and exception monitoring thread in the main process, creating task proxy thread and decompression thread in the file extraction child process, adopting parallel working mode and using preset resource quota and heartbeat notification mechanisms, parallelization of compressed bun files is achieved through preset resource quota and heartbeat notification mechanisms Unzip and release, avoiding temporary files' lag and multi-level subdirectory creation.
Improves CPU utilization, reduces the time for temporary files to occupy disk space, avoids multi-level subdirectory creation, and ensures that target files are not missed in the super-large compressed package file scenario.
Smart Images

Figure CN118897826B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of extracting compressed package sub-files, and in particular to a method, system and medium for extracting compressed package sub-files according to screening rules. Background Art
[0002] With the rapid development of information technology, the data generated by social and economic activities and personal production and life has exploded, and the storage, management and transmission of electronic data have put forward higher and higher requirements on storage cost and transmission efficiency. Data compression technology is increasingly valued by organizations and individuals, and the proportion of data stored in the form of compressed files is increasing.
[0003] Software products that mainly process user data, such as decompression software, antivirus software, system forensics software, and security and confidentiality software, basically support the identification, reading, and content analysis of sub-files inside compressed files. In order to improve processing efficiency, the product supports the function of filtering compressed package sub-files. In the process of filtering compressed package sub-files, in order to avoid the operational risks brought to the main software program by unknown data inside the compressed package, the processing of the compressed package is mainly isolated into a separate sub-process.
[0004] However, the above solution has the following problems during implementation: (1) After the main process initiates the task, it needs to wait for the completion notification of the child process before it can carry out subsequent business processing. The child process will not notify the main process until all target sub-files have been released. The main process is always in a waiting state, and the CPU is not fully utilized. (2) Releasing temporary sub-files requires additional disk space. The child process needs to release all target sub-files before sending a notification. When the main process just receives the notification, the disk space usage reaches its peak. (3) There are multiple levels of temporary directories created, and the speed of sub-file release is slow. (4) When the child process is blocked due to extremely special reasons, the main process needs to set an additional timeout mechanism. The selection of the timeout value is not universal, which will cause the child process to be forcibly terminated due to the timeout mechanism being triggered due to excessive time, and then some sub-files cannot be released normally and are omitted. Summary of the invention
[0005] In response to the above-mentioned deficiencies in the prior art, the present application provides a system, method and medium for extracting compressed package sub-files according to screening rules, so as to solve the problems of low CPU utilization in serial working mode, excessive disk space occupied by temporary files, creation of multi-level sub-directories under the temporary directory, and possible omission of some target files in ultra-large compressed package file scenarios in the existing solutions.
[0006] In the first aspect, the present application provides a method for extracting compressed package sub-files according to screening rules, the method comprising: creating a business main thread and an exception monitoring thread in a main process; creating a task proxy thread in a file extraction sub-process; obtaining a screening range, starting the main process, and reading local files within the screening range; when the local file is a compressed file, initiating a new decompression task to the task proxy thread through the business main thread, and then creating a decompression thread corresponding to the new decompression task through the task proxy thread; wherein the decompression thread is configured with at least a preset resource quota, a path corresponding to the compressed file, and a preset file screening condition, and the new decompression task includes the path corresponding to the compressed file; traversing the sub-files in the path corresponding to the compressed file through the decompression thread, when the current sub-file is a compressed file, initiating a new decompression task to the task proxy thread, and then creating a decompression thread corresponding to the new decompression task through the task proxy thread; when the current sub-file meets the preset file screening condition and the preset resource quota is not less than the preset quota threshold, releasing the current sub-file to a temporary directory, and Based on the capacity of the released sub-file, the value of the preset resource quota is reduced, and a sub-file release notification is sent to the business main thread at the same time; wherein, the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory; when the business main thread processes the sub-file in the temporary directory and deletes the sub-file based on the sub-file release notification and the built-in processing program, the business main thread sends a temporary file deletion notification to the decompression thread corresponding to the sub-file; and then based on the capacity of the deleted file in the temporary file deletion notification, the value of the preset resource quota corresponding to the decompression thread is increased; during the operation of the decompression thread, a compressed file decompression progress notification is generated based on a preset time interval and sent to the task agent thread; based on the compressed file decompression progress notification, the task agent thread sends a heartbeat notification to the abnormal monitoring thread; when the heartbeat notification is not received within the preset notification time period, the abnormal monitoring thread starts the timer, and when the heartbeat notification is not received within the preset timing waiting time period, the execution file extraction sub-process is controlled to exit the process.
[0007] Furthermore, after traversing the sub-files in the path corresponding to the compressed file through the decompression thread, the method further includes:
[0008] When the preset resource quota is lower than the preset quota threshold, the corresponding decompression thread suspends releasing the sub-files in the compressed file until the preset resource quota is no lower than the preset quota threshold.
[0009] Furthermore, before creating a decompression thread corresponding to the new decompression task through the task proxy thread, the method further includes:
[0010] The main business thread obtains the file size of the compressed file corresponding to the new decompression task; based on the preset corresponding relationship between the file size and the preset quota threshold, determines the preset quota threshold corresponding to the current decompression thread.
[0011] Furthermore, before creating a decompression thread corresponding to the new decompression task through the task proxy thread, the method further includes:
[0012] The preset file screening condition is obtained through the task proxy thread, and then the preset file screening condition is sent to the task proxy thread, and then the preset file screening condition corresponding to the decompression thread is configured through the task proxy thread.
[0013] Furthermore, after the exception monitoring thread starts the timer, the method further includes:
[0014] When a heartbeat notification is received within a preset timed waiting period, the timer is reset and the next preset notification period is monitored again to see whether a heartbeat notification is received.
[0015] Further, releasing the current subfile to a temporary directory specifically includes: obtaining the file suffix information of the current subfile and the index value of the current subfile in the compressed package; determining the index value to be the name of the current subfile, and keeping the file suffix information unchanged.
[0016] In the second aspect, the present application provides a system for extracting compressed package sub-files according to screening rules, and the system includes: a creation module, which is used to create a business main thread and an exception monitoring thread in the main process; create a task proxy thread in the file extraction sub-process; a startup module, which is used to obtain the screening range, start the main process, and read the local files within the screening range; the business main thread, which is used to initiate a new decompression task to the task proxy thread when the local file is a compressed file; the task proxy thread, which is used to create a decompression thread corresponding to the new decompression task; wherein the decompression thread is configured with at least a preset resource quota, a corresponding path of the compressed file, and a preset file screening condition, and the new decompression task includes the corresponding path of the compressed file; the decompression thread, which is used to traverse the sub-files in the corresponding path of the compressed file, and when the current sub-file is a compressed file, initiate a new decompression task to the task proxy thread, and then create a decompression thread corresponding to the new decompression task through the task proxy thread; when the current sub-file meets the preset file screening condition and the preset resource quota is not lower than the preset quota threshold, release the current sub-file to the temporary directory, and based on the capacity of the released sub-file, reduce the value of the preset resource quota, and send a sub-file release notification to the business main thread at the same time; wherein the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory; the business main thread is used to process the sub-file in the temporary directory and delete the sub-file based on the sub-file release notification and the built-in processing program, and then send a temporary file deletion notification to the decompression thread corresponding to the sub-file; and then based on the capacity of the deleted file in the temporary file deletion notification, increase the value of the preset resource quota corresponding to the decompression thread; the decompression thread is used to generate a compressed file decompression progress notification based on a preset time interval during operation and send it to the task agent thread; based on the compressed file decompression progress notification, the task agent thread sends a heartbeat notification to the abnormal monitoring thread; the abnormal monitoring thread is used to start the timer when the heartbeat notification is not received within the preset notification time period, and when the heartbeat notification is not received within the preset timing waiting time period, control the execution file extraction sub-process to exit the process.
[0017] Furthermore, the main business thread is also used to obtain the file size of the compressed file corresponding to the new decompression task; based on the preset corresponding relationship between the file size and the preset quota threshold, determine the preset quota threshold corresponding to the current decompression thread.
[0018] Furthermore, the task proxy thread is also used to obtain preset file screening conditions, and then send the preset file screening conditions to the task proxy thread, and then configure the preset file screening conditions corresponding to the decompression thread through the task proxy thread.
[0019] In a third aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which when executed implement a method for extracting compressed package sub-files according to screening rules as described above.
[0020] Those skilled in the art can understand that the present application has at least the following beneficial effects:
[0021] (1) Solved the problem of low CPU utilization in serial working mode:
[0022] In the improvement scheme proposed in the present application, an instant communication link between processes is added, making it possible for the main process and the file extraction sub-process to work in parallel: when the decompression thread releases a sub-file, it immediately sends a sub-file release notification to the business main thread, and then continues to release subsequent sub-files. After receiving the notification, the business main thread can immediately go to the temporary directory to read the sub-file for business processing, without having to wait until all sub-files are released before processing.
[0023] Due to the limiting effect of the preset resource quota in the decompression thread, the sub-file release notification buffer queue of the main business thread will not expand infinitely.
[0024] From the above working process, it can be seen that the main process and the sub-process are in parallel working mode at a macro level, which effectively solves the problem of low CPU utilization.
[0025] (2) Improve the problem of temporary files taking up too much disk space:
[0026] The decompression thread of the subprocess of this application releases temporary files to consume disk space, while the main thread of the main process deletes temporary files to reclaim disk space. The two work in parallel to minimize the retention time of temporary files in the file system. Even in the scenario where the file release speed is faster than the processing speed, due to the limitation of the preset resource quota mechanism, there will be no problem of excessive consumption of disk space.
[0027] (3) Solved the problem of creating multiple subdirectories under the temporary directory:
[0028] In the "sub-file release notification" message sent by the sub-process decompression thread to the main process business main thread, key information is attached: the path information of the file in the temporary directory and the path information of the file in the compressed package. The former can locate and read the data of the file, and the latter can obtain the location information of the sub-file in the compressed package.
[0029] From the above, we can see that the main thread of the main process business does not need to restore the location information of the file in the compressed package through the file location of the temporary directory. Therefore, the sub-process decompression thread no longer needs to create multi-level sub-directories under the temporary directory. It only needs to release the sub-files directly to the root directory of the temporary directory. This strategy can effectively reduce disk I / O pressure and improve the overall business processing speed.
[0030] In addition, directly releasing the sub-files to the root directory of the temporary directory requires solving the problem of file name conflicts in the same directory: there may be multiple target files with the same file name but distributed in different sub-directories inside the compressed package. If the original file name is used when releasing the file, if the previous target file with the same name has not been cleaned up, there will be a file name conflict problem. The solution provided by this application is: when releasing the file, the original file name of the target file is not used, but only the file suffix information is retained, and the file name is named using its index value inside the compressed package. The index value of each sub-file is unique inside the compressed package, so the problem of naming conflicts can be completely solved.
[0031] (4) Solved the problem that some target files may be missed in the scenario of extremely large compressed files:
[0032] In the improved solution of the present application, the child process sends heartbeat notifications to the main process regularly. If the decompression thread of the child process can regularly report progress data to the task proxy thread, it means that the thread is working normally. The task proxy thread uses this as a basis to regularly send heartbeat notifications to maintain the abnormal monitoring timer on the main process side from being triggered. If the decompression thread is blocked for a long time or enters an abnormal state such as an infinite loop, it will not be able to report progress data. The task proxy thread will delay sending the heartbeat notification until the timer on the main process side is triggered, and the entire business will enter the timeout exception handling. From the above process, it can be seen that after the improvement of this application, the problem of missing the target file will not occur in the scenario of super-large compressed files. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:
[0034] Figure 1 It is a flow chart of a method for extracting compressed package sub-files according to screening rules provided in an embodiment of the present application.
[0035] Figure 2 It is a schematic diagram of the internal structure of a system for extracting compressed package sub-files according to screening rules provided in an embodiment of the present application. DETAILED DESCRIPTION
[0036] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.
[0037] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0038] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0039] The present application embodiment provides a method for extracting compressed package sub-files according to screening rules, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:
[0040] Step 110: Create a business main thread and an exception monitoring thread in the main process; create a task proxy thread in the file extraction subprocess; obtain the screening range, start the main process, and read the local files within the screening range.
[0041] It should be noted that the main business thread is the core business bearer of compressed file data processing in the software function. It is responsible for starting the file extraction subprocess, creating an exception monitoring thread, dispatching compressed package decompression tasks, traversing and processing the target files in the temporary directory, deleting temporary files and sending notifications to the subprocess. The exception monitoring thread is responsible for monitoring the running status of the file extraction subprocess and is created by the main business thread of the main process. After the thread is started, it creates a timer and then listens to the "heartbeat notification" message sent by the subprocess. Every time a heartbeat notification is received, the monitoring thread resets the timer and restarts the timing. If the heartbeat notification is not received within the specified time, the timer will be triggered. At this time, the monitoring thread determines that the subprocess is abnormal, forcibly terminates the subprocess, and notifies the main business thread for subsequent exception processing.
[0042] The task agent thread is the core thread responsible for management tasks in the file extraction sub-process. It is responsible for listening to new task notifications from the main process and interpreting task parameters, starting a dedicated decompression thread for new tasks, monitoring the working status of each working thread, and regularly sending "heartbeat notifications" to the main process to report the working status of the sub-process to prevent triggering timeout exception notifications.
[0043] Among them, the process of obtaining the screening range can be determined by the operator according to actual needs, and this application does not limit this.
[0044] Step 120: When the local file is a compressed file, a new decompression task is initiated to the task proxy thread through the business main thread, and then a decompression thread corresponding to the new decompression task is created through the task proxy thread.
[0045] It should be noted that the decompression thread is configured with at least a preset resource quota, a path corresponding to the compressed file, and a preset file screening condition, and the new decompression task includes a path corresponding to the compressed file.
[0046] The decompression thread is the specific undertaker of the compressed file decompression work. It is created by the child process task proxy thread. In the scenario of supporting multi-layer nested compression, multiple decompression threads can exist at the same time, and each decompression thread is responsible for processing an independent compressed package file.
[0047] Step 130, traverse the sub-files in the path corresponding to the compressed file through the decompression thread, and when the current sub-file is a compressed file, initiate a new decompression task to the task agent thread, and then create a decompression thread corresponding to the new decompression task through the task agent thread; when the current sub-file meets the preset file screening conditions and the preset resource quota is not lower than the preset quota threshold, release the current sub-file to a temporary directory, and at the same time, based on the capacity of the released sub-file, reduce the value of the preset resource quota, and send a sub-file release notification to the business main thread.
[0048] It should be noted that the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory. The preset file screening condition can be any feasible condition determined by those skilled in the art according to actual conditions, for example, picture data.
[0049] It can be seen here that the sub-file release process will gradually consume the "preset resource quota" of the decompression thread. When the quota is insufficient, the decompression thread suspends file release and waits for the "temporary file deletion notification" sent back by the business main thread. This type of notification is equivalent to "quota return". When enough quota is returned, the decompression thread continues to execute subsequent decompression tasks.
[0050] That is, after traversing the sub-files in the path corresponding to the compressed file through the decompression thread, the method further includes:
[0051] When the preset resource quota is lower than the preset quota threshold, the corresponding decompression thread suspends releasing the sub-files in the compressed file until the preset resource quota is no lower than the preset quota threshold.
[0052] In addition, the present application can adjust the preset quota threshold according to the size of the decompression package corresponding to the decompression thread. As an example, before creating a decompression thread corresponding to a new decompression task through the task agent thread, the business main thread obtains the file size of the compressed file corresponding to the new decompression task; based on the preset corresponding relationship between the file size and the preset quota threshold, the preset quota threshold corresponding to the current decompression thread is determined.
[0053] The preset corresponding relationship may be that the larger the file is, the larger the preset quota threshold is.
[0054] In addition, any feasible preset file screening condition may be determined by a person skilled in the art according to actual conditions. As an example, before creating a decompression thread corresponding to a new decompression task through a task proxy thread, the preset file screening condition is obtained through the task proxy thread, and then the preset file screening condition is sent to the task proxy thread, and then the preset file screening condition corresponding to the decompression thread is configured through the task proxy thread.
[0055] In addition, releasing the current subfile to a temporary directory may specifically include: obtaining the file suffix information of the current subfile and the index value of the current subfile in the compressed package; determining the index value to be the name of the current subfile, and keeping the file suffix information unchanged.
[0056] It is understandable to those skilled in the art that directly releasing a sub-file to the root directory of a temporary directory requires solving the problem of file name conflicts in the same directory: there may be multiple target files with the same file name but distributed in different sub-directories inside the compressed package. If the original file name is used when releasing the file, if the previous target file with the same name has not been cleaned up, there will be a file name conflict problem. The solution provided by the present application is: when releasing the file, the original file name of the target file is not used, but only the file suffix information is retained, and the file name is named using its index value inside the compressed package. The index value of each sub-file is unique inside the compressed package, so the problem of naming conflicts can be completely solved.
[0057] Step 140: After the business main thread processes the sub-files in the temporary directory and deletes the sub-files based on the sub-file release notification and the built-in processing program, the business main thread sends a temporary file deletion notification to the decompression thread corresponding to the sub-files; and then increases the value of the preset resource quota corresponding to the decompression thread based on the capacity of the deleted files in the temporary file deletion notification.
[0058] It should be noted that each decompression thread will receive the "preset resource quota" from the startup parameters when it is created, that is, the maximum number of sub-files currently allowed to be released or the maximum amount of disk space allowed to be occupied. After each sub-file is released to the temporary directory, the decompression thread will reduce its own "preset resource quota" according to the file size, and then send a sub-file release notification to the main process business main thread. The message notification is accompanied by the internal path of the compressed package of the sub-file and the file name in the temporary directory. After receiving the notification, the main process will go to the temporary directory to read and process the sub-file. After processing and deleting the file, the main process needs to send a "temporary file deletion notification" to the decompression thread.
[0059] Step 150: During the operation of the decompression thread, a compressed file decompression progress notification is generated based on a preset time interval and sent to the task agent thread; based on the compressed file decompression progress notification, the task agent thread sends a heartbeat notification to the abnormality monitoring thread.
[0060] It should be noted that, through step 150, the sub-process (task agent thread) periodically sends heartbeat notifications to the main process (abnormal monitoring thread). If the sub-process decompression thread can regularly report progress data to the task agent thread, it means that the thread is working normally. The task agent thread uses this as a basis to regularly send heartbeat notifications to maintain the abnormal monitoring timer on the main process side from being triggered.
[0061] Step 160: When no heartbeat notification is received within a preset notification time period, the abnormal monitoring thread starts a timer. When no heartbeat notification is received within a preset timing waiting time period, the execution file extraction subprocess is controlled to exit the process.
[0062] It should be noted that if the decompression thread is blocked for a long time or enters an abnormal state such as an infinite loop, the progress data cannot be reported, and the task agent thread will delay sending the heartbeat notification until the timer on the main process (abnormal monitoring thread) side is triggered, and the entire business will enter the timeout abnormality processing.
[0063] In addition, after the exception monitoring thread starts the timer, the method further includes:
[0064] When a heartbeat notification is received within a preset timed waiting period, the timer is reset and the next preset notification period is monitored again to see whether a heartbeat notification is received.
[0065] besides, Figure 2 A system for extracting compressed package sub-files according to screening rules is provided in an embodiment of the present application. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:
[0066] The creation module 210 is used to create a business main thread 310 and an exception monitoring thread 320 in the main process 300; and to create a task proxy thread 410 in the file extraction sub-process 400.
[0067] The startup module 220 is used to obtain the screening range, start the main process, and read the local files within the screening range;
[0068] The business main thread 310 is used to initiate a new decompression task to the task proxy thread 410 when the local file is a compressed file.
[0069] The task proxy thread 410 is used to create a decompression thread 411 corresponding to the new decompression task; wherein the decompression thread 411 is configured with at least a preset resource quota, a compressed file corresponding path, and a preset file screening condition, and the new decompression task includes a compressed file corresponding path.
[0070] The task proxy thread 410 is further used to obtain preset file screening conditions, and then send the preset file screening conditions to the task proxy thread 410, and then configure the preset file screening conditions corresponding to the decompression thread 411 through the task proxy thread 410.
[0071] The decompression thread 411 is used to traverse the sub-files in the path corresponding to the compressed file. When the current sub-file is a compressed file, a new decompression task is initiated to the task agent thread 410, and then the decompression thread 411 corresponding to the new decompression task is created through the task agent thread 410; when the current sub-file meets the preset file screening conditions and the preset resource quota is not lower than the preset quota threshold, the current sub-file is released to the temporary directory, and based on the capacity of the released sub-file, the value of the preset resource quota is reduced, and a sub-file release notification is sent to the business main thread 310; wherein, the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory.
[0072] The main business thread 310 is used to process the sub-files in the temporary directory and delete the sub-files based on the sub-file release notification and the internal processing program, and then send a temporary file deletion notification to the decompression thread 411 corresponding to the sub-file; and then increase the value of the preset resource quota corresponding to the decompression thread 411 based on the capacity of the deleted files in the temporary file deletion notification.
[0073] The main business thread 310 is also used to obtain the file size of the compressed file corresponding to the new decompression task; based on the preset corresponding relationship between the file size and the preset quota threshold, determine the preset quota threshold corresponding to the current decompression thread 411.
[0074] The decompression thread 411 is used to generate a compressed file decompression progress notification based on a preset time interval during operation and send it to the task agent thread 410; based on the compressed file decompression progress notification, the task agent thread 410 sends a heartbeat notification to the abnormality monitoring thread 320.
[0075] The abnormal monitoring thread 320 is used to start the timer when no heartbeat notification is received within the preset notification time period, and to control the execution file extraction subprocess 400 to exit the process when no heartbeat notification is received within the preset timing waiting time period.
[0076] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a method for extracting compressed package sub-files according to screening rules as described above is implemented.
[0077] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.
Claims
1. A method for extracting compressed package sub-files according to screening rules, characterized in that: The method comprises: Create a business main thread and an exception monitoring thread in the main process; create a task proxy thread in the file extraction subprocess; Get the filter range, start the main process, and read the local files within the filter range; When the local file is a compressed file, a new decompression task is initiated to the task proxy thread through the business main thread, and then a decompression thread corresponding to the new decompression task is created through the task proxy thread; wherein the decompression thread is configured with at least a preset resource quota, a path corresponding to the compressed file, and a preset file screening condition, and the new decompression task includes a path corresponding to the compressed file; The decompression thread traverses the sub-files in the path corresponding to the compressed file. When the preset resource quota is lower than the preset quota threshold, the corresponding decompression thread suspends the release of the sub-files in the compressed file until the preset resource quota is not lower than the preset quota threshold. When the current sub-file is a compressed file, a new decompression task is initiated to the task proxy thread, and then a decompression thread corresponding to the new decompression task is created through the task proxy thread; when the current sub-file meets the preset file screening conditions and the preset resource quota is not lower than the preset quota threshold, the current sub-file is released to a temporary directory, and at the same time, based on the capacity of the released sub-file, the value of the preset resource quota is reduced, and a sub-file release notification is sent to the business main thread; wherein the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory; After the main business thread processes the sub-files in the temporary directory and deletes the sub-files based on the sub-file release notification and the built-in processing program, the main business thread sends a temporary file deletion notification to the decompression thread corresponding to the sub-file; then, based on the capacity of the deleted file in the temporary file deletion notification, the value of the preset resource quota corresponding to the decompression thread is increased; During the operation of the decompression thread, a compressed file decompression progress notification is generated based on a preset time interval and sent to the task agent thread; based on the compressed file decompression progress notification, the task agent thread sends a heartbeat notification to the abnormality monitoring thread; When no heartbeat notification is received within the preset notification time period, the abnormal monitoring thread starts the timer. When no heartbeat notification is received within the preset timing waiting time period, the execution file extraction subprocess is controlled to exit the process.
2. The method for extracting compressed package sub-files according to screening rules according to claim 1, characterized in that: Before creating a decompression thread corresponding to the new decompression task through the task proxy thread, the method further includes: The main business thread obtains the file size of the compressed file corresponding to the decompression new task; Based on a preset corresponding relationship between the file size and the preset quota threshold, a preset quota threshold corresponding to the current decompression thread is determined.
3. The method for extracting compressed package sub-files according to screening rules according to claim 1, characterized in that: Before creating a decompression thread corresponding to the new decompression task through the task proxy thread, the method further includes: The preset file screening condition is obtained through the task proxy thread, and then the preset file screening condition is sent to the task proxy thread, and then the preset file screening condition corresponding to the decompression thread is configured through the task proxy thread.
4. The method for extracting compressed package sub-files according to screening rules according to claim 1, characterized in that: After the abnormal monitoring thread starts the timer, the method further includes: When a heartbeat notification is received within a preset timed waiting period, the timer is reset and the next preset notification period is monitored again to see whether a heartbeat notification is received.
5. The method for extracting compressed package sub-files according to screening rules according to claim 1, characterized in that: Release the current sub-file to a temporary directory, including: Get the file suffix information of the current sub-file and the index value of the current sub-file in the compressed package; The index value is determined to be the name of the current sub-file, and the file suffix information remains unchanged.
6. A system for extracting compressed package sub-files according to screening rules, characterized in that: The system comprises: Create a module to create a business main thread and an exception monitoring thread in the main process; create a task proxy thread in the file extraction subprocess; The startup module is used to obtain the filter range, start the main process, and read the local files within the filter range; The main business thread is used to initiate a new decompression task to the task proxy thread when the local file is a compressed file; The task proxy thread is used to create a decompression thread corresponding to the new decompression task; wherein the decompression thread is configured with at least a preset resource quota, a path corresponding to the compressed file, and a preset file screening condition, and the new decompression task includes a path corresponding to the compressed file; A decompression thread is used to traverse the sub-files in the path corresponding to the compressed file. When the preset resource quota is lower than the preset quota threshold, the corresponding decompression thread suspends the release of the sub-files in the compressed file until the preset resource quota is not lower than the preset quota threshold. When the current sub-file is a compressed file, a new decompression task is initiated to the task proxy thread, and then a decompression thread corresponding to the new decompression task is created through the task proxy thread; when the current sub-file meets the preset file screening conditions and the preset resource quota is not lower than the preset quota threshold, the current sub-file is released to a temporary directory, and based on the capacity of the released sub-file, the value of the preset resource quota is reduced, and a sub-file release notification is sent to the business main thread; wherein the sub-file release notification includes the internal path of the compressed package of the released sub-file and the file name in the temporary directory; The main business thread is used to process and delete the sub-files in the temporary directory based on the sub-file release notification and the internal processing program, and then send a temporary file deletion notification to the decompression thread corresponding to the sub-file; and then increase the value of the preset resource quota corresponding to the decompression thread based on the capacity of the deleted file in the temporary file deletion notification; The decompression thread is used to generate a compressed file decompression progress notification based on a preset time interval during operation and send it to the task agent thread; based on the compressed file decompression progress notification, the task agent thread sends a heartbeat notification to the abnormal monitoring thread; The abnormal monitoring thread is used to start the timer when no heartbeat notification is received within the preset notification time period, and to control the execution file extraction subprocess to exit the process when no heartbeat notification is received within the preset timing waiting time period.
7. The system for extracting compressed package sub-files according to screening rules according to claim 6, characterized in that: The main business thread is also used to obtain the file size of the compressed file corresponding to the new decompression task; based on the preset corresponding relationship between the file size and the preset quota threshold, determine the preset quota threshold corresponding to the current decompression thread.
8. The system for extracting compressed package sub-files according to screening rules according to claim 6, characterized in that: The task proxy thread is also used to obtain preset file screening conditions, and then send the preset file screening conditions to the task proxy thread, and then configure the preset file screening conditions corresponding to the decompression thread through the task proxy thread.
9. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the method for extracting compressed package sub-files according to screening rules as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
File decompression method and device
CN113111040A
Data processing method and device, computer readable medium and electronic equipment
CN113901000A
Mass data browsing method and device
CN117873975A