A method and device for archiving electronic files based on multi-thread fair scheduling
By segmenting archiving tasks through feature analysis, allocating thread pools, and setting scheduling weights and time slices, the problem of low efficiency in electronic archive archiving was solved, fair use of resources and rapid fault recovery were achieved, and data consistency was ensured.
Patent Information
- Application Number
- CN202511235260.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-01
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2045-09-01
AI Technical Summary
Existing electronic record archiving methods are inefficient when dealing with massive, high-concurrency, and heterogeneous archives. They are prone to thread starvation, uneven resource utilization, high failure recovery costs, and difficulty in guaranteeing data consistency.
By segmenting archived tasks into different types through feature analysis, allocating corresponding thread pools, setting scheduling weights and time slices, and creating fault recovery checkpoints, fair scheduling and rapid fault recovery of heterogeneous tasks can be achieved.
It improves the archiving time and efficiency of small-volume files, increases resource utilization, reduces fault recovery costs, and ensures data integrity and consistency.
Smart Images
Figure CN120743478B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data processing, in particular to an electronic archive archiving method and device based on multi-thread fair scheduling, electronic equipment and storage medium. BACKGROUND
[0002] With the deepening of informatization construction, the number and types of electronic archives are growing explosively. The existing archiving method usually adopts a simple multi-thread processing model, such as a first-in first-out (FIFO) queue or a scheduling strategy based on fixed priority, so that the traditional electronic archive archiving system faces severe challenges in processing efficiency and stability when facing mass, high concurrency and heterogeneous types of archive (such as text documents, high-definition pictures, large-capacity videos, etc.) batch submission.
[0003] Based on the above problems, the applicant proposes the technical solution of the present application. SUMMARY
[0004] In order to achieve the above purpose, the present application discloses an electronic archive archiving method based on multi-thread fair scheduling, comprising the following steps:
[0005] Receiving an electronic archive to be archived, performing feature analysis on the electronic archive to form a feature analysis result, and dividing the electronic archive into an archiving task type according to the feature analysis result, wherein the archiving task type at least includes CPU intensive, IO intensive and mixed type;
[0006] According to the archiving task type corresponding to the electronic archive, a corresponding thread pool is allocated to the electronic archive, a scheduling weight is calculated according to the feature analysis result, and a scheduling time slice is set according to the scheduling weight, wherein the thread pool at least includes a CPU intensive thread pool, an IO intensive thread pool and a mixed thread pool;
[0007] The electronic archive is distributed to the thread of the corresponding thread pool for archiving processing by using a scheduling engine. Before the archiving processing is executed, a checkpoint for fault recovery is created. After the archiving processing is executed, the archiving task is marked as completed, and the thread occupied by the electronic archive is automatically released.
[0008] In one embodiment, the feature analysis result at least includes file size, processing complexity and business priority. Before calculating the scheduling weight, the corresponding adjustable coefficient is set for the file size, the processing complexity and the business priority respectively. The scheduling weight is obtained by multiplying the file size, the processing complexity and the business priority with the respective corresponding adjustable coefficient.
[0009] In one embodiment, the scheduling time slice is inversely proportional to the scheduling weight, so that the electronic archive with high scheduling weight obtains short scheduling time slice.
[0010] In one embodiment, before receiving the electronic archives to be archived, the method further comprises: initializing various thread pools, and starting a monitoring service.
[0011] In one embodiment, in the execution of the archiving process, if the archiving task is not executed within a standard time slice, the checkpoint of the archiving task is updated, and after the standard time slice is exhausted, the archiving task is put back into the scheduling queue, and the scheduling weight and scheduling time slice of the archiving task are recalculated.
[0012] In one embodiment, when a fault occurs in the execution of the archiving process, the unfinished archiving task is identified, the context is recovered from the checkpoint of the archiving task, and the archiving process is continued from the interruption.
[0013] The application also discloses an electronic archive archiving device based on multi-thread fair scheduling, comprising the following modules:
[0014] An archive receiving module is configured to receive electronic archives to be archived, perform feature analysis on the electronic archives to form feature analysis results, and divide the electronic archives into archiving task types according to the feature analysis results, wherein the archiving task types at least include CPU-intensive, IO-intensive and mixed types.
[0015] A weight calculation module is configured to assign a corresponding thread pool to the electronic archives according to the archiving task types corresponding to the electronic archives, calculate a scheduling weight according to the feature analysis results, and set a scheduling time slice according to the scheduling weight, wherein the thread pool at least includes a CPU-intensive thread pool, an IO-intensive thread pool and a mixed thread pool.
[0016] A process execution module is configured to distribute the electronic archives to threads of the corresponding thread pool by using a scheduling engine to execute archiving processes, create a checkpoint for fault recovery before executing the archiving processes, mark the completion of the archiving task after executing the archiving processes, and automatically release threads occupied by the electronic archives.
[0017] The application also discloses an electronic device comprising at least one processor, a memory, wherein the memory stores a computer program, and the at least one processor executes the computer program stored in the memory, so that the at least one processor executes the above-mentioned electronic archive archiving method based on multi-thread fair scheduling.
[0018] The application also discloses a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned electronic archive archiving method based on multi-thread fair scheduling.
[0019] The application further discloses a computer program product comprising program codes for executing the electronic file archiving method of opportunity multithread fair scheduling as described above when a computer runs the computer program.
[0020] Compared with the prior art, the application has the following beneficial effects:
[0021] 1. The application divides electronic files into archiving task types, sets scheduling weights and scheduling time slices, so that electronic files of large and small volumes can be fairly scheduled according to their own characteristics, the thread starvation phenomenon can be eliminated, and the archiving time and efficiency of small-volume files are greatly improved, especially in a high-concurrency scenario.
[0022] 2. The application adopts heterogeneous task type scheduling by corresponding allocation of thread pools, can effectively isolate the archiving of tasks of different resource consumption types, avoids mutual contention and waiting for CPU and IO resources, and significantly improves the overall resource utilization.
[0023] 3. The application sets a fault recovery checkpoint, so that in the fault recovery process, only the fault recovery needs to be performed from the interruption, the consistency of the archived data during fault recovery is ensured, the fault recovery cost is greatly reduced, and the recovery efficiency and data integrity are improved.
[0024] The concept, specific structure and generated technical effects of the application will be further described below with reference to the accompanying drawings, so as to fully understand the purpose, features and effects of the application. BRIEF DESCRIPTION OF DRAWINGS
[0025] Figure 1 is a processing flowchart of the electronic file archiving method based on multithread fair scheduling in the application;
[0026] Figure 2 is another processing flowchart of the electronic file archiving method based on multithread fair scheduling in the application;
[0027] Figure 3 is a processing flowchart of the fault in the electronic file archiving method based on multithread fair scheduling in the application;
[0028] Figure 4 is a processing flowchart of the monitoring optimization in the electronic file archiving method based on multithread fair scheduling in the application;
[0029] Figure 5 is a hierarchical architecture schematic diagram of the electronic file archiving device based on multithread fair scheduling in the application. DETAILED DESCRIPTION
[0030] In order to make the technical means, creative features, purposes and effects of the application easy to understand, the application will be further described below in combination with specific drawings. However, the application is not limited to the following embodiments.
[0031] It should be understood that the structures, proportions, sizes, etc. shown in the drawings attached to the present specification are only used to cooperate with the content disclosed in the specification for understanding and reading by those skilled in the art, and do not have technical substantive significance, and any modification of the structure, change of the proportion relationship or adjustment of the size, without affecting the effects and purposes that can be achieved by the application, should still fall within the scope of the technical content disclosed by the application.
[0032] The existing archiving method for electronic archives usually adopts a simple multi-thread processing model, such as a first-in-first-out queue or a scheduling strategy based on fixed priority, which is prone to thread starvation in the case of resource competition. When there are a large number of archives submitted at the same time, such as video transcoding, large file encryption, etc., the archives with large volume or complex processing will occupy the processing thread for a long time, resulting in the archives with small volume and simple processing, such as ordinary text and table, being in a long queuing state and unable to be processed in time, thus causing the phenomenon of thread starvation. The uncontrollability of processing delay seriously affects the overall efficiency of archiving and user experience. Archiving of archives usually includes multiple types of sub-tasks, such as CPU-intensive encryption or decryption tasks, IO-intensive file reading and writing and network transmission tasks, and format conversion tasks with both CPU and IO. The traditional thread pool does not distinguish between these heterogeneous tasks, resulting in uneven allocation of CPU and IO resources, and often causing CPU resource idle while IO blocking, or CPU full load while IO resource utilization low, thus resulting in low overall resource utilization efficiency. During the long batch archiving process, it may be terminated unexpectedly due to hardware failure, software error or network interruption. The traditional archiving lacks effective task state persistence and recovery mechanism, and once the fault occurs, the task state being processed will be lost, resulting in inconsistent data. After the fault is repaired, the interrupted task often needs to be retried in full, which not only wastes a lot of computing resources, but also further prolongs the archiving time, resulting in low archiving efficiency.
[0033] Based on the above technical problems, the first embodiment of the present application provides an electronic archive archiving method based on multi-thread fair scheduling. As shown in the figure, the method comprises the following steps: Figure 1
[0034] Step S1, receiving an electronic archive to be archived, performing feature analysis on the electronic archive, forming a feature analysis result, and dividing the electronic archive into an archiving task type according to the feature analysis result, the archiving task type at least including CPU intensive, IO intensive and mixed type.
[0035] Specifically, after receiving the electronic archive to be archived, the electronic archive enters the scheduling queue. Before scheduling, the electronic archive is analyzed according to its internal properties and external requirements. The feature analysis includes file volume which can represent the actual size of the archive, processing complexity set according to archiving task processing type, which includes encryption, format conversion, metadata extraction, etc., and business priority set according to user demand, which is generally divided into high, medium and low priority.
[0036] According to the feature analysis result, the electronic archive can be divided into archiving task type. In this embodiment, the archiving task type is divided into three types, namely CPU intensive, IO intensive and mixed type. For file encryption, complex calculation and other tasks, it belongs to CPU intensive task. For large file reading and writing, network transmission and storage medium migration, it belongs to IO intensive task. For file format conversion, it belongs to mixed task which has CPU calculation and IO reading and writing.
[0037] Step S2, according to the archiving task type corresponding to the electronic archive, the corresponding thread pool is allocated for the electronic archive, the scheduling weight is calculated according to the feature analysis result, and the scheduling time slice is set according to the scheduling weight, the thread pool at least includes CPU intensive thread pool, IO intensive thread pool and mixed thread pool.
[0038] Specifically, in this embodiment, a plurality of thread pools are provided, which are divided into CPU intensive thread pool, IO intensive thread pool and mixed thread pool according to processing characteristics. When receiving a certain archiving task, the archiving task is allocated to the corresponding thread pool for processing according to the type of the archiving task. For example, CPU intensive task is allocated to CPU intensive thread pool, CPU intensive thread pool is CPU dedicated thread, IO intensive task is allocated to IO intensive thread pool, IO intensive thread pool is IO asynchronous thread, and mixed task is allocated to mixed thread pool, and mixed thread pool is elastic thread. Thus, according to the type of the archiving task, the corresponding thread pool is allocated for processing, which can greatly improve the archiving processing efficiency.
[0039] In order to further refine the processing, the scheduling efficiency is improved by calculating the scheduling weight and scheduling time slice before entering the thread pool processing. The feature analysis result includes file volume, processing complexity, and business priority. Before calculating the scheduling weight, the corresponding adjustable coefficient is set for the file volume, the processing complexity, and the business priority. The scheduling weight is obtained by multiplying the file volume, the processing complexity, and the business priority with the corresponding adjustable coefficient respectively.
[0040] In one example, the calculation of the scheduling weight can be represented as W = f(α×log2(V), β×C, γ×P), where α, β, γ are dynamically adjustable coefficients to adapt to different system loads and business demands, V represents the file volume, C represents the processing complexity, and P represents the processing priority.
[0041] A reference configuration is as follows:
[0042] File volume (V): the actual size of the archive, in MB.
[0043] Processing complexity (C): the estimated processing complexity according to the task type, with a value range of [1, 10]:
[0044] - Basic operation (such as file copying): C = 1
[0045] - Metadata extraction: C = 3
[0046] - Text format conversion: C = 5
[0047] - Image format conversion: C = 7
[0048] - Encryption / decryption operation: C = 8
[0049] - Video transcoding: C = 10
[0050] Business priority (P): priority specified by the user or system policy, with a value range of [1, 5]:
[0051] - Low priority: P = 1
[0052] - Normal priority: P = 3
[0053] - High priority: P = 5
[0054] Related coefficient configuration:
[0055] - α: file volume weight coefficient, with a value range of [0.1, 0.5]
[0056] - β: complexity weight coefficient, with a value range of [0.2, 0.6]
[0057] - γ: priority weight coefficient, value range [0.2, 0.6]
[0058] The calculation formula of the scheduling weight is: W = a x log2(V) + β x C + γ x (6-P).
[0059] Suppose there are the following three tasks in the scheduling queue:
[0060] 1. Small file text conversion task
[0061] - V = 1MB, C = 5, P = 3 (normal priority)
[0062] - W1= 0.3 x log2(1) + 0.4 x 5 + 0.3 x (6-3) = 0 + 2.0 + 0.9 = 2.9
[0063] 2. Medium picture conversion task
[0064] - V = 50MB, C = 7, P = 5 (high priority)
[0065] - W2= 0.3 x log2(50) + 0.4 x 7 + 0.3 x (6-5) = 1.7 + 2.8 + 0.3 = 4.8
[0066] 3. Large file video transcoding task
[0067] - V = 500MB, C = 10, P = 1 (low priority)
[0068] - W3= 0.3 x log2(500) + 0.4 x 10 + 0.3 x (6-1) = 2.7 + 4.0 + 1.5 = 8.2
[0069] After obtaining the scheduling weight, the scheduling time slice is set according to the scheduling weight. The scheduling time slice is inversely proportional to the scheduling weight, so that the scheduling time slice of the electronic archive with high scheduling weight is short. The time slice calculation formula is: TimeSlice = BaseTimeSlice x (Wmax / W), wherein:
[0070] - TimeSlice: actual allocated time slice length (milliseconds)
[0071] - BaseTimeSlice: base time slice length, default is 50ms
[0072] - Wmax: maximum weight value in the current scheduling queue
[0073] - W: weight value of current task
[0074] Set BaseTimeSlice = 50ms, Wmax = 8.2 (the maximum weight in current scheduling queue), then the time slices obtained by the three tasks are respectively:
[0075] 1. Small file task: TimeSlice1= 50 × (8.2 / 2.9) ≈ 141ms
[0076] 2. Medium task: TimeSlice2= 50 × (8.2 / 4.8) ≈ 85ms
[0077] 3. Large file task: TimeSlice3= 50 × (8.2 / 8.2) = 50ms
[0078] In this way, the small file or high priority archiving task has a low calculated scheduling weight, and a long scheduling time slice is set for it. The large file or low priority archiving task has a high calculated scheduling weight, and a short scheduling time is set for it.
[0079] If the scheduling time slice of an archiving task is longer, even longer than a standard time slice in the thread pool, then the archiving progress of the archiving task needs to be judged when the standard time slice is about to end. If the archiving task cannot be executed within a standard time slice, the archiving task temporarily releases the thread and reenters the scheduling queue to wait for the archiving task to continue in the subsequent standard time slice. In this way, small tasks can be quickly completed, and large tasks can be executed in slices to avoid long-term exclusive use of resources by a single task during archiving, and to improve fair scheduling.
[0080] Step S3, using the scheduling engine to distribute the electronic archives to the threads of the corresponding thread pool for archiving processing. Before archiving processing, a checkpoint for fault recovery is created, and after archiving processing, the archiving task is marked as completed, and the thread occupied by the electronic archives is automatically released.
[0081] Specifically, in order to guarantee the integrity of the archive and the system fast recovery ability, the embodiment introduces incremental state snapshot and fast fault recovery for the archiving process. The incremental state snapshot is to divide the archiving process into multiple independent sub-processing steps, such as receiving files, encryption completion, metadata extraction completion, writing storage completion, etc. When a key step is successfully completed, a system event is triggered to atomically persist the ID, completion status and related context information of the current task to the checkpoint database. The fast fault recovery is that when the system is restarted after a fault, the system first scans the checkpoint database to identify all tasks that are not marked as "completely successful". For the interrupted task, the latest successful snapshot is read, all completed sub-steps are skipped, and the archiving process is retried from the interrupted part. When a fault occurs during the execution of the archiving process, the unfinished archiving task is identified, the checkpoint recovery context of the archiving task is recovered, and the archiving process is continued from the interrupted part. In this way, the cost of full retry is minimized, the recovery time is greatly shortened, and the atomicity of each archive archiving process is ensured.
[0082] As shown in Figure 2 Before receiving the electronic archives to be archived, various thread pools need to be initialized, various configuration parameters need to be loaded, and a monitoring service needs to be started to wait for the submission of electronic archives. When an electronic archive is submitted, the archiving task is received, feature analysis is performed on the archiving task, the archiving task is classified, CPU-intensive archiving tasks are assigned to CPU dedicated threads, IO-intensive archiving tasks are assigned to IO asynchronous threads, and mixed archiving tasks are assigned to elastic threads. Before thread execution, the scheduling weight and scheduling time slice are calculated, and dynamic allocation is performed according to the standard time slice. Then a checkpoint is created, and the archive processing execution is entered. The archiving task is checked on the checkpoint. If the archiving task is completed, the task is marked as completed, the processing log is recorded, the resources are cleaned up, and the archiving is ended. If the archiving task is not completed, the checkpoint is updated. If the standard time slice is exhausted at this time, the archiving task is put back into the scheduling queue, the scheduling weight is updated, and the next scheduling is waited for the archiving task execution phase. If the standard time slice is not exhausted, the archiving task execution is continued.
[0083] As shown in Figure 3 When a running fault is encountered, the system saves the current running state, scans the checkpoint database after the system is restarted, and identifies the unfinished tasks. The identification result includes completed, partially completed and not started. For the completed archiving task, the task is skipped and the next task is checked. For the partially completed task, the context is recovered from the latest checkpoint and the archiving process is continued from the interrupted part. For the not started task, the scheduling is restarted. This way of setting checkpoints and identifying tasks and recovering context from the interrupted part avoids full retry and realizes fast recovery.
[0084] As shown in Figure 4 During the entire archiving operation, the started monitoring service continuously works, the monitoring thread continuously collects data such as the waiting and execution time of the task, calculates the waiting time of each task, and analyzes the execution time allocation. Once it is detected that the waiting time of a task is too long, the scheduling strategy is automatically adjusted, and the scheduling weight of the task is temporarily increased to ensure the long-term fairness of scheduling. When it is detected that there is a task with too long waiting time, the scheduling strategy is challenged, the weight of the hungry task is increased, the time slice is redistributed, and then the monitoring data is updated, the performance report is generated, and the next monitoring period is waited.
[0085] The second embodiment of the application provides an electronic archive archiving device based on multi-thread fair scheduling, which comprises an archive receiving module, a weight calculation module and a processing execution module.
[0086] The archive receiving module is used for receiving electronic archives to be archived, performing feature analysis on the electronic archives, forming a feature analysis result, dividing the electronic archives into archiving task types according to the feature analysis result, and the archiving task types at least include CPU intensive, IO intensive and mixed type.
[0087] The weight calculation module is used for assigning a corresponding thread pool to the electronic archives according to the archiving task type corresponding to the electronic archives, calculating a scheduling weight according to the feature analysis result, setting a scheduling time slice according to the scheduling weight, and the thread pool at least includes a CPU intensive thread pool, an IO intensive thread pool and a mixed thread pool.
[0088] The processing execution module is used for distributing the electronic archives to the threads of the corresponding thread pool for archiving processing by using a scheduling engine, creating a checkpoint for fault recovery before archiving processing is executed, marking the completion of the archiving task after archiving processing is executed, and automatically releasing the threads occupied by the electronic archives.
[0089] As shown in Figure 5As shown, the device architecture is composed of four layers, namely a data reading layer, a business logic layer, a core scheduling layer and a data persistence layer, which work cooperatively. The file receiving module is located in the data reading layer and is responsible for receiving various archiving tasks. The business logic layer belongs to the core function implementation area, and the weight calculation module is located in the business logic layer and is responsible for calculating the scheduling weight, including a multi-dimensional weight calculator, a system load monitor, a weight coefficient adjuster and a weight standardizer. The business logic layer also includes other key modules, such as a task analysis module, a scheduling engine module, a processing execution module and a monitoring and optimization module. The task analysis module performs in-depth analysis on the received tasks, which is composed of a file feature analyzer, a task classifier, a complexity evaluator and a priority manager, to determine the properties of the tasks. The scheduling engine module makes scheduling decisions according to the weight and task type, which is composed of a task scheduler, a time slice manager, a thread pool manager and a resource allocator. The processing execution module is responsible for executing specific archiving operations and interacting with the final data persistence layer. The monitoring and optimization module continuously monitors the system state and scheduling fairness and adjusts the scheduling engine strategy in reverse.
[0090] In the core scheduling layer, efficient and fair scheduling is implemented, mainly including a multi-thread scheduler, a time slice allocator, a resource manager and a load balancer. The multi-thread scheduler includes a scheduling control center and three special thread pools for different task types, namely a CPU-intensive thread pool, an IO-intensive thread pool and a hybrid thread pool, which can isolate and optimize heterogeneous tasks. The time slice allocator, resource manager and load balancer work closely with the multi-thread scheduler to complete the dynamic allocation of resources.
[0091] In the data persistence layer, basic support is implemented. The data persistence layer is responsible for storing the state and metadata during system operation, including checkpoint data for fault recovery, task status data for recording task progress, configuration management data and log data, etc.
[0092] Since the first embodiment corresponds to the present embodiment, the present embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in the present embodiment, and the technical effects achieved in the first embodiment can also be achieved in the present embodiment. In order to reduce repetition, they will not be described here. Correspondingly, the related technical details mentioned in the present embodiment can also be applied in the first embodiment.
[0093] The third embodiment of the present application provides an electronic device, including at least one processor, a memory, the memory stores a computer program, and the at least one processor executes the computer program stored in the memory, so that the at least one processor executes the electronic file archiving method based on multi-thread fair scheduling as described in the first embodiment.
[0094] The fourth embodiment of the present application provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is used for implementing the electronic file archiving method based on multi-thread fair scheduling when executed by a processor.
[0095] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make various modifications and changes without any creative work based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope defined by the claims.
Claims
1. An electronic archive archiving method based on multi-threaded fair scheduling, characterized in that, Includes the following steps: The system receives electronic archives to be archived, performs feature analysis on the electronic archives based on their inherent attributes, generates feature analysis results, and classifies the electronic archives into archiving task types based on the feature analysis results. The archiving task types include at least CPU-intensive, I / O-intensive, and hybrid types. Based on the archiving task type corresponding to the electronic archive, a corresponding thread pool is allocated to the electronic archive. The scheduling weight is calculated based on the feature analysis results. The scheduling time slice is set according to the scheduling weight. The thread pool includes at least a CPU-intensive thread pool, an I / O-intensive thread pool, and a hybrid thread pool. The electronic archives are distributed to threads in the corresponding thread pools using a scheduling engine for archiving. Before archiving is executed, checkpoints for fault recovery are created. After archiving is executed, the archiving task is marked as complete, and the threads occupied by the electronic archives are automatically released. The feature analysis results include at least file size, processing complexity, and business priority. Before calculating the scheduling weight, corresponding adjustable coefficients are set for the file size, processing complexity, and business priority, respectively. The scheduling weight is obtained by multiplying the file size, processing complexity, and business priority by their respective adjustable coefficients. The scheduling time slice is inversely proportional to the scheduling weight, so that electronic files with high scheduling weights get shorter scheduling time slices, and both large and small electronic files can be fairly scheduled according to their own characteristics, thereby eliminating thread starvation. The scheduling weight can be calculated as W = f(α*log(V), β*C, γ*P), where α, β, and γ are dynamically adjustable coefficients to adapt to different system loads and business needs, V represents the file size, C represents the processing complexity, and P represents the processing priority. The processing complexity is set according to the archiving task processing type, which is divided into encryption, format conversion, and metadata extraction.
2. The electronic archive archiving method based on multi-threaded fair scheduling according to claim 1, characterized in that, Before receiving the electronic archives to be archived, the method further includes: initializing various thread pools and starting the monitoring service.
3. The electronic archive archiving method based on multi-threaded fair scheduling according to claim 1, characterized in that, During the archiving process, if the archiving task is not completed within a standard time slice, the checkpoint of the archiving task is updated. After the standard time slice is exhausted, the archiving task is put back into the scheduling queue, and the scheduling weight and scheduling time slice of the archiving task are recalculated.
4. The electronic archive archiving method based on multi-threaded fair scheduling according to claim 1, characterized in that, When a failure occurs during archiving, the incomplete archiving task is identified, the context is restored from the checkpoint of the archiving task, and archiving is resumed only from the point of interruption.
5. An electronic archive archiving device based on multi-threaded fair scheduling, characterized in that, Includes the following modules: The archive receiving module is used to receive electronic archives to be archived, perform feature analysis on the electronic archives according to their inherent attributes, generate feature analysis results, and classify the electronic archives into archiving task types according to the feature analysis results. The archiving task types include at least CPU-intensive, IO-intensive, and hybrid types. The weight calculation module is used to allocate a corresponding thread pool to the electronic archive according to the archiving task type corresponding to the electronic archive, calculate the scheduling weight according to the feature analysis results, and set the scheduling time slice according to the scheduling weight. The thread pool includes at least a CPU-intensive thread pool, an I / O-intensive thread pool, and a hybrid thread pool. The processing and execution module is used to distribute the electronic archives to the threads of the corresponding thread pool for archiving processing using the scheduling engine. Before the archiving process is executed, a checkpoint for fault recovery is created. After the archiving process is executed, the archiving task is marked as completed and the threads occupied by the electronic archives are automatically released. The feature analysis results include at least file size, processing complexity, and business priority. Before calculating the scheduling weight, corresponding adjustable coefficients are set for the file size, processing complexity, and business priority, respectively. The scheduling weight is obtained by multiplying the file size, processing complexity, and business priority by their respective adjustable coefficients. The scheduling time slice is inversely proportional to the scheduling weight, so that electronic files with high scheduling weights get shorter scheduling time slices, and both large and small electronic files can be fairly scheduled according to their own characteristics, thereby eliminating thread starvation. The scheduling weight can be calculated as W = f(α*log(V), β*C, γ*P), where α, β, and γ are dynamically adjustable coefficients to adapt to different system loads and business needs, V represents the file size, C represents the processing complexity, and P represents the processing priority. The processing complexity is set according to the archiving task processing type, which is divided into encryption, format conversion, and metadata extraction.
6. An electronic device, characterized in that, It includes at least one processor and a memory, the memory storing a computer program, the at least one processor executing the computer program stored in the memory, such that the at least one processor performs the electronic record archiving method based on multi-threaded fair scheduling as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, is used to implement the electronic archive archiving method based on multi-threaded fair scheduling as described in any one of claims 1 to 4.
8. A computer program product, characterized in that, Includes program code that, when the computer runs the computer program, executes the electronic archive archiving method based on multi-threaded fair scheduling as described in any one of claims 1 to 4.