File upload completion judgment method and terminal
By initiating monitoring tasks in file system events, detecting file size changes according to sampling periods, and combining stable counting and timeout protection, the problem of inaccurate completion judgment during large file uploads is solved, achieving efficient and accurate file upload completion judgment, and improving system reliability and monitoring efficiency.
Patent Information
- Application Number
- CN202610323394.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-17
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies cannot accurately determine whether a file has been completely written during large file uploads. This can cause the monitoring program to start processing before the file is fully written, resulting in problems such as reading incomplete files, parsing failures, or program crashes.
The monitoring task is initiated in response to file system events. The file size attribute is obtained according to the preset sampling period, and the current size attribute is compared with the historical attribute to update the stability count. When the stability count reaches the preset threshold, the file writing status is determined to be completed. Combined with multi-level filtering and timeout protection mechanisms, the upload completion time is accurately identified.
It enables accurate identification of when file uploads are completed, avoiding waste of system resources and misjudgments, improving monitoring efficiency and accuracy, and is applicable to various file writing scenarios, thus enhancing the robustness and reliability of the system.
Smart Images
Figure CN122457590A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of file management, and in particular to a method and terminal for determining file upload completion. Background Technology
[0002] In scenarios such as file management systems, NAS (Network Attached Storage), and cloud storage, it is necessary to monitor directory file changes in real time and automatically trigger subsequent processing, such as metadata extraction and video transcoding, after a file upload is completed. Traditional technologies rely on the operating system's file event mechanism (such as inotify and ReadDirectoryChangesW) to detect file creation or write events, but these events only reflect file operation behavior and cannot accurately determine whether the file has been completely written.
[0003] Especially in scenarios involving the transfer of large files over a network, file writing is a continuous process, and the operating system may trigger events multiple times before the file is fully written. If the monitoring program starts processing at this time, it will lead to problems such as reading incomplete files, parsing failures, and program crashes. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a method and terminal for determining the completion of file upload, so as to accurately identify the timing of file upload completion.
[0005] A method for determining file upload completion, the method comprising: In response to file system events of the target file, initiate a monitoring task for the target file; In the monitoring task, the size attribute of the target file is acquired according to a preset sampling period; The currently acquired size attribute is compared with the historically acquired size attribute, and the stable count is updated based on the comparison result; When the stability count reaches a preset stability threshold, the writing status of the target file is determined to be completed.
[0006] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows: A terminal for determining file upload completion includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it performs the following steps: In response to file system events of the target file, initiate a monitoring task for the target file; In the monitoring task, the size attribute of the target file is acquired according to a preset sampling period; The currently acquired size attribute is compared with the historically acquired size attribute, and the stable count is updated based on the comparison result; When the stability count reaches a preset stability threshold, the writing status of the target file is determined to be completed.
[0007] The beneficial effects of this invention are as follows: By initiating a monitoring task in response to file system events of the target file, this invention avoids the waste of system resources caused by continuous polling. Subsequent processes are triggered only when a file is actually created or written, thus improving monitoring efficiency. In the monitoring task, the size attribute of the target file is acquired at a preset sampling period, and the currently acquired size attribute is compared with the historically acquired size attribute. The stability count is updated based on the comparison result, enabling the system to continuously track size changes during file writing. Only when the stability count reaches a preset stability threshold does the system finally determine that the writing status of the target file is complete. This method, based on multiple consecutive stability checks, effectively avoids false judgments caused by single static events due to factors such as instantaneous network fluctuations, disk cache refreshes, and process scheduling pauses. Compared to traditional schemes that only determine whether the file size has changed, stability verification through multiple consecutive sampling periods improves the accuracy of identifying the completion timing in scenarios such as large file uploads and slow network transmissions. Attached Figure Description
[0008] Figure 1 A flowchart illustrating the steps of a file upload completion determination method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating the steps of an application embodiment of the file upload completion determination method provided in this invention. Figure 3 This is a schematic diagram of the structure of a terminal for determining file upload completion according to an embodiment of the present invention; Label Explanation: 1. A terminal for determining file upload completion; 2. A processor; 3. A memory. Detailed Implementation
[0009] Table 1 below provides a detailed explanation of the definitions of terms used in this application.
[0010] Table 1. Glossary of Terms
[0011] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.
[0012] In existing technologies, scenarios such as file management systems, network attached storage, and cloud storage synchronization often require real-time monitoring of file changes in a specified directory and automatic triggering of subsequent processing flows after a new file is uploaded. Traditional technologies typically rely on the file system event mechanism provided by the operating system to detect file operations. However, these events only reflect file operation behavior and cannot accurately determine whether the file has been completely written. Especially in scenarios where large files are transmitted over a network, file writing is a continuous process, and the operating system may trigger events multiple times before the file is completely written. If the monitoring program immediately starts processing logic at this stage, it will lead to incomplete file reading, parsing failure, or even program crashes. Related solutions, such as temporary extension mechanisms, file locking mechanisms, or callback notification mechanisms, all have limitations such as strong dependence on client behavior, poor cross-platform compatibility, and susceptibility to false positives and false negatives. Therefore, how to accurately identify the timing of file upload completion without relying on client cooperation has become a key technical challenge for improving the reliability of automated processing systems.
[0013] To at least address the aforementioned issues, this invention initiates a monitoring task in response to file system events of the target file. It acquires file size attributes at a preset sampling period, compares the current size attribute with historical data, and updates the stability count. When the stability count reaches a preset threshold, it determines that the file writing is complete. In this way, without relying on client behavior or file locking mechanisms, it can accurately identify the upload completion time through stability detection over multiple consecutive time windows, thus improving the accuracy of the judgment.
[0014] Please refer to Figure 1 A method for determining file upload completion includes steps 110 to 140.
[0015] Step 110: In response to file system events of the target file, initiate a monitoring task for the target file. For example, when the operating system detects a file creation or write event in the target directory, the listening module captures the event and parses the file path. After the file filtering module filters out the business target files that meet the conditions according to preset rules, it initiates an independent asynchronous monitoring task for the filtered files.
[0016] Step 120: In the monitoring task, acquire the size attribute of the target file according to the preset sampling period. For example, after the monitoring task starts, acquire the size attribute of the target file at the beginning of each sampling period.
[0017] Step 130: Compare the currently acquired first size attribute with the previously acquired second size attribute, and update the stable count based on the comparison result. For example, compare the currently acquired first size attribute of the target file with the previously acquired second size attribute, and update the stable count based on the comparison result.
[0018] Step 140: When the stability count reaches the preset stability threshold, change the write status of the target file to "completed". For example, when the updated stability count reaches the preset stability threshold, change the write status of the target file to "completed".
[0019] As described above, the beneficial effects of this invention are as follows: by initiating a monitoring task in response to file system events, continuously monitoring file size changes according to a preset sampling period, and accurately identifying the upload completion time based on size stability detection over multiple consecutive sampling periods. This method does not rely on specific upload client behaviors, file locking mechanisms, or temporary naming conventions, and is applicable to various file writing scenarios; the built-in timeout protection mechanism effectively prevents the monitoring task from being suspended indefinitely due to network interruptions or client anomalies, improving system robustness and reliability; it can be seamlessly integrated into existing file processing systems as an independent middleware module, offering the advantages of high versatility, high accuracy, and low invasiveness.
[0020] Furthermore, step 130 also includes steps 131 to 133.
[0021] Step 131: Compare the currently acquired first size attribute with the previously acquired second size attribute. For example, after acquiring the current size of the target file in each sampling period, the monitoring task compares the current size attribute with the second size attribute acquired in the previous sampling period to determine whether the file size has changed within this sampling interval.
[0022] Step 132: If the comparison result shows that the first size attribute and the second size attribute are the same, then perform an accumulation operation on the stability count. For example, if the current size is the same as the previous size, it indicates that the file size has not changed in the current sampling period, and the file writing may be in a temporary or completed state. In this case, the stability count value is incremented by 1.
[0023] Step 133: If the comparison result shows that the first size attribute and the second size attribute are different, then the stability count is reset, and the first size attribute is used as the new second size attribute. For example, if the current first size attribute is different from the historical second size attribute, it indicates that the file is still being written. In this case, the stability count is reset, and the currently acquired size attribute is updated to the global state table as the historical comparison benchmark for the next sampling period.
[0024] Table 2 Global Monitoring Status Table
[0025] As described above, the comparison and counting mechanism across multiple consecutive sampling periods effectively distinguishes between brief pauses during file writing and true write completion. In file upload scenarios, factors such as instantaneous network congestion, disk buffer flushing, and process scheduling often cause temporary pauses in file size. A simple judgment based on a single instance of unchanged size can easily misjudge such false stability as upload completion. This solution sets a stability threshold N, requiring the file size to remain completely unchanged for N consecutive sampling periods, fundamentally filtering out these interference factors. Whenever a size change is detected, the stability count is reset to zero, ensuring that only files that have truly completed writing and have not changed for an extended period trigger a completion determination. Furthermore, this mechanism works in conjunction with timeout protection to degrade long-term pauses caused by network interruptions, client crashes, or other anomalies, preventing monitoring tasks from being suspended indefinitely and improving the accuracy and stability of the automated file processing system.
[0026] Furthermore, steps 111 and 112 are included before starting the monitoring task in response to a file system event of the target file.
[0027] Step 111: Filter the target files that trigger file system events according to preset filtering conditions to obtain the filtering results, and exclude temporary files from the filtering results according to preset naming rules to obtain candidate files. For example, after the listening module captures a file system event, it first parses the full path, directory name, file name, and extension of the file; then it matches the file path with the preset business path whitelist, and at the same time matches the file extension with the extension whitelist that the business is concerned about. Only files that meet both the path and extension conditions are processed further. On this basis, it further excludes various temporary files and intermediate files based on common naming characteristics of temporary files, such as suffixes like ".tmp", ".part", ".uploading", etc., or temporary flags in file attributes. After the above multi-level filtering, the files that are finally retained are the candidate files that need to be judged for upload completion.
[0028] Step 112: Start monitoring tasks for candidate files. For example, for the candidate files selected in Step 111, the system assigns them independent asynchronous monitoring tasks and registers the file information to the global monitoring status table so that the file size changes can be continuously monitored according to the preset sampling period.
[0029] As described above, by setting up a multi-level filtering mechanism before starting the monitoring task, the target files of business interest can be accurately identified: First, business rules are matched based on path whitelists and extension whitelists to ensure that only files of specified types in specific directories are processed; then, various temporary files, intermediate files, and cache files are excluded based on the naming characteristics and attribute flags of temporary files, avoiding the mistaken inclusion of incompletely uploaded files in the monitoring scope. This mechanism effectively eliminates interference from irrelevant files to the monitoring system, avoids consuming system thread resources, memory resources, and I / O overhead by creating monitoring tasks for a large number of non-target files, improves monitoring efficiency and system performance, and ensures that limited system resources can be concentrated on the target files that truly need attention.
[0030] Furthermore, steps 113 to 115 are included before step 110 starts the monitoring task for the target file.
[0031] Step 113: Query the global monitoring status record to determine if the target file already has a corresponding monitoring task running. For example, when a file is identified as a candidate file through the filtering mechanism, the system queries the global monitoring status table using the file path as the key to check if the file has been registered and if a corresponding monitoring task is running.
[0032] Step 114: If no corresponding monitoring task is running for the target file, a monitoring status record is created and a monitoring task for the target file is started. For example, if the query result shows that the file has not yet been monitored, the system creates a new status record for the file in the global monitoring status table, initializes fields such as monitoring start time and stability count to zero, and allocates an independent monitoring coroutine or thread from the asynchronous task scheduler to be specifically responsible for the periodic sampling and stability testing of the file.
[0033] Step 115: If no corresponding monitoring task is running for the target file, create a monitoring status record and start the monitoring task for the target file; otherwise, do not start the monitoring task for the target file. For example, if the query results show that the file is currently under monitoring, it means that a previous event has already triggered a monitoring task. In this case, there is no need to create a duplicate monitoring task for the same file; simply abandon the processing of this event to avoid wasting resources and data inconsistency.
[0034] As described above, by querying the global monitoring status record before starting a monitoring task, the deduplication management of monitoring tasks can be effectively achieved, avoiding the creation of duplicate monitoring tasks due to multiple event triggers of the same file, and preventing waste of system resources and status chaos. At the same time, by centrally maintaining the monitoring status of each file, a unified data access point is provided for subsequent periodic sampling and stability judgment, ensuring the coordination and reliable operation of the entire monitoring process.
[0035] Further, step 120 includes steps 121 and 122.
[0036] Step 121: At the beginning of each sampling period, suspend the monitoring task for a preset sampling interval. For example, after the monitoring task starts, it enters a loop execution state. At the beginning of each period, the task actively suspends its own thread or coroutine, waiting for the preset sampling interval to end. The sampling interval can be dynamically configured according to the business scenario, such as setting it to 2 seconds, to balance the real-time performance of monitoring and system I / O overhead, and to avoid excessively frequent attribute queries from putting unnecessary pressure on the file system and storage devices.
[0037] Step 122: After the suspension ends, call the system interface to obtain the current first size attribute of the target file. For example, after the suspension time expires, the monitoring task resumes execution and reads the current first size attribute of the target file through file attribute acquisition interfaces provided by the operating system, such as stat() and GetFileSize().
[0038] As described above, by suspending monitoring tasks at preset intervals to achieve periodic sampling, the continuity and regularity of file size monitoring are ensured, while avoiding excessive consumption of system performance due to uninterrupted polling. The configurability of the sampling interval allows this solution to flexibly adapt to different business scenarios with varying requirements for real-time performance and resource consumption, ensuring both accuracy of judgment and system operating efficiency.
[0039] Furthermore, step 110, which initiates the monitoring task for the target file, also includes steps 1101 to 1103.
[0040] Step 1101: When starting a monitoring task, record the start time of the monitoring task. For example, when the system creates a monitoring task for a candidate file and registers it to the global monitoring status table, it simultaneously records the current system timestamp as the start time of the monitoring task and stores it in the status record corresponding to that file, serving as the time base for subsequent timeout judgment.
[0041] Step 1102: During the execution of the monitoring task, obtain the time span between the current time and the start time. For example, after completing the acquisition of the size attribute and the update of the stable count in each sampling period, the monitoring task reads the current system time, calculates the time difference with the start time, and obtains the total duration of continuous monitoring of the file, which is used to determine whether timeout protection has been triggered.
[0042] Step 1103: If the time span reaches the preset timeout threshold, terminate the monitoring task. For example, compare the calculated time span with the system's preset maximum monitoring timeout, such as 10 minutes; if the total monitoring duration has reached or exceeded the timeout threshold, regardless of the current stable count value, immediately terminate the monitoring task, release related resources, mark the file as an abnormal monitoring state, and trigger the preset degradation process, such as recording alarm logs, notifying the administrator, or moving the file to the pending review queue for manual intervention.
[0043] As described above, introducing a timeout protection mechanism into monitoring tasks can effectively address various abnormal file write stall states. In scenarios such as network transmission interruptions, client crashes, or unexpected termination of the upload process, file sizes may remain unchanged for extended periods, preventing the stable count from reaching the preset threshold. Without timeout protection, monitoring tasks will be suspended indefinitely, causing continuous system resource consumption and task backlog. This solution sets a maximum monitoring timeout threshold, defining a clear lifecycle limit for each monitoring task. Regardless of the file write status, resources are forcibly released after the timeout, preventing monitoring backlog and system performance degradation caused by abnormal files. Simultaneously, the degradation handling mechanism after timeout provides a flexible processing path for the subsequent handling of abnormal files, improving the system's robustness, self-healing ability, and maintainability, ensuring the long-term stable and reliable operation of the entire file monitoring system.
[0044] Furthermore, after step 1103 terminates the monitoring task, steps 1104 and 1105 are also included.
[0045] Step 1104: Mark the target file as an abnormal monitoring status. For example, after a monitoring task is forcibly terminated due to timeout, the system updates the status field corresponding to the file to "monitoring abnormal" in the global monitoring status table or a separate abnormal record table, and records information such as the reason for the abnormality ("upload timeout"), the timestamp of the abnormality, and the file size at the time of termination, providing data support for subsequent problem tracking and statistical analysis.
[0046] Step 1105: Trigger the preset degradation process and move the target file into the review queue. For example, based on the degradation policy configured in the business, the system automatically triggers the corresponding processing actions: push the file path and related abnormal information to the review message queue to notify the administrator or monitoring platform for manual verification; or move the file to a dedicated abnormal file storage directory to await subsequent scanning or reprocessing mechanisms; at the same time, detailed alarm logs can be recorded to facilitate operations and maintenance personnel in locating and analyzing whether there are abnormalities in network status, client behavior, or file system.
[0047] As described above, by introducing anomaly marking and degradation handling mechanisms after timeout termination, the file monitoring system's anomaly handling capabilities are further improved. When a file cannot be uploaded for an extended period due to network interruption, client crash, or other reasons, the system can not only release resources promptly through timeout protection but also incorporate such abnormal files into a separate tracking and management process to prevent files from being missed or forgotten. Anomaly status marking provides basic data for problem tracking and statistical analysis, helping to identify business paths where system bottlenecks or frequent network failures occur. The degradation handling process gives the system the ability to flexibly respond to anomalies, allowing it to choose different handling methods such as manual intervention, delayed retry, or automatic ignoring based on business importance, ensuring that critical business operations are not affected by individual abnormal files. This significantly improves the system's robustness, observability, and operational friendliness in the face of complex network environments and uncontrollable client behavior.
[0048] Furthermore, after step 140 determines that the writing status of the target file is complete, step 141 is also included.
[0049] Step 141: Terminate the monitoring task and remove the monitoring status of the target file from the global status record. For example, when the stable count reaches a preset threshold and the file upload is completed, the system immediately terminates the monitoring coroutine or thread corresponding to the file and stops subsequent periodic sampling; at the same time, the status record of the file, including all monitoring information such as start time, last size, and stable count, is deleted from the global monitoring status table using the file path as the key. The above cleanup operation is usually guaranteed by delayed calls or final execution blocks to ensure that the resource release action can be reliably executed regardless of whether subsequent processing is abnormal.
[0050] As described above, by promptly terminating the monitoring task and removing the status record after the file upload is completed, effective reclamation of system resources and timely release of the state space are achieved. As a long-running, lightweight process, the monitoring task will continuously occupy system thread resources and memory space if not terminated promptly; if the global status record only increases and never decreases, the status table will expand indefinitely, affecting query efficiency and system stability. This solution performs a cleanup operation immediately after the judgment is completed, avoiding the risks of resource leakage and monitoring backlog. More importantly, after removing the file entry from the global status record, if a new file is written to the same path, the entire monitoring process can be retried, ensuring the system's ability to correctly handle scenarios of repeated file writes. Through delayed calls or final execution blocks, the reliability and fault tolerance of the cleanup operation are further enhanced, providing a solid foundation for the long-term stable operation of the entire file monitoring system.
[0051] Furthermore, after step 140 determines that the writing status of the target file is completed, steps 142 to 145 are also included.
[0052] Step 142: Trigger the subsequent business processing flow of the target file by invoking a pre-defined post-processing function or publishing an event message. For example, the system selects a synchronous or asynchronous triggering method based on the integration requirements of the business scenario: for locally integrated processing modules, a pre-registered callback function can be directly called; for distributed systems or microservice architectures, a file ready event is published through a message queue to notify downstream services to process it.
[0053] Step 143: The post-processing function is used to directly trigger subsequent business processing of the target file. For example, in monolithic applications or tightly coupled systems, the system passes the file path as a parameter to a preset processing function. This function synchronously performs subsequent operations such as metadata extraction or index building within the same process space, and the caller waits for the processing to complete before continuing execution. This approach is simple to implement and suitable for lightweight processing scenarios with high real-time requirements.
[0054] Step 144: Publish event messages to asynchronously trigger subsequent business processing flows for the target file. For example, in a distributed deployment or loosely coupled architecture, the system publishes structured messages containing information such as file path, size, and modification time to a message queue or Webhook callback interface. Multiple downstream processing services subscribe to the message and execute different business logics in parallel, such as video transcoding, content scanning, and backup archiving. This approach achieves decoupling of the processing flow and parallelization of tasks, improving the system's scalability and throughput.
[0055] Step 145: The subsequent business processing flow includes at least one of the following: metadata extraction, index building, content scanning, backup archiving, and video transcoding. For example, for media files, FFmpeg can be used to extract metadata such as duration and resolution and generate thumbnails; for document files, an OCR engine can be called to recognize text content and build a full-text search index; for various types of files, virus scanning, encryption detection, or automatic backup to remote storage can be performed; the system can flexibly combine multiple processing flows according to business needs to achieve one-stop automated processing after upload.
[0056] As described above, by providing a flexible triggering mechanism and rich business processing capabilities to determine file upload completion, this solution achieves an organic connection between file monitoring and business processing. Synchronous post-processing functions are suitable for scenarios with immediate dependencies on processing results, ensuring that files are processed immediately upon readiness. Asynchronous event messages endow the system with high scalability and modularity, supporting the parallel execution and independent evolution of multiple processing tasks. Optional business processing flows cover the most common automation needs in file management, enabling this solution to be seamlessly embedded as a general-purpose middleware into various business systems. By combining accurate upload completion judgment with a flexible business triggering mechanism, this solution provides complete technical support for building a highly reliable and automated file processing pipeline, improving the efficiency and stability of the entire link from file upload to business processing.
[0057] Please refer to Figure 2 The following describes specific application embodiments of the present invention. The present invention can apply the above solution to various scenarios requiring real-time monitoring of file uploads and automatic triggering of subsequent processing, particularly in the field of file processing platforms such as network attached storage and cloud storage synchronization services. Taking a company's media asset management system as an example, the specific implementation includes the following steps: S201. Taking the example of a media producer uploading a 4K high-definition video file to a monitoring directory via FTP, when the upload operation begins, the operating system detects a file creation event. During the upload process, as data packets are continuously written, the operating system continuously triggers file write events. The event listening module captures these raw events and parses information such as event type, file path, file name, and extension. For example, capturing the write event for the file " / uploads / 4k_sample.mp4" reveals the directory as " / uploads", the file name as "4k_sample.mp4", and the extension as ".mp4". This is equivalent to step 110 above.
[0058] S202. The system first filters according to preset configuration rules: it matches the file path with the business path whitelist, such as " / uploads", and compares the file extension with the list of media file extensions that the business is concerned about, such as ".mp4", ".mov", and ".avi". Only when both match successfully is the file identified as a target file. For the ".mp4" file mentioned above, since both its path and extension meet the conditions, it is determined to be a target file. Subsequently, the system queries the global monitoring status table to confirm that the file has not yet been monitored, so it creates a monitoring status record for it and starts an independent asynchronous monitoring task. If the file is a temporary file, such as one with a ".tmp" suffix, or the path is not within the scope of concern, the system directly ignores the event and does not start a monitoring task. This is equivalent to steps 111 to 112 and steps 113 to 115 above.
[0059] The system configures the sampling interval to 2 seconds based on the business's real-time requirements. At the beginning of each sampling period, the monitoring task actively suspends its own coroutine and waits for 2 seconds to avoid uninterrupted polling of the file system. For example, for a 4K video file that is being uploaded, the monitoring task wakes up every 2 seconds to prepare to obtain the latest file size information. This is equivalent to step 121 above.
[0060] After the monitoring task is activated, it calls an operating system interface such as stat() to obtain the current byte size of the target file. During the initial sampling, the current size is recorded as the previous size, and the stability counter is initialized to 0. In each subsequent cycle, the current size is compared with the previous size recorded in the status table to determine if the file size has changed within the sampling interval. This is equivalent to steps 122 and 131 above.
[0061] S203. If the current size is the same as the previous size, it indicates that the file size has not changed within the current sampling period, and the system increments the stability counter by 1. For example, if data transmission is paused due to momentary network congestion during the upload process, and the file size remains unchanged for two consecutive sampling periods, the stability counter accumulates to 2. If the current size is different from the previous size, it indicates that the file is still being written to, and the system resets the stability counter to 0 and updates the current size to the new previous size as the comparison benchmark for the next period. For example, if the file size increases from 100MB to 150MB after the upload resumes, the system detects the change and resets the counter to 0. This is equivalent to steps 132 to 133 above.
[0062] S204. The system's preset stability threshold is 3, which means that the file size must remain completely unchanged for 3 consecutive sampling periods before the upload can be considered complete. At the same time, the preset maximum monitoring timeout is 10 minutes to prevent the task from being suspended indefinitely due to network interruptions or other abnormalities.
[0063] After the counter is updated in each sampling period, the system first checks whether the stable counter is greater than or equal to a preset threshold N. If the stable counter reaches 3, it indicates that the file size has not changed for 6 consecutive seconds, and the system determines that the file upload is complete. For example, after a 4K video file is fully uploaded, its size remains stable for 3 consecutive sampling periods, and the system confirms that the file is ready.
[0064] If the stability counter has not yet reached the threshold, the system further determines whether the total monitoring duration has exceeded the preset timeout. The system records the start timestamp when the monitoring task starts and calculates the difference between the current time and the start time each cycle. If the total monitoring duration reaches 10 minutes and the stability counter still has not reached the threshold, it indicates a possible network interruption, client crash, or other abnormal situation. The system immediately terminates the monitoring task to avoid prolonged resource occupation. For example, if a network failure during the upload process causes the file to stall at 50% progress for 10 minutes, the system triggers timeout protection to terminate the task. This is equivalent to steps 1101 to 1103 and step 140 above.
[0065] S205. For uploaded files, the system terminates the monitoring task and removes the file entry from the global status record. Simultaneously, it calls the preset post-processing interface or publishes an event message to initiate subsequent processing. For example, the system pushes the file path " / uploads / 4k_sample.mp4" to the message queue. After the media processing service subscribes to this message, it performs parallel metadata extraction, such as parsing video duration, resolution, and bitrate information; initiates a transcoding task to generate playback versions at multiple resolutions; calls the intelligent analysis service for scene recognition and content review; and archives the original file and transcoded output to object storage. The entire processing flow is automated and requires no manual intervention.
[0066] For tasks that time out, the system marks the file as being in an abnormal monitoring state, records the exception log, and triggers a degradation process, such as moving the file to a pending review directory and notifying operations personnel to investigate network or client issues. Regardless of whether the file upload is completed normally or the timeout occurs, the system ensures that the monitoring task is terminated, the status record is removed, and resources are reclaimed, preparing for the next round of file monitoring. For example, after processing, the system waits for the next file system event and continues to loop through the above steps. This is equivalent to steps 141 to 145 and steps 1104 to 1105 above.
[0067] Through the above application examples, this invention implements a universal and highly reliable solution for determining file upload completion, achieving closed-loop management of the entire process from event listening, file filtering, asynchronous monitoring, periodic sampling, continuous stability testing, timeout protection to business triggering. This solution effectively addresses typical pain points of traditional file monitoring solutions, such as strong reliance on client behavior, poor cross-platform compatibility, susceptibility to false positives and false negatives, and insufficient system robustness. It has significant engineering practical value in improving the automation level of file processing, reducing manual intervention costs, and ensuring the accuracy of business processing.
[0068] Please refer to Figure 3A terminal 1 for determining file upload completion includes a memory 3, a processor 2, and a computer program stored on the memory 3 and running on the processor 2. When the processor 2 executes the computer program, it implements each step of the above-mentioned text upload completion determination method.
[0069] In summary, this invention provides a method and terminal for determining file upload completion. This method initiates an asynchronous monitoring task in response to file system events, continuously monitors file size changes according to a preset sampling period, and introduces a stability detection mechanism with multiple consecutive time windows, thereby achieving accurate identification of the timing of file upload completion.
[0070] In terms of event monitoring and file filtering, the system captures file creation or writing events, parses file paths and extensions, performs multi-level filtering by combining business path whitelists and extension whitelists, and excludes temporary files according to naming rules to ensure that monitoring tasks are only started for business target files that meet the conditions; it also manages task deduplication by querying global monitoring status records to avoid creating monitoring tasks repeatedly for the same file, effectively saving system resources.
[0071] For stability detection, the system assigns an independent asynchronous monitoring task to each target file, periodically acquiring the file size attribute according to a configurable sampling interval. Each sampling period compares the currently acquired first size attribute with the historical second size attribute: if they are the same, it indicates that the file size has not changed within the period, and the stability counter is incremented; if they are different, it indicates that the file is still being written, and the stability counter is reset to zero, and the current size is updated to the new historical comparison benchmark. By requiring the stability counter to continuously accumulate to a preset threshold before determining upload completion, this solution effectively filters out temporary false stability caused by network congestion, disk buffer flushing, etc., significantly improving the accuracy of the judgment.
[0072] Regarding timeout protection and anomaly handling, the system records the start time when a monitoring task is initiated and calculates the total monitoring duration in each sampling period. If the total duration exceeds the preset maximum timeout threshold and the stable count still fails to meet the standard, the monitoring task is forcibly terminated to prevent the task from being suspended indefinitely due to network interruptions, client crashes, or other anomalies. At the same time, the file is marked as being in an abnormal monitoring state, triggering a degradation process such as moving it to a pending review queue or notifying the administrator to intervene, thereby enhancing the robustness and maintainability of the system.
[0073] In terms of business triggering and resource recovery, the system promptly terminates the monitoring task and removes the file entry from the global status record after the file upload is completed, ensuring that new files on the same path can re-trigger the monitoring process. At the same time, it flexibly triggers subsequent business processing processes, including metadata extraction, index building, content scanning, backup archiving, and video transcoding, by calling preset post-processing functions or publishing event messages, thus achieving a seamless connection from the completion of file upload to automated business processing.
[0074] By constructing a closed-loop management system covering the entire process from event listening, file filtering, asynchronous monitoring, periodic sampling, continuous stability testing, timeout protection to business triggering, this invention effectively solves the core technical problems of traditional file monitoring solutions, such as strong reliance on client behavior, poor cross-platform compatibility, susceptibility to false positives and false negatives, and insufficient system robustness. It provides a universal and reliable solution for determining file upload completion for file management systems, cloud storage platforms, and media asset processing systems.
[0075] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A method for determining file upload completion, characterized in that, include: In response to file system events of the target file, initiate a monitoring task for the target file; In the monitoring task, the size attribute of the target file is acquired according to a preset sampling period; Compare the currently acquired first size attribute with the historically acquired second size attribute, and update the stable count based on the comparison result; When the stability count reaches the preset stability threshold, the writing status of the target file is changed to "completed".
2. The method for determining file upload completion according to claim 1, characterized in that, The step of comparing the currently acquired first size attribute with the historically acquired second size attribute and updating the stable count based on the comparison result includes: Compare the currently acquired first size attribute with the previously acquired second size attribute; If the comparison result shows that the first size attribute and the second size attribute are the same, then an accumulation operation is performed on the stable count; If the comparison result shows that the first size attribute and the second size attribute are different, then a reset operation is performed on the stable count, and the first size attribute is used as the new second size attribute.
3. The method for determining file upload completion according to claim 1, characterized in that, The process, which includes responding to a file system event of the target file but before starting the monitoring task, further includes: The target files that trigger the file system event are filtered according to preset filtering conditions to obtain filtering results, and temporary files are excluded from the filtering results according to preset naming rules to obtain candidate files. The monitoring task is initiated for the candidate file.
4. The method for determining file upload completion according to claim 1, characterized in that, Before initiating the monitoring task for the target file, the following steps are also included: Query the global monitoring status records to determine whether the target file already has a corresponding monitoring task running; If the target file does not have a corresponding monitoring task running, then a monitoring status record is created and the monitoring task for the target file is started; If the target file has a corresponding monitoring task running, then the monitoring task for the target file will not be started.
5. The method for determining file upload completion according to claim 1, characterized in that, The step of acquiring the size attribute of the target file according to a preset sampling period in the monitoring task includes: At the beginning of each sampling period, the monitoring task is suspended for a preset sampling interval duration; After the suspension ends, the system interface is called to obtain the current first size attribute of the target file.
6. The method for determining file upload completion according to claim 1, characterized in that, The step of initiating the monitoring task for the target file also includes: When the monitoring task is started, the start time of the monitoring task is recorded; During the execution of the monitoring task, the time span between the current time and the start time is obtained; If the time span reaches a preset timeout threshold, the monitoring task is terminated.
7. The method for determining file upload completion according to claim 6, characterized in that, After terminating the monitoring task, the method further includes: The target file is marked as being in an abnormal monitoring state; The preset downgrade process is triggered, and the target file is moved into the waiting queue for review.
8. The method for determining file upload completion according to claim 1, characterized in that, After determining that the writing status of the target file is complete, the process further includes: Terminate the monitoring task and remove the monitoring status of the target file from the global status record.
9. The method for determining file upload completion according to claim 1, characterized in that, After determining that the writing status of the target file is complete, the process further includes: The subsequent business processing flow of the target file is triggered by calling a preset post-processing function or publishing an event message; The post-processing function is used to directly trigger the subsequent business processing flow of the target file; The published event message is used to asynchronously trigger the subsequent business processing flow of the target file; The subsequent business processing flow includes at least one of the following: metadata extraction, index creation, content scanning, backup archiving, and video transcoding.
10. A terminal for determining file upload completion, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements each step of the file upload completion determination method according to any one of claims 1 to 9.