11 results about "Temporary file" patented technology

The application discloses a Flink-based unstructured file transmission method and system. The method comprises the following steps: reading a file from object storage and encapsulating the file into a data object through a self-defined Source operator; splitting the data object into a data block stream containing block serial numbers through a flatMap operator; grouping the data block stream based on a file name keyBy, so that data blocks of the same file are routed to the same task instance; initializing an expected block serial number, an out-of-order block cache, a completion flag and a temporary file path through a KeyedProcessFunction, sequentially writing a local temporary file and processing out-of-order blocks, uploading the temporary file to a target object storage after completion, and cleaning up the state. The application fully utilizes the distributed parallel capability and state management mechanism of Flink, realizes efficient parallel transmission, out-of-order recombination, integrity check and breakpoint resume of unstructured files, simplifies the integration with a Flink stream batch integrated architecture, and improves transmission efficiency and reliability.

This invention discloses a workshop production scheduling method and system based on deep reinforcement learning and multimodal multi-objective computation. The method includes the following steps: obtaining a population size N, an objective function, and a maximum number of iterations; randomly generating an initial population based on the population size and the objective function, defining the state of the initial population, and constructing a deep Q-network; setting a counter and comparing the counter with a stagnation check period; adaptively selecting operators for the initial population using the deep Q-network to generate a child population; generating three temporary files based on the child population and merging them to obtain a merged file; performing environment selection on the merged file to obtain a new population and calculating state-reward pairs; updating the deep Q-network every 50 iterations based on the state-reward pairs; re-comparing the counter with the stagnation check period, and outputting the scheduling result if the counter reaches the stagnation check period.

The application relates to the technical field of big data processing, and provides a big data export method, a big data export system, a big data export device and a storage medium. The method comprises the following steps: receiving an export request; sampling and counting to-be-exported data in a time range to obtain data distribution characteristics; dynamically calculating a shard boundary according to the data distribution characteristics, generating at least one shard task based on the shard boundary; monitoring a system resource occupation state, dynamically adjusting a concurrent execution degree according to the system resource occupation state, and concurrently executing each shard task according to the concurrent execution degree; after all shard tasks are executed, downloading temporary files corresponding to each shard from object storage, and merging the downloaded temporary files into a final export file. The application can realize efficient and optimized format big data export under the conditions of adaptive data distribution, dynamic adaptation of system resources and support for breakpoint continuation.

This invention discloses a method and system for data caching and breakpoint resumption in IoT devices. This invention relates to the field of IoT technology and solves the problem of rapid depletion of local storage space and accelerated wear of storage media during network outages due to continuous writing. This invention scans the free space at the end of the last sector of each file in the file system and writes the data to be cached into these previously unused fragmented areas. This eliminates the need to create temporary files or allocate formal storage space, thus avoiding additional occupation of formal storage areas during network outages and effectively alleviating the problem of storage capacity shortage. Furthermore, by utilizing the free space at the end of files and unallocated clusters for cache writing, it does not modify the file system's allocation bitmap and metadata. All write operations occur in physical areas that have been marked as idle, significantly reducing the number of erase / write cycles on the storage media and extending the lifespan of IoT devices.

The application discloses an HTTP request scheduling method and system. The scheduling method establishes a global request dispatcher independent of a page instance in a JavaScript engine of an embedded device to centrally manage the HTTP request life cycle. A request description block carrying a page handle is generated in response to a script layer request and is added to a message queue for processing according to a scheduling strategy. When the request is completed, the native layer checks the validity of the page handle, and the callback is executed only when the page handle is valid, thereby avoiding wild pointers and system crashes caused by data transmission to a released page. For a download request, the dispatcher calls a power-off resume transmission interface, and the device automatically resumes the download progress according to a temporary file after resetting, thereby ensuring the reliability of large data transmission. The method decouples the request from the page life cycle, solves the problem of interrupt loss caused by page closing, and significantly improves the system stability and efficiency.

The application discloses a sequencing duplicate read marking method and system based on block-level division and external sorting, and the method comprises the following steps: logically dividing a BAM / CRAM file, and allocating a continuous and data-equivalent file block to each thread; each thread constructs a ReadEnds data structure containing read position information, sequence numbers and sequencing quality sums in parallel, and outputs the ReadEnds data structure to a temporary file after being sorted and compressed; performing external multi-path merge sorting on the temporary file to generate a globally ordered data stream, and performing parallel scanning to identify duplicate sets, mark duplicate reads and extract sequence numbers; performing merging processing on the boundary duplicate sets, and performing external sorting to obtain a sequence number data stream; and each thread writes back the original file according to the sequence number data stream in parallel, sets a mark on the duplicate reads and outputs a result file in the original sequence. The application realizes full-process parallel processing, controllable memory and high I / O efficiency, and can stably and efficiently process super-large-scale sequencing data.

This invention discloses a fully automated SaaS system and implementation method for script-based movie video generation, belonging to the field of artificial intelligence content generation and software service technology. The system includes a user management module, a script parsing module, an image generation module, a post-processing module, a task scheduling module, a permission and security module, a commercial service module, and a resource cleanup module. This invention automatically splits user-input script text into shot sequences, calls a text-to-image model to generate visuals, and completes automatic subtitles, dynamic camera movement, speech synthesis, and integrated audio-visual synthesis. It also integrates commercial functions such as user packages, three-tier distribution, WeChat payment, commission settlement, and backend management. System stability and security are ensured through concurrency control, model loading timeout protection, failure retries, automatic temporary file cleanup, and temporary token anti-leeching. This invention achieves one-stop fully automated generation from script to movie-quality video, forming a SaaS platform that can be directly launched and operated, suitable for automated video production scenarios such as short dramas and promotional videos.

The application belongs to the field of end-cloud collaborative computing and lightweight virtualization, and proposes a lightweight application distribution and isolation running method and system for end-cloud collaboration. WASM bytecode is used as the only format for application distribution, and the application is compiled into a WASM module on the cloud side and distributed to the end-side device. The end-side device uses OverlayFS to merge the host read-only configuration data and the memory temporary file system Tmpfs into a virtual file system in the WASM sandbox at the file level. At the symbol level, a dynamic WASI bridge is introduced, and according to the requirements of the import section in the WASM module, the native capabilities of the host system are injected into the WASM runtime in the form of a host function (HostFunction) in real time, adapting to the heterogeneous hardware characteristics of the end-side device. The application is suitable for end-side environments with immutable operating systems as the base, and is compatible with cloud container deployment scenarios.

A system, responsive to successfully authenticating a requestor user, can receive sender user information of a sender user, generate a link corresponding to a file repository, generate a packet, wherein the packet corresponds to a location at the file repository where a file uploaded responsive to interaction with an upload link is stored, generate a packet access token that comprises a packet identifier associated with the packet, embed the packet access token with the link to create the upload link, wherein the upload link is associated with the packet; and output the upload link to enable the sender user to access the packet to upload the file to the location at the file repository that corresponds to the packet. Once the sender user uploads the file using the upload link, a file access link can be provided to a recipient user to access the file.

This invention discloses a general method for scanning and sending data to applications, relating to the field of data processing technology. The method includes the following steps: scanning device sending emails, parsing and filtering emails, data processing, file processing and storage, file sending and encapsulation, process control, and log management. In this invention, the SMTP email receiving service module is used to parse the email structure, filter valid scanned data, and block spam; the data processing and parsing module is used to extract user identity information and scan metadata, completing identity matching and metadata standardization; the file processing module is used to manage temporary file storage, realize intelligent conversion of scanned file formats, and ensure compliance with application requirements; the file sending service module is used to encapsulate files and metadata, call application interfaces for transmission, and determine the sending status; the process control module is used to monitor the process, perform dynamic retries, and clean up temporary files; and the log management unit records the entire process to ensure traceability.