23 results about "Asynchronous I/O" patented technology

The invention provides an end-side multi-modal large model accelerated reasoning method and system, and the method comprises the steps: carrying out the two-stage screening and rearrangement of visual tokens based on the CLS attention and text-to-visual attention in a visual encoder and pre-filling stage, and constructing a sparse attention and sparse key value cache; in a decoding stage, an important neuron set is judged according to activation gating or historical statistics, only a corresponding feedforward network weight is pulled and calculated, missed weights are loaded on demand through asynchronous I / O, and hot neurons are maintained in a high-speed memory to utilize model sparsity, so that video memory / memory occupancy and calculation overhead are remarkably reduced on an end side; throughput and time delay performance are improved. According to the method, the internal memory and computing resources required by reasoning of the multi-modal large language model are reduced from two dimensions by utilizing the endogenous sparsity of the end-side large language model in input and the model, so that a higher reasoning speed is achieved by utilizing fewer resources on the premise of keeping the size of the model unchanged, and the performance of the whole system is improved.

The invention relates to the field of data access, and particularly provides an MPT-based block chain database system, a data access method, a terminal and a medium, the system comprises a native MPT storage engine used for realizing an MPT node structure in a disk and a memory, and the node structure comprises branch nodes, extension nodes and leaf nodes; the asynchronous I / O module is used for realizing non-blocking disk operation based on Linux iouring; the file system bypass module is used for supporting direct access to the block device; the versioning concurrency control module is used for realizing lock-free reading through an immutable Trie structure and atomic pointer updating; and the dynamic compression module is used for dynamically adjusting a historical data retention strategy according to the disk space utilization rate. The MPT structure storage engine is realized, basic data support is provided for a high-performance block chain system, serialization overhead is eliminated, and storage efficiency is improved.

The invention provides a CODESYS-based HID equipment adaptive data interaction method and system and a storage medium, and the method comprises the steps: firstly, capturing an HID equipment plugging event in real time, and extracting equipment fingerprints containing VID, PID and protocol types; secondly, white list verification is carried out during operation, an exclusive annular buffer area is created for legal equipment, and a standardized function block is dynamically generated; then, a communication mode is configured based on the equipment type, and direct data transmission is realized through a zero-copy channel by adopting a polling or fragmentation transmission mode; and finally, the application layer calls the function block through a declarative interface, and executes data retransmission or fusing protection according to a CRC check result. According to the method, middleware dependence is eliminated, asynchronous I / O and shared memory technologies are adopted, data transmission delay is reduced, and hot plug automatic identification of equipment is also realized; according to the invention, a high-real-time and high-reliability HID equipment access solution is provided for the field of industrial control.

Embodiments of the present disclosure provide a data read / write method, system, and apparatus, a computing device, and a storage medium. The data read / write method comprises: receiving, by means of an asynchronous I / O layer, a data read / write request sent from a user space, and storing the data read / write request in a request queue; when the asynchronous I / O layer determines that the data read / write request is of a pass-through type, sending the data read / write request to a driver layer; and parsing the data read / write request by means of the driver layer, and sending request information of the data read / write request to a target processing end device for read / write processing. The data read / write request sent from the user space is received by means of the asynchronous I / O layer, and is sent to the driver layer after the data read / write request is determined to be of the pass-through type, thereby reducing context switching and data copying overhead, and improving the processing efficiency of I / O operations. The request is parsed at the driver layer and sent to the processing end device for read / write processing, thereby optimizing the scheduling of I / O tasks, reducing disk addressing time and I / O latency, and improving overall performance and system stability.

The application discloses a large model firewall fine-grained dynamic scheduling method and system. The application defines a unified abstract detection interface, each security module is used as an implementation class, and scheduling information is recorded through the static attribute of the interface; three types of execution resource pools, including a CPU thread pool, a GPU inference queue and an asynchronous I / O thread pool, are constructed; the security module is associated to the corresponding resource pool according to the scheduling information; a module performance probe is deployed, running indexes are collected after module execution, and the real-time load of each resource pool is calculated respectively; when the load of any resource pool exceeds a preset upper threshold, the security module in the pool is started to degrade according to the static attribute; and when the load of the resource pool is lower than a lower threshold and lasts for a preset length of time, the security module in the pool which has been degraded is tried to recover. The application realizes fine-grained scheduling of resource pool isolation and module-level perception, reduces the risk of false degradation, and improves the system throughput and recovery efficiency.

The invention discloses a plug-in type data communication middleware design implementation method and device based on ZeroMQ, and the method comprises the steps: constructing a ZeroMQ asynchronous communication core through an asynchronous I / O multiplexing and message serialization optimization algorithm, and achieving the direct communication of decentralized nodes; a plug-in management framework is constructed by a plug-in dependency analysis and sandbox isolation algorithm, and module modularization and dynamic expansion are supported; building a visual protocol adaptation interface by adopting an intelligent field mapping and real-time conversion preview algorithm, generating conversion rules and scripts, packaging the conversion rules and scripts into adaptation plug-ins, and loading the adaptation plug-ins to a communication core; and integrating a script engine, realizing user-defined logic and conversion rule collaboration through security sandbox execution and a dynamic compiling optimization algorithm, and generating a modular plug-in which is dynamically plugged into a communication core. According to the method, the problems that a traditional integrated interface is tedious, high in coupling and difficult to adapt can be solved, and rapid seamless integration of the heterogeneous system is achieved.

This application discloses an asynchronous I / O call method, apparatus, storage medium, and computer device for business code. The method includes: creating a reactor thread and initializing thread-local resources when the framework starts; creating a request object in response to a detected business request; executing the business process in the business processing function of the request object; switching state variables and calling an asynchronous command sending function when an I / O call operation is executed; creating a callback data object and constructing an asynchronous operation object through the asynchronous command sending function; executing the asynchronous call operation through the asynchronous operation object; the response to the asynchronous call operation drives the business processing function to re-execute with the current state variable after the lifecycle of the callback data object ends; processing response data after the response to the asynchronous call operation returns; and sending the response content after all response data has been processed. This method, through an automatic callback-driven mechanism, enables asynchronous scheduling and non-blocking execution of business logic.

The invention discloses a plug-in type communication architecture supporting multiple industrial protocols. The plug-in type communication architecture comprises a protocol adapter dynamic loading module, a protocol adapter dynamic loading module and a protocol adapter dynamic loading module, wherein the protocol adapter dynamic loading module is used for dynamically loading and unloading adapters of the multiple industrial protocols during operation; the container arrangement module is used for realizing plug-in deployment and automatic discovery of services through a containerization technology; the asynchronous processing and connection pool collaborative optimization module is used for reducing communication time delay through asynchronous I / O processing and connection pool management; and the heartbeat detection and automatic fault recovery module is used for improving the reliability of the system through a heartbeat detection and automatic recovery mechanism. According to the architecture, industrial protocol plug and play and flexible service deployment are realized through protocol adapter dynamic loading and container arrangement, and the system expansibility is improved; through asynchronous processing and connection pool collaborative optimization, the communication time delay is reduced, and the throughput capacity is improved; the reliability and the self-healing capability of the system are enhanced through heartbeat detection and an automatic fault recovery mechanism, and finally collaborative optimization of time delay, multi-protocol compatibility and reliability is realized in a complex industrial environment.

Embodiments of the present application provide a high-concurrency network file processing system, method, computer device and storage medium. The system comprises a DPDK data plane module configured to receive network data packets from a network card transceiver queue and store the network data packets in an MBUF memory pool; a coroutine scheduling module configured to preprocess the network data packets based on an event loop and generate corresponding files; a task dispatching module configured to add a file writing task to a task queue and assign the task to a target file cache shard according to system load conditions; a file cache module configured to receive and cache data segments of the writing task and reorder the data segments according to offset information; and an asynchronous I / O module configured to submit the reordered data to a storage device interface in an asynchronous manner. The introduction of the DPDK enables user mode direct access to the network card transceiver queue, avoids the overhead of system calls and kernel protocol stacks, and significantly improves network I / O performance.

The edge-side server (20) transmits a first response to the request to the player terminal (30) upon receiving the request transmitted from the player terminal (30), and transmits an aggregated result aggregated with a result of processing based on the request to the cloud-side server (10) every certain period of time. Then, the edge-side server (20) receives a result of processing based on the aggregated result executed at the cloud-side server (10), and transmits the result of the processing as a second response to the player terminal (30). The player terminal (30) receives the first response and the second response from the edge-side server (20) in sequence via asynchronous I / O for a single request.

The invention relates to a distributed knowledge base file storage system based on a high-speed optical fiber network. A hardware architecture comprises storage nodes, an optical fiber switch and an application server. The system optimizes communication through a remote direct memory access technology, and a software architecture comprises a high-speed network driving layer, a joint file system daemon process and a user space file system framework. The united file system daemon process provides a unified virtual directory view, and supports standard file operation and data consistency. And the network optimization synchronization module optimizes data transmission by using a zero copy technology and asynchronous I / O. And the dynamic configuration management module supports dynamic node management and persistent configuration. And the control management interface module provides a UNIX Socket interface, supports JSON format instructions and realizes dynamic scheduling of storage resources. The user space file system framework is responsible for daemon process management, provides a unified storage view service and supports large model knowledge base application. The system provides an efficient and extensible storage solution for a large-scale knowledge base.

The invention relates to the field of semiconductor manufacturing, and discloses a general communication driving method and system for a semiconductor wafer transmission system, and the method comprises the steps: loading and analyzing a preset external protocol configuration file; starting a multi-thread asynchronous I / O communication model based on the mapping relation; analyzing the return data received by the independent receiving thread in real time so as to distinguish whether the return data is used as a response message of a physical instruction sent by the main control thread or an asynchronous event message actively reported by the semiconductor wafer transmission system; and establishing and maintaining a state cache region for storing the real-time operation state of the semiconductor wafer transmission system in the memory, and updating the state cache region by adopting a dual-mode state cache mechanism. According to the method and the device, zero-code adaptation of heterogeneous equipment is realized through the external protocol configuration file, and communication overhead is optimized while data synchronization is ensured through a dual-mode state cache mechanism, so that comprehensive problems which cannot be solved in the prior art are fundamentally solved.

The invention relates to a global atmospheric spectrum mode I / O optimization method, system and device and a medium. The method comprises the steps that a global atmospheric spectrum mode model and an asynchronous I / O frame integrated with an XIOS are constructed, in the asynchronous I / O frame, the global atmospheric spectrum mode model serving as a client is only used for executing a model calculation process, a server manages data buffering, model calculation data aggregation and parallel writing, and meanwhile model calculation continues to be conducted; the asynchronism of the client-server ensures that model calculation is not hindered by I / O tasks to a great extent, and network congestion and communication delay are reduced. Model calculation data are firstly cached in a buffer area through the buffer area arranged in the asynchronous I / O framework and a sending optimization mechanism based on the buffer area, so that the overlapping of the I / O operation and the model calculation process is realized to the maximum extent, the overhead of frequent I / O operation is reduced, and the global atmospheric spectrum mode I / O bottleneck is successfully relieved.

The invention provides an information processing method and device, equipment, a storage medium and a program product. The method comprises the steps that a first data set is obtained and comprises application layer index data, system layer index data and hardware layer index data, the application layer index data is used for representing service load characteristics, the system layer index data is used for representing resource utilization rate and kernel behaviors, and the hardware layer index data is used for representing physical equipment performance states; performing data preprocessing on the first data set to obtain a processed second data set; and according to the second data set, utilizing a machine learning model based on Gaussian process regression to predict and obtain configuration parameters of the asynchronous I / O interface. According to the method, multi-level factors of an application layer, a system layer and a hardware layer are comprehensively considered, asynchronous I / O configuration can be adjusted in time according to real-time performance changes of the system through the machine learning model based on Gaussian process regression, and high-performance operation of the system under different workloads is kept.

This invention relates to the field of object storage technology and discloses an object storage method, apparatus, device, and medium applied to an object storage gateway. The method includes: when a data processing request is received from a client, obtaining the client's historical load information and the storage platform's historical health information; processing the historical load information and historical health information to determine the target asynchronous I / O load corresponding to the current data processing request; if the data processing request is a data upload request, determining the number of asynchronous I / O channels and the data block size corresponding to the data upload request based on the target asynchronous I / O load; and segmenting and uploading the upload object corresponding to the data upload request based on the data block size and the number of asynchronous I / O channels. This invention determines the current asynchronous transmission load by combining historical data when a data upload request is received, thereby determining an appropriate data block size and the number of asynchronous I / O channels, ensuring transmission performance during data upload.

The present invention relates to a security threat detection system and method that minimizes the impact on network traffic performance by performing security threat detection on web requests using an asynchronous I / O method, and detects and blocks various security threats in real time through a machine learning model. The system comprises: a firewall execution unit that processes web requests on a network asynchronously, requests security threat detection for received web requests, and blocks or forwards said web requests to a destination web server according to the security threat detection result; and a security threat detection unit that analyzes web request data through a machine learning model according to the request of said firewall execution unit to detect whether there is a security threat and transmits the result.

The present disclosure provides a request processing method and device based on an asynchronous storage engine, equipment and a storage medium, relating to the technical field of distributed storage. The specific implementation scheme is: in the user mode, in response to a current request obtained by a current thread, a current cooperative I / O instance bound with the current thread is determined; wherein the current cooperative I / O instance is an asynchronous I / O interface instance; the current request is submitted to a submission queue of the current cooperative I / O instance; in the kernel mode, the current request is processed asynchronously through a current thread pool; wherein the current thread pool is a thread pool of a current virtual I / O instance shared by the current cooperative I / O instance.

The present application relates to weak network environment batch user migration management method, device and storage medium, read operation task data from local offline file to execute operation task based on task data; Group operation task according to network address of baseboard management controller, adopt asynchronous I / O concurrent processing mechanism, concurrent execution operation task of different network address of baseboard management controller, the present application realizes concurrent operation to user, improves operation efficiency;In the same baseboard management controller, according to the definition order of local offline file, operation task is executed in turn, to avoid concurrent conflict;The business logic of all operation tasks is concentrated in the core module, and the calling module is established as the program entry and the dispatcher to call the core module to execute the operation task, so that the weak network environment batch user migration management is completed.The present application can improve the operation efficiency, especially in the weak network environment, the performance advantage of the present application is more significant compared with the traditional scheme.

The application relates to a Xen full-virtualization agentless real-time backup method and system and a storage medium, and belongs to the technical field of data disaster recovery backup. The method comprises the following steps: initializing a backup driver; intercepting an asynchronous I / O request; acquiring write position information, a disk file path and disk file change data; creating and loading a backup structure body, and writing backup structure body data into a kernel buffer area for temporary storage; and asynchronously sending kernel buffer area data to a backup service program. The system comprises the following modules: an initialization module; an interception module; a data acquisition module; a data loading module; and a sending module. The application continuously protects data of a virtual machine using a full-virtualization technology in Xen, can capture changes in real time when the virtual machine is running, and can synchronize the changes to backup storage, thereby reducing the risk of data loss.

The application relates to a global atmospheric spectrum model I / O optimization method, system, device and medium. The method comprises the following steps: constructing an asynchronous I / O framework integrated with a global atmospheric spectrum model and XIOS, in the asynchronous I / O framework, the global atmospheric spectrum model as a client is only used for executing a model calculation process, data buffering, model calculation data aggregation and parallel writing are managed by a server while the model calculation is continuously carried out, and the asynchrony of the client-server ensures that the model calculation is largely hindered by I / O tasks, and network congestion and communication delay are reduced. Through a buffer area arranged in the asynchronous I / O framework and an optimization mechanism based on the buffer area, model calculation data is first buffered in the buffer area, so that the overlapping of I / O operation and the model calculation process is maximized, the overhead of frequent I / O operation is reduced, and the global atmospheric spectrum model I / O bottleneck is successfully relieved.

The application discloses a PXIe high-speed data zero-copy processing and real-time streaming disk method for a domestic multi-core processor and relates to the technical field of PXIe bus data processing and storage. The VFIO technology is used to realize user space mapping of a DMA buffer, and zero-copy data transmission is completed in combination with 2MB large page memory; the AMP mode is used to divide multi-core control surfaces and data surfaces, and the multi-core scheduling is optimized by using a lock-free queue, core-interrupt affinity processing; the streaming disk link bypasses a file system through bare I / O, and massive data real-time parallel streaming disk is realized in combination with io_uring asynchronous I / O, a pre-allocated memory pool and a lock-free producer-consumer model. The application eliminates the performance bottleneck of each link of data processing, fully utilizes the parallel advantages of the 16GB / s high bandwidth of the PXIe bus and the domestic multi-core processor, adapts to the national production software and hardware system, and improves the real-time performance and throughput of data processing.

This invention relates to a server data caching and acceleration method based on edge computing. First, historical access data is acquired and the access frequency of various data types is statistically analyzed. High-frequency access data is stored in a distributed cache on edge nodes. The data access frequency is updated in real time, and the priority weight of cache items is dynamically adjusted based on an adaptive LRU algorithm. Upon receiving a data access request, a hash lookup algorithm is used to check if the data is in the cache. If a match is found, a direct response is provided; otherwise, the data is forwarded to the central server via asynchronous I / O. When data is accessed, it is moved to the head of a linked list; when the cache reaches its capacity, data at the tail of the list is evicted. The accessed data is preprocessed. The central server performs big data analysis using Spark and a federated learning framework, and stores the data in a NoSQL database and synchronizes it using gRPC. This method improves data access efficiency, cache utilization, and data quality, optimizes big data analysis results, and is suitable for server data caching and acceleration in edge computing scenarios.

The invention relates to the field of data transmission, in particular to a high-speed USB transmission method based on asynchronous I / O. The method comprises the following steps: firstly, realizing a USB Gadget FFS mode in USB equipment, and creating a service program for responding to a read-write request of an upper computer; and when the PC initiates a read request, the service program responds to the read request, communicates with the kernel in a multi-queue asynchronous I / O (Input / Output) mode, processes the read request of the upper computer and returns data to the PC. Through the FFS function provided in the USB Gadget driver, the method for directly realizing the USB function by the user space program is realized, and the data transmission performance of the USB3.0 is greatly improved by creating the service program, responding to the read-write request of the upper computer and adopting a multi-queue asynchronous IO data transmission method when responding to the read-write request.