36 results about "Zero-copy" patented technology

The invention belongs to the technical field of computers, and provides a video display and storage method and system based on a wayland protocol and a storage medium in order to solve the problems that an existing wayland video system is high in rendering delay, high in CPU load, low in storage efficiency and poor in cross-platform compatibility. Video frames are directly written into an Overlay layer through DRM-KMS and mixed by bypassing a synthesizer, meanwhile, CPU memory amplitude overhead is eliminated through zero copy transmission, an ISP-GPU-VPU direct transmission assembly line is constructed, full-link DMA-BUF direct transmission is achieved, and delay is greatly reduced; iSP preprocessing, GPU shader scaling and VPU coding are all executed by special hardware, so that the load is reduced, and the multi-path processing capability is improved; vPU dynamic code rate compression is utilized, segmented storage is carried out according to events / time, metadata is synchronized to a database, redundant storage is avoided, and space is saved.

The invention belongs to the technical field of data transmission and processing, and discloses a zero-copy-based multi-thread high-throughput data stream channel separation method and system, and the method comprises the steps: directly writing a mixed data stream into a front-end buffer region configured as an annular structure through a data receiving module by adopting direct memory access; then, a multi-thread processing module dynamically allocates a plurality of processing threads from a thread pool to separate channel data in parallel, each thread adopts a zero copy algorithm based on pointer offset, positions the channel data in a memory, creates pointer reference and associates the channel data to a corresponding rear-end buffer area, and logic separation is achieved without physical copy; and finally, the data storage module efficiently writes the separated data into persistent storage in an asynchronous I / O mode. According to the method, zero-copy, multi-thread parallel and two-stage dynamic buffering strategies are combined, the data separation efficiency is remarkably improved, CPU occupation and memory bandwidth are greatly reduced, and the real-time performance and stability of high-throughput data processing are guaranteed.

The invention discloses a deterministic soft bus communication method for a virtualized industrial control system, and relates to the field of industrial automation. According to the method, a soft bus communication mode is adopted, so that the containers are interacted through the intermediate message bus, and direct connection does not need to be directly established; all data streams are managed by a unified bus, so that the number of connections is reduced; and meanwhile, a shared memory zero copy mechanism is adopted to optimize data transmission, and the data is directly shared between a producer and a consumer by using a shared memory and a direct memory mapping mode, so that unnecessary copy operation is avoided, the overhead of data transmission is reduced, and the communication efficiency is improved. According to the invention, a unified and efficient communication channel can be established among a plurality of industrial edge applications on the edge device, real-time communication among local containers is supported, automatic routing capability of remote communication is realized, and the communication requirements of high concurrency and high real-time performance in a virtualized industrial control system are met.

A computer-implemented system for routing artificial intelligence (AI) queries. The system utilizes a zero-copy data pipeline, which processes prompts in memory-mapped buffers to eliminate at least one memory copy operation, thereby reducing latency relative to conventional serialization pipelines. The system continuously verifies AI provider compliance by injecting synthetic prompts containing invisible, Ed25519-signed Unicode watermarks. Algorithmic bias is detected by generating counterfactual “digital twin” prompts and applying Fisher exact statistical testing.Routing decisions for multi-tier autonomous systems are governed by safety-level requirements (ASIL-D, ASIL-B, QM) and may be constrained by external routing directives received via a meta-identifier. A hash-chained manifest, cryptographically signed using Ed25519 and consumed by downstream gateways, is generated for each routing decision, with its Merkle root asynchronously anchored to a blockchain to create a tamper-evident audit trail for regulatory compliance.

The invention discloses a large-bandwidth network flow protocol identification and analysis method and system, and relates to the technical field of network flow protocol analysis processing. The method comprises the following steps of: receiving original traffic of a network, storing the original traffic in a kernel mode buffer area in a data frame form, and directly mapping the data frame in the kernel mode buffer area to a user mode memory area based on a zero copy technology, logic classification is conducted on the data flow, the data flow is sent to the corresponding calculation processing cores through a dynamic distribution mechanism, and each calculation processing core completes layer-by-layer identification and analysis on a flow protocol and extracts corresponding metadata from a protocol identification result. According to the method, a protocol identification method of a three-level pipeline architecture is adopted in a computing processing core, layer-by-layer identification and analysis of a traffic protocol are completed, high-complexity deep analysis of all traffic is avoided, the whole identification process does not need to be reconstructed in the face of a newly-added protocol or a complex protocol, and each stage is independently optimized.

The invention discloses a data stream processing method and device, electronic equipment and a storage medium, and relates to the technical field of distributed storage. According to the data stream processing method and device, the data stream transmission mode is pre-judged to be intra-node transmission or cross-node transmission based on a hardware topology mapping table, and read-write caches are dynamically allocated according to the pre-judged data stream transmission mode; data received by the front-end network card is directly written into a controller memory buffer area of the target equipment through point-to-point direct memory access for data streams transmitted in the nodes, and is read and forwarded from the target equipment by the rear-end network card; the sending cache and the receiving cache of the data stream transmitted across the nodes are locked to the non-uniform memory access node to which the target network card belongs, and the data are sent to the target node through the remote direct memory access register memory, so that the problem that the data stream processing is not optimized in combination with hardware topology characteristics can be solved; the technical effects of reducing data replication overhead, avoiding redundant access across hardware nodes, improving data stream transmission efficiency and realizing end-to-end zero replication transmission are achieved.

The embodiment of the invention provides a big data communication method and device under an intelligent driving system, equipment and a storage medium. The method comprises the following steps: after monitoring new bare data, a producer determines a target memory block according to the states of a plurality of shared memory blocks, writes the bare data into the target memory block, modifies the state of the target memory block, and synchronizes an index of the target memory block to a consumer annular queue through an Nvscissync synchronization mechanism, and a consumer queues the target memory block based on the index of the target memory block. And accessing the data in the target memory block in a read-only manner. By means of the method, zero copy is achieved, simultaneous access of multiple downstream modules is supported, the data distribution efficiency is improved, delay is reduced, and system space resources are saved.

Provided herein are systems and methods for a zero-copy clone of a DT. A method includes performing a clone operation on a dynamic table (DT) to generate a cloned DT. The DT is based on a query applied on a base table. The cloned DT is based on the query applied on a cloned base table corresponding to the base table. A first delta is determined based on at least one change in the base table between a first version of the base table used by the DT at a time of the clone operation and a second version of the base table generated prior to the clone operation. A first refresh operation of the cloned DT is performed based on the first delta.

The invention provides an RDMA-based big data transmission and storage system and method and a semiconductor detection device.The big data transmission and storage system comprises a hardware layer and a software layer, the hardware layer comprises an NVMe SSD storage array, an NVMe RAID acceleration card based on a GPU architecture and an RDMA network card, and the software layer comprises an NVMe SSD storage array, an NVMe RAID acceleration card based on the GPU architecture and an RDMA network card; the software layer comprises a multi-thread transmission module, a self-adaptive memory pool module, a transmission control module, a data verification and retransmission module, a scheduling and optimizing module and a disk falling and storage management module. According to the method, multi-channel transmission parallelization is achieved through data direct connection, zero copy transmission, RAID acceleration and multi-thread parallel RDMA transmission in a hardware link, data is directly connected and transmitted from a memory to the NVMe SSD storage array, data receiving and disk falling thread decoupling directly access the NVMe SSD storage array through asynchronous writing, network overload or storage blockage is avoided, and the data transmission efficiency is improved. The data transmission efficiency and reliability are high, and popularization and application in scenes needing large-scale image transmission processing such as semiconductor detection are facilitated.

A multi-processor communication framework (MCF) based on Transmission Control Protocol / Internet Protocol (TCP / IP) architecture is defined that can be used for inter-processor communication between heterogenous processing nodes. The MCF includes an application layer, a transport layer, a network layer, and an inter-processor communication (IPC) layer for each processing node. The IPC layer can be used to facilitate data transfer between two processing nodes via a physical channel within an electronic device, or through shared memory. The MCF can support multicast, broadcast, unicast, and zero-copy features like the TCP / IP stack.

The invention relates to the technical field of computer networks, and discloses a DPDK-based high-speed data isolation method, which comprises the following steps of: S1, directly taking over a data packet transceiving function of a network card through a DPDK driver, bypassing a kernel protocol stack of an operating system, and establishing a user mode data channel; s2, based on a multi-core CPU architecture, distributing the received data packets to independent processing cores according to a preset rule, and binding each processing core to an exclusive memory pool to realize physical-level data isolation; and S3, directly transmitting the isolated data to a target application module through a zero copy technology in the user mode, and verifying the data integrity in real time through a hardware verification module. According to the method, the network card is directly taken over through the DPDK driver, the user state data channel is established, an operating system kernel protocol stack is bypassed, the zero copy technology is combined, data does not need to be copied in the whole process from receiving, processing to transmission, the end-to-end delay is reduced to 50 microseconds or below, and the method is suitable for a high throughput scene.

Disclosed in some examples are systems, devices, machine-readable mediums, and methods for customizing an in-memory versioning mode for each memory location according to a predicted access behavior to optimize memory device performance. Usage data in a previous time period may be utilized along with policy rules to determine whether to configure a particular memory address as a zero copy or direct copy mode. For example, memory addresses that are read frequently may be configured as direct copy mode to reduce the read latency penalty. This improves the functioning of the memory system by reducing read latency for memory addresses that are frequently read but written to less frequently, and reduces write latency for memory locations that are frequently written to, but not read as frequently.

The invention discloses a data processing method and device, a storage medium and a computer program product. The method comprises the steps that in a computing layer, a first device receives a first processing request sent by a second device, the first processing request is used for requesting to write first data into a storage layer of the first device, and the first data is stored in a first buffer area; the first processing request comprises address information of the first buffer area and feature information of the first data; and writing the first data into the storage layer by utilizing the address information of the first buffer area and the feature information of the first data. According to the technical scheme provided by the invention, the data is written or read by transmitting the address information of the data buffer area between the devices so as to realize zero copy of the data, so that the occupation, overhead and processing delay of resources can be reduced, and the performance of the devices is greatly improved.

The invention relates to the technical field of embedded computing and system collaboration, in particular to a multi-operating system heterogeneous collaboration method based on a lightweight message bus and related equipment. According to the method, a cross-operating system data exchange basis is established by initializing a shared memory annular buffer area and setting access authority; a message bus application interface is used for completing multi-system service registration in the unified namespace, and a service identifier mapping table is generated; the sender operating system efficiently writes the message data into a target buffer area based on a zero copy mechanism and triggers a notification signal; the receiver operating system analyzes the message according to the read-write pointer and outputs a processing state; and finally, efficient communication and running state monitoring among multiple operating systems are realized through event synchronization and a watchdog mechanism. According to the method, efficient heterogeneous cooperative execution among multiple operating systems is realized in a light-weight, low-delay and low-power-consumption manner, and the cross-system data interaction efficiency and the operation stability are remarkably improved.

The present application belongs to the technical field of computers. Disclosed are a distributed dial test monitoring method and system, and an electronic device and a storage medium. The method comprises: registering a dial test request connection event to a connection request event multiplexer, so as to obtain a first event set; then performing polling to determine a first connection event, and registering the first connection event to a data reading event multiplexer, so as to obtain a second event set, and then performing polling to determine a ready data reading event; reading dial test response data on the basis of a zero copy method; performing encapsulation processing on the dial test response data, creating a data analysis and processing module connection event, and registering the data analysis and processing module connection event to a connection request event multiplexer, so as to obtain a third event set, and then performing polling to determine a ready data analysis and processing module connection event; and on the basis of the ready data analysis and processing module connection event, sending the dial test response data to a data analysis and processing module for data analysis and processing, so as to obtain a data analysis result. The present application can perform distributed dial test monitoring with high levels of efficiency and low delay.

The invention provides an aircraft access device and method based on a zero-copy technology, and the device comprises a communication adaption module which dynamically adapts to multiple types of peripherals of an aircraft through a modular interface, and the modular interface integrates multiple communication protocols and drive programs, and achieves the data format translation and interaction among the multiple types of peripherals; the shared memory module constructs a shared memory area based on a POSIX protocol, and controls access synchronization through semaphore; and the plurality of application programs in the user mode and the processing module in the kernel mode read and write the shared memory area through a zero copy technology, and release service information for controlling the aircraft through the shared memory area. The decentration networking module dynamically matches different components of the aircraft through service information to form a publisher and a subscriber; the data interaction module drives a publisher to write data into the shared memory based on a subscription-publishing mechanism and notifies a subscriber through a synchronization mechanism, and the subscriber reads the data from the shared memory according to a preset strategy to execute corresponding processing.

Systems and methods for zero-copy forwarding for network function virtualization (NFV). An example method comprises: receiving, by a supervisor of a host computer system, a definition of a packet filter originated by a virtual execution environment running on the host computer system; responsive to validating the packet filter, associating the packet filter with a vNIC of the virtual execution environment; receiving, by the supervisor, a network packet originated by the vNIC; and responsive to matching the network packet to a network connection specified by the packet filter, causing the packet filter to forward the network packet via the network connection.

The network interface card (NIC) comprises a controller and a local memory, and the network interface card (NIC) comprises the controller and the local memory. The controller is used for receiving a data message. The data packet includes a data payload, a header, and metadata. The controller is further configured to select a 0 copy buffer from a 0 copy descriptor ring having a host physical address (HPA), and to select a 0 copy buffer from a 0 copy descriptor ring having a host physical address (HPA). The controller is further configured to write the data payload of the data packet to the 0 copy buffer. The controller is also configured to parse the data packet to identify a virtual destination. The controller is also to select a virtual buffer with a virtual address from a descriptor ring of the virtual destination. The controller is also configured to map the virtual address of the selected virtual buffer to the HPA of the 0 copy buffer.

In some implementations, a compute express link (CXL) compliant memory system may configure a portion of a memory as a shared memory region directly accessible by multiple fabric-attached processing units. The CXL compliant memory system may establish, with a first and second fabric-attached processing unit, a first and second device direct access link, respectively, to the shared memory region. The CXL compliant memory system may receive, via the first device direct access link and from the first fabric-attached processing unit, communication information associated with communications between the multiple fabric-attached processing units. The CXL compliant memory system may store the communication information in the shared memory region. The CXL compliant memory system may permit, via the second device direct access link and by using a zero-copy operation, access to the communication information by the second fabric-attached processing unit.

The invention discloses an Android container application display method based on a cross-system zero-copy rendering link, and the method comprises the steps: taking a gap as a host and an Android as a container, building encrypted communication between the two parties through a memory coordination service and an agent daemon process, completing capability negotiation, and generating a cross-system capability matrix; creating a secure shared buffer area pool by the host according to the secure shared buffer area pool, and distributing a unique identifier and a DRM Prime handle for the buffer area; a container side application initiates a buffer area rendering request, a host matches and distributes a buffer area, and a handle is imported into the container side to bind a rendering context; after the application finishes rendering, synchronizing an object mark completion state through the DRM and notifying the host; and the host verifies the data and waits for a synchronization signal, and submits a buffer area handle to a display controller in a zero copy mode at the VSync opportunity to complete on-screen operation.

The invention relates to a distributed knowledge base file storage system based on a high-speed optical fiber network. A hardware architecture comprises storage nodes, an optical fiber switch and an application server. The system optimizes communication through a remote direct memory access technology, and a software architecture comprises a high-speed network driving layer, a joint file system daemon process and a user space file system framework. The united file system daemon process provides a unified virtual directory view, and supports standard file operation and data consistency. And the network optimization synchronization module optimizes data transmission by using a zero copy technology and asynchronous I / O. And the dynamic configuration management module supports dynamic node management and persistent configuration. And the control management interface module provides a UNIX Socket interface, supports JSON format instructions and realizes dynamic scheduling of storage resources. The user space file system framework is responsible for daemon process management, provides a unified storage view service and supports large model knowledge base application. The system provides an efficient and extensible storage solution for a large-scale knowledge base.

The invention relates to an industrial control network flow generation system based on a zero copy mechanism, the system comprises a support layer, a service capability layer, an interaction layer and a test device, the support layer is constructed on a basic environment of a Linux operating system and used for managing and driving bottom layer physical resources, and the service capability layer is constructed on the basis of the basic environment of the Linux operating system. An equipment driving library, a software development library and a configuration management tool are provided to support an upper-layer function operation environment, WEB service, database service and storage service operation environments are established, and L7-layer application simulation service is provided; the service capability layer is used as a configuration and execution layer of firewall test items; the interaction layer comprises a WEB GUI interface and a Console CLI interface, and the WEB GUI interface is used for receiving a user instruction, sending the user instruction to the service capability layer and receiving and displaying test process data and a test result; the Console CLI interface is used for a professional user to execute equipment maintenance operation. According to the invention, delay and overhead caused by a network stack can be greatly reduced.

The application relates to the technical field of big data processing and distributed computing, in particular to a distributed big data batch processing optimization method based on SpringBatch, which comprises the following steps: multi-dimensional entropy analysis and fragmentation: performing random pre-sampling calculation key value information entropy, dynamically adjusting histogram bucket number, and establishing a dynamic fragmentation strategy based on a work-stealing mode; self-adaptive feedback control: collecting JVM heap memory usage, dynamically adjusting data grabbing size by using an incremental PID algorithm and a variable step limiter, and forming a self-adaptive back pressure mechanism; zero copy and reuse: performing zero copy conversion by using off-heap memory mapping, and reducing object creation frequency by using object pool reuse technology; asynchronous submission and fault tolerance: combining and submitting large transactions based on an asynchronous disk brushing strategy, and performing hierarchical retry operation; the application realizes statistical perception self-adaptive control of a batch processing process, and significantly improves system throughput and data consistency.

The invention relates to multiple processing unit communication using zero copy fixed compute fast link memory. In some embodiments, a compute fast link (CXL) compliant memory system may configure a portion of a memory as a shared memory region directly accessible by multiple fabric attached processing units. The CXL compatible memory system may establish first and second device direct access links with first and second fabric attachment processing units, respectively, to the shared memory region. The CXL compatible memory system may receive communication information associated with communication between the plurality of fabric-attached processing units from the first fabric-attached processing unit via the first device direct access link. The CXL compatible memory system may store the communication information in the shared memory region. The CXL compatible memory system may allow the second fabric attachment processing unit to access the communication information directly via the second device and by using a zero copy operation.

The invention discloses a high-real-time power equipment inter-board communication method and system based on a zero copy technology, and relates to the technical field of power system communication, and the method comprises the steps: defining a communication equipment logic instance through an enumeration type, and binding communication equipment through registering a callback function; the registered communication equipment is polled in the high-speed sampling interruption, and a message to be analyzed is received; distinguishing the message type through the type code in the field of the to-be-analyzed message; directly calling a registered callback function to process the data of the message to be analyzed based on the message type; wherein the callback function is only used for data copying. According to the invention, the technical problems of single communication equipment type, poor expansibility, high data transmission delay and insufficient system reliability in the prior art are solved.

Deep neural network checkpoint optimization system and method based on non-volatile memory are provided, where the client module and the server module register corresponding network structures in non-volatile memory and create data indexes and data communication protocols based on remote direct memory access (RDMA) before the start of training deep neural networks, and during the neural network training process, provide zero-copy, asynchronous, end-to-end neural network data persistence, which allows users to perform fine-grained checkpointing to ensure fault tolerance and data persistence without affecting the training speed.

The invention discloses a Java-based big data stream-oriented computing framework and an optimization method thereof. The optimization method comprises the following steps: pre-distributing a GC visible out-of-heap region outside a JVM (Java Virtual Machine), and registering the GC visible out-of-heap region to a G1 / Shendoah region table; automatically generating a data difference log and a difference abstract in TCP transceiving soft interrupt by utilizing eBPF, and locking an old page by combining hardware copy-on-write; calculating a bandwidth time delay product in real time according to a TCP congestion window, and directly driving operator parallelism degree expansion and contraction; in the recovery stage, the physical page frame number is remapped through user faultfd, and cross-Pod thermal migration is completed within 200 ms. A unified SQL interface is exposed to the outside by the framework, direct memory byte codes are automatically generated, a user can enjoy zero copy through zero perception, GC is free of pause, a back pressure instant feedback effect is achieved, throughput is improved by 30-50% under the same hardware, end-to-end delay is reduced by 20-40%, and global pause time is reduced to lt; and 5 seconds per hour.

The invention discloses a memory application method and system for reducing message copy in communication middleware, which comprises the following steps: acquiring an actual allocated memory based on a request memory space and a reserved space, storing user data in the request memory space to obtain a memory block, acquiring an actual memory block address based on the request memory space, and storing the actual memory block address in the reserved space; obtaining a user available address based on the actual memory block address and the reserved space, when header information needs to be added to forward a message, writing the header information into the reserved space to obtain a new message initial address, the length of the message being the length of the user data and the header information, and when the memory block needs to be called again, writing the header information into the reserved space to obtain a new message initial address; and releasing the memory block based on the actual memory block address. The present invention enables zero copy message forwarding. According to the method, the memory management concept of one-time distribution and zero-copy forwarding is put forward for the first time, and the memory performance bottleneck in a message forwarding scene is fundamentally solved through an innovative pre-distribution strategy and an address calculation mechanism.

The present application relates to a method, system and medium for implementing RDMA services, wherein the method includes: an abstract layer for providing socket interface semantic functions is encapsulated on the verbs interface of the RDMA service, and the data received from the sender is cached into the RDMA memory area by calling the verbs interface through the socket interface semantic functions in the abstract layer; the data in the memory area is directly read for serialization, and stored in a preset database using the socket interface. Through this application, it is realized to provide socket semantics on the verbs interface, and then provide RDMA services for existing databases using the socket interface, and the serialization of the socket interface and the database are collaboratively extended to support the zero copy and kernel bypass characteristics of the RDMA service, solving the problem of how to implement RDMA services on existing software using the socket interface.

The invention discloses an efficient asynchronous caching method and system for DAS high-frequency data, and the method comprises the steps: caching acoustic data into a data packet, writing a data pointer of the data packet which is fully written into a data block, and writing a pointer of the data block which is fully written into a data queue. By using the two-stage data structure of the data packet and the data block to store the data, the snapshot can be efficiently created, the lock-free, zero-copy and low-delay (1 millisecond delay) can be realized, the requirement of a continuous memory is greatly reduced, the method is suitable for high-frequency (10khz-100khz) acoustic data of the DAS and a high-efficiency data storage algorithm, and the resource overhead of a CPU (Central Processing Unit) is greatly reduced.