30results about How to "Reduce data copying" patented technology

The invention discloses a network data packet parallel processing method based on an Intel DPDK (Data Plane Development Kit). The method comprises the following steps: step 1, establishing an Intel DPDK running environment; step 2, creating a memory pool, and creating a RX memory ring for receiving the network data packet and a TX memory ring for transmitting the network data packet; step 3, setting a network data packet scheduling strategy, and preparing to perform network data packet scheduling; step 4, creating multiple worker threads, and performing parallel processing of the network data packet. In the method provided by the invention, performance advantage of a multi-core CPU is used fully, and by lock-free data structure design, parallel network processing framework and high-efficiency network data packet distribution, parallel processing of the network data packet on multiple CPU cores is realized.

The invention relates to the field of set top boxes and discloses an audio-video stripping method and a system for TS (transport stream) media. The method comprises the steps that 1, a set top box receives and demultiplexes a TS; 2, a basic code stream of a digital television is identified according to PIDs (packet identifiers) of audio data and video data acquired during a demultiplexing course; and 3, If a node of a buffer queue has the audio data, then the node is added to a audio data queue; if the node of the buffer queue has the video data, then the node is added to a video data queue; and if the node of the buffer queue has the audio data and the video data, then the node is added to both the audio data queue and the video data queue. TS data after demultiplexing is subjected to attribute marking, and the audio data and the video data of a single program are stripped and reconstituted according to attributes. The function is completely accomplished by CPU (central processing unit) software and does not require a hardware support. Meanwhile, a shared memory buffer method is adopted, so that data copy is reduced and the processing efficiency is high.

The invention discloses a distributed non-volatile file system, which comprises a plurality of metadata servers and a plurality of storage servers. One or more non-volatile memories are arranged in each storage server, and all the non-volatile memories are mapped into a whole memory space. Wherein, any one of the metadata servers is configured to divide the target file into a plurality of stripesupon receiving a storage request for the target file; according to the size of each strip, the corresponding memory space is allocated for each strip through a query space allocatable table. Record the starting address and offset of each strip in memory space in the metadata of the destination file. The technical proposal provided by the embodiment of the invention can realize fast metadata access, reduce searching time of metadata, reduce data copy and reduce read-write access delay by utilizing non-volatile memory for file storage.

The invention includes M pieces of node machines for storage service, one node machine of metadata and N pieces of client node machines, which are connected to each other through VIA communication network. Running on the node machines for storage service, the modules include system initiation module, module for monitoring request, and module for processing request. Running on the node machine of metadata, the modules include metadata initiation module, monitoring module for request from metadata, and state detection module. Running on the client node machine, the modules include one or more device drive modules of network module based on VIA. The invention realizes zero copy of data transmission, shortened path of network stored data, as well as solves main issues of network storage: network bandwidth, storage access speed and interoperability.

The invention provides a method for high-speed data transmission between multiple cores. The method comprises the following steps of building a section of shared memory with a fixed size between inner cores, designing the shared memory into a form of an annular buffering region, initializing a CPPI (communications port programming interface), then judging whether the CPPI is provided with an unoccupied buffering region if a sender inner core intends to send data, taking out a CPPE (communications port programming element) pointer from the CPPI through a receiver inner core if the CPPI is provided with the unoccupied buffering region, and directly processing the data to implement data transmission. According to the method, data copying can be reduced, and the problem of data read-write conflicts of the shared memory can be solved; the transmission efficiency and the transmission stability of the data are improved under a condition of high data rate.

The invention relates to the technical field of network communication, in particular to a method and a device for quick packet mirroring forwarding of network equipment. The method comprises the stepsthat: the network equipment carries out mirroring of an original message through using a mirroring network port after receiving an original message; the original message is analyzed to obtain a message header and bearing data of the original message; a message header of a forwarding message is reconstructed according to the message header of the original message and a data packet modification requirement of the forwarding message; and the newly constructed message header and the bearing data of the original message are combined and sent to finish message forwarding. For the situation that theoriginal message needs to be modified before the message is forwarded, the method can ensure that mirroring and forwarding of the data do not influence each other, unnecessary data copying is reducedwhile the correctness of mirroring and forwarding of the data packet is ensured, and the performance of equipment is greatly improved.

The invention discloses a virtualization carrier wave migration method based on L3 and L2 layers protocol, and the method mainly solves a problem of low scheduling efficiency of C-RAN base-band pool resource in prior art. The scheme comprises: a source end transmits a migration request to a carrier wave migration and control module; the carrier wave migration and control module transmits migration environment preparation message to a target end; the carrier wave migration and control module receives a message that the migration environment preparation of the target end is finished, and a migration command is transmitted; the last round of data migration copy is performed on L3 and L2 layers protocol carrier wave virtual machines by the source end, and a L1 layer protocol virtual machine caches the processed data; the carrier wave migration and control module receives a message that the data copy and migration is finished, and the module transmits the above message to the target end; the target end starts the L3 and L2 layers protocol carrier wave virtual machines generated because of migration and clears the cache data, and source end business data is transmitted to the target end through an exchange network. The method realizes the efficient utilization of resources and can be applied in C-RAN base-band pool resource scheduling.

The invention relates to a method for improving handling capacity performance of a solid-state disk. The method comprises the steps that (1) a type of a solid-state disk interface is judged, and if the solid-state disk is an SATA interface or an SAS interface, the step (2) is started; if the solid-state disk is a PCIe interface, the step (3) is started; (2) the number of lines and the line depth of each line are increased, a user-state storage driver and a user-state TCP / IP protocol stack are established, a user-state timing polling mode is adopted, a state register of the solid-state disk is inquired in real time, and a necessary I / O operation is conducted; (3) an IO scheduling layer is removed, a simplified NVMe protocol is directly hung on an upper interface layer, and the discovery and maintenance of the solid-state disk are achieved through driving of a PCIe bus; after simulation verification is completed, the handling capacity of the solid-state disk can be effectively improved, and compared with a traditional IO protocol stack, the handling capacity can increase by 20% or above.

The invention provides a method and an apparatus for forwarding data of a virtual switching device, and a computer device. The method for forwarding the data of the virtual switching device comprisesthe steps of: receiving, by using a vhost interface, first network data transmitted by a virtual terminal, and transmitting the first network data to a network bridge; performing encapsulation of a preset protocol on the first network data in the network bridge according to a preliminarily established encapsulation flow table to generate a forwarding data packet, and transmitting the forwarding data packet to a Data Plane Development Kit (dpdk) port; and sending the forwarding data packet to a target terminal via a network card at the dpdk port. According to the method for forwarding the dataof the virtual switching device provided by the invention, one network bridge in a one-off data forwarding process can be reduced and the data copies of a memory of a one-off virtual switching deviceare reduced; and therefore, the forwarding performance of the virtual switching device is improved.

The invention provides an IPSec VPN high-performance data synchronization method; the method comprises the steps of setting a to-be-synchronized queue and a synchronizing queue, adopting batch synchronization based on a timer, and actively linking to the to-be-synchronized queue when data in a data information item changes, thereby realizing timing synchronization; the to-be-synchronized queue is used for linking all data information items which are changed but do not start synchronization; the synchronizing queue is used for maintaining all data information items which are changed and are being synchronized; and the timer is responsible for initiating a data synchronization function according to a set timing time threshold. The meaningless waste due to a fact that the traditional timing inquiry method cannot obtain whether the synchronous data is changed and inquires all items is saved; and meanwhile, the data copy due to timing inquiry is also reduced.

The invention relates to the field of set top boxes and discloses an audio-video stripping method and a system for TS (transport stream) media. The method comprises the steps that 1, a set top box receives and demultiplexes a TS; 2, a basic code stream of a digital television is identified according to PIDs (packet identifiers) of audio data and video data acquired during a demultiplexing course; and 3, If a node of a buffer queue has the audio data, then the node is added to a audio data queue; if the node of the buffer queue has the video data, then the node is added to a video data queue; and if the node of the buffer queue has the audio data and the video data, then the node is added to both the audio data queue and the video data queue. TS data after demultiplexing is subjected to attribute marking, and the audio data and the video data of a single program are stripped and reconstituted according to attributes. The function is completely accomplished by CPU (central processing unit) software and does not require a hardware support. Meanwhile, a shared memory buffer method is adopted, so that data copy is reduced and the processing efficiency is high.

The invention relates to a multi-process communication method based on a shared memory backup mechanism. The method comprises the following steps: receiving communication character string data and a data cache length; performing hash value calculation processing according to the communication character string data, and opening up N shared memory blocks with the same storage space according to a hash value and a data cache length sequence; enabling the write process to sequentially query the state information of each shared memory block according to the index address; when the shared memory blocks are in an idle state or a write-in state, enabling the write process to point a write data pointer to the shared memory blocks until all the shared memory blocks are fully written, and enabling the write process to point the write data pointer to the first shared memory block; meanwhile, enabling the reading process to sequentially query the state information of each shared memory block according to the index address; when the shared memory block is in an idle state or a read state, enabling the read process to point a read data pointer to the shared memory block, and reading write-in data in the shared memory block and / or a pointer address of the read data pointer.

The invention discloses a reliable transmission method based on broadcast communication, a storage medium and a terminal. The method comprises the following steps: sending data to a to-be-sent devicein a broadcast or multicast form; and sending the data to a to-be-sent device which is not fed back within a timeout threshold in a unicast manner. According to the method disclosed by the invention,the form of preferentially broadcasting the data and adding unicast transmission to the device which is not successfully received is adopted, so that the purpose of reducing data copy under the condition that the transmission quality is ensured under a one-to-many reliable transmission scene is achieved.

Provided is an efficient fault-tolerant data collecting method. According to the method, large-scale data flow of a satellite is processed in an assembly line mode, and circular buffering serves as an assembly shop of an assembly line. Error codes in a satellite data transmission process can be skillfully limited within a buffering length by circular buffering, and robustness is enhanced. In addition, in the method, the uncompressing row missing circumstance in each satellite row is indicated by using bits, actual copying operation is replaced with an index row position uncompressing method, and the improvement can improve satellite data real-time processing ability to the maximum extent.

Disclosed is an apparatus for distributed processing of an identical packet in high-speed network security equipment, including: a plurality of analysis modules for each determining whether vulnerability analysis is required by analyzing a received packet; a circular queue for receiving the packet from an analysis module initially determining that the vulnerability analysis is required and storing the received packet as a bucket structure; and a plurality of analysis engines for each performing different vulnerability analyses for the packet acquired from the circular queue based on a packet address of the bucket structure, in which the bucket structure includes a packet data storage unit and packet use information storage units which are as many as the plurality of analysis engines, and the packet use information storage units store packet use information of the plurality of respective analysis engines, respectively.

The invention discloses a linux system efficient packet receiving and sending method suitable for an industrial control environment, and the method comprises the following steps: S1, enabling a linuxsystem to carry out the caching of an original data packet when the linux system carries out the packet receiving operation; s2, receiving the message to a user state through an AF _ PACKET socket; s3, enabling the kernel mode and the user mode to share a ring buffer, andcompleting packet receiving operation; s4, when the linux system carries out packet sending operation, directly sending the original data message, and S5, sending the data message through the kernel protocol stack to complete the packet sending operation. According to the invention, the advantages of an AF _ PACKET packet receiving scheme and the advantages of an NFQUEUE packet sending scheme are combined, and packet receiving and sending can be carried out efficiently. A kernel packet forwarding function is fully utilized, and packet sending processing is carried out at a data packet forwarding point of a kernel, so that the processing time delay of a network data packet is effectively reduced, and the real-time requirement of an industrial control system is met.

The invention discloses a virtualized carrier migration method based on L3 and L2 layer protocols, which mainly solves the problem of low scheduling efficiency of C-RAN baseband pool resources in the prior art. The scheme is: the source end sends a migration request to the carrier migration control module; the carrier migration module sends a migration environment preparation message to the destination; the carrier migration control module sends a migration command to the source after receiving the destination migration environment preparation message; When the terminal performs the last round of data migration and copying for the L3 and L2 layer protocol carrier virtual machines, the L1 layer protocol virtual machine caches the processed data; after the carrier migration control module receives the data copy migration completion message, it sends the data migration copy to the destination end Completed message; the L3 and L2 layer protocol carrier virtual machines generated by the migration of the destination end are started, and the cached data is cleared, and the source end service data is sent to the destination end through the switching network. The invention realizes efficient utilization of resources, and can be applied to C-RAN baseband pool resource scheduling.

The invention discloses a high-speed data acquisition and transmission device, which comprises a programmable logic device data acquisition module, an RAM (Random Access Memory) reading and writing coordination control module, a communication module and an Ethernet data transmission module, wherein the programmable logic device data acquisition module is used for high-speed acquisition and buffer; the RAM reading and writing coordination control module is used for high-speed buffer and reading of programmable logic device control data and is used for reading data of a certain data length from an RAM and buffering in a programmable logic device when a single-chip microprocessor has a data request; the communication module is used for realizing communication of the programmable logic device and the single-chip microprocessor and realizing reading of data between the single-chip microprocessor and the programmable logic device by a pair of backward stage FIFO (First Input First Output) and SPI (Single Program Initiation) communication of ping-pong operation; and the Ethernet data transmission module is used for optimizing a UIP (Universal Interface Processor) protocol stack, acquiring data quantity which is easy for IP (Internet Protocol) sub-packet transmission, simplifying a data transmission process into a copy packet header and indirectly reading the data cached in the RAM to the network buffer region through the programmable logic device. According to the high-speed data acquisition and transmission device, high-speed AD (Analog Digital) acquisition is realized, the copy frequency for uploading the data to Ethernet is reduced, and the problem of low CPU (Central Processing Unit) utilization rate is solved.

The invention discloses a network data packet parallel processing method based on an Intel DPDK (Data Plane Development Kit). The method comprises the following steps: step 1, establishing an Intel DPDK running environment; step 2, creating a memory pool, and creating a RX memory ring for receiving the network data packet and a TX memory ring for transmitting the network data packet; step 3, setting a network data packet scheduling strategy, and preparing to perform network data packet scheduling; step 4, creating multiple worker threads, and performing parallel processing of the network data packet. In the method provided by the invention, performance advantage of a multi-core CPU is used fully, and by lock-free data structure design, parallel network processing framework and high-efficiency network data packet distribution, parallel processing of the network data packet on multiple CPU cores is realized.

The invention relates to a cloud-based rail transit integrated data bus system communication method and system, and the method specifically comprises the steps of maintaining a theme data structure which comprises at least one partition data structure, wherein the partition data structure is a message sequence with a sequence, the message is a data structure used for transmitting information, andwherein the partition data structure has a unique partition number in the theme data structure to which the partition data structure belongs, and the message has a unique message number in the partition data structure to which the message belongs; and enabling the producer of the message to transmit the message to the corresponding theme data structure, so that all consumers subscribing the themecan obtain the message from the theme data structure. Therefore, the invention has the following advantages: 1, low delay is realized while the throughput of the data bus is improved; 2, through message persistence, flexible message processing is realized; and 3, various requirements of message orderliness, message disorder and the like are supported.

The invention discloses a network packet receiving and sending path optimization method and device and a storage medium, and the method comprises the steps: calling a recv interface to read a data packet stored in a pre-constructed annular queue during network packet receiving, and returning the address of the data packet to a user program, so that the user program can operate the data packet according to the address; during network packet sending,calling an alloc interface, returning a first available memory address from the annular queue, constructing an Ethernet frame on the memory, and then calling a send interface to send a data packet. According to the invention, the packet is received in a polling mode during packet receiving, interruption processing is not needed any more, a kernel protocol stack is not bypassed any more, and data copying is reduced; according to the network packet receiving and sending method, the network packet receiving and sending path is optimized, and the network packet receiving and sending performance is effectively improved.

The invention provides a deterministic airborne high-performance file transmission method and system, the system comprises a server and at least one client, and at least one application software runson the client; a long message channel and a short message channel are arranged between the server and the client; by utilizing a distributed priority scheduling control mechanism based on static configuration, the file transmission service quality of different security level applications is ensured; and by utilizing the characteristic that an airborne network resource short message channel and a long message channel coexist, a message is controlled to pass through the short message channel, a data message passes through the long message channel, a response message transmission protocol is simplified, and the file transmission performance is improved. Short and long message resources are fully utilized, a transmission protocol response message is simplified, the file closing frequency is reduced, data copying is reduced, and the airborne file transmission efficiency is improved; and through the priority round-robin scheduling of the client and the server, the transmission service quality of applications with different security levels is ensured.

The invention provides an IPSec VPN high-performance data synchronization method; the method comprises the steps of setting a to-be-synchronized queue and a synchronizing queue, adopting batch synchronization based on a timer, and actively linking to the to-be-synchronized queue when data in a data information item changes, thereby realizing timing synchronization; the to-be-synchronized queue is used for linking all data information items which are changed but do not start synchronization; the synchronizing queue is used for maintaining all data information items which are changed and are being synchronized; and the timer is responsible for initiating a data synchronization function according to a set timing time threshold. The meaningless waste due to a fact that the traditional timing inquiry method cannot obtain whether the synchronous data is changed and inquires all items is saved; and meanwhile, the data copy due to timing inquiry is also reduced.

The invention relates to the field of set top boxes and discloses an MP3 streaming media sound mixing method of a set top box. The method comprises the following step of: 1) a data downloading process; 2) a data decoding process: obtaining a decodable MP3 data buffer zone, obtaining an available pulse code modulation (PCM) data buffer zone and decoding MP3 data in the PCM data buffer zone; and assigning the decoded MP3 data buffer zone to the data downloading process, assigning the PCM data buffer zone to a PCM injection process and applying for a next decodable MP3 data buffer zone; and 3) the PCM injection process. The method has the advantages that since multiple processes share an internal memory to realize data caching and the MP3 data is downloaded segment by segment, the occupation rate of the internal memory is reduced; since the access and the protection of the shared internal memory adopt an assigning method, the data copies are reduced and the efficiency is improved.

The invention relates to a task processing method and device, computer equipment and a storage medium. Atomic attribute nodes of a task sequence are identified, an atomic attribute task sequence is obtained based on the atomic attribute nodes, tasks contained in the atomic attribute task sequence are continuously executed on target equipment; due to the fact that each task in the atomic attribute task sequence can reuse the data on the devices, the method of the embodiment can not only reduce data exchange between the devices in the task execution process, but also reduce data copying when the tasks need to be continuously executed.