Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method for quickly receiving and storing data in shared file system

A technology for sharing files and receiving data. It is applied in file systems, transmission systems, file access structures, etc. It can solve the problems of NAS performance degradation, inability to guarantee real-time performance, and exhaustion of CPU resources, so as to break performance limitations and improve performance. Concurrent access efficiency, reducing the effect of page fault interruption

Active Publication Date: 2018-03-02
BEIJING JINGYI QIANGYUAN TECH
View PDF6 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In education recording and broadcasting systems or video surveillance systems, there are often hundreds or even more IP cameras, and dozens of clients need to write and operate files. Facing the storage of such huge video data, traditional NAS The file system generally has the following obvious deficiencies: Because of the traditional network card driver method, when multiple clients access multiple files in the NAS file system at the same time, the performance of the NAS is greatly reduced, and ultimately cannot meet the needs of users
The traditional network card driver is a kernel-mode network card driver, and the message needs to be copied between the user mode and the kernel mode multiple times. When the amount of data increases, the overhead is particularly obvious; the traditional network card driver uses the interrupt mode driver, and a large number of CPU resources will be exhausted, resulting in a serious performance drop; in traditional network card drivers, each data thread and control thread is scheduled back and forth between each CPU core, wasting resources
The above shortcomings caused that when multiple clients access multiple files in the NAS file system at the same time, the speed drops seriously, and the real-time performance cannot be guaranteed.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for quickly receiving and storing data in shared file system
  • Method for quickly receiving and storing data in shared file system
  • Method for quickly receiving and storing data in shared file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0040] This embodiment is a method for quickly receiving and storing data in a shared file system. The hardware system used in the method includes: multiple clients are connected to a Linux-based NAS subsystem through a LAN network, and the NAS subsystem The system includes a storage server and a storage disk array, and each client is connected to a plurality of IP cameras through a LAN network, such as figure 1 shown.

[0041]The above-mentioned client can be an ordinary PC workstation or a server, capable of connecting to a LAN network, and can collect video and audio data from video and audio sources. The LAN network described in this embodiment is composed of Ethernet switches and uses TCP / IP as the communication protocol. It is a broadband network with a bandwidth exceeding 1G and can transmit high-definition video files. The video and audio source is a high-definition IP camera, which can provide a large bit-rate high-definition video stream. The storage system is a Li...

Embodiment 2

[0053] This embodiment is an improvement of the first embodiment, and is a refinement of the "step of sending video and audio data" of the first embodiment. The "step of sending video and audio data" described in this embodiment includes the following sub-steps, the process is as follows: image 3 Shown:

[0054] The sub-steps of establishing a connection: the client sends a connection request to the storage server, and waits for the response from the storage server. If the connection is successful, enter the following sub-steps, otherwise exit.

[0055] The sub-step of sending the command to open the file: the client specifies the information to be written into the file, and sends it to the storage server according to the preset private transfer protocol, and waits for the response. If the file is opened successfully, enter the following sub-step, otherwise quit.

[0056] The sub-step of opening up the cache space: if the file is successfully opened, open up a cache space l...

Embodiment 3

[0060] This embodiment is an improvement of the above embodiment, and is a refinement of the "step of receiving video and audio data" of the above embodiment. The "step of receiving video and audio data" described in this embodiment includes the following sub-steps:

[0061] Substep of setting up huge page memory: Used for storage servers to set up the system to support huge page memory.

[0062] The sub-steps of PCI device initialization: the storage server scans PCI devices, registers PCI drivers, and uses Linux UIO technology to register the network card used to receive data as a UIO device, so that user-mode applications can directly use the network card to send or receive data.

[0063] The sub-step of creating a memory pool: the storage server applies for several fixed-size memory blocks in the huge page memory to form a memory pool for caching network packets.

[0064] The sub-step of initializing the data queue: it is used for the storage server to initialize 2 metada...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a method for quickly receiving and storing data in a shared file system. The method comprises the steps of: obtaining videos and audios; processing the videos and audios; sending video and audio data; receiving the video and audio data; and writing in a file. According to the method disclosed by the invention, by utilization of a large page memory, a user-state network card driving technology and a data and metadata separation technology, a storage server can quickly receive multiple channels of data flows sent by multiple client sides, and write the data flows in thefile; and thus, the purpose of storing hundreds of video flows at the same time can be achieved. By means of the method disclosed by the invention, the concurrent access efficiency of the shared filesystem is effectively increased; simultaneously, the more access files are, the more obvious the effect is; and, for the common education recording industry or video monitoring industry, the video storage efficiency is increased to a great degree.

Description

technical field [0001] The invention relates to a method for quickly receiving and storing data in a shared file system, a method for storing video data in a shared file system, and a method for quickly receiving and storing data applicable to the field of video storage. Background technique [0002] At present, a common NAS shared file system is composed of a NAS server and multiple clients through a LAN network. The client connects and communicates with the NAS server through the tcp / ip protocol under the LAN network. When the client wants to perform file writing operations, it needs to continuously send data to the NAS server through the LAN network. [0003] NAS storage has the advantages of separating servers and storage, reducing the burden on servers, facilitating data sharing, good adaptability to existing network environments, and high scalability. It has been widely used in the field of video storage, including an important application scenario : Multiple clients...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08G06F17/30
CPCH04L67/1097G06F16/13G06F16/176H04L67/561
Inventor 徐荣波黄贤强许珊珊张广东赵子兰张跃
Owner BEIJING JINGYI QIANGYUAN TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com