Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

50results about How to "Reduce I/O operations" patented technology

Method and system for obtaining dynamic feed index

The invention discloses a method and system for obtaining a dynamic feed index. Better query performance can be obtained on the occasion of high concurrent query and large data amount of an index table. An index table cache is established by a storage structure by means of a key value, and a feed index in the index table cache is consistent with a corresponding feed index in a relation type database. The method includes that a client server sends a query request to the index table cache and the relation type database, and takes query results fed back by the index table cache and the relation type database as the obtained feed index.
Owner:BEIJING FEINNO COMM TECH

Dynamic taint analysis device and application thereof to document format reverse analysis

The invention relates to a dynamic taint analysis device and application of the dynamic taint analysis device to document format reverse analysis, wherein the dynamic taint analysis device comprises a dynamic instrumentation executive logging module and a static snapshoot analysis taint tracking module, wherein the dynamic instrumentation executive logging module is used for calling and executing a tested program by using a binary program instrumentation platform, monitoring opening, analyzing and closing behaviors of a data document including original taint data in a process of executing the tested program, and acquiring snapshoot logs of all command processes, context information and memory access information in the process of executing the tested program according to the process of executing the tested program; the static snapshoot analysis taint tracking module is used for analyzing the snapshoot logs and simulating replay execution of a progress according to information obtained by analysis and recording processing and spread information of original taint data in the data document to obtain a taint data stream path. According to the dynamic taint analysis device, I / O (Input / Output), time and space overhead of dynamic taint analysis in a dynamic execution process can be reduced, an extended instruction set can be supported, and the continuity and incidence relation of original taint data can be obtained.
Owner:BEIJING UNIV OF POSTS & TELECOMM

Log data writing method and log system

The invention discloses a log data writing method and a log system. The method comprises the following steps of: obtaining generated log data and writing the log data into a memory; when the log data in the memory reach a preset number, writing all log data stored in the memory into a disk; or, after obtaining the generated log data and writing the log data into the memory and before writing all log data into the disk, writing all log data into the disk if the log data stored in the memory exist when a preset disk writing time is finished. Through the adoption of the log data writing method and the log system disclosed by an embodiment of the invention, the operation of immediately writing the generated log data into the disk once the log data are generated in the prior art can be reduced, I / O (Input / Output) operations of the disk are reduced, and the frequency of writing the log data into the disk for filing under a high-concurrency application of the log data is reduced, so that the time of writing the log into the disk is reduced; meanwhile, the power consumption of the system during writing the data into the disk is reduced and finally a purpose of quick response is achieved.
Owner:深圳市金蝶精斗云网络科技有限公司

Data dimension reduction method based on parallel principal component analysis (PCA) algorithm

InactiveCN107273917AOvercome the problem of not being able to load into memory at one timeImprove processing efficiencyCharacter and pattern recognitionHigh dimensionalEuclidean vector
The invention discloses a data dimension reduction method based on a parallel principal component analysis (PCA) algorithm. The method comprises the steps of: S1, constructing a sample data matrix D<nxm> by high-dimensional data of which dimensions are to be reduced; S2, calculating a covariance matrix C<mxm> of the sample data matrix D<nxm>; S3, calculating m feature values of the covariance matrix C<mxm> and m corresponding feature vectors; S4, determining the number k of principal components according to the feature values and the feature vectors; and S5, utilizing the feature vectors, which correspond to the top-k greater feature values, to construct a transformation matrix, and utilizing the transformation matrix to calculate a principal component matrix, wherein the principal component matrix is data of which the dimensions are reduced. According to the method, the problem that according to a traditional stand-alone principal component analysis algorithm, the data cannot be loaded into a memory at once because a data size is too large is overcome, I / O operations are reduced, and the processing efficiency of data dimension reduction is improved.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Vector data preorder quadtree coding and indexing method based on Key / Value type NoSQL (Not only SQL)

The invention relates to a vector data preorder quadtree coding and indexing method based on the Key / Value type NoSQL (Not only SQL). According to the method, the data physical storage order and the spatial scale are consistent in continuity, the data major key order and the data physical storage order are consistent, and therefore input / output (I / O) operation is reduced during spatial query, and query efficiency is improved. The method comprises the steps of complete quadtree space division and preorder quadtree node coding, vector data prefix coding and index establishment, and preorder quadtree indexing.
Owner:NANJING NORMAL UNIVERSITY

OWGA memory cache based mass remote sensing tile data quick releasing method

The invention discloses an OWGA memory cache based mass remote sensing tile data quick releasing method. The method is characterized in that considering three attributes including the access time, theaccess frequency, and the key value of tile data; establishing an OWGA operator according to the access time and the access frequency of a requesting target tile, and using the key value as an uniqueidentifier of a tile, wherein the key value consists of the hierarchy, a line number and a row number of the tile; adopting a Map < the key value of the tile and a tile object > a chain table to manage the tile object in a memory; preferentially performing descending sorting according to the OWGA value of the tile object, and performing ascending sorting according to the key value so as to save the tile data which is frequently requested by a client recently and has high resolution in the memory all the time, and reduce time-consuming file I / O operations. The method provides a strategy that aplurality of tiles near the requesting target tile are preloaded in a memory cache so as to predetermine a purpose of a client request, can further improve the cache hit ratio, and can accelerate theresponse speed of a releasing server.
Owner:北京四维新世纪信息技术有限公司

Read-write solution for tens of millions of small file data

The invention provides a read-write solution for tens of millions of small file data. According to the solution, the mode of creating large-block continuous disk space is used for storing a large number of small files when the small files are stored, namely, logically continuous data are stored on the continuous space of a disk array as far as possible; the disk space is divided into a plurality of blocks, and the size of each block is 64 KB. According to the basic thought, each small file can only be stored in the single block and cannot be stored by crossing two blocks, each folder can be provided with one or more blocks, the blocks are only used for storing the data of the corresponding folder, and each piece of file data is stored on the continuous disk space. Compared with the prior art, the logically continuous data are stored on the continuous space of each physical disk as far as possible, the cache technology is used for playing the role of a metadata server, the cache utilization rate is improved through simplified file information nodes, and thus the access performance of the small files is improved.
Owner:LANGCHAO ELECTRONIC INFORMATION IND CO LTD

Migration and erasure code-based reconstruction coupled rapid prediction repair method and implementation

The invention discloses a migration and erasure code-based reconstruction coupled rapid prediction repair method. The method comprises the steps of monitoring a node state by adopting a node failure prediction algorithm; organizing the blocks on the nodes about to fail into a plurality of reconstruction sets; and determining a coupling strategy of the migration and erasure code-based reconstruction according to the number of blocks contained in the reconstruction sets. According to the teaching of the above embodiment, the migration and erasure code-based reconstruction coupled rapid prediction repair can be realized by combining the failure prediction, thereby not only accelerating the data restoration, but also reducing the I / O operation needed by the repair compared with the reconstruction based on the erasure codes. The invention further discloses a migration and erasure code-based reconstruction coupled rapid prediction repair device.
Owner:云链网科技(广东)有限公司

Open platform and system for processing reports and report processing method

The embodiment of the invention discloses an open platform and a system for processing reports and a report processing method. The open platform for processing the reports is accessed into an in-memory database to serve as a data source of an on-line report, so that the data of the in-memory database can be synchronized, visited and statistically processed, real-time of the data is ensured and meanwhile processing and generating efficiencies of the reports are greatly improved, the use ratio of computer resources is optimized, meanwhile consistency of the in-memory database and mainframe source data is maintained by a data synchronization module, the data of the in-memory database can be visited and processed by a report analysis engine, I / O operation when in data synchronization and processing is reduced, real-time and higher operation parallelism of the data are ensured, and performance of the whole system is improved.
Owner:CHINA CONSTRUCTION BANK

I/O method for iSCSI target module block device

The invention discloses a method used in the I / O of iSCSI target device block equipment. The method comprises the steps that page high speed cache is used and a Radix-Tree-based searching algorithm is adopted between interfaces of the iSCSI target device and the storage equipment. The method aims at enhancing the I / O performance of a storage system, exchanging time by using space, as well as using high-efficient searching algorithm to enhance executing efficiency. The invention adopts the method for improving a software system, basically has nothing to do with the characteristics of hardware equipment and has good universal external interfaces.
Owner:WUHAN HAIHENG INFORMATION STORAGE

Data storage control method and device, equipment and storage medium

An embodiment of the invention discloses a data storage control method, a data storage control device, equipment and a storage medium. The data storage control method comprises the steps of: when to-be-stored target data is received, acquiring at least one item of data attribute information matched with the target data according to at least one preset filtering type; processing the data attributeinformation by adopting a filtering processing rule matched with the filtering type to generate a filtering comparison identifier matched with the target data; judging whether the filtering comparisonidentifier is stored in a pre-established identifier database or not; if so, performing data filtering on the target data; otherwise, storing the target data in a target database, and updating the identifier database according to the filtering comparison identifier. According to the data storage control method and the data storage control device of the invention, massive data can be cleaned according to user requirements, and meanwhile, the data is filtered before being stored in the database, so that IO resources are saved, and data storage control performance is improved.
Owner:ZHEJIANG UNIVIEW TECH CO LTD

Data caching method and device, equipment and storage medium

The invention discloses a data caching method and device, equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps that a data acquisition request initiated by a target client for target data is acquired, after the data acquisition request reaches a gateway, the gateway judges whether a first-level cache located in a local memory hits the target data or not according to a target caching rule, and if yes, the first-level cache in the local memory hits the target data; if the target data is hit, obtaining the target data from the first-level cache and returning the target data to the target client, if the target data is not hit, judging whether the target data exists in a second-level cache of the preset cache middleware or not, and if the target data exists in the second-level cache, obtaining the target data from the second-level cache and returning the target data to the target client, and if not, obtaining the target data from the service system and returning the target data to the target client. According to the method, network transmission and I / O operation of a disk are reduced in a multi-level cache mode, the client request efficiency, the CPU resource utilization rate and the query frequency processed by the system in unit time are improved, and meanwhile cache data can be prevented from being lost.
Owner:四川万网鑫成信息科技有限公司

Data processing method and apparatus

Embodiments of the invention disclose a data processing method and apparatus. The method comprises the steps of obtaining multiple pieces of to-be-processed data; defining the to-be-processed data in a plurality of preset intervals; determining approximate values of the to-be-processed data in the intervals; and according to the approximate values, determining quantiles corresponding to the to-be-processed data. By utilizing the method and the apparatus, the approximate values are used for replacing the numerical values of the to-be-processed data defined in any interval, so that the quantity of data participating in a calculation process can be reduced. Therefore, the traversal frequency during quantile calculation can be reduced; the I / O operation on a database can be further reduced; and the efficiency of the quantile calculation process can be improved to a certain extent.
Owner:ADVANCED NEW TECH CO LTD

Video storage and retrieval method

The invention discloses a video storage and retrieval method, and the method comprises the following steps: obtaining a video stream, and converting the video stream into an H264 bare stream; identifying a key frame in the H264 bare stream; calculating the frame number of the key frame and the absolute deviation between the key frame and the initial position; establishing a key frame index queue according to the frame number of the key frame and the absolute offset; writing the key frame index queue into a storage file; the efficiency of video positioning, fast forward and slow backward is greatly improved, and the delay of dragging the video progress bar is reduced.
Owner:NANJING UNIV OF POSTS & TELECOMM

Real-time data report statistical method under massive data

The present invention discloses a real-time data report statistical method under the massive data in the field of data statistics. The method is characterized in that: while application data is storedin a database, statistical data is written into an in-memory database in real time according to statistical characteristics of the data; according to the statistical characteristics of the data, fromthe current statistical period, the statistical data of the previous statistical period is stored by the in-memory database in the database; and when a user views statistical report information, a data collection processor performs statistical analysis according to different statistical characteristics, extracts data from the database or the in-memory database, and returns a statistical result. According to the method disclosed by the present invention, the purposes of collecting statistics of real-time data and having relatively high performance results are achieved, and the user experienceis further improved.
Owner:SHENZHEN SEENTECH INFORMATION TECH DEV

Telemetering data access method for ground station of unmanned airship system

The invention relates to a telemetering data access method for a ground station of an unmanned airship system, which comprises the following steps: saving the telemetering data of the unmanned airship system as an internal memory mapping data base comprising a plurality of data files, simultaneously establishing a corresponding external memory data base, and establishing worksheets corresponding to the data files of the internal memory mapping data base one to one in the external memory data base; then setting a historical variable and a current variable for each data file in the internal memory mapping data base, wherein the historical variables are used for marking recording positions in the external memory data base, and the current variables are used for marking the current recording position in the internal memory mapping data base; and finally, saving updated data to the internal memory mapping data base and saving the data files from the historical variables to the current variables in the internal memory mapping data base to the external memory data base at regular time to replace the complex and expensive internal memory mapping data base, realize the real-time monitoring on the unmanned airship and meet the demands on telemetering data access in the aspects of real-time performance, reliability and usability.
Owner:SHANGHAI JIAO TONG UNIV

Maximum frequent item set mining method based on adjacency list

The invention relates to a maximum frequent item set mining method based on an adjacency list, and the method employs a storage mode combining the adjacency list and a Hash table to reduce the scanning frequency of a database and the spatial scale of traversal, can trim out an item set smaller than a support degree threshold as early as possible, and avoids the generation of all non-empty subsetsof a longer maximum frequent item set. According to the method, the established adjacency list is fully utilized, the original database only needs to be scanned once, and the method has the advantagesof low time complexity, memory consumption and the like.
Owner:NORTHWESTERN POLYTECHNICAL UNIV

A method, device and system for allocating port rule information

The invention discloses a method, device and system for allocating port rule information, which are applied in a network system including a server device, a port rule database and at least one client device, wherein the method includes the following steps: the server device Obtain all port rule information in the port rule database from the port rule database; the server device stores the obtained port rule information; the server device submits a query port rule request according to the client , sending the corresponding port rule information to the client device. By using the invention, the I / O operation from the server end to the port database is saved, and at the same time, high concurrency of data transmission can be achieved, and a large number of monitoring requests can be supported.
Owner:TENCENT TECH (SHENZHEN) CO LTD

Line migration elimination method and device, computer equipment and storage medium

The invention relates to the technical field of cloud data, and provides a line migration elimination method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of detecting a recording line with line migration, determining a line identifier corresponding to the recording line, searching a current data block where the recording line is located after line migration and an original data block where the recording line is located before line migration based on the line identifier, and performing line migration on the current data block and the original data block. And determining a residual free space of the original data block and a storage space occupied by the record line, when the residual free space is not smaller than the storage space, migrating the record line in the current data block into the original data block, and deleting a pointer pointing to the current data block in the original data block, the additional I / O operation of the database is reduced, and the data processing performance of the database is improved on the whole.
Owner:PING AN SECURITIES CO LTD

Buffering capacity allocation method for massive logs

The invention discloses a buffering capacity allocation method for massive logs. The method comprises an eleventh step of reading the log in a sublist in real time; a twelfth step of counting citation times of an offset of a domain appears for the first time; a thirteenth step of establishing citation volume of each segment, and computing total citation volume of the sublist; a fourteenth step of performing linear fitting on each segment of citation volume; and a fifteenth step of allocating a preset total buffering capacity in the sublist for each segment. The method has the beneficial effects of reasonably allocating the buffering capacity, occupying less memory resources, and reducing input / output (I / O) operation due to citation of domain offset.
Owner:SURFILTER NETWORK TECH

Application server constant value dynamic update method and system

The invention provides a method for dynamically updating the constant values of an application server. The method comprises the steps of: reading and analyzing constant files in a hard disk, and storing data in the constant files into a memory in a key-value mode, wherein the key represents a constant object, and the value represents a constant value; scanning the constant files in the hard disk, judging whether the constant files are changed or not, and if changed, reading the changed constant files; analyzing according to the key-value mode so as to obtain the changed constant objects and constant values; searching all the original constant values, which correspond to the changed constant objects, from the memory according to the changed constant objects; and clearing all the researched original constant values which correspond to the changed constant objects from the memory, and loading the changed constant values into the memory to replace the original constant values. The invention further provides a system for dynamically updating the constant values of the application server. By utilizing the method and the system for dynamically updating the constant values of the application server, the constant values of the application server can be updated in real time.
Owner:ALIBABA GRP HLDG LTD

Dynamic taint analysis device and its application in file format reverse analysis

The invention relates to a dynamic taint analysis device and application of the dynamic taint analysis device to document format reverse analysis, wherein the dynamic taint analysis device comprises a dynamic instrumentation executive logging module and a static snapshoot analysis taint tracking module, wherein the dynamic instrumentation executive logging module is used for calling and executing a tested program by using a binary program instrumentation platform, monitoring opening, analyzing and closing behaviors of a data document including original taint data in a process of executing the tested program, and acquiring snapshoot logs of all command processes, context information and memory access information in the process of executing the tested program according to the process of executing the tested program; the static snapshoot analysis taint tracking module is used for analyzing the snapshoot logs and simulating replay execution of a progress according to information obtained by analysis and recording processing and spread information of original taint data in the data document to obtain a taint data stream path. According to the dynamic taint analysis device, I / O (Input / Output), time and space overhead of dynamic taint analysis in a dynamic execution process can be reduced, an extended instruction set can be supported, and the continuity and incidence relation of original taint data can be obtained.
Owner:BEIJING UNIV OF POSTS & TELECOMM

A very large-scale image feature point matching method and system

The invention discloses a super-large scale image feature points matching method and system thereof. The method comprises the following steps: firstly, image matching pairs are obtained by caring out image near neighbor search; images are acted as nodes, the edges of constitution among the image neighbors form an undigraph, the undigraph is caring out priority sorting of breadth, then the images and image pairs are obtained; the feature information of the images are sorted again according to the sorting result, and the feature information of the images is divided and stored as binary files; the binary files with the feature information are read according to the order of sequence; according to the images after being sorted, the characteristics match is being carried out in sequence, and the subsequent useless feature information is released in time; the feature information is iteratively read and the feature matching is being carried out until all images finish matching. The method and system has the advantages that one read and local full both of image feature points in the process of image matching are ensured, memory is not spilled and at the same moment, the efficiency of algorithm is ensured.
Owner:HUAZHONG UNIV OF SCI & TECH

A data interaction method and device

The embodiment of the present invention provides a data interaction method and device. The method includes: if a data export instruction is received, searching for the mapping relationship with the relational database management system RDBMS in the target hard disk area by retrieving multiple mapping relationships recorded in the mapping list. The first internal file corresponding to the first external table in ; export the first internal file to the distributed file system HDFS; if a data import instruction is received, then determine the target hard disk by retrieving multiple mapping relationships recorded in the mapping table Whether there is already a second internal file to be imported from HDFS in the area; if not, import the second internal file into the target hard disk area, and generate a second external table in the RDBMS; establish a second external table in the mapping list The mapping relationship between the table and the second internal file. The invention effectively reduces the input / output operation of the RDBMS on the disk where the database table is located, thereby effectively ensuring the original business resources of the RDBMS and improving the processing efficiency and resource utilization of the RDBMS.
Owner:DATANG MOBILE COMM EQUIP CO LTD

A method and system for transmitting packaged files based on cloud computing environment

The invention discloses a method and system for transmitting packaged files based on a cloud computing environment. The method receives a command to transfer folders; obtains the file types and file sizes of all files in the folder; divides all files into files suitable for packaging and files unsuitable for packaging according to the file types; The files suitable for packaging are put into the packaging queue, and the files whose file size is smaller than the file threshold among the files that are not suitable for packaging are put into the packaging queue; among the files that are not suitable for packaging, the files whose file size is greater than or equal to the file threshold performing transmission; compressing and packaging the files in the packaging queue, and transmitting the packaged files formed after packaging. It greatly shortens the transfer time of the entire folder, improves the file transfer efficiency, and also reduces the I / O operations during the file transfer process, which can greatly improve the data throughput rate and service efficiency of the data center, and increase the service life of the server.
Owner:柏科数据技术(深圳)股份有限公司

A high-speed digital printing method based on multi-core processor and dma technology

The invention discloses a high-speed digital printing method based on a multi-core processor and the DMA technology. The problem that an existing digital printing machine still cannot carry out real-time processing on a large number of images due to the limitations of the PC processor performance and parallel interface data transmission is solved. Firstly, a multi-core processor is used for quickening processing on printing commands and printing images, then the DMA technology is used for achieving the input / output operation mode that automatic in-batch data exchanging is carried out between a high-speed peripheral device and a main memory so as to reduce CPU interference to the greatest extent, the efficiency of a whole computer system is greatly improved, finally the RAID technology is used for improving the access speed of a hard disk, and therefore the efficiency of a whole digital printing system is promoted.
Owner:HANGZHOU WENSLI SILK DIGITAL PRINTING CO LTD

A key-value storage method based on log-structure merged tree

ActiveCN105468298BImprove throughputReduce read and write I/O operationsInput/output to record carriersBloom filterOperating system
The invention discloses a key value storage method based on a log-structured merged tree. The method comprises: 1, setting a cache component in a memory for each data storage component, setting a Bloom filter for each cache component, and setting a Bloom filter for each data block of each SSTable; 2, carrying out jump judgment on each key value pair of current data blocks of to-be-merged data storage components Ci stage by stage by adopting the corresponding Bloom filter, wherein if the key value pair jumps to a C<i+M>, no version of key value pair corresponding to keys exists in the C<i+M> and each stage of component in the front of the C<i+M>, a B<i+M+1> comprises a corresponding version of key value pair and then the key value pair is set into a B<i+M>, and if the key value pair is jumped to a B<i+N>, no version of key value pair corresponding to keys in the B<i+N> and each stage of component in the front of the B<i+N> and a C<i+N> comprises a corresponding version of key value pairs; and 3, processing the to-be-merged key value pairs in the Bi by adopting a similarity method.
Owner:INST OF INFORMATION ENG CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products