Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

40results about How to "Reduce I/O" patented technology

File version control system and file version control method

The invention discloses a file version control system which comprises a local library and a control module. The local library comprises a working area, a meta database and a version library; the working area is used for saving files managed by a system, the meta database is used for saving a file tree, a version tree and a version list, and the version library is used for saving version files in the working area. The control module comprises an abstract file system interface module, a file operation module, a meta data access module, a recovery manager module and a working area monitor module. The files are backed up to a server after being encrypted so that privacy disclosure of the server side can be avoided. Version information of the files is maintained automatically so as to bring convenience for users to recover the files to previous version after the files are in error operation. Besides, since the file tree, the version tree and the version list are saved by means of tables and the tables are then converted into key values to save to disk files, modification and delete of cells of the tables are achieved by means of new value writing, random access to the disk is avoided, Input / output (I / O) of the disk is reduced, and network overhead is also decreased.
Owner:JIANGSU UNIV

Full-text database architecture, bitmap index creation method, data query method, server and medium

The invention discloses a full-text database architecture, a bitmap index creating method, a data query method, a server and a medium. The full-text database architecture of the invention comprises: a data module, which is used for recording database table names of a full-text database and index field information thereof; a storage module, which includes a plurality of independent data blocks, and used for storing data and index information of database tables; a loading module, which is used for loading the data of all the database tables into the data blocks according to the index information in the metadata module, and when one of the data blocks is written fully, marking the data block as a steady state; an index creation module, which is used for generating an index of the data block marked as the steady state; and a query module, which is used for carrying out searching in the metadata module according to a query statement, searching indexes of all the data blocks if a matching index field is found, and then determining, according to the index of the data block, whether the data block meets query conditions. The architecture greatly increases the query speed and database throughput.
Owner:INST OF INFORMATION ENG CAS

Method, apparatus, system and computer program for reducing I/O in a messaging environment

Requests from a client to put messages into an input queue are intercepted before reaching the input queue. The intercepted requests which refer to a message destined for the server application are established. Information associated with the server application indicates that the server application will process such a message according to a predetermined manner. A request associated with the server application for retrieving such a message is transactional, and the server application uses a one-phase commit for retrieving such a message. Such a message can then be matched with the server application's request such that the server application can retrieve the message. In this way, such messages by-pass the input queue, and are, therefore, not logged, resulting in a reduction of I / O.
Owner:IBM CORP

Distributed buffer management method based on dynamic index

The invention discloses a distributed buffer management method based on a dynamic index, applied in a database cluster system, wherein each node of the database cluster system is provided with an independent buffer; when a data block is firstly loaded to the buffer of the node by the certain node, information of the data block is firstly loaded in a dynamic index table of a global buffer manager, the attribute of the buffer is set to be a local resource, and is held by the node; when the other nodes need to access the data block, the nodes firstly search in the global buffer manager, and under the condition that the data block is loaded and held by the certain node is acquired through the dynamic index table, the nodes modify the attribute of the buffer to be a global resource, and apply to the node which holds the data block for transferring the data block for accessing. The invention realizes multi-version assess control of the data block through a dynamic index mechanism, reduces disk I / O (Input / Output), and improves overall performance of the database cluster system.
Owner:北京人大金仓信息技术股份有限公司

An internal control system management method with a process structure as a core

InactiveCN109034602AQuick buildFast implementation of constructionResourcesSystems managementWork content
The invention provides an internal control system management method with a process structure as a core, which includes: constructing a process architecture, wherein the process architecture includes single or multiple architecture nodes, the architecture node is the business action after the management actions such as approval and approval are removed from the process of organization application,and each architecture node is composed of basic attribute, form attribute and risk attribute; when the process architecture is created successfully, copying the process architecture into different process templates according to different departments, so that different users can select the process templates to copy and form the proprietary process management; The process starts to flow automatically. After completing the work content of the node, whether to approve or not is judged according to the approval content of the process template node. The data of the architecture node is exchanged with the back-end by using no-refresh mechanism. The invention realizes the combination of process management and internal control management in software, realizes the management duplication quickly, andsimplifies the examination and approval process.
Owner:北京中泰合信管理顾问有限公司 +1

Data storage method and device based on LSM Tree structure and storage engine

The invention provides a data storage method and device based on an LSM Tree structure and a storage engine. The method comprises the steps of processing to-be-stored data in a computer memory, and analyzing keys of to-be-stored data and values corresponding to the keys; storing the values of the to-be-stored data and obtaining storage addresses of the values of the to-be-stored data; assigning the obtained storage addresses to the keys of the to-be-stored data, and writing the assigned keys into the LSM Tree structure. In this way, the keys of the to-be-stored data and the corresponding values of the keys are stored separately, so that the values of the data are not directly stored in the LSM Tree structure, instead, the keys and the corresponding storage addresses of the values are stored, thus in subsequent data merging, the processing procedures of a large quantity of values can be effectively reduced, and the magnetic disk IO can also be reduced.
Owner:BEIJING QIHOO TECH CO LTD

Asymmetrical disk array caching dispatching method

The invention discloses an asymmetrical disk array caching dispatching method, belonging to the technical field of computer storage. According to the method, a disk not involved in reconstitution is selected as a sacrificed disk, cache size of the sacrificed disk is reduced, and the cache size of a disk involved in reconstitution is correspondingly increased, so that number of times of input / output (I / O) requests to invalid disks and disks involved in reconstitution are reduced, the reconstitution speed of the invalid disk is improved, I / O response time of users is shortened, the availability and reliability of a storage system are improved, and the performance of the system is improved.
Owner:HUAZHONG UNIV OF SCI & TECH

Smart grid WAMS-based time-series big data storage method

The invention discloses a smart grid WAMS-based time-series big data storage method, and belongs to the technical field of real-time databases. According to the method, a corresponding key-value storage model is built as a data model according to a WAMS scene; a storage unit is created for each measure point per second; the length of the storage unit is set according to the sampling frequency of WAMS data; a data mid-value is compressed by a write-ahead logging technology in a data loading process while the key is kept invariable; and database division is carried out on the data according to the loaded data scale and the loading frequency. The smart grid WAMS-based time-series big data storage method can meet rapid loading and querying requirements of the time-series big data of a smart grid WAMS scene and has very high reliability.
Owner:CHINA REALTIME DATABASE

Spark-based parallel association mining optimization method

The invention discloses a Spark-based parallel association mining optimization method. According to the method, first, a transaction databases is preprocessed, data cleaning is performed according to a business demand, brief effective information is extracted, all transaction item data is read into a memory after being encoded, and the encoded transaction item data is converted into an RDD model; in the process of generating a frequent one-item set, a new data structure is constructed to store transaction serial numbers of the one-item set; in the process of connecting and pruning frequent item sets to generate candidate sets, a generation process of candidate item sets is abandoned, and the item sets with the transaction serial numbers meeting minimum support after connection are screened out; and the process is repeated till a larger item set meeting the requirement is not generated. Through the method, the defect of an Apriori algorithm is overcome, and mining efficiency is improved.
Owner:NANJING UNIV OF POSTS & TELECOMM

Object aggregation method, device and equipment and readable storage medium

ActiveCN111176578AReduce I/OImprove data reading performanceInput/output to record carriersComputer hardwareClustered data
The invention discloses an object aggregation method, which comprises the steps of: acquiring a target object, and calculating a correlation index corresponding to the target object, wherein the target object is an object file of which the file volume is smaller than a preset threshold value; generating aggregation task by utilizing the correlation index, and putting the aggregation tasks into anaggregation task queue; and acquiring target aggregation tasks from the aggregation task queue according to a preset correlation sequence, constructing an aggregation object by utilizing first targetobjects corresponding to the target aggregation tasks, and storing the aggregation object into a storage system. According to the object aggregation method, the target aggregation tasks are acquired from the aggregation task queue according to the preset correlation sequence, and the aggregation object is constructed by utilizing the first target objects corresponding to the target aggregation tasks and stored in the storage system, thus aggregation storage of the objects is completed, cluster IO is reduced during data reading and writing, and cluster data reading performance is enhanced; in addition, the invention further provides an object aggregation device, object aggregation equipment and a computer readable storage medium, which further have the above beneficial effects.
Owner:LANGCHAO ELECTRONIC INFORMATION IND CO LTD

Method for batch query of HBase (Hadoop Database) data for Servlet in TTLB (time to last byte) based on Lua

ActiveCN104572865AGuaranteed multi-thread safetyReduce I/OSpecial data processing applicationsByteThread safety
The invention discloses a method for batch query of HBase (Hadoop Database) data for Servlet in TTLB (time to last byte) (request response time) based on Lua. The method comprises a Wrapper type, a LuaScript type and a ReadHBase type, and in the Wrapper type, using Java API (Application Program Interface) to package Lua API of the LuaScript, and supporting Servlet calling; caching a query request into a Lua table, and finding a query result in the Lua table; in the LuaScript type, packing all threads in the TTLB, sending the query request, calling the Java API of the ReadHBase type to complete batch query of HBase table records, unpacking returning results, and generating the returning result of each thread; in the ReadHBase type, unpacking parameters sent by the LuaScript, and calling a function HTable.get (List<Get>) of the Java API; packing returning results of the get function; packing and unpacking the batch query request and the batch query result, so as to realize the batch query of any HBase table, and query any HBase table record. The method has the advantage that the method can be used for the Servlet to search any HBase table data, and the batch query request of the Servlet can be cached through the Lua table, so as to guarantee the thread security of the Servlet.
Owner:LUZHOU MEDICAL COLLEGE

Distributed consistency protocol submission method and system based on sequential verification

The invention discloses a distributed consistency protocol submission method and system based on sequential verification, and belongs to the field of distributed database development. The method comprises the following specific steps: S1, establishing long connection with a distributed database cluster by using a client according to an agreed communication mode; s2, enabling the database to execute an SQL statement query process; s3, after SQL statement analysis is completed, converting related operations into Request of a corresponding type. According to the method, the idea of high cohesion and low coupling is utilized, user data is decoupled from the distributed consensus module, an independent user data processing module is formed, the independent user data processing module can be parallel to the distributed consensus module, the performance overhead of the distributed consensus module is reduced, the consensus reaching efficiency of distributed consensus is improved, and therefore, the overall processing capacity of the distributed database is improved, the performance of the distributed database is improved, and meanwhile the portability and reusability of the consensus module are improved.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Data cube-oriented operable graphic perspective table construction method

The invention relates to a data cube-oriented operable graphic perspective table construction method, which comprises the following steps of: projecting a cube into a two-dimensional data perspectivetable, and quickly indexing member information of dimensions by adopting a B + tree according to the dimensions and measurement information displayed by horizontal and longitudinal axes of the perspective table configured by a user, and in combination with a table algebra calculation table structure and a table header structure, obtaining a row and column structure of a table data area and a numerical attribute of each data unit, and constructing an MDX query statement according to the calculated attributes to query a corresponding specific numerical value in the data cube. When the user operates the level of the dimension hierarchy, the user only needs to combine the B + tree of the dimension and the table algebra again to quickly calculate the table structure and index the data. The inventionadapts to the characteristics of simplicity and intuition of a data perspective table, has the characteristics of agile perspective table operation response, efficient processing of a large amount of data and the like, and can quickly calculate and reduce system I / O.
Owner:ZHEJIANG UNIV OF TECH

Distributed transaction processing method and device, medium and equipment

The invention relates to a distributed transaction processing method and device, a medium and equipment. And when a distributed transaction processing request is received, distributed transaction processing is realized based on a TCC protocol. Based on a branch transaction command record table and a global transaction command record table stored in a business database, storage of configuration and operation record information related to a corresponding transaction command is realized, and then according to an execution result of an interface in each branch transaction attempt stage, and according to the branch transaction command record table and the global transaction command record table, the determined distributed transactions and the state of each branch transaction, processing of the confirmation stage and the revocation stage of the distributed transactions is achieved, and therefore distributed transaction processing is achieved. Due to the fact that the branch transaction command record table and the global transaction command record table are stored in the same database with the service data, distributed transaction processing can be achieved based on the same-database coordination mode, network IO is effectively reduced, and therefore the system performance is effectively improved.
Owner:HANGZHOU NETEASE CLOUD MUSIC TECH CO LTD

Load release characteristic influence factor analysis method

The invention relates to a load release characteristic influence factor analysis method, which comprises the following steps: collecting first power data and a plurality of influence factors thereof in a power supply area, and forming a first influence factor group by the plurality of influence factors; based on the Python language, cleaning the first power data and the first influence factor group to obtain second power data and a second influence factor group; and determining a main influence factor of the second power data from the second influence factor group based on the rough set theory. The method is based on big data research, and utilizes the rough set theory to research the influence of the relevant index data of each cell on the electrical load change of the cell. On one hand,various data indexes of a researched area are collected, a python language is used for cleaning data, big data summarization is obtained, and the flexibility of big data mining and the data processingefficiency are improved; and on the other hand, main influence factors are selected from all the factors based on the rough set theory, the accuracy is high, and the application prospect is good.
Owner:HUAZHONG UNIV OF SCI & TECH +1

Method and system for storing analytic measurement data format MDF file

The invention provides a method and system for storing an analytical measurement data format MDF file. The method comprises the steps of collecting and uploading an MDF data file; sorting the collected MDF data files; extracting metadata from the sorted MDF data files and storing the extracted metadata; converting the electric signal value of each record in the sorted MDF data file into a physicalvalue; generating a self-parsed column type storage format file and storing the generated column type storage format file in a distributed storage system; and calculating the stored column type storage format file by utilizing a distributed calculation system.
Owner:昆易电子科技(上海)有限公司

Average elevation plane-based image rapid geometric preprocessing method

The invention relates to an average elevation plane-based image rapid geometric preprocessing method. According to the method, two steps of sensor correction and system geometric correction are integrated, and only one system geometric correction product based on an average elevation plane is output. The product has the characteristics of a sensor correction product and a system geometry correction product, and comprises an image with geographic positioning information and a rational polynomial model coefficient. On one hand, the image contains geographic positioning information and can be directly used as a system geometric correction product; and on the other hand, the user can also use the rational polynomial model coefficient to carry out geometric finishing on the image to obtain an image product containing geographic positioning information. According to the image rapid geometric preprocessing system based on the average elevation plane, CPU and I / O resources are saved, the timeliness is improved, and the usability of a product is also improved.
Owner:WUHAN UNIV

A Distributed Buffer Management Method Based on Dynamic Index

The invention discloses a distributed buffer management method based on a dynamic index, applied in a database cluster system, wherein each node of the database cluster system is provided with an independent buffer; when a data block is firstly loaded to the buffer of the node by the certain node, information of the data block is firstly loaded in a dynamic index table of a global buffer manager, the attribute of the buffer is set to be a local resource, and is held by the node; when the other nodes need to access the data block, the nodes firstly search in the global buffer manager, and under the condition that the data block is loaded and held by the certain node is acquired through the dynamic index table, the nodes modify the attribute of the buffer to be a global resource, and apply to the node which holds the data block for transferring the data block for accessing. The invention realizes multi-version assess control of the data block through a dynamic index mechanism, reduces disk I / O (Input / Output), and improves overall performance of the database cluster system.
Owner:北京人大金仓信息技术股份有限公司

A single data disk failure rapid reconstruction method with three-disk failure erasure code

The invention discloses a quick single-disk failure recovery method for triple-erasure-correcting codes. The method includes: dividing each stripe unit of each stripe into p-1 sub-blocks during recovery of single stripes; acquiring a total number R<idea1> of the sub-blocks, used for recovering a failed disk, in each stripe; in an array of p+2 disks, initializing a feasible solution value for failure of the failed disk I into 3<p-1>; initializing a counter j=0; converting j into a recovered sequence which is ternary and which shares p-1 elements by means of the division algorithm; calculating a sum of bits with 0, 1 and 2 values in the recovered sequence of p-1 bits; comparing the three numbers pairwise; judging whether or not absolutes of differences are all smaller than or equal to 1; if yes, starting an effectiveness test function and judging whether or not test blocks, represented with the recovered sequence, are able to recover all data blocks in the failed disk i. The letter p is a prime number, with p+2 being equal to the number of disks in an RAID (redundant array of independent disks), and R<idea1>=R<low>+w%. The quick single-disk failure recovery method for triple-erasure-correcting codes has the advantages that disk recovery process can be quickened and system reliability is improved.
Owner:HUAZHONG UNIV OF SCI & TECH

Data storage method, device and storage engine based on lsm Tree structure

The present invention provides a data storage method, device and storage engine based on the LSM Tree structure, wherein the method includes processing the data to be stored in the computer memory, parsing out the key of the data to be stored and the value corresponding to the key . Store the value of the data to be stored, and obtain the storage address of the value of the data to be stored. Assign the obtained storage address to the key of the data to be stored, and write the assigned key into the LSM Tree structure. Therefore, by separately storing the key of the data to be stored and its corresponding value value, the LSM Tree structure does not directly store the value value of the data, but stores the key and the corresponding value value storage address for subsequent data processing. Merging can not only effectively reduce a large number of value processing processes, but also reduce disk IO.
Owner:BEIJING QIHOO TECH CO LTD

Path dynamic detection method and device and equipment

The invention discloses a path dynamic detection method and device, equipment and a computer readable storage medium. The method comprises the steps that each designated transmission path is detectedto obtain a corresponding detection result; whether the detection results indicate that all the transmission paths are in a normal working state is judged; if yes, the sum of a current time interval and a preset time interval is determined as a current time interval; if not, a preset minimum time interval is determined as a current time interval, wherein the initial value of the current time interval is the minimum time interval; and the step of detecting each designated transmission path is returned to and executed after the current time interval from the moment when the detection results areobtained. Through the path dynamic detection method and device, the equipment and the computer readable storage medium, resource waste can be avoided to some extent, and abnormal path repair conditions can be known in time.
Owner:ZHENGZHOU YUNHAI INFORMATION TECH CO LTD

A data processing method and related equipment

The embodiment of the present application provides a data processing method and related equipment, which can comprehensively utilize the advantages of solid-state equipment, tile recording disks, and log-structured merge trees to optimize the stored data. The method includes: acquiring target data; determining a target ordered key-value pair table corresponding to the target data; determining a second ordered key-value pair table according to the target ordered key-value pair table; The value pair table determines the first ordered key-value pair table; merges the second ordered key-value pair table into the first ordered key-value pair table; when the merged first ordered key-value pair table When the table meets the preset condition, write the merged first ordered key-value pair table and the third ordered key-value pair table as an array set into the target dynamic area whose size dynamically changes in the tile record disk .
Owner:WENHUA UNIV

Architecture method and query method of time sequence database and time sequence database

The invention discloses a time sequence database architecture method, a time sequence database query method and a time sequence database, and belongs to the technical field of databases, and the time sequence database architecture method comprises the following steps: S1, according to the characteristic that time sequence data has a sequence, reserving sequence writing; and S2, statistical information layering is carried out according to time granularity, each layer is directly accessed, a root node or an upper layer node does not need to be returned, and the statistical information is managed according to a unified data block. According to the method, I / O can be effectively reduced, and database query performance is remarkably improved.
Owner:CHENGDU SOBEY DIGITAL TECH CO LTD

A method and system for storing and analyzing measurement data format mdf files

The invention provides a method and system for storing and analyzing measurement data format MDF files. The method includes: collecting and uploading MDF data files; sorting the collected MDF data files; extracting metadata from the sorted MDF data files and storing the extracted metadata; Convert the electrical signal value of each record in the file into a physical value; generate a self-parsed columnar storage format file and store the generated columnar storage format file in a distributed storage system; and use a distributed computing system to analyze all Stored columnar storage format files for calculation.
Owner:昆易电子科技(上海)有限公司

Fog storage data redundancy removal method based on repeated data deletion

The invention relates to a fog storage data redundancy removal method based on repeated data deletion, which comprises the following steps: aiming at data with higher access frequency in a fog node, constructing an index table in a memory, and taking a red-black tree corresponding to each index value as a structure for storing data fingerprints, so that I / O (Input / Output) between a disk and the memory is reduced, and the query speed is increased; judging whether the data blocks with the same data fingerprint are repeated or not by using a cyclic redundancy code (CRC) technology, and storing conflict data blocks in fingerprint nodes by using a linked list structure, so that the hash conflict problem generated during calculation of the data fingerprint is solved; persistently storing the fingerprint table in the memory, and dividing the fingerprint table into a mapping file of the memory at a certain moment and a log file for recording update to prevent sudden crash of the system; the data redundancy elimination scheme conforming to the actual situation of the fog node is provided, disk I / O is reduced in a network with high requirements for communication efficiency, and the fingerprint query speed is increased.
Owner:SHANGHAI UNIVERSITY OF ELECTRIC POWER
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products