Patents
Literature
Hiro is an intelligent assistant for R&D personnel, combined with Patent DNA, to facilitate innovative research.
Hiro

290 results about "Data efficiency" patented technology

Data Efficiency refers to efficiency of the many processes that can be applied to data such as storage, access, filtering, sharing, etc., and whether or not the processes lead to the desired outcome within resource constraints.

OLAP (on-line analytical processing) data storage and query method based on Hadoop

The invention discloses an OLAP (on-line analytical processing) data storage and query method based on Hadoop. For the data storage, firstly, a new column file storage format HCFile (Hadoop column file) is defined, and then, a datasheet storage method based on the HCFile is given. In the scheme, when a column of data is read, only a plurality of HCFile needs to be read, the visit to other columns of data is not needed, and the I/O (input/output) efficiency is greatly improved than that of the storage according to lines; and meanwhile, when one column of data is added, only new files need to be added, and the extension is very easy. For the aggregation computation, firstly, the data index based on the inverted structure is created, then, MapReduce is utilized for realizing the basic aggregation computation of the OLAP, the basic aggregation computation comprises summation, maximum/minimum value computation, counting and the like, other aggregation computation can be realized by the basic aggregation computation, and the aggregation computation performance is obviously improved through the efficient data index. Compared with the prior art, the OLAP data storage and query method has the advantages that the data storage and query efficiency is effectively improved, in addition, hardware resources are saved, the time and the hardware cost are reduced, and meanwhile, the application is more convenient and flexible.
Owner:SOUTHEAST UNIV +1

Fast processing method for large area 3D seismic data volume

The invention discloses a method for rapidly processing a large-area three-dimensional seismic data body, relating to the technical field of seismic exploration data processing and interpretation. The method comprises seismic data pre-stack stage processing, seismic data stack stage processing and seismic data pre-stack time migration stage processing. The seismic data pre-stack stage processing comprises the following steps of: subdividing pre-stack data into a plurality of processing units; and realizing surface consistent amplitude and convolution processing in the event of processing the plurality of subdivided processing units. The seismic data stack stage processing comprises the following steps of: extracting a CMP (Common Middle Point) trace gather; stacking subdivision dynamic correction; and processing residual static correction. The seismic data pre-stack time migration stage processing comprises the following steps of: according to offset requirements and available resources, selecting an offset distance; blocking data; calculating nodes; and, defining and operating the nodes and I/O (Input/Output) resources. According to the method, the node resources can be effectively utilized; the data I/O efficiency is improved; the node idleness ratio is reduced maximally; and the efficiency for processing large-area three-dimensional seismic data can be multiplied.
Owner:CHINA NAT PETROLEUM CORP CHUANQING DRILLING ENG CO LTD

A data processing method and device

ActiveCN109597567AAvoid the problem of low efficiency in storing data to be storedInput/output to record carriersTransmissionUser identifierData efficiency
The embodiment of the invention relates to the field of data processing, in particular to a data processing method and device, and is used for solving the problems that in the prior art, physical nodes are added in a cluster during capacity expansion, data migration in the cluster is caused, and then the data storage and reading efficiency of a user is improved. In the embodiment of the invention,the method is suitable for a distributed storage system, the distributed storage system comprises a plurality of clusters, and each cluster belongs to at least one virtual group; The method comprisesthe steps of obtaining a user identifier to which to-be-stored data belongs; Determining a virtual group for storing the to-be-stored data according to the user identifier of the to-be-stored data; Determining a target storage cluster for storing the to-be-stored data according to a storage rule of the virtual group of the to-be-stored data, Wherein the target storage cluster is one of the plurality of clusters; And storing the to-be-stored data into the target storage cluster. According to the embodiment of the invention, the capacity expansion or reduction is realized by adding or deletingthe cluster in the virtual group, so that the data migration caused by adding or deleting the nodes in the cluster is avoided.
Owner:CHINANETCENT TECH

Traffic condition prediction method, electronic equipment and storage medium

The invention relates to a traffic condition prediction method, electronic equipment and a storage medium, and belongs to the field of artificial intelligence. The traffic condition prediction method comprises the steps of obtaining time sequence traffic data collected by an ETC portal system, carrying out the data preprocessing of the time sequence traffic data, and obtaining time sequence traffic features; writing the time sequence traffic characteristics into a big data platform through Spark Streaming; using spark Jar on a big data platform for reading time sequence traffic characteristics, and training a flow model and a speed model through an expansion causal convolutional neural network algorithm; and predicting the traffic flow and the vehicle speed based on the trained flow model and speed model. According to the embodiment of the invention, the collected vehicle data is comprehensive, and the accuracy of traffic condition prediction can be improved; a distributed data processing technology is applied, so that the real-time performance of the data can be improved, a large amount of data can be quickly processed, and the efficiency is high; and an expansion causal convolutional neural network algorithm is adopted, so that resources can be saved, and the training speed is increased.
Owner:TONGDUN HLDG CO LTD +1
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products