Method and device for caching massive data
A high-speed, massive data technology, applied in the field of data processing, can solve the problems of data redundancy, unsuitable for statistical analysis platform data processing, etc., to achieve the effect of improving efficiency and breaking through capacity limitations
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0024] figure 1 The implementation flow of the massive data caching method provided by the first embodiment of the present invention is shown, and the process is described in detail as follows:
[0025] In step S101, the original data acquired in the database is converted into data in a standardized matrix format, and the converted data is cached in a mat file format.
[0026] In this embodiment, the data in the standardized matrix format is matlab standard matrix data. The mat file is a standard format for matlab data storage. mat files are standard binary files that can be saved and loaded in ASCII code.
[0027] Among them, converting the raw data obtained in the database into a standardized matrix format specifically includes:
[0028] Obtain data in the cell format from the database; specifically, obtain the original data from the database through multiple data adapters, the original data is obtained from the database to the matlab platform in a cell matrix format, eac...
Embodiment 2
[0055] figure 2 An example of the architecture of the massive data cache device provided by the second embodiment of the present invention is shown, as shown in figure 2 As shown, the architecture of the massive data cache device includes a database layer, an adapter layer, a data interface routing layer, a data extraction and cache scheduling layer, and a graphic user interaction layer from bottom to top. The specific applications of each layer are as follows:
[0056] 1) Database layer
[0057] The database layer mainly stores original data through the database, and extracts the original data through a series of stored procedures.
[0058] 2) Adapter layer
[0059] The adapter layer mainly extracts original data from the database, and converts the extracted original data into matlab standard matrix data. The adapter layer includes a plurality of data adapters, each of which handles a type of data.
[0060] The specific application of the adapter layer is as follows: fi...
Embodiment 3
[0073] image 3 The composition structure of the mass data cache device provided by the third embodiment of the present invention is shown, and for the convenience of description, only the parts related to the embodiment of the present invention are shown.
[0074] The mass data cache device can be a software unit, a hardware unit or a combination of software and hardware running in the terminal device, or it can be integrated into the terminal device as an independent pendant or run in the application system of the terminal device middle.
[0075] The massive data cache device includes a first cache unit 31 , a judgment unit 32 , a data acquisition unit 33 and a data filling unit 34 . Among them, the specific functions of each unit are as follows:
[0076] The first cache unit 31 is used to convert the raw data obtained in the database into data in a standardized matrix format, and cache the converted data in a mat file format;
[0077] A judging unit 32, configured to jud...
PUM
Abstract
Description
Claims
Application Information
- R&D Engineer
- R&D Manager
- IP Professional
- Industry Leading Data Capabilities
- Powerful AI technology
- Patent DNA Extraction
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com