Data node-based parallel massive data processing method
A mass data processing and data node technology, applied in the field of data processing, can solve the problems of slow mass data processing and low development efficiency, and achieve the effects of reducing development and maintenance costs, saving resources, and avoiding data loss
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0030] The data node-based parallel massive data processing method of the present invention, such as figure 2 As shown, it includes the following steps:
[0031] Compress and encapsulate data information into a data object, which integrates data compression, storage and access, and is also the basic unit of data processing, that is, a data object is used to compress, store and access a monitoring of a certain measurement point Indicators, through the reading interface of the data object, all the historical data of the monitoring indicator can be obtained, and the value of the monitoring indicator at a certain time can be stored through the writing interface of the data object, and the data object stores the data information encapsulated An access interface is provided, and the data objects can classify and store massive data, and can perform parallel access to data objects in multiple data nodes.
[0032] Divide massive data into a series of data nodes according to actual bu...
Embodiment 2
[0037] On the basis of the data node-based parallel mass data processing method described in Embodiment 1, the bus is an ORB bus of CORBA, and ORB (Object Request Broker) is an object request agent. Based on the ORB mechanism, distributed, interoperable object structures and interoperable applications can be fully utilized. This advantage is obvious. The details of data communication are completely encapsulated, and users can use different development languages and environments to build shared resource applications without knowing the details of the implementation of the interaction, which is portable.
[0038] The access interface is a GID interface, and the GID interface is an interface of the IEC61970 GID (Generic Interface Definition) standard, which provides a set of general interfaces for data access and exchange between systems, and solves the application of the system and the sharing of data resources Problems, especially for the data interaction between those closel...
Embodiment 3
[0040] On the basis of the above embodiments, each data object corresponds to a database file, and storage space is pre-allocated for each database file to prevent uncertain situations such as insufficient disk space during operation, and to prevent disk fragmentation. , to ensure the stability and efficiency of data access performance.
[0041] The distributed storage of database files is currently difficult for database systems to associate multiple databases, and mass data processing technology can configure database files for complex data requirements to realize the construction of distributed storage networks, which can conveniently store important data Backup to avoid data loss or uncheckability caused by server failure; the huge storage structure is only a simple CORBA interface to the outside world, making it easy for users to use; in addition, distributed deployment can enhance the availability of database services and prevent partial network paralysis The resulting d...
PUM

Abstract
Description
Claims
Application Information

- R&D
- Intellectual Property
- Life Sciences
- Materials
- Tech Scout
- Unparalleled Data Quality
- Higher Quality Content
- 60% Fewer Hallucinations
Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.
© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com