72 results about "NoSQL" patented technology

In one exemplary embodiment, a method of a distributed database system includes the step receiving a query in a query language from a client with a distributed database system. An index that matches the query is located. The index is pre-generated from a database table in the distributed database system. A map function of a MapReduce programming model is implemented using the index. A reduce function of the MapReduce programming model is implemented using the output of the map function. Optionally, a finalize function can be implemented using the output of the reduce function. The distributed database system can be a scalable NoSQL database. The reduce function can be optional when the value of the output of the map function is guaranteed to be unique.

Embodiments of the invention disclose a database and file system access method and apparatus. The method comprises the steps of determining a source database or source file system type corresponding to a table name and a keyword in an input standard SQL (Structured Query Language) statement according to metadata; when the source database or source file system type of the table name and the keyword in the standard SQL statement is an RDBMS (Relational Database Management System) or an MPP (Massive Parallel Processing) system, correspondingly transmitting the standard SQL statement in which the table name and the keyword are located to the RDBMS or the MPP system; when the source database or source file system type of the table name and the keyword in the standard SQL statement is a distributed computing Hadoop system or an NoSQL (Not Only Structured Query Language) database, analyzing the table name and the keyword, correspondingly generating an access statement capable of running in the Hadoop system or the NoSQL database, and correspondingly transmitting the access statement capable of running in the Hadoop system or the NoSQL database to the Hadoop system or the NoSQL database; and receiving an access processing result from the RDBMS, the MPP system, the Hadoop system or the NoSQL database.

The invention belongs to the technical field of zoology and remote sensing information science and relates to a distributed remote sensing data organization query method based on the NoSQL database. The method includes the steps: preprocessing an original remote sensing data file, and outputting an initial remote sensing image file according to an appointed format, extracting data of each pixel and data element information in the initial remote sensing image file; reconstructing the extracted data of each pixel and data element information as unstructured text data; storing the unstructured text data in the distributed NoSQL database, and setting up query indexes; and returning corresponding remote sensing data to users according to query conditions inputted by the users, and regenerating image files. The distributed remote sensing data organization query method based on the NoSQL database can be used for effectively storing and managing massive remote sensing data and can be applied to correlated fields of remote sensing ecological monitoring, species distribution forecast, remote sensing data inversion and the like.

The present invention discloses a distributed OLAP analysis method and system based on pre-computation. The method mainly comprises: constructing a data model based on the distributed data warehouse, and defining data cubes according to the data model; starting pre-computation tasks for the given data cubes so as to pre-build the cubes in a parallel computation manner, and storing results in a distributed key storage system; through a series of steps, converting multi-dimensional analysis operations into key-value query operations on the data cubes, directing obtaining analysis results from the built cubes, and displaying the results in the form of rich and diversified charts; and using the NoSQL to carry out cache optimization on the OLAP query operation. According to the method and system disclosed by the present invention, the powerful processing performance of the Hadoop platform is fully exerted, and data cubes are pre-built, the problem that the query is slow due to that the traditional method needs a large amount of computation from the original data in each query is overcome, so that OLAP analysis efficiency and system performance are improved.

Logically arranged hierarchy or tiered storage may comprise a layer of storage being a faster access storage (e.g. solid state drive (SSD)) and another (e.g., next) layer being a traditional disk (e.g. HDD). In one embodiment, compaction occurs within the higher layer, e.g., until there is no more room and then during the compaction sequence the data may be moved down to the lower layer. In another embodiment, compaction and migration to a lower layer may occur within the higher layer, e.g., based on one or more policies, even if the higher layer is not full. In one embodiment, the data between layers are maintained as disjoint. In one embodiment, the more recent versions are always in the higher layer and the older versions are always in the lower layer.

Embodiments of the present invention provide a system for generative adversarial network training and feature extraction for biometric authentication. The system collects electronic biometric data of a user from one or more data sources, and stores the collected electronic biometric data as a biometric user profile for the user in a personal NoSQL database library associated with the user. A generative adversarial neural network system then determines improved biometric feature selection and improved model refinements for existing biometric authentication models based on the biometric profile for the user in the personal library associated with the user. The system can then determine user exposure levels for different authentication channels, including certain biometric authentication channels. A custom adversarial strategy for general adversarial network attacks is then established based on the user exposure levels to generate a biometric authentication process that is more accurate and secure.

The invention discloses an automatic nosql cluster configuration system and method based on cloud computing. The automatic configuration system comprises a decision analysis module, a cloud communication module, an nosql management module, a cluster monitoring module and a cloud platform, wherein the cluster monitoring module acquires motion state parameters of an nosql cluster in real time, sends the parameters to the decision analysis module for decision analysis and sends a command for adjusting the nosql cluster to the nosql management module according to the analysis result, and the nosql management module configures the adjusted nosql cluster. With the adoption of the system and the method, high concurrency, extendibility and dynamic scalability required by storage and management of cloud computing big data in the prior art are realized, load condition changes can be responded dynamically, burdens of management staff can be reduced, and the utilization rate of system resources can be increased.

The invention provides a method for supporting uniform SQL (Structured Query Language) query of an embedded document. The method is characterized in that an upper-layer interface continues to use an SQL query language which is familiar to a user, and query of a non-structured embedded document is realized functionally. By adopting the method, unified SQL operation can be performed on relational data and an embedded document in NoSQL database MongDB through distributed database middleware specific to the characteristics of big-data multi-source heterogeneity, data in a bottom-layer database are acquired, and operation is correct; meanwhile, no matter data in the bottom-layer database exist in which form, the data can be operated in a relational way at a front end, and a result can be presented to a user in a relational form, so that the usability of the middleware is enhanced.

The invention discloses a big data indexing method and system and relates to the technical field of database index. The method comprises the following steps of: sending data generated by all terminal devices connected with a data center server unit into an NoSQL (Not Only Structured Query Language) database unit; establishing a pLSM (physical Log-Structured Merge) indexing engine unit; and carrying out indexing operation by a user. The system comprises the data center server unit, the NoSQL database unit and the pLSM indexing engine unit, wherein the data center server unit is used for receiving the data generated by all the connected terminal devices and sending the data into the NoSQL database unit; the NoSQL database unit is used for storing the data sent by the data center server unit; and the pLSM indexing engine unit is used for taking a COLA (Cache-Oblivious Lookahead Array) as an external storage assembly, taking a Skip List as an internal storage assembly and additionally arranging Delete Filter modules in an internal storage to support delete operation on the external storage assembly.

Automatically moving NoSQL data store to a relational database system. Based on discovered data structure schema of a NoSQL file and query plans, attribute usage and association relationships may be determined. Trunk tables may be defined based on trunk part of the data structure schema determined based on the attribute usage. Trunk tables may be validated and relational database tables are generated that correspond to the trunk tables. NoSQL trunk template is generated based on the trunk tables. The relational database tables are loaded with data filtered from the NoSQL file according to the NoSQL trunk template.

The embodiment of the invention provides a distributed session management method and device based on a NoSQL (Not Only SQL). The method comprises the steps of receiving session requests sent by a client, and obtaining operation types of the session requests and session IDs of the session requests from the session requests; routing the session requests to a master node of a cluster server if the session requests are write operation, thereby enabling the master node to store the session IDs and session data carried by the session requests, and synchronize the session IDs and the session data toeach slave node of the cluster server; and if the session requests are read operation, after it is judged that the session data corresponding to the session IDs is synchronized completely, routing thesession requests to the slave nodes according to a preset load balancing policy, thereby enabling the client to read the session data corresponding to the session IDs from the slave nodes according to the session IDs. According to the embodiment of the invention, consistency when data synchronization is carried out can be ensured, service can be provided in a high concurrency scene at high performance, and business performance is high.

The invention discloses a method for establishing a full-space-oriented data model based on Hash geocoding, which comprises the following steps of: S1, scheduling P2P protocol files based on Hash plaintext geocoding indexes, and respectively generating Hash-based geocoding for nodes and files for indexing; s2, establishing a data node network based on a Hash geocoding index; s3, building and evaluating a global virtual environment data sharing prototype and an experimental platform; s4, making a multi-granularity coding rule; s5, spatial object big data storage based on Hash geocoding is carried out; s6, designing NoSQL database. The method has the advantages that support is provided for establishment of a multi-granularity space-time database or a full-space information system; and unification of relative positions between the data storage environment and objects in the real world is realized.

Automatically moving NoSQL data store to a relational database system. Based on discovered data structure schema of a NoSQL file and query plans, attribute usage and association relationships may be determined. Trunk tables may be defined based on trunk part of the data structure schema determined based on the attribute usage. Trunk tables are validated and relational database tables are generated that correspond to the trunk tables. NoSQL trunk template is generated based on the trunk tables. The relational database tables are loaded with data filtered from the NoSQL file according to the NoSQL trunk template.

The invention discloses a not only structured query language (NoSQL)-based method for realizing message-oriented middleware. The method comprises the following steps of: forwarding a cache and restoring shutdown. In a NoSQL, the access control of data is performed in an internal memory, and the read-write speed is high; messages are stored in the NoSQL, so that the high efficiency of message caching can be ensured; when searched, the messages can be positioned quickly by utilizing the advantage of the read-write speed of the internal memory; and by matching with a journal file, the lost of the messages can be avoided after the shutdown of a message-oriented middleware server, and the messages before the shutdown can be restored after restarting.

The invention discloses a NoSQL (not only SQL) based medical information system database implementation method. The implementation method includes the steps that A, an application program is driven to be connected to a JDBC (java database connectivity) interface through JDBC; B, the JDBC interface is called and connected to a Server module through Pyro; C, the Server module performs AST (glutamic oxalacetic transaminase) analysis upon SQL (structured query language) to obtain an AST structure tree; D, the AST structure tree is analyzed, if an association list is obtained, an Overlay operation is performed, and if an independent list is obtained, a Cache operation is performed; E, the AST structure tree is converted to form an executing script; F, the executing script is mapped to an NoSQL database through an API (application program interface). By the application of the implementation method, shortcomings in the prior art can be overcome, and rapid access with high performance, high expandability and a high-redundancy database can be realized.

The invention discloses an NoSQL-based massive small file storage structure for aviation logistics and a storage method of the NoSQL-based massive small file storage structure. The NoSQL-based massive small file storage structure for aviation logistics comprises a data cache region, a data active region and a data permanent region. The file storage method of the NoSQL-based massive small file storage structure for aviation logistics comprises three processes including file storage, file access and permanent file processing. According to the NoSQL-based massive small file storage structure for aviation logistics and the storage method of the NoSQL-based massive small file storage structure, a multi-stage storage mechanism is designed, a file pre-storing mechanism is adopted, accordingly, the storage efficiency of small files is effectively improved, the memory consumption of a cluster NameNode is remarkably reduced, and the utilization rate of a disk is effectively improved.

A computer-implemented method executed on one or more processors is provided for processing a query for a NoSQL (non-structured query language) database. The computer-implemented method includes periodically monitoring active transactions that started and are committing dirty versions before a specific time, sending a query to fetch a state of the active transactions, and receiving a query result. The computer-implemented method further includes, if the query result includes active transactions, performing a normal query that scans dirty and committed versions. The computer-implemented method further includes, if the query result includes no active transactions, due to a time window indicating a specified staleness, sending a simplified query that scans only committed versions.

In one exemplary embodiment, a method of a distributed database system includes the step receiving a query in a query language from a client with a distributed database system. An index that matches the query is located. The index is pre-generated from a database table in the distributed database system. A map function of a MapReduce programming model is implemented using the index. A reduce function of the MapReduce programming model is implemented using the output of the map function. Optionally, a finalize function can be implemented using the output of the reduce function. The distributed database system can be a scalable NoSQL database. The reduce function can be optional when the value of the output of the map function is guaranteed to be unique.

The invention discloses a cloud distributed real-time database system which comprises a real-time database and a real-time service bus. The real-time database comprises a storage layer with a distributed hierarchical structure, a positioning layer and an interface layer; the storage layer stores a real-time model based on a distributed NoSQL database, and the real-time model corresponds to a relational model of a relational database of the power grid system; the positioning layer is used for storing fragmented memory snapshots, and the fragmented memory snapshots comprise monitoring model snapshots, numerical section snapshots, transaction snapshots and the like; the interface layer provides a transaction type component and is used for realizing establishment, access and processing of a fragmented memory snapshot; and the real-time service bus comprises a model loading service, a fat client service and the like. Based on the intelligent power grid regulation and control system CIM model grouping and slicing technology, fat client service real-time calculation processing and thin client quasi-real-time access are achieved, model checking and synchronization functions are expanded, and concurrent control and extension expansibility of the system are improved.

The invention discloses a method for indexing spatial data in a cloud computing environment. The method comprises the following steps of: 1) establishing a data storing table on the basis of the spatial data; 2) building an R+-tree index on the basis of spatial object distribution; 3) building a grid index on the basis of spatial object coverage range; 4) carrying out dimensionality reduction andcoding on the R+-tree index and the grid index by utilizing a Z curve; 5) establishing a code mapping table of R+- trees and grids; 6) establishing an index table on the basis of R+- tree codes and spatial objects ID; and 7) carrying out spatial range query. The method has the beneficial effects of making full use of the data organization characteristics of a NoSQL (Not Only Structured Query Language) database, optimizing the traditional spatial index and realizing the efficient access of the spatial data in the cloud environment.

The invention relates to a fusion method and system of a relational geological database and an NoSQL, the method comprising a structure obtaining step, a data storing step and a data retrieving step.Firstly, the NoSQL storage structure of the designed unstructured data is obtained, and the relational connection table is designed according to the storage structure. Secondly, the unstructured datais stored in NoSQL according to the storage structure of the designed unstructured data, the index information of the internal elements of the unstructured data is obtained, and the index informationof the internal elements of the unstructured data is stored in a relational join table. SQL is used to retrieve the index information of the corresponding unstructured data from the relational join table, and the element information of the unstructured data or the corresponding unstructured data is obtained according to the index information. According to the invention, the flexible and free management function of NoSQL to the unstructured data is utilized, and the powerful retrieval function of the relational database is also utilized, so that the geological data which contains not only a large amount of structured data but also a large amount of unstructured data is more convenient and efficient in management.

The invention relates to an invoice authenticity inquiry method, which comprises the following steps of receiving an invoice authenticity inquiry request; converting the inquiry request to be NoSQL (Not Only Structured Query Language) database inquiry language, and inquiring in a preset NoSQL database; returning an inquiry result and analyzing the inquiry result to be information corresponding to the inquiry request; judging whether the analyzed information is empty or not; if yes, confirming an invoice as a fake invoice. By adopting the method, the authenticity of the invoice can be quickly and efficiently judged when large-scale data is inquired parallely. In addition, the invention also provides an invoice authenticity inquiry system.

The invention provides an index creating method and device based on a mobile terminal NoSQL (Not Only Structured Query Language) database, a data query method and device, computer equipment and a computer readable storage medium, and relates to the technical field of data query. The creating method comprises the following steps that: according to the attribute of pre-stored data in the NoSQL database, setting index information; according to the type of the pre-stored data of a corresponding attribute, determining to store the index information corresponding to the corresponding attribute intoa numeric type index storage structure or a character type index storage structure; and storing the address of the NoSQL database, the address of the numeric type index storage structure and the address of the character type index storage structure in an NoSQL connection pool. By use of the scheme, through the attribute of data to be pre-stored in the NoSQL database, index creating is carried out,an index is established out of the NoSQL database to carry out relationship query, so that complex design does not need to be carried out on an application side so as to be irrelevant to the specificimplementation of the NoSQL, and the method exhibits transportability.

The invention further provides a distributed picture storage service method. A picture storage service system (300) stores uploaded pictures. In the storage process, firstly, a file path generation module (303) transforms pictures into numbers to generate multi-level subdirectories for storing the pictures, and an HTTP (hyper text transfer protocol) load balancing module (302) detects and calculates the sizes, the use frequency and the number of picture files, the capacity and the state of NoSQL (not only structured query language) database modules (306 and 308) and the like, and distributes storage and reading paths of the picture files. When a client uploads or requests picture services, the HTTP load balancing module (302) makes a judgment by detection and calculation, and determines supply of the picture services for the client by the special NoSQL database modules (306 and 308) or a client picture caching module (311) or storage hard disk (309). Compared with the prior art, the distributed picture storage service method has the advantages that picture storage is separated from Web services, and the storage paths are reasonably configured by load balancing, so that picture transmission speed and concurrency are effectively improved.

The embodiment of the invention discloses a data management method and a data management system for a centralized clearing system. The data management method according to the embodiment of the invention comprises the following steps of acquiring message data to be written; parsing the message data to be written to obtain message relational data and message content data; storing the message relational data in an RDBMS (relational database management system) database, storing the message content data in an NoSQL (not only structured query language) database, and establishing a message content data query index in the RDBMS database; acquiring a message reading request; acquiring the message relational data and the message content data query index from the RDBMS database according to the message reading request, and querying the message content data in the NoSQL database; reading message information after the message relational data and the message content data are combined. According to the embodiment of the invention, the IO (input / output) overhead of the RDBMS database is reduced, the query cache hit ratio of the RDBMS database is increased, and the master-slave synchronization efficiency of the RDBMS database is improved; meanwhile, the size of data stored in the RDBMS database is reduced, so that the backup and restoration speed of the data in the RDBMS database is increased.

The invention relates to a database, and discloses a method and a system for inquiring a database with a user defined function (UDF). The method includes steps as follows: detecting whether an instruction for calling a UDF transmission parameter value is received; starting a first interface of the UDF if the instruction for calling the UDF transmission parameter value is received; receiving the parameter value through the first interface; transmitting the received parameter value to an external data source; and returning and detecting whether an instruction for calling the UDF to transmit the parameter value is received. With the adoption of the method, data interactivity between a RDBMS (relational database management system) and a NoSQL (not only SQL) data source can be effectively improved.

The present invention aims to provide a NoSQL-based single field section index query method for distributed Internet of things data. The method is characterized by comprising the following steps: step S0: creating a NoSQL database and importing Internet of things data into the database; step S1: creating an index table corresponding to the database, wherein the index table stores indexes and a field attribute in the database; step S2: directly searching, according to a field section, the index table to obtain the field attribute of the database; and step S3: when the database is added, deleted or modified, updating the index table so as to achieve data consistency. According to the NoSQL-based single field section index method for the distributed Internet of things data provided by the present invention, a search for the field attribute of a NoSQL database can be implemented by directly searching the index table generated by the database, thereby reducing the wasted time of searching the NoSQL database, and improving the data search.