142results about How to "Improve query performance" patented technology

The invention provides a natural language SQL (Structured Query Language) conversion method based on a data platform and a large language model, which relates to the technical field of data processing and comprises the following steps of: performing vectorization storage on metadata, blood relationship, indexes and historical query information, performing intention judgment and vector matching on a user query request, and determining a query condition based on a matching result. An association path is constructed according to data consanguinity, and an optimal SQL statement is generated by utilizing a historical query optimization table connection sequence and combining index matching. According to the method, intelligent conversion from the natural language to the efficient SQL can be realized, the query efficiency is improved, and the use threshold of a user is reduced.

The invention relates to the technical field of medical care, in particular to an intelligent query method for a nursing knowledge base, which comprises the following steps of: performing multi-form standardized processing on a nursing problem input by a user, deeply analyzing a query request by utilizing a semantic understanding model, and obtaining an intelligent query result based on an inference algorithm; the method comprises the following steps: automatically selecting related literatures, clinical data and nursing experience data from a nursing knowledge base, generating a preliminary nursing scheme candidate set in combination with a nursing professional knowledge graph, and performing personalized screening and optimization on the scheme by using an intelligent recommendation model in combination with the current illness state, treatment progress and historical nursing data of a patient; the intelligent recommendation model is self-optimized through user feedback, and the nursing scheme recommendation precision is continuously improved. The nursing knowledge base automatically fuses newest medical research results and clinical nursing practice data. The problems that in the prior art, nursing scheme recommendation accuracy is low, updating is not timely, and the individuation degree is insufficient are effectively solved.

The invention discloses a database-based data indexing and query optimization system and method, belongs to the technical field of databases, and aims to solve the technical problem of how to improve the data indexing efficiency and query performance of a database system. According to the technical scheme, the system comprises a query optimization module, an index construction module and a data storage module; the data storage module is used for storing data records in a database; data records are stored in a column mode, the data compression rate and query performance storage are improved, and disk I / O operation is reduced; the index construction module is used for constructing an index according to the data record in the data storage module, the index adopts a B + tree design, and the number of I / O operations during data transmission between a disk and a memory is reduced; and the query optimization module is used for receiving a query request and optimizing a query path according to the index constructed by the index construction module.

The invention discloses a self-adaptive construction method of a database index. The method comprises the following steps: S1, extracting a query information set based on a database log; s2, obtaining a high-frequency query statement based on TF through the information set; s3, obtaining importance values of all fields in the high-frequency query statement; s4, constructing a database index according to the importance value of the field; and S5, periodically obtaining the importance value of the database log constructed in the step S4, and dynamically adjusting the database index so as to improve the query efficiency and adaptability of the database system.

The invention relates to the technical field of database establishment, in particular to an establishment method and system based on an aircraft aerodynamic model database. The method comprises the following steps: obtaining an aircraft pneumatic data storage demand and an aircraft pneumatic data query demand, and carrying out multi-level storage architecture design to obtain a basic data layer, a processing data layer and a high-level model data layer; the method comprises the following steps: collecting aircraft original aerodynamic data sets of wind tunnel experiments, flight tests and CFD numerical simulation, and transmitting and storing the aircraft original aerodynamic data sets to a basic data layer; carrying out aerodynamic load, flow field parameter and aerodynamic pressure distribution analysis on the original aerodynamic data set of the aircraft in the basic data layer, and transmitting and storing the aerodynamic load, flow field parameter and aerodynamic pressure distribution to the processing data layer; and carrying out aircraft aerodynamic model construction on the data in the processing data layer, transmitting and storing the data to a high-level model data layer, and carrying out R-tree multi-dimensional index connection to obtain an aircraft aerodynamic model multi-level storage database. According to the invention, an efficient and intelligent aerodynamic model database can be established.

The invention provides an SQL (Structured Query Language) query method and device, electronic equipment and a storage medium, and the method comprises the following steps: querying a current to-be-processed SQL statement in a preset cache region; if the cache data matched with the SQL statement is not queried in the cache region, generating an execution plan for the SQL statement, and storing outline data of an optimizer in the execution plan generation process as the cache data of the SQL statement in the cache region; and if the cache data matched with the SQL statement is queried in the cache region, generating an execution plan for the SQL statement based on the cache data matched with the SQL statement.

The invention relates to the technical field of data storage management, in particular to a data management method and system for an automobile transaction platform. The method comprises the following steps: obtaining a to-be-stored data sequence corresponding to an automobile transaction platform, and carrying out index pair classification extraction and storage space limit quantification calculation to obtain a storage release memory space limit degree corresponding to each type of index field; performing sequence storage division on the corresponding to-be-stored data sequence to obtain a storage data sequence segment corresponding to each type of index field; obtaining a query access rate and a storage interval approach access degree of the corresponding sequence segment, and carrying out storage activity evaluation analysis and index storage priority measurement analysis to obtain an index storage priority measurement of each type of index fields for each sequence segment; and carrying out priority storage management on the corresponding storage data sequence segments. According to the invention, mass data in the automobile transaction platform can be managed efficiently and accurately, so that the storage process of the data is optimized.

The invention relates to an insurance industry-oriented customer full-life-cycle event data modeling method and system. The method comprises the steps of collecting customer full-life-cycle event data through a customer contact channel and a business processing link of an insurance industry; preprocessing the collected data, and converting the data into structured data; key elements are extracted from the preprocessed structured detail data; performing hierarchical and classified storage on the structured data based on a hierarchical and classified event data model, and respectively establishing a trunk index model and an expansion detail model in combination with the key elements; generating a unique event ID (Identity) for each event from the key elements; and performing data query optimization and providing data service based on the trunk index model, the expansion detail model and the event ID. Compared with the prior art, the method has the advantages that massive customer event data can be efficiently integrated and analyzed, the data processing and query capability is improved, more efficient and flexible data services are provided, and decision support is provided for the insurance industry.

The invention discloses a data compression and fusion method and device for a digital earth space, and a medium, and relates to the technical field of digital earth spaces. The method comprises the following steps: performing feature extraction, classification and coding on bit vectors stored in a cloud to obtain an index directory set; reading each data unit in the multi-source heterogeneous data stream, and searching the longest matching character string in the index directory set; under the condition that the longest matching character string is found, the codes in the index directory set are output, and a compressed code stream is obtained; reading the code stream, and searching corresponding geographic space data, meteorological data and interaction records in the index directory set; and according to the timestamps, fusing the geographic space data, the meteorological data and the interaction records to obtain multi-source fusion data. By means of the method, the index directory set is constructed and used at the same time in the compression and decompression processes, all data modes do not need to be known in advance, and occupied storage space in a system is effectively reduced.

The invention discloses a ClickHouse-based fused CDN (Content Delivery Network) business data processing method, device and equipment, and the method comprises the steps: obtaining business data of a corresponding fused CDN manufacturer through calling a data interface of each fused CDN manufacturer in real time; performing preprocessing including data format standardization, field expansion and invalid data filtering on the service data by using Spark, packaging the preprocessed service data and sending the packaged service data to a Kafka cluster; the method comprises the following steps: creating a distributed data table based on a ReplacingMergeTree engine in a ClickHouse database, consuming service data in a Kafka cluster in real time by utilizing Spark, and writing the service data into the distributed data table of the ClickHouse database in batches; and retrieving target business data from the ClickHouse database according to a data query request of a user, and converting the target business data into a JSON format to serve as a query result to be returned. The query performance, the storage efficiency, the expandability and the write-in performance can be improved.

The invention discloses a database query method, device and system and a medium, and the method comprises the steps: receiving a database query statement, analyzing the query statement to generate an execution plan of a tree structure, segmenting the execution plan, and obtaining a plurality of execution plan subtrees; matching the plurality of execution plan sub-trees with the intermediate result set in the cache, and determining whether intermediate result set sub-trees equal to the execution plan sub-trees exist in the intermediate result set or not; if an intermediate result set sub-tree equal to the execution plan sub-tree exists in the cache, replacing the successfully matched execution plan sub-tree with the equal intermediate result set sub-tree, and generating a replacement execution plan; and performing scheduling execution according to the replacement execution plan to obtain a corresponding database query result. The subtrees in the execution plan are matched and replaced through the intermediate result set in the cache, so that the intermediate result set in the cache can be directly used in subsequent execution operators, resource consumption and execution time are reduced, and database query performance is improved.

The invention discloses an analysis query method and device for a database, and the method comprises the steps: obtaining data corresponding to a query instruction from a pre-constructed preset storage structure through a preset index algorithm when the query instruction is received in a data query process of a database system, the preset storage structure is obtained by optimizing a storage structure design layer of a multi-version concurrency control mechanism, the preset storage structure is used for avoiding volume expansion caused by multi-version control and improving the performance of data analysis and query, and data is compressed through a data compression algorithm. And storing the compressed data in a preset storage structure in a column storage mode.

The invention relates to the technical field of data processing, in particular to a data statistical method based on a large data volume, which comprises the following steps of: S1, defining a many-to-many first subject table and a second subject table of different data magnitudes in a relational database; s2, acquiring a mapping relation between the first main body table and a second main body table, traversing the second main body table, acquiring IDs of all records corresponding to each record in the second main body table in the first main body table, and splicing an ID list of nodes on a path by traversing a path from a node where the ID is located to a root node, after splicing is completed, an array is formed and stored in a field added to a corresponding record in the first main body table, and storage of the data mapping relation is completed; and S3, completing storage of a data mapping relationship between the first main body table and the second main body table for subsequent adjustment, statistics or query calling. According to the method, the data storage cost can be greatly reduced, the data maintenance cost can be reduced, and real-time statistics and query of a large amount of data can be realized.

The invention discloses an Internet of Vehicles data processing method and system based on TD engine, and the method comprises the following steps: S1, building a TD engine time sequence database, and carrying out the initialization; s2, modifying data related services of the Internet of Vehicles, and adding TD engine related configurations; s3, carrying out a multi-dimensional test on the TD engine time sequence database; and S4, monitoring and storing related performance indexes of the Internet of Vehicles data, and after the performance indexes are stabilized for a preset time, independently supporting data storage and query through a TD engine time sequence database. According to the method, the data processing efficiency, the data writing capability and the query performance are improved by using the TDengine, the dependence on an additional message queue system can be reduced, and the system complexity and the maintenance cost are reduced.

The invention relates to the field of data processing, in particular to a tunnel structure monitoring and detection data standardized database construction method. The method comprises the following steps: acquiring multi-source monitoring and detection data from a tunnel monitoring system; performing cold and hot separation processing on the multi-source monitoring and detection data to obtain detection data; the cold and hot separation treatment is related to the importance degree of a tunnel structure corresponding to each deployment position in the tunnel monitoring system; selecting a plurality of groups of detection data, and distributing index hierarchical identifiers corresponding to the plurality of groups of detection data; the index hierarchical identifier is obtained by mixing at least two of a spatio-temporal index, an inverted index and a multi-level index; and according to the index hierarchical identifier, performing hierarchical storage on the multiple groups of detection data to obtain a target database corresponding to the tunnel monitoring system. According to the method, efficient data management of the tunnel monitoring system can be realized, the storage cost and the data access speed are optimized, and the query performance, the processing efficiency and the expandability of the system are also improved.

The invention discloses a business data management method and system, electronic equipment and a storage medium. The method comprises the following steps: constructing a preset business index library; wherein the preset business index library comprises a first storage unit and a second storage unit; obtaining to-be-stored business data; wherein the business data to be stored comprises index metadata and index numerical data, and the index metadata corresponds to the index numerical data; storing the index metadata to the first storage unit, storing the index numerical data to the second storage unit, and constructing to obtain a target business index library; and when it is determined that the query index request is received, querying from the target business index library according to the query index request to obtain target business data. According to the embodiment of the invention, the management of the business data can be realized, and the query efficiency and query performance of the business data are effectively improved. The method can be widely applied to the technical field of big data.

The present invention provides a method and system for automatically generating database query rewriting rules, including: step 1: modeling the query plan and query rewriting rules of the database into an enumerable form; step 2: enumerating the constraints between each pair of query plan templates to generate a first query rewriting rule. Compared with the prior art, the present invention can generate new rewriting rules that do not exist in the existing database system, can match and rewrite more complex queries generated by frameworks such as object-relational mapping in Internet applications, and improve the database query rewriting capability and the performance of database queries under Internet applications.

This invention relates to an index construction method, device, program product, and storage medium, and pertains to the technical field of vector databases. The method includes: acquiring query vectors; calculating the coordinate region identifiers of each query vector in the vector space; statistically analyzing the query frequency of each coordinate region identifier within a preset time window to obtain regional query frequency distribution data; calculating the frequency growth rate of the regional query frequency distribution data of each coordinate region identifier at the current moment compared to the regional query frequency distribution data at the previous moment; calculating a query popularity fluctuation coefficient based on each frequency growth rate; monitoring the load parameters of the vector database system in real time and calculating a comprehensive system load index based on the load parameters; and constructing an index for each query vector based on the query popularity fluctuation coefficient and the comprehensive system load index. Implementing the technical solution provided in this application can achieve a balance between indexing effectiveness and system stability.

The invention relates to a data query method and device, electronic equipment and a storage medium. The data query method comprises the steps that a query statement corresponding to a query request is acquired, and query elements of the query statement are determined; judging whether the query element exists in a query cache or not, wherein the query cache is used for storing historical query elements and historical query results; in response to the query element existing in the query cache, obtaining a query result corresponding to the query element in the query cache; and in response to the situation that no or part of the query elements exist in the query cache, performing query based on the data sources corresponding to the query elements, and obtaining a query result. According to the method and the device, the query result is determined to be obtained or the query result is obtained after query, so that the query performance is improved, and the query pressure on the cache data is reduced.

This application provides one or more embodiments of a relational database management method and apparatus. The method includes: obtaining a creation instruction for creating a target data table in the relational database; wherein the creation instruction includes column information corresponding to basic data columns in the target data table; responding to the creation instruction, creating the target data table in the relational database, including the basic data columns and hidden data columns, based on the column information; wherein the hidden data columns are extended data columns independent of the basic data columns; after the target data table is created, establishing a unique tree-structured index for the target data table based on the data in the hidden data columns; wherein the data in the hidden data columns corresponding to data rows in the target data table includes monotonically increasing values.

The invention relates to the field of knowledge graphs, and discloses a bitmap-based graph data storage and query method, system, device and medium, the method adopts a unified bitmap model to store graph data, converts graph structure data into a bitmap suitable for parallel computing, maps a graph query request into a bitmap operation expression tree, and stores the graph data in the bitmap operation expression tree; therefore, graph query is converted into bitmap operation, and the query performance of high-concurrency and complex graph query is remarkably improved; moreover, a reasonable computing resource scheduling decision is dynamically set according to an input graph query request, and for high-concurrency and complex graph query, thousands of cores of a large-scale parallel computing engine are used for parallel processing, so that the random access bottleneck and the CPU computing power bottleneck of traditional graph traversal are thoroughly avoided, the query speed of order of magnitude can be increased, and the query efficiency is improved. A plurality of complex atlas analysis queries can be processed at the same time, and the real-time requirement of complex atlas query can be met.

The invention discloses a historical data archiving implementation method and device and a medium. The method comprises the following steps: creating a system architecture and a table; under the condition that table creation is completed, archiving conditions are configured; under the condition that the archiving condition configuration is completed, the archiving task is executed regularly; and aiming at the execution process of the archiving task, the query function is optimized. The method has the beneficial effects that the comprehensive functions of reducing the data volume of the table, improving the query performance, supporting flexible condition configuration, reducing the storage space occupied by the table, avoiding associated data migration, querying archived historical data, querying archived and unarchived total data and periodically deleting useless data through configuration are realized.

A file system path query acceleration method using distributed KV, in which a full-path cache domain is created for the file system in a distributed KV storage system, the domain includes multiple co-processing nodes, which are responsible for storing all directory access information of the file system, allowing the system to complete the query operation of the entire path through any co-processing node in one network request, significantly improving the query efficiency and system performance. In addition, the present invention utilizes the multi-key transaction characteristics of distributed KV storage to uniformly cache all directory access information of the same file system, avoiding performance problems caused by client cache limitations and uneven loads between nodes in traditional methods. Through this method, not only the processing time of metadata access is reduced, but also the input / output performance of the distributed KV storage system is optimized, providing strong technical support for efficient management and fast access of massive files.

The invention relates to the technical field of data storage, in particular to a data processing method and device, equipment and a storage medium, and is used for improving the deletion efficiency of different keys in an LSM number. The method comprises the following steps: for any hierarchy of a log structure merging LSM tree, obtaining each to-be-processed file, namely a read-only memory table or a source SST associated with the hierarchy, sorting and traversing a first type of keys in the files, and determining whether the keys are invalid or not according to a preset judgment condition; wherein the judgment condition comprises at least one of the following conditions: the key itself or the value corresponding to the key itself meets a first value taking condition, and the key itself or the value corresponding to the key itself and the preorder key itself or the value corresponding to the key itself meets a second value taking condition; the first type of keys are different keys in the to-be-processed files; and for each to-be-processed file, generating a target SST through the key value pair data of the effective key.

The application discloses a system and method for accelerating reading compressed files based on A Virtual File System, and belongs to the technical field of time series databases; the technical problem to be solved by the application is how to realize column format compressed storage and reading of time series data types, reduce IO, memory and CPU consumption caused by decompression, and improve the ability of the system to process time series data; the technical scheme adopted is as follows: in the time series engine of the open database ZNBase, an execution engine generates an execution plan according to SQL semantic analysis, calls a storage layer interface of ZNBase, and queries accurate time series history partition compressed data; the storage layer completes fast execution of the query in the storage layer based on various compressed query interfaces provided by AVFS, and accelerates the query of the database; the specific steps are as follows: pre-analysis; compressed file pre-reading; reading compressed data.

The invention discloses a multi-channel hardware environment cloud broadcasting method and system based on multi-type warning, and the method comprises the steps: constructing an access frame, accessing a plurality of hardware devices through the access frame, and receiving warning messages triggered by the hardware devices; building a distributed cluster, performing parallel processing and asynchronous processing on each alarm message through the distributed cluster to obtain a cloud broadcast task, and performing cloud broadcast notification according to the cloud broadcast task; and storing the alarm messages through a distributed full-text retrieval engine. By constructing a unified access framework, the access process of each hardware device can be simplified, the fluency and compatibility of data interaction are improved, parallel processing and asynchronous processing are performed on each alarm message through a constructed distributed cluster, each alarm message is stored through a distributed full-text retrieval engine, and the security of data interaction is improved. The response time of the alarm message can be shortened, millisecond-level response of inquiring hundred million-level data is achieved, and the method can be widely applied to the technical field of the Internet of Things.

This application discloses a data aggregation method, data aggregation device, electronic device, and storage medium, relating to the field of computer technology. It can improve the query performance of data after it is stored in the database, and reduce the difficulty and cost of querying. The specific solution includes: acquiring network data stream traffic data; generating at least one first data log and at least one second data log based on the traffic data; aggregating all first data logs according to a first aggregation rule based on the link identifier corresponding to the first data log to obtain at least one first aggregated log; and aggregating all second data logs according to a second aggregation rule based on the link identifier corresponding to the second data log to obtain at least one second aggregated log.

The present invention relates to a query system based on data analysis, and in particular to the technical field of database management. The query system comprises a user information management module for implementing dynamic authority management for users, a data source management module for managing user requests, caches, data sources and data security, and a database management module for managing a number of databases. The present invention analyzes the query request of the user to determine whether the user request exists in the cache, so as to determine the selection of the query mode for the query request, thereby making flexible selections in the query process, and setting different supervision strengths in the corresponding query mode to supervise the query process, thereby improving the accuracy of control over the data query process.

The invention relates to the technical field of data storage, in particular to a distributed file storage method. In particular to a distributed file storage method based on a ShardingSphere distributed SQL (Structured Query Language) transaction and query engine, an object storage service of a MinIO open source protocol and a Mysql database. The advantages of all middleware are comprehensively integrated, MinIO serves as the basis of file storage, ShardingSphere and Mysql serve as auxiliary storage file meta-information, the capacity of uploading, querying and deleting open files is achieved through a storage service application API, and the distributed file storage method which is expansible, easy to backup and high in performance is provided for mass file storage.

The invention discloses a database distributed version synchronization and intelligent compression method and system, and relates to the technical field of distributed databases. The method comprises the following steps: a database fragment management mechanism in a distributed environment divides a database into a plurality of fragments, and each fragment independently maintains a versioned storage manager; a cross-node version coordination mechanism ensures the consistency of global version numbers; according to a multi-copy synchronization mechanism, the consistency of multiple copies is ensured by adopting a two-stage submission protocol; an intelligent copy selection algorithm; a distributed garbage recycling mechanism; the version compression mechanism is used for analyzing historical version data and storing the historical version data by adopting increment; intelligently combining strategies, and determining a compression strategy according to the data access frequency. According to the method, the problem of consistency of data version synchronization in a distributed environment is solved, and meanwhile, storage space occupation is reduced through intelligent compression.