17 results about "Full table scan" patented technology

The invention discloses an index data query method, device and equipment, and relates to the technical field of computers, the method comprises the following steps: receiving a query index request initiated by a user, and obtaining a query dimension associated with a query index; judging whether query dimensions associated with the query indexes are materialized and stored in a materialized view of a fast area or not; if the query dimensions associated with the query indexes are materialized and stored in the materialized view of the fast area, reading data of the query indexes from the materialized view of the fast area; otherwise, reading the data of the query index from the common area in a full-table scanning mode; and integrating the data of the plurality of query indexes read from the fast area and the common area, and generating a unified query result based on preset service logic. According to the method, the existence of all dimensions associated with each query index in the fast zone materialization view is dynamically judged, the optimal data reading path is intelligently selected for different indexes, and the overall query response time is shortened.

The invention discloses a query statement processing method and device, equipment, a storage medium and a product. The invention relates to the technical field of databases. The method comprises the steps that a slow query type corresponding to a target slow query statement is determined according to an execution plan of the target slow query statement, the first type corresponds to full-table scanning, full-index scanning and the situation that the number of scanning lines is larger than a preset numerical value, the second type corresponds to the full-partition scanning situation, and a lexical analysis mode determined according to the slow query type is utilized, performing lexical analysis on the target slow query statement to obtain a first analysis result, and displaying first processing information in response to the condition that the first analysis result comprises a preset slow query reason of the target slow query statement, the first processing information comprises at least one of a preset slow query reason, a recommended optimization writing method corresponding to the preset slow query reason and optimization suggestion information corresponding to the preset slow query reason. By adopting the technical scheme, the optimization efficiency of the slow query statement can be improved.

The application provides a SQL statement query optimization method, device, medium and product. The SQL statement query optimization method comprises the following steps: obtaining a SQL statement; if it is found that there is an inequality predicate in the SQL statement, extracting the inequality predicate to obtain an original predicate structure; rewriting the original predicate structure into an equivalent predicate structure to obtain a candidate predicate structure; and determining an execution path by comparing the query costs of the original predicate structure and the candidate predicate structure. In this way, the inequality predicate structure which cannot be applied to partition pruning and index can be rewritten into a predicate structure which can be applied to partition pruning and index, thereby helping to reduce the number of areas that need to be scanned, or helping to avoid full table scanning and reducing disk I / O.

The invention discloses a data and performance verification method and device of a database, equipment, a medium and a product, and relates to the technical field of big data. The method comprises the steps of obtaining a table list; controlling production platform equipment according to the table list, and generating first metadata of the production environment database and a first execution plan statement of the first query statement; obtaining and checking whether the first metadata is consistent with second metadata corresponding to the test environment database to obtain a checking result; the production platform equipment is controlled, and the first execution plan statement is operated to screen out a first target query statement for full-table scanning; generating statistical information of tables involved in the first target query statement; displaying the verification result and the statistical information; wherein the production environment database and the test environment database are in network isolation. Under the condition that the plaintext information in the production environment database cannot be obtained, the data and performance of the production environment database and the test environment database which are in mutual network isolation are verified.

The invention discloses a progressive visibility mapping upgrading and freezing optimization method and device for a database. The method comprises the following steps of: additionally arranging a PDVMFROZEN flag bit at the head of a visibility mapping VM block to mark a new version (including all-frozen bit); setting a vmversion field in a table memory structure Relation, and dynamically detecting new and old VM versions; modifying a VM interface and adding a VMALLFROZEN macro compatible new version and an old version; automatically marking a new version for the new VM block, and gradually upgrading the old VM block; in the VACUUM process, a frozen block is skipped through VMALLFROZEN, and an all-open bit is automatically marked. According to the method, full-table scanning overhead is reduced, I / O and CPU occupation is reduced, online smooth upgrading is supported, long-term performance and operation and maintenance convenience of the database are improved, and compatibility of new and old versions and system stability are guaranteed.

The application provides a database operation statement optimization method, medium, product and equipment. The method comprises the following steps: obtaining a database operation statement to be optimized, wherein the database operation statement comprises a target null judgment function, and the target null judgment function is an expression comprising a null value processing logic; analyzing the target null judgment function to determine the parameter type and operation logic of the target null judgment function; rewriting the target null judgment function into an equivalent conditional judgment expression according to the parameter type and operation logic; and replacing the target null judgment function in the database operation statement with the conditional judgment expression, thereby obtaining an optimized database operation statement. Through the method, the problems of large cardinality estimation error and inaccurate execution cost estimation caused by the use of the null judgment function in the existing database can be effectively solved, the optimizer can correctly identify the query condition and normally enable the index path to perform index scanning, and resource waste caused by full table scanning can be avoided.

The invention discloses a statistical acceleration method based on a data template, and relates to the field of big data and real-time computing, when data is stored in a database, bitmap templates are generated according to historical data in a table, each template has an ID as a data virtual column, and the virtual columns are persistently stored in an FRC; when query is carried out, corresponding coding values constructed according to conditions are read firstly, intersection and union calculation is carried out in a template to find a conforming template ID set, whether a conforming template ID exists in each FRC file or not is judged, and then filtering optimization is achieved; the query performance is remarkably improved, efficient balance of storage and query is achieved, dynamic expansion and high-dimensional data processing are supported, automatic and intelligent template generation is achieved, an existing system architecture is compatible, and the problems that when an existing database system processes query under the multi-column null value combination condition, an efficient data filtering mechanism is lacked, the query performance is low, and the query efficiency is high are solved. In a mass data scene, the problems of frequent full-table scanning, large I / O overhead, long response time and the like occur.

The invention discloses a semantic retrieval method and system for structured data, and relates to the technical field of data analysis, and the method comprises the following steps: setting a label of field information in a relational database, expanding a field initial code according to input data, generating a field expansion code, and querying a jump order, and updating the Markdown document in combination with the field extension code and the query jump order, obtaining jump field information, and constructing a corresponding relationship between the query jump order and the second timestamp. According to the method, the retrieval range is dynamically adjusted by querying the jump order, full-table scanning is avoided, redundant calculation is remarkably reduced, field codes are automatically expanded according to input data, service rapid iteration is adapted, document changes are marked through the first timestamp, the latest effective fields are positioned in combination with the jump order, and historical data are prevented from being repeatedly processed; full-link automation from label setting to document generation, such as Markdown document automatic update field description.

The invention discloses a data splitting method and device based on customer number prefixes, and the method comprises the steps: splitting a customer number interval according to the length of the customer number prefixes, and obtaining a plurality of prefixes; replacing the content in the LIKE condition of the SQL query statement with a prefix to obtain a query statement of the prefix; querying the total number of clients corresponding to all prefixes from a client information table by using the query statements of all prefixes; determining the number of target number segments according to the total number of clients and the node processing capability; and distributing the total number of customers according to the number of the target number segments and the prefix set to obtain a plurality of customer number segments. Data splitting is carried out based on customer number prefixes, and loads are reasonably distributed by comprehensively analyzing the prefix data size, the node processing capacity and the number of target number segments. In the query level, the LIKE condition is replaced by a specific prefix by means of a dynamic SQL to realize targeted query, so that full-table scanning is avoided fundamentally, I / O overhead is reduced greatly, and normal execution of a query task is guaranteed.

The invention discloses a cross-library updating optimization method and device and electronic equipment. The method comprises the steps that in the cross-database association updating process, execution plans of SQL in a source database and a target database are detected respectively, execution plan detection results are obtained, and the execution plan detection results comprise a remote interaction detection result corresponding to the source database and a full-table scanning and index detection result corresponding to the target database; under the condition that the execution plan detection result indicates that the driven table in the target database has index missing, the scanning cost of the driven table is determined according to the return row number of the driving table in the target database and the corresponding single scanning time; and adding an index in the driven table according to the scanning cost. According to the method and the device, the technical problems that when performance degradation occurs in large-batch cross-library updating, cross-library updating optimization in related technologies generally depends on manual intervention, an automatic optimization means is lacked, and high consistency of data cannot be ensured are solved.

Embodiments of the present application disclose a voucher generation method and related equipment, which are used for solving the problems of large database pressure and batch processing efficiency bottleneck caused by using database exclusive lock and full table scanning voucher serial number in the prior art. The method of the embodiments of the present application comprises: in response to a voucher generation request, obtaining voucher serial number cache information under a target accounting period and a target voucher serial number corresponding to the voucher generation request; based on the voucher serial number cache information, determining an occupied interval and a broken serial number interval of a historical voucher serial number; based on the broken serial number interval and the occupied interval, allocating at least one target voucher serial number for each voucher generation request; and generating a voucher corresponding to each target voucher serial number for the target voucher serial number allocated for each voucher generation request.

The invention relates to the technical field of data warehouses and multi-dimensional analysis, in particular to an automatic cubic table construction method based on metric dependency analysis, which comprises the following steps of: classifying metric fields, identifying and calculating a dependency relationship, and constructing a directed acyclic graph; based on the directed acyclic graph, generating a measurement construction path which depends on order; constructing a dimension level by adopting recursive aggregation from bottom to top, and caching an intermediate result to reduce full-table scanning; metric calculation is executed according to paths, parallel calculation of non-dependency / low-dependency metrics and increment construction are supported, and data consistency is guaranteed by combining updating-inserting double-stage writing. The method improves cubic table construction efficiency and data consistency, reduces resource consumption, adapts to multi-dimensional analysis requirements in the fields of education, finance, retail and the like, and is suitable for data warehouses and business intelligent systems.

The invention provides a report query method and device, equipment, a storage medium and a program product, relates to the technical field of data processing, and can reduce the error rate of manually configuring SQL query statements and improve the query efficiency. According to the specific technical scheme, when a user inputs a query request, a system firstly understands the intention of the user and generates a corresponding SQL statement; secondly, a preliminary query scheme is formulated according to the actual situation of the database, index scanning or full-table scanning is determined, and an SQL statement structure is optimized; and then predicting the calculation complexity, the data scanning amount and the parallel feasibility of the query, and finally determining and executing a query execution plan in combination with the load condition of the current database. In the execution process, the database load can be checked in real time, and the execution strategy is dynamically adjusted. Finally, a visual report is generated according to a query result, and chart display and export functions are supported.

The invention discloses a statistical method for residence duration of Internet of Things equipment and a terminal. The statistical method comprises the following steps: acquiring a station identifier, an equipment identifier and a state event of each piece of Internet of Things equipment; taking the station identifier and the equipment identifier of each piece of Internet of Things equipment as index identifiers, and storing the latest timestamp of the state event of each piece of Internet of Things equipment; in response to a statistical requirement for the target Internet of Things equipment, reading the latest timestamps corresponding to the state events of the target Internet of Things equipment according to the index identifiers, and sorting the state events according to the sequence of the timestamps; and determining the state event with the top timestamp in the sequence as the current state of the target Internet of Things equipment, and taking the difference value between the current time and the timestamp of the current state as the residence duration of the current state of the target Internet of Things equipment. According to the method, the dependence on CPU and memory resources is greatly reduced, and the performance risk caused by full-table scanning is reduced.

The invention relates to an SCADA (supervisory control and data acquisition) system historical data query optimization system and method, in particular to the field of industrial automation, and is characterized in that a storage architecture is dynamically optimized through a data thermal layering module, so that high-frequency access data automatically resides in a high-speed storage layer, and the hotspot data query efficiency is remarkably improved; the intelligent preloading module prejudges data requirements based on a working condition context, converts passive query into active loading, and effectively reduces response delay of a key operation scene; the multi-dimensional index construction module accurately locates data through a dynamic index strategy, so that full-table scanning resource consumption is avoided; and the query execution cache module is combined with an intelligent compression and self-adaptive adjustment mechanism to construct an efficient closed loop from query request to result return. The whole system greatly reduces time consumption of complex condition retrieval while guaranteeing query stability, and is particularly suitable for high-concurrency industrial monitoring scenes.

The application provides a multi-table query method and device, obtains a page number P of a page needing to be displayed and a maximum data number Z that can be displayed per page, queries a total number of data meeting a condition from a plurality of data tables, the process is fast in query speed, then a first data set composed of the first P×Z data of each data table is obtained, full table scanning of all data tables is avoided every time a page is skipped, resource consumption is saved, and query efficiency is improved. Then, the first data sets of all data tables are merged and sorted to obtain a second data set, the number of sorting is small, and sorting cost is reduced. Finally, Z data is intercepted from the second data set, the number of data sets for interception is also reduced, and interception efficiency is higher. Therefore, the method uses conditional query, avoids adding indexes that greatly increase the performance of data table addition, deletion and modification and consume a large amount of memory for storing the indexes, and greatly improves query efficiency.

The invention provides an index creating method, computing equipment, a storage medium and a product, and the index creating method comprises the steps that column access information is acquired, and target column access information is acquired by at least one structured query statement; determining at least one candidate data column according to the column access information; obtaining a database table corresponding to each candidate data column, and obtaining row information of the database table corresponding to each candidate data column and column information of each candidate data column; according to the row information of the database table corresponding to each candidate data column and the column information of each candidate data column, determining an index creation weight corresponding to each candidate data column; and according to the index creation weight corresponding to each candidate data column, determining at least one target data column, and creating an index for the database table corresponding to each target data column. The database performance problem caused by index creation lagging or omission is solved, artificial resources do not need to be consumed, and the high-frequency access data columns of full-table scanning caused by lack of indexes can be automatically recognized.