13results about How to "Reduce query latency" patented technology

The invention relates to the technical field of data storage management, in particular to a data management method and system for an automobile transaction platform. The method comprises the following steps: obtaining a to-be-stored data sequence corresponding to an automobile transaction platform, and carrying out index pair classification extraction and storage space limit quantification calculation to obtain a storage release memory space limit degree corresponding to each type of index field; performing sequence storage division on the corresponding to-be-stored data sequence to obtain a storage data sequence segment corresponding to each type of index field; obtaining a query access rate and a storage interval approach access degree of the corresponding sequence segment, and carrying out storage activity evaluation analysis and index storage priority measurement analysis to obtain an index storage priority measurement of each type of index fields for each sequence segment; and carrying out priority storage management on the corresponding storage data sequence segments. According to the invention, mass data in the automobile transaction platform can be managed efficiently and accurately, so that the storage process of the data is optimized.

The invention discloses a database distributed version synchronization and intelligent compression method and system, and relates to the technical field of distributed databases. The method comprises the following steps: a database fragment management mechanism in a distributed environment divides a database into a plurality of fragments, and each fragment independently maintains a versioned storage manager; a cross-node version coordination mechanism ensures the consistency of global version numbers; according to a multi-copy synchronization mechanism, the consistency of multiple copies is ensured by adopting a two-stage submission protocol; an intelligent copy selection algorithm; a distributed garbage recycling mechanism; the version compression mechanism is used for analyzing historical version data and storing the historical version data by adopting increment; intelligently combining strategies, and determining a compression strategy according to the data access frequency. According to the method, the problem of consistency of data version synchronization in a distributed environment is solved, and meanwhile, storage space occupation is reduced through intelligent compression.

The invention discloses a translation lookaside buffer query acceleration system, method and device, a medium and a product, and relates to the technical field of electric digital data processing, and the system comprises a multi-stage translation lookaside buffer and a translation lookaside buffer filter which are in communication connection with each other. The translation backup buffer filter comprises a hash function module and a digit array module which are in communication connection with each other. The multi-level translation lookaside buffer is used for receiving an address translation request and querying target address mapping corresponding to the address translation request, the translation lookaside buffer filter and the multi-level translation lookaside buffer work in parallel, at least one hash abstract value is generated according to the address translation request, and the at least one hash abstract value is sent to the multi-level translation lookaside buffer. And comparing the hash digest value with a historical hash digest value in the digit array module to judge whether the target address mapping exists in the multi-level translation lookaside buffer, if not, interrupting the on-going query operation of the multi-level translation lookaside buffer, performing memory access to obtain the target address mapping, and if not, performing memory access to obtain the target address mapping. And the query efficiency of the translation lookaside buffer is improved.

The application relates to the technical field of data caching, in particular to a social caching retrieval system fusing hot spot analysis, which comprises a hot degree modeling unit, a social association unit and a caching resource allocation unit. The hot degree modeling unit is used for obtaining a hot spot event set, performing hot degree modeling on each hot spot event in the hot spot event set, and obtaining hot degree characteristics of the hot spot event. The social association unit is used for obtaining a user set and constructing a user-content bipartite graph according to interactive behaviors between the hot spot event set and the user set. Through the hot degree modeling unit and the propagation calculation unit, the hot spot event can be modeled according to user behaviors and historical data, and the hot degree of the hot spot event at a future moment can be predicted by using a hot degree prediction model, so that the events which will become hot spots can be identified in advance, and more caching resources can be allocated to the events according to the prediction results.

The invention discloses a database construction method based on hybrid heap allocation and versioned storage, and relates to the technical field of databases. The method comprises the following steps: establishing a database file structure system comprising a data file, a metadata file, a function file, a mode file and an extension file; the method comprises the following steps: constructing a versioned storage manager adopting a hybrid heap allocation algorithm, dividing a heap memory into memory pages and memory blocks with fixed sizes, managing the free memory pages by using a free linked list, recording the allocation state of the memory blocks in a bitmap, and adding version header information for the memory blocks; establishing a version index mechanism based on a persistent B + tree; and a historical data tracking function is realized. According to the method, the advantages of the idle linked list and the bitmap are combined, the memory allocation efficiency is guaranteed, the damage resistance of the system is improved, and meanwhile, complete historical tracking of the data is achieved through a versioning storage mechanism.

The invention discloses a distributed database query method, a distributed database, a device, a medium and a product; the method comprises: when a query request satisfies a target condition, a distributed database portal determines a storage node to be queried and a corresponding sub-query request, and sends the sub-query request to the corresponding storage node to be queried; each storage node to be queried queries a data table in the storage node to be queried in parallel through a main connection and at least one auxiliary connection between the storage node to be queried and the distributed database portal, and feeds back to the distributed database portal; the distributed database portal performs connection calculation on the query result, determines the target query request, and executes the target query request on the main connection corresponding to each storage node to be queried and a client, determines the query result, solves the problem of high latency when the distributed database queries data rows of multiple storage nodes, reduces the query latency of the distributed database executing complex SQL, and improves the query efficiency.

The present invention discloses a memory access control system for an RDMA network card, comprising: a memory access control software module and a memory access control logic module; the memory access control software module is arranged in a host RDMA network card driver, and is used to register and deregister memory areas and memory windows, and to map virtual address spaces to physical address spaces; the memory access control logic module is arranged in the RDMA network card, and is used to determine whether the access rights requested by the RDMA are legal, and when the access rights are legal, the virtual address is converted to the physical address, and information is provided for the DMA operation of the RDMA network card. The present invention is beneficial in that the memory access control system for the RDMA network card provided unloads the query logic related to the virtual-physical address conversion and the memory access right control to the RDMA network card hardware, which not only helps to give full play to the performance of the high-speed transmission of the network card, improves the transmission bandwidth of the network card, reduces the delay of the RDMA read and write operations, but also reduces the storage space occupation.

The invention relates to the field of data request processing, and discloses a reading method based on Hudi data, which comprises the following steps: receiving a request for accessing a target object to a Hudi storage system, and extracting an identifier of the target object from the reading request; querying a data cache region of the Hudi storage system by using the identifier to determine whether an update record merge file of the target object is stored in a read-time merge layer of the data cache region or not; if the update record merged file corresponding to the target object is stored, retrieving an initial data record of the target object from the MOR table according to the identifier; and sending the retrieved initial data record to a data cache region, and merging the initial data record and the update record merging file to generate a final data file of the target object in response to the reading request. By introducing the data cache region and the merging layer during reading, the situation that a large number of historical records need to be traversed during reading each time is avoided, and therefore the query efficiency is improved.

The present invention belongs to the field of database optimization, and in particular to a model-accelerated database retrieval optimization system and method based on a knowledge graph. The system collects and preprocesses database cache and query data to build a real-time updated distributed multi-level knowledge graph, and uses a path generation algorithm to enhance query efficiency. Secondly, a high-frequency pre-cached sub-database is linked with a graph node, and a prediction model is combined to preload high-frequency query results, and regular updates are performed to ensure data timeliness. Thirdly, when querying, the system provides real-time recommendations, pre-caches intent queries, and parses global queries and generates query paths when intent queries are missed. Finally, a path selection unit is used to determine the optimal path to execute the query, quickly return the results, and perform secondary query optimization on the query results based on the returned results and secondary query statements. The overall process significantly improves the database retrieval speed and user experience.

The invention discloses a method for improving the recall rate of a vector database by using text extension training. The method comprises the following steps: S1, generating and extending questions; s2, performing similarity evaluation and problem screening; s3, linking and embedding questions and answers; s4, constructing and optimizing a vector database; and S5, performing real-time retrieval. In the preprocessing stage, not only can a plurality of similar problems be generated when the original problems are expanded, but also the similarity of each generated problem is evaluated by introducing a text neighbor algorithm review mechanism. Only questions passing the similarity evaluation criteria can be linked with original answers and embedded into a vector database. And during query, the problem of the user is directly matched with the optimally stored data, so that the load in a real-time query stage is reduced, and the overall efficiency of the system is improved.

The invention relates to the field of data management, in particular to an intelligent logistics cloud platform data management system and method. The method comprises the following steps: firstly, carrying out data preprocessing on collected original data and bound timestamps and position coordinates to obtain standardized data; based on the standardized data, introducing a space-time grid XOR perturbation coding algorithm to obtain space-time characteristic values; and then, based on the standardized data and the space-time characteristic values, introducing a linear consistency Hash variant algorithm based on space-time disturbance to obtain a main Hash value and an auxiliary virtual Hash value, and copying and storing the standardized data to corresponding physical data storage nodes. The technical problems of data load imbalance, hotspot node overload, performance bottleneck, single-point fault risk and insufficient fault-tolerant capability are solved.

This invention discloses an artificial intelligence-driven method and system for constructing a disease gene pharmacology association database, belonging to the interdisciplinary fields of computer software, bioinformatics, and precision medicine. This invention collects multi-source heterogeneous data through a deep reinforcement learning intelligent crawler, and achieves data standardization and quality control through a GNN-DTI model and a three-level labeling system; it employs a MySQL hierarchical storage engine to support high concurrency and large-scale expansion; it combines a T5 model and D3.js to provide natural language retrieval and visual interaction, and uses a multi-language SDK and containerized deployment to achieve pluggable expansion. This invention realizes a closed loop of data acquisition, processing, storage, and application, with a data confidence level exceeding 0.85, millisecond-level query response, and supports six major scenarios including precision medicine, providing efficient decision support for medical, pharmaceutical, and scientific research fields.

The invention relates to a high-performance IP-to-geographic position method and system based on an IP offline library, and the method comprises the steps: querying geographic position information in a Redis cache in response to an IP query request, and outputting a query result if the corresponding geographic position information is queried in the Redis cache; if the corresponding geographical location information is not queried in the Redis cache, querying the MongoDB cache, and if the corresponding geographical location information is queried in the MongoDB cache, outputting a query result; if the corresponding geographical location information is not queried in the MongoDB cache, converting a character string of the IP into a shaping number; and performing retrieval in an index area of the IP offline library based on the reshaped digits, obtaining corresponding offset of the IP in the index area, and extracting geographical location information corresponding to the IP from a recording area of the IP offline library based on the offset. According to the method, the accuracy and the query efficiency of the geographic position information are remarkably improved by analyzing the IP through the offline library, and the query delay is reduced and the stability and the practicability of the system are ensured through the Redis cache and the MongoDB cache mechanism.