53results about How to "Improve retrieval speed" patented technology

The invention discloses a context learning-based database query generation method and system and a storage medium, and the method comprises the steps: extracting keywords and phrases from a natural language query, retrieving values similar to the keywords in a database, and returning values most related to the keywords and corresponding columns thereof, retrieving description information related to the keywords and the phrases in a database directory; performing mode selection aiming at the most relevant value of each keyword and the corresponding column, namely screening out the column irrelevant to the natural language query, further selecting a table necessary for generating the SQL query, and finally reducing the mode to a minimum column set necessary for generating the SQL query; and correcting logic and grammar errors of the method model through the database mode, the natural language query, the generated candidate SQL query and the execution result thereof. According to the method, the efficient SQL query statement is generated by intelligently understanding the data context and the query requirement.

The invention discloses a judicial field class case retrieval method and system based on a large model. The method comprises the following steps: constructing a judicial document vector database by combining a vector model with an obtained judicial document; performing semantic recognition and question enhancement on judicial case or judicial question texts through the large model; generating vectors for the enhanced judicial case or judicial question text through a vector model, and performing similarity retrieval in a judicial document vector database to obtain a primarily screened judicial document set; reordering the primarily screened judicial document set through a Reranker model to generate a finely arranged judicial document set; and inputting the enhanced judicial case or judicial problem text and all judicial documents in the finely arranged judicial document set into a large model for correlation analysis, outputting the judicial documents most related to the judicial case or judicial problem text, and sorting the judicial documents according to similarity to form a data set as a retrieval result. According to the method, the accuracy and efficiency of class case retrieval can be improved.

The invention discloses a large-scale embedded index data desensitization method suitable for a forestry database, and belongs to the technical field of data desensitization, and the method comprises the steps: obtaining a forestry data description document D1, splitting the document D1 into sentences, and processing the sentences into embedded vectors; a user customizes a query sentence containing the desensitization word, and a query vector is generated; performing hash coding on the embedded vector and the query vector to obtain a corresponding hash function cluster; searching data to be desensitized through cosine similarity calculation; and inputting to-be-desensitized data and the selected desensitization method into the large language model to generate desensitized data, and replacing the corresponding sentences in the D1 to obtain a desensitized forestry data description document D2. According to the method, the input data is converted into the high-dimensional vector through the embedding layer, so that the semantic features of the data can be effectively expressed in the vector space, the retrieval speed and accuracy of large-scale data are improved by means of the locality sensitive hashing technology, the calculation overhead is reduced, and a technical guarantee is provided for real-time desensitization processing of a large-scale data set.

The invention discloses a similar surgery picture retrieval method, system and equipment for minimally invasive surgery based on deep learning and a medium, and relates to the technical field of image processing. Respectively extracting original image features in the preprocessed video stream and original text features in the preprocessed text content; performing interactive fusion on the original image features and the original text features by adopting a feature fusion module; using a feature codebook module to map the fusion feature into a codebook vector; a mapping feature decoding module is adopted to decompose the codebook vector into compressed image features and compressed text features, finally, the compressed image features are decoded into similar images, and the compressed text features are decoded into text descriptions corresponding to the similar images; and the operation picture can be quickly and accurately retrieved.

The invention relates to the technical field of archive storage and management, and discloses an archive classification management method which comprises the following steps: constructing an alliance chain property block chain network and an I PFS file system; for the received multi-dimensional electronic archive set, performing clustering analysis on the electronic archive set to obtain an archive classification result of the electronic archive set; based on the number of the electronic archives in the archive classification result, the storage nodes are classified to obtain node classification results, and the node classification results are in one-to-one correspondence with the archive classification results; according to a node classification result, uploading and storing the classified electronic archive file through the corresponding storage node, generating a hash ID table related to the storage node, and classifying the hash ID table; and the block chain network generates the Hash ID table into a block according to the category and a block chain network operation mechanism, and stores the block into the block chain network.

The human cost data storage method for the power grid engineering system comprises the steps that human cost data related to cost data in the power grid engineering system are acquired, and the human cost data comprise work type data, man-hour data and salary data of constructors; determining a DBSCAN clustering radius range based on the average man-hour salary of each type of work; based on a DBSCAN clustering radius range, performing clustering analysis in a man-hour-salary two-dimensional plane formed by the human cost data to obtain a preliminary classification result; when it is determined that the isolated data points exist based on the preliminary classification result, secondary classification is carried out on the isolated data points based on the distance between the isolated data points and the known clustering cluster, and a final classification result is obtained; and realizing partition storage of the cost data based on a final classification result. By implementing the method, optimized storage of the cost data of the power grid engineering system can be realized.

The invention relates to the technical field of environmental monitoring, and discloses an exclusive encrypted file format of a universal water-gas-sound-residue pollutant data acquisition instrument. File structures including a file header setting section, a data storage format definition section, a self-checking section and an encryption algorithm section are sequentially stored in a file to form an exclusive encryption file format of the universal water, gas, sound and residue pollutant data acquisition instrument, various encryption and checking measures are adopted, and the safety and integrity of data in the transmission and storage process are ensured. The method has the advantages that data are stored in the database and prevented from being tampered and stolen, data storage efficiency and retrieval speed are increased due to reasonable data storage formats and index designs, data management and analysis are facilitated, file damage or data tampering can be timely discovered through CRC32 check and data integrity check codes, data reliability is guaranteed, and data storage efficiency and retrieval speed are increased. And on the basis of an XML structured storage mode and an open file format design, subsequent addition of new pollutant parameters or function extension is facilitated, and good expandability is achieved.

The present application discloses a method for constructing a knowledge graph based on a large language model and a vector library, which relates to the field of knowledge graph technology, including: using a pre-trained RoBERTa model to perform word segmentation, part-of-speech tagging and semantic analysis on the collected natural language text to obtain a semantic representation vector of the natural language text; obtaining the association between the semantic representation vector and the corresponding named entity through the Seq2Seq framework and the attention mechanism, and obtaining the named entity through the obtained association; constructing a knowledge graph; fusing the constructed knowledge graph with a relational database of a preset domain to generate a fused knowledge graph; using dependency syntactic analysis to obtain the named entity and the corresponding attribute, obtaining the entity relationship of the named entity, and constructing a structured triple of the named entity, attribute and entity relationship of the fused knowledge graph. In view of the difficulty of entity linking in the prior art, the present application converts the knowledge graph into a low-dimensional dense vector representation, and constructs an inverted index library, etc., to reduce the difficulty.

The embodiment of the invention provides an archive management method and system based on big data analysis, and belongs to the technical field of data processing. The method comprises the steps of obtaining initial text data and initial image data corresponding to an initial user; performing deduplication processing on the initial text data to obtain target text data, and performing deduplication processing on the initial image data to obtain target image data; performing association relationship analysis on the target text data and the target image data to obtain a target relationship corresponding to the target text data and the target image data; performing text classification on the target text data to obtain a target type corresponding to the target text data; and performing data storage on the target text data and the target image data according to the target type and the target relationship to obtain a target storage result. The problems that in the prior art, the occupied storage space is increased and the data access and retrieval efficiency is reduced due to the fact that texts and images are often used as independent information units to be processed and an effective integration mechanism is lacked are solved.

The invention discloses a document key information retrieval method based on a large language model, a retrieval result is output according to a multi-stage input document and a question input by a user, and the method comprises the following steps: constructing a domain knowledge graph; constructing and training a question extraction model, and obtaining key information and intention of the question input by the user through the question extraction model; retrieving in a knowledge graph according to the key information and the intention to form paragraph description of the key information; screening the document and splitting the document into a plurality of text blocks to obtain a plurality of text blocks most related to the content input by the user; obtaining a corresponding text block with the highest embedding similarity by using a large language model; and integrating and reasoning the corresponding text blocks and paragraph descriptions by adopting a large language model to obtain information about the user input question in the document. The method has the beneficial effects that the accuracy and efficiency of information extraction can be improved, specific requirements of different fields can be met, and powerful technical support is provided for document automatic processing and knowledge management.

The invention discloses a comprehensive archive management system and management method, and relates to the technical field of archive management, the comprehensive archive management system comprises a data entry module, an analysis adjustment module, a picture processing module, a data processing module and a data storage module, and the matched comprehensive archive management method is provided. According to the method, analysis processing and angle adjustment processing are carried out on the obtained archive picture, the angle of the picture is corrected, follow-up checking by related technicians and computer recognition are facilitated, then stain removal processing is carried out, so that the content on the picture is clearer, character filling processing is further carried out, the obtained archive content is complete, and the efficiency is improved. In general, clear and complete archive picture data can be obtained, and the method has a good use prospect.

Embodiments of the invention provide a document processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a first text of a current dialogue; extracting N first features of the first text; splicing the N first features of the first text into a second feature; acquiring a first hash code of the second feature; in a current document extension tree, a second hash code is retrieved, the similarity between the second hash code and the first hash code is larger than or equal to a first threshold value, the document extension tree comprises a plurality of nodes, each node is associated with at least one document, and the hash codes of the documents associated with the same node are the same; under the condition that the second hash code is retrieved, a document associated with the first target node is determined as a document matched with the current dialogue, and the first target node is a node to which the second hash code belongs; and adjusting the first target node as a root node of the document extension tree. Therefore, according to the embodiment of the invention, the document retrieval speed can be increased.

The invention relates to the technical field of three-dimensional data processing, and discloses a three-dimensional grid data processing and multiplexing method and system, and the method comprises the steps: S1, obtaining and analyzing grid data, obtaining three-dimensional grid original data, and carrying out the format recognition, geometric analysis and topology analysis of the three-dimensional grid original data; s2, carrying out reuse-oriented standardization processing; s3, constructing multiplexing description information; s4, grid resource storage and index management; s5, determining a subset based on the request; s6, on-demand conversion and result generation; and S7, unified scheduling control. By constructing multiplexing description information containing a unique identifier, a hierarchy mapping relationship, a block mapping relationship, version information and a service adaptation mark, unified management and efficient multiplexing of grid data under different hierarchy and service scenes are realized, and unified data semantic support is provided for hierarchical retrieval, on-demand extraction and multi-service collaboration; therefore, the reusability of the three-dimensional grid data and the system cooperation efficiency are remarkably improved.

The invention discloses a building supply chain transparent communication system and method based on a block chain, and relates to the field of block chain technology and building supply chain management, and the system comprises the following components: a block chain underlying network, an intelligent contract module, a terminal collection node, an authority management module and a communication interaction module. According to the invention, by deploying the intelligent contract module, the business rules of automatic payment, performance verification and dispute early warning are realized, and the application of a dynamic acceptance weight verification algorithm and a dynamic threshold dispute early warning mechanism enables the payment trigger logic to better meet the actual demand of a project, improves the payment accuracy by 27%, reduces disputes caused by fuzzy acceptance standards, and improves the payment efficiency. Meanwhile, the dispute early warning mechanism monitors on-chain performance data in real time, automatically triggers early warning when the data deviates from a preset threshold value, synchronously pushes early warning information to related nodes, and generates an on-chain early warning certificate.

The invention relates to the field of big data processing, and discloses an intelligent SQL (Structured Query Language) database processing method, which comprises the following steps of: when first data is stored, judging whether content which is overlapped with the first data and exceeds a second threshold value of the content of the first data exists in second data or not; and when the first data is called, synthesizing the third data corresponding to the first data stored in the data list and the second data into the first data and outputting the first data. According to the method, the overlapped part of the first data and the second data with relatively large storage capacity is not stored, so that the storage capacity pressure of the SQL database is reduced, and the first data and the second data are distributed in a non-overlapped manner as the third data to be independently stored in the SQL database. By means of the method, storage with smaller capacity can be achieved, and it is guaranteed that the relative calling speed is high.

The invention relates to an atmospheric ocean data retrieval optimization method and device based on a column database. The method comprises the following steps: carrying out multi-dimensional on-demand cutting extraction and data decoding on historical atmospheric ocean data; in the element dimension, performing data storage and element dimension data retrieval optimization in a determinant database through a single element data table obtained by element segmentation; in the spatial dimension, data storage in a determinant database is extracted through metadata information, and spatial dimension data retrieval optimization is carried out based on constructed spatial codes; in the time dimension, time codes obtained through time discretization subdivision are subjected to data storage in a determinant database and time dimension data retrieval optimization; on the basis of atmospheric ocean data retrieval optimization of element dimensions, spatial dimensions and time dimensions, data storage is further carried out based on data blocks with fixed sizes. By adopting the method, the retrieval and access efficiency of massive historical atmospheric ocean data can be improved.

The invention relates to the technical field of question and answer text data processing, and provides a question and answer text data processing method and device, electronic equipment and a medium. The method comprises the steps of obtaining a question text sequence set; performing identification processing on each question text sequence in the question text sequence set through a large language model to obtain an answer text sequence corresponding to each question text sequence; determining a positive sample question text sequence corresponding to each question text sequence according to each question text sequence and the answer text sequence corresponding to each question text sequence; generating a corresponding negative sample problem text sequence based on the positive sample problem text sequence corresponding to each problem text sequence; and training a vector retrieval model based on the positive sample problem text sequence and each negative sample problem text sequence corresponding to each problem text sequence by adopting a loop iteration mode. By improving the training and application mode of the vector retrieval model, the accuracy and efficiency of information retrieval are remarkably improved.

This application provides a data storage method, apparatus, and business system across an HBase cluster. The method includes: upon receiving a data upload message, obtaining the location information, first time information, and first identification information of the data to be stored; encoding the location information, first time information, and first identification information to obtain an index number, which uniquely identifies the data to be stored; reversing the index number to obtain the row key value of the data to be stored; and storing the data to be stored in a first target partition on a target server according to the row key value. The first target partition is the storage partition corresponding to the byte of the first identification information in the row key value, and the byte order of the row key value and the index number is reversed. This method solves the problem in existing technologies where storing files sequentially in the same area causes hotspot phenomena and reduces data storage speed.

The invention belongs to the technical field of file retrieval. According to the sensitive file retrieval method and system based on the white list database, before file retrieval and detection are started, the white list database is queried, and whether a file path is recorded in a record which passes detection of the white list database or not is judged; if the file path exists in the detected record of the white list database, directly skipping a file corresponding to the file path, and entering a detection process of a next file; otherwise, executing strategy detection; after strategy detection is passed, file paths and corresponding strategy identifiers are recorded in a white list database, and files in the white list database are directly skipped during retrieval; according to the method, after the file passes through the detection strategy, the path of the file is recorded in the white list in the database, and when the same detection strategy is executed subsequently, the system can directly skip the recorded file, so that the retrieval efficiency is remarkably improved.

The present invention provides a method and system for retrieving data of a pig gene bank, and a storage medium. When a preset condition is met, a statistical analysis of a pig gene bank query record is triggered, hot spot information of the pig gene bank corresponding to the query request is obtained, and an index tree of the hot spot field is constructed; the order of the fields in the joint index is determined according to the number of occurrences of the fields in the hot spot joint field, and a B+ tree is constructed according to the order as the index tree of the hot spot field. When a user queries the pig gene bank, it is determined whether to query through the index tree according to the SQL statement corresponding to the query request. If so, the query is made through the index tree, otherwise a full table scan query is performed. The present invention can automatically construct an index of the gene bank when the preset condition is met, and two different index trees are constructed by judging the hot spot information, thereby improving the efficiency of searching the index, and thus improving the query speed of the pig gene bank.

The invention discloses a digital identity identification method and system based on an industrial internet and a medium, belongs to the technical field of industrial internet, and is used for solving the technical problems that an existing identity authentication system faces a centralized management bottleneck and is insufficient in safety and poor in privacy protection. The method comprises the following steps: performing identity registration in an identity registration module based on personal identity information to generate a digital identity label; storing the generated digital identity label and the corresponding personal identity information in a distributed storage module; performing authority control on the user according to the digital identity label; and based on the identity verification request, retrieving corresponding personal identity information in the distributed storage module, and performing identity authentication. A unique digital identity label is generated for each user, the uniqueness of the identity is ensured, the risk of false use of the identity is avoided, the data is stored on a plurality of distributed nodes, single-point faults and data tampering are avoided, and the safety and reliability of data storage are improved.

The invention provides a method and device for establishing a space grid system and a remote sensing image processing system. The method belongs to the technical field of remote sensing images, and carries out unified structured management on remote sensing image data by constructing a three-dimensional holographic management framework composed of a time grid, a space grid and a spectrum grid. The space grids adopt global subdivision grids and are endowed with unique codes, the time grids establish indexes according to years, seasons and months, and the spectrum grids are divided according to professional spectrum segments. The corresponding device comprises a data receiving module, a grid coding module, a storage module, a scheduling module and a processing module. The system supports micro-service cluster deployment, visual low-loss compression and open API integration. According to the method, pixel-level scheduling, dynamic processing and real-time service release of the remote sensing data are realized, the bottlenecks that the traditional file-level management efficiency is low, the preprocessing is tedious, and the instant service demand of massive high-resolution images is difficult to meet are overcome, and the data management, processing and service efficiency is remarkably improved.

This invention provides a database optimization method and terminal for storing game items. The method involves obtaining general attributes of item data and extended attributes for each type of item data; constructing a general table based on the general attributes and an identifier field; and constructing an extended table based on the extended attributes and the identifier field. An extensible key-value pair field is added to the extended table to store variable attributes. This invention avoids the redundancy problem of distributing general attributes of different types of item data across multiple tables by storing them in a general table, thus improving retrieval speed. Furthermore, the addition of a key-value pair field to the extended table to store variable attributes allows for direct addition of new attributes using the key-value pair field without modifying the extended table structure, further reducing the impact on the original table structure, lowering the difficulty of extending attributes, and increasing the flexibility of modifying item data.

Embodiments of the present application disclose a picture retrieval method and device, a storage medium and an electronic device, and relate to the technical field of picture processing. According to a feature point code table, sample picture feature points in each sample picture are quantitatively coded, and a same picture index is constructed according to sample quantization values of the sample picture feature points. After target picture feature points of a target picture are extracted, the target picture feature points can also be quantitatively coded according to the feature point code table to obtain target quantization values, and then a sample picture corresponding to the target quantization values is searched in the same picture index, that is, a retrieval result corresponding to the target picture is obtained. Since the calculation amount of the quantitative coding is much smaller than that of directly comparing the feature points in the sample picture and the target picture, the calculation amount when retrieving similar pictures can be reduced, and the retrieval speed when retrieving similar pictures can be effectively improved.

This invention provides an intelligent storage chip integrating an edge computing data processing system, relating to the field of electronic digital data processing. It includes a data acquisition module, an edge computing module, a data storage module, and a chip management module. The data acquisition module receives externally input information data; the edge computing module performs computational analysis on the data; the data storage module stores the processed data; and the chip management module controls and manages the chip's computational and storage functions. By adding an edge computing module, this chip utilizes computing power to process edge tasks while simultaneously managing data storage, effectively improving the retrieval efficiency and access speed of stored data.

The invention discloses a sensitive data governance method based on big data analysis, which comprises the following steps: firstly, establishing a dynamic data governance framework including dynamic role and authority management, data consanguinity recording, automatic data discovery and classification and data security management; a hierarchical management data center is established, and standardized management of data is realized; and a self-adaptive data governance strategy: constructing a data governance strategy based on big data analysis for the dynamic data governance framework. According to the invention, intelligent role and authority management, comprehensive data consanguinity recording, automatic data discovery and classification and enhanced data security measures are realized, and a governance strategy can be continuously optimized according to a real-time data analysis result, so that the efficiency, accuracy and security of data governance are improved.

A storage device is disclosed, comprising a memory and a computing unit, wherein the memory is used to store first characteristic information of at least one storage object, at least one storage data corresponding to each storage object, and storage information of the at least one storage data, and the first characteristic information of the storage object is associated with the storage information of the at least one storage data; the computing unit is used to obtain second characteristic information of the retrieval data, determine target characteristic information according to the comparison result between the second characteristic information and the first characteristic information in the memory, and obtain target retrieval data corresponding to the storage information according to the storage information associated with the target characteristic information. The storage device can access the corresponding storage data using the first characteristic information as an index, store the first characteristic information of the storage object and the storage data in different areas, reduce data handling, and improve the speed of data retrieval.

The present invention relates to the technical field of text processing, and in particular to a standard intelligent association method based on text mining, comprising: pre-processing each text file to obtain corresponding vocabulary sets; obtaining corresponding keyword sets based on each vocabulary set; pre-screening the text file to be mined based on the keyword set to obtain a number of coarse-grained text files to form a coarse-grained file set; performing pointing processing on each keyword of each keyword set in the coarse-grained file set to form each keyword group set; calculating the similarity between the several keyword sets corresponding to the coarse-grained file set based on each keyword group set to obtain a fine-grained file set; determining the association logic map of each text file in the fine-grained file set according to the attributes of the keyword set corresponding to each text file in the fine-grained file set. The present invention determines similar files by a two-step method of coarse-grained screening and fine-grained screening, thereby improving the accuracy of similarity calculation.

The invention discloses a part category identification method and system based on deep learning and a knowledge graph, and the method comprises the steps: obtaining a part picture data set, and carrying out the preprocessing and data enhancement of the data set; a recognition model is constructed and trained, the trained model is used to perform feature extraction on the part pictures and store the features, and feature vectors contained in different part pictures are obtained; feature information is extracted from a part picture needing to be recognized through a model, and the feature information is compared with feature vectors of existing pictures to obtain a most similar result; and automatically searching corresponding information in the knowledge graph by using the query statement according to a comparison result, and feeding back the information. The detailed information is obtained from the knowledge graph by identifying the part picture, and the method can play a role in the aspects of equipment operation and maintenance and the like. According to the method, a large amount of required information is rapidly fed back according to a single picture, the information retrieval speed under the scenes of equipment maintenance, part replacement, automatic overhaul and the like is effectively increased, and the method has good reference significance for improving the part information query efficiency.

This invention discloses a method for identifying the operating conditions of a pumping unit based on orthogonal depth hashing and graph reasoning, comprising: (1) acquiring, labeling, classifying, and training an orthogonal depth hashing network; (2) using the trained orthogonal depth hashing network to extract features from each dynamometer in the sample set, generating and saving bit-independent binary hash codes; (3) constructing an inverted index library based on the hash codes of different sample graphs, storing the storage node information of different sample graphs in the corresponding hash buckets; (4) acquiring the load and displacement data collected by the dynamometer of the pumping well to be detected, and drawing the dynamometer; (5) performing pumping condition fusion reasoning judgment based on a graph convolutional network with dual sorting. The binary hash codes generated by this invention are mutually uncorrelated in each dimension, maximizing the information entropy of the encoding. This ensures the uniform distribution of massive samples in Hamming space, significantly improving the recall rate and retrieval speed in the coarse screening stage.