Patents
Literature
20results about How to "Optimize storage structure" patented technology
Efficacy Topic
Property
Owner
Technical Advancement
Application Domain
Technology Topic
Technology Field Word
Patent Country/Region
Patent Type
Patent Status
Application Year
Inventor
Method, system and equipment for generating SQL (Structured Query Language) statement based on large model
PendingCN119127913ARealize the structureOptimize storage structureBiological modelsInference methodsEngineeringMetadata description
The invention relates to the field of natural language processing, and discloses an SQL statement generation method, system and device based on a large model. Respectively carrying out vectorization processing and document processing on the metadata and the metadata description text information from the database, and storing the metadata and the metadata description text information into a vector database and a full-text retrieval library; in response to the query instruction, respectively performing query matching on the vector database and the full-text retrieval library, screening matching results which are ranked in the front and are preset ranks according to the similarity from high to low, and determining an optimal SQL statement list and optimal metadata; on the basis of the current language of the query instruction, generating a cue word of the current language in combination with the optimal SQL statement list and the optimal metadata; the large model of the preset type is determined according to the language type of the cue word, the preset large model is called to conduct reasoning on the cue word, and the SQL statement is generated. Based on the large model and the generative AI technology, the accuracy of the generated SQL statement is improved, the use threshold of non-technical personnel is lowered, and the query efficiency of the SQL statement is also improved.
Owner:广域铭岛数字科技有限公司 +1
Data acquisition method and device of industrial Internet of Things, and storage medium
ActiveCN119622221Aaccurate collectionIntegrity guaranteedInput/output to record carriersResource allocationIndustrial equipmentData acquisition
The invention relates to the field of technical data acquisition, and particularly discloses a data acquisition method and device for an industrial Internet of Things and a storage medium, and the method comprises the following steps: S1, constructing a dynamic sensor network, and collecting the operation data of industrial equipment in real time through a multi-modal sensor; s2, dynamically adjusting a data acquisition frequency # imgabs2 # based on an environment noise coefficient # imgabs0 # and an equipment health index # imgabs1 #; s3, performing hierarchical compression and encryption on the acquired data, and allocating a compression rate # imgabs4 # and encryption strength # imgabs5 # according to data type importance # imgabs3 #; s4, executing load balancing through an edge computing node, and distributing a computing task to an optimal node; and S5, storing the processed data into a distributed database according to spatial-temporal correlation, and generating a multi-copy disaster recovery index. Through dynamic acquisition, intelligent processing, optimized storage and adaptive strategy adjustment, accurate and efficient data acquisition is realized, data security and processing performance are flexibly guaranteed, storage query efficiency and disaster tolerance capability are improved, and storage resources are utilized to the maximum extent.
Owner:CHENYANG CHENGSHUO ELECTRONIC TECHNOLOGY CO LTD
Power plant data management method and system based on SIS system
PendingCN119537358AImprove management efficiencyimprove accuracyData processing applicationsDatabase management systemsData integrityAnalysis data
The invention discloses a power plant data management method and system based on an SIS system, and relates to the technical field of data management, and the method comprises the steps: obtaining power plant operation data, and carrying out the unified formatting processing of the power plant operation data; preprocessing the formatted data, and removing noise and invalid data; performing quality verification on the preprocessed data, and checking data integrity and consistency; classifying, marking and storing the data subjected to quality verification into a database; and extracting and analyzing data from the database according to operation requirements of the power plant, and generating a decision report. According to the power plant data management method based on the SIS system, through unified formatting and time alignment processing, data format difference and time disorder between different devices are eliminated, and consistency and accuracy of data are ensured. And noise elimination, interpolation processing and anomaly detection are adopted, so that the data quality is guaranteed, and the reliability of an analysis result is improved. And aiming at data with different access frequencies.
Owner:HUANENG JINGTAI THERMAL POWER CO LTD
Data storage method and system based on large model
ActiveCN119148946AImprove storage efficiencyImprove access speedInput/output to record carriersData compressionData set
The invention relates to the technical field of large-scale data management, in particular to a data storage method and system based on a large model, and the method comprises the following steps: collecting multiple items of data in a large model data set, recording the occurrence frequency of each type of data, calculating the entropy value of each type of data, determining the volatility and diversity of the data, and generating an entropy value calculation result. According to the method, the data are sorted and stored in a classified mode according to the entropy values, so that the high-entropy-value data are located in a high-density storage area, the low-entropy-value data are stored in a low-density area, the utilization rate of the storage space is optimized, unnecessary storage cost is reduced, stratum superposition is simulated according to hierarchical storage of the data and sorting of time stamps, and the data storage efficiency is improved. The storage structure is optimized, rapid positioning and access of new data are also facilitated, adjustment of data compression and storage parameters simulates the geologic stratum compaction process, the storage capacity is effectively managed, the energy efficiency of a data center is enhanced, and the response capability to a complex data environment is enhanced.
Owner:BEIJING SHENZHOU BANGBANG TECH SERVICE CO LTD
ClickHouse-based fused CDN (Content Delivery Network) service data processing method, device and equipment
PendingCN119271888AGuaranteed accuracyImprove efficiencyDatabase updatingDatabase management systemsDatasheetEngineering
The invention discloses a ClickHouse-based fused CDN (Content Delivery Network) business data processing method, device and equipment, and the method comprises the steps: obtaining business data of a corresponding fused CDN manufacturer through calling a data interface of each fused CDN manufacturer in real time; performing preprocessing including data format standardization, field expansion and invalid data filtering on the service data by using Spark, packaging the preprocessed service data and sending the packaged service data to a Kafka cluster; the method comprises the following steps: creating a distributed data table based on a ReplacingMergeTree engine in a ClickHouse database, consuming service data in a Kafka cluster in real time by utilizing Spark, and writing the service data into the distributed data table of the ClickHouse database in batches; and retrieving target business data from the ClickHouse database according to a data query request of a user, and converting the target business data into a JSON format to serve as a query result to be returned. The query performance, the storage efficiency, the expandability and the write-in performance can be improved.
Owner:XIAMEN ANJIEYUN NETWORK CO LTD
Data management method and system for multi-modal database
ActiveCN119149783AOptimize storage structureImprove generalization abilityBiological modelsOther databases indexingAnalytic modelEngineering
The invention discloses a data management method and system for a multi-modal database, and relates to the technical field of data management. A data management system of a multi-modal database comprises a multi-modal data analysis module, a multi-modal data storage module and a multi-modal data updating module. According to the method, deep-level features are extracted by using the multi-modal data feature analysis model, so that the accuracy and depth of data identification are improved; database storage allocation is carried out through a feature matching technology, a data storage structure is optimized, and query efficiency is improved; a feature adaptive threshold factor Z and CNN model improvement based on a knowledge graph are introduced, so that the generalization ability and adaptability of the model are enhanced; the updating model is managed through the multi-modal database, dynamic updating of database partitions is achieved, user operation changes are responded, timeliness and correlation of the database are kept, and user experience is enhanced.
Owner:HANGZHOU HUABAI COMPUTER CO LTD
Archive management method and system based on big data analysis
PendingCN119669326Aeasy to integrateimprove relevanceDatabase management systemsBiological modelsText categorizationData access
The embodiment of the invention provides an archive management method and system based on big data analysis, and belongs to the technical field of data processing. The method comprises the steps of obtaining initial text data and initial image data corresponding to an initial user; performing deduplication processing on the initial text data to obtain target text data, and performing deduplication processing on the initial image data to obtain target image data; performing association relationship analysis on the target text data and the target image data to obtain a target relationship corresponding to the target text data and the target image data; performing text classification on the target text data to obtain a target type corresponding to the target text data; and performing data storage on the target text data and the target image data according to the target type and the target relationship to obtain a target storage result. The problems that in the prior art, the occupied storage space is increased and the data access and retrieval efficiency is reduced due to the fact that texts and images are often used as independent information units to be processed and an effective integration mechanism is lacked are solved.
Owner:HUAIAN YIDOU TECHNOLOGY CO LTD
Safety verification device for electronic archive data management
InactiveCN119091934APrevent tamperingprevent leakageDigital data protectionRecord information storageData integrityConfidentiality
The invention relates to the technical field of archive management, and provides a safety verification device for electronic archive data management, which comprises a storage box, a storage device, a safety verification module, an access control unit, a display module and an alarm unit, a plurality of groups of connecting rotating shafts are mounted on the outer side wall of the storage box, and the outer side walls of the plurality of groups of connecting rotating shafts are rotatably connected with a sealing door plate; the outer side wall of the sealing door plate is fixedly connected with a connecting handle, and the sealing door plate is used for sealing the storage box; the storage device is installed in the storage area of the storage box and used for storing electronic archive data, and a connecting socket is installed on the outer side wall of the storage device; through the combination of data integrity verification and a data encryption technology, electronic archive data are effectively prevented from being tampered and leaked, the integrity and confidentiality of the data are ensured, the flexibility and reliability of data integrity verification are improved by supporting multiple Hash algorithms and regular automatic verification, and the security of the electronic archive data is improved. And the electronic archive data can be kept complete and correct at any time.
Owner:LAIWU VOCATIONAL & TECHNICAL COLLEGE
Real-time storage optimization method based on block data
PendingCN121958403ASolve floodingOptimize storage structureResource allocationDatabase management systemsMemory footprintParallel computing
The invention discloses a real-time storage optimization method based on block data, which relates to the technical field of big data processing and distributed storage, introduces a block data concept, realizes block-by-block writing, in-block ordering, block-level ordering and an intelligent flush trigger mechanism, and realizes real-time writing with high throughput, low delay and low memory occupation. The method specifically comprises the steps that block data is introduced into a data memory model to serve as a basic processing unit, multiple rows of data are aggregated into a block according to an aggregation key CKValue, row data in the block are sequenced according to a primary key PK, and write-in, namely aggregation is achieved; writing data, aggregating according to blocks, and uniformly sorting in the blocks; one block, namely multiple rows of data, generates an FRC file, and small files are reduced; exception processing: supporting block-level recovery to skip a failed block; the core problems of too high memory occupation, large CPU consumption, high writing delay and the like in a high-concurrency and high-throughput real-time writing scene can be solved, and the method is widely applied to scenes with extremely high requirements on data real-time performance and system stability.
Owner:XIAN FIBERHOME SOFTWARE TECH CO LTD
Vector matrix multiplication acceleration method, unit, acceleration unit and system on chip
ActiveCN114398593BOptimize storage structureeasy to handleDigital data processing detailsComplex mathematical operationsComputer hardwareComputational science
Embodiments of the present application provide a vector matrix multiplication acceleration method, unit, acceleration unit and system on chip, the method is applied to a fifth generation reduced instruction set processor, the method comprises the following steps: obtaining the number N of elements of a first matrix stored in a vector register according to the data length of the elements of the first matrix and the length of the vector register, N is a positive integer; performing rearrangement processing on the first matrix according to the number N to obtain a third matrix which can process N data in parallel; and performing multiplication calculation on the vector and the third matrix by using a single instruction multiple data stream instruction. The present scheme can improve the operation speed of vector matrix multiplication.
Owner:C SKY MICROSYST CO LTD
City planning monitoring core database construction method
InactiveCN119311914AImprove ventilationeasy to handleVideo data indexingCharacter and pattern recognitionTraffic flowInternet of Things
The invention relates to the technical field of data monitoring and database construction, in particular to an urban planning monitoring core database construction method. The method comprises the following steps: acquiring multi-source city data by using a sensor; real-time traffic flow collection is carried out through the unmanned aerial vehicle and the camera, and a real-time traffic flow monitoring video is obtained; performing spectrum separation on the real-time traffic flow monitoring video to generate image spectrum data; performing image enhancement processing on the real-time traffic flow monitoring video according to the image spectrum data to obtain an enhanced image; performing video reconstruction on the enhanced image to generate a real-time traffic flow monitoring enhanced video; video frame feature extraction is carried out on the real-time traffic flow monitoring enhanced video, and a real-time monitoring image matrix is constructed; therefore, through the Internet of Things technology, the data analysis technology, the image processing technology and the prediction analysis technology, the prediction capability for the potential problem of the multi-source city data is improved, and the data retrieval speed of the core database is increased.
Owner:SHANGHAI YOUYU IND CO LTD XINYI BRANCH
Enterprise information management method and system based on cloud platform
InactiveCN119398531AGuaranteed accuracyOptimize storage structureMathematical modelsInput/output to record carriersData setPerformance index
The invention discloses an enterprise information management method and system based on a cloud platform, and relates to the technical field of data migration. According to the method, enterprise performance index data and metadata are extracted in real time and preprocessed, and a management performance data set is obtained. Summarizing and calculating to obtain a data migration coefficient sqy according to the information data group, and performing migration evaluation on the data migration coefficient sqy and a preset data migration threshold A to determine whether the data needs to be migrated to a cold storage layer or not. A comprehensive migration risk assessment coefficient QYF (t) is calculated and compared with a preset migration risk threshold value B; and when the migration risk exists, constructing a migration risk regulation and control model through a Bayesian network model, calculating a risk control regulation coefficient fxt (t), and regulating the migration risk. And carrying out migration risk assessment on the adjusted risk coefficient QYT (t) and the migration risk threshold B to ensure that the data is safely migrated to the cold storage layer. According to the method, the security and efficiency of data migration are improved, and enterprise information management is optimized.
Owner:JIANGXI UNIVERSITY OF FINANCE AND ECONOMICS
APT (Advanced Persistent Threat) data compression and detection method fusing spatial-temporal characteristics and graph embedding technology
PendingCN119249311AImprove processing efficiencyImprove acceleration performanceEnergy efficient computingData compressionAttack
The invention relates to the field of network security and machine learning, in particular to an APT data compression and detection method fusing spatial-temporal characteristics and a graph embedding technology. The method comprises the following steps: constructing a traceability graph by using compressed log data, and creating a bipartite graph according to the traceability graph; executing a first random walk process based on the traceability graph to obtain a walk matrix; based on the bipartite graph, executing a second random walk process for multiple times according to the normalized walk matrix to generate a plurality of node sequences, and obtaining an embedded vector of each node in the node sequences; the average value of the embedded vectors of each node is calculated to obtain the final embedded vector of each node, a clustering method is adopted to obtain a cluster center, and when the distance between the final embedded vector of the current node and the cluster center exceeds a threshold value, the current node is identified as an abnormal node. According to the method, the storage structure is optimized, the requirement of the storage space is reduced, the automation capability of the system is improved, and the analysis efficiency and accuracy of APT attack tracing are improved.
Owner:ZHEJIANG UNIV OF TECH
Wardrobe convenient to move
ActiveCN224112320USolve the inconvenience of movingReduce risk of damageWardrobesFeetStructural engineeringMechanical engineering
Owner:罗宝
A data management method and system for multimodal database
ActiveCN119149783BOptimize storage structureImprove generalization abilityBiological modelsOther databases indexingAnalytic modelEngineering
The present invention discloses a data management method and system for a multimodal database, and relates to the technical field of data management. A data management system for a multimodal database includes: a multimodal data analysis module, a multimodal data storage module, and a multimodal data update module. The present invention improves the accuracy and depth of data recognition by extracting deep-level features using a multimodal data feature analysis model; optimizes the data storage structure and improves query efficiency by performing database storage allocation using feature matching technology; introduces a feature adaptive threshold factor Z and a CNN model improvement based on a knowledge graph to enhance the generalization ability and adaptability of the model; and realizes dynamic updates of database partitions through a multimodal database management update model, responds to changes in user operations, maintains the timeliness and relevance of the database, and enhances user experience.
Owner:HANGZHOU HUABAI COMPUTER CO LTD
A method for storing big data databases in an energy management system
ActiveCN115658691Bcapacity controltimely depositError detection/correctionEnergy efficient computingData classEnergy management system
This invention provides a method for storing big data databases in an energy management system. The database storage process is as follows: S1, create a heap table according to the data type; S2, store the data obtained from the terminal into the heap table of the corresponding data type; S3, determine if it is time for the execution of the scheduled task on the next natural day, and if so, proceed to step S4; S4, convert the heap table generated on the previous natural day into an index-organized table and store it in the database; S5, complete the storage conversion process; S6, perform a backup operation. The beneficial effects of this invention are: a method for storing big data databases in an energy management system can effectively control the storage capacity of a single database table within an ideal range.
Owner:TIANJIN KEYVIA ELECTRIC CO LTD
Method and system for dynamically determining copy position of HDFS (Hadoop Distributed File System) based on node historical state
PendingCN119513061Aload balancingimprove accuracyDigital data information retrievalEnergy efficient computingDistributed File SystemDistributed computing
The invention provides a method and system for dynamically determining a copy position of an HDFS (Hadoop Distributed File System) based on a node historical state, and the method comprises the steps: constructing a node load evaluation index, recording the current load of a node at regular time, carrying out the training through an LSTM neural network, and constructing a node load prediction model for predicting the future load of a cluster node, determining a copy creation position of the file in the cluster according to the predicted value; when a file copy is written, the load of each node at the next moment is predicted, and nodes with relatively low loads in the future are written preferentially. According to the method, the write-in positions of the file copies in the HDFS cluster are dynamically set, so that load balancing in a heterogeneous environment can be realized. Compared with an existing fixed copy strategy, the method is more flexible, and the availability of the system is enhanced. Compared with an existing replica placement strategy based on the current node state, burst state interference is eliminated, the balance is higher, meanwhile, the prediction accuracy is higher through the neural network, and the overall performance of the system can be better improved.
Owner:CHINA TELECOM CLOUD TECH CO LTD
Method, system and equipment for detecting database naming specifications and medium
PendingCN119127862Aachieve consistencyeasy to operateNatural language translationSpecial data processing applicationsTable (database)Engineering
The invention discloses a method, system and device for detecting database naming specifications and a medium. The method comprises the steps that meta-information is synchronized, and the meta-information is synchronized from a database of a service system; standard vocabulary query: querying whether the description information has a corresponding English name in a standard vocabulary library; performing word segmentation processing, namely performing word segmentation processing on the description information of which the corresponding English name is not found by using a word segmentation device; vocabulary filtering and translating: filtering the segmented vocabularies, obtaining English abbreviations corresponding to the vocabularies, and splicing the English abbreviations into field name candidates; result comparison and judgment: comparing results after word segmentation of each word segmentation device; and naming specification judgment and translation failure processing. By means of the method, it can be ensured that table names and field names in the database follow the unified naming specification, the standardization level of the whole data warehouse or database is improved, data consistency and interoperability between different systems can be achieved, and element names in the data warehouse are clearer and more significant due to the unified naming specification.
Owner:HENAN ZHONGYUAN CONSUMER FINANCE CO LTD
Event content verification method, apparatus and storage medium based on large model
This invention discloses a method, apparatus, and storage medium for event content verification based on a large model, belonging to the field of data processing technology. The method includes: identifying events in target text using a large model, rewriting the events in the target text according to target rewriting rules to obtain reference events; the target text is standard data that has been reviewed; storing the reference events in an event knowledge base; retrieving target reference events associated with the event to be verified from the event knowledge base based on the event to be verified; and comparing the event to be verified and the target reference events using the large model to obtain a comparison result characterizing whether the event to be verified is correct. This application utilizes the powerful semantic understanding capability of the large model to accurately identify events in standard data and rewrite the events to form structurally unified and semantically clear reference events, thereby constructing a high-quality event knowledge base and improving the accuracy of event content verification.
Owner:BEIJING TOPWALK INFORMATION TECH CO LTD +1
A data storage method and system based on large model
The present invention relates to the technical field of large-scale data management, specifically to a data storage method and system based on a large model, comprising the following steps: collecting multiple data in a large model data set, recording the number of occurrences of each data, calculating the entropy value of each data, determining the volatility and diversity of the data, and generating an entropy value calculation result. In the present invention, by sorting and classifying the data according to the entropy value, high entropy value data is located in a high-density storage area, while low entropy value data is stored in a low-density area, optimizing the utilization of storage space, reducing unnecessary storage costs, simulating stratigraphic superposition according to the hierarchical storage of data and sorting by time tags, not only optimizing the storage structure, but also facilitating the rapid positioning and access of new data, data compression and storage parameter adjustment simulate the compaction process of geological layers, effectively managing storage capacity, enhancing the energy efficiency of data centers, and enhancing the responsiveness to complex data environments.
Owner:BEIJING SHENZHOU BANGBANG TECH SERVICE CO LTD