31 results about "Hierarchical storage management" patented technology

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical treatment and health and the like, and discloses a data hierarchical storage method, device, equipment and medium based on access characterization. Writing the data into a matched first target storage device according to a preset device performance corresponding relation; after the data are stored, continuously monitoring subsequent access characteristics of the data, and dynamically judging whether the heat state is changed or not; and if the popularity state is changed and a specific migration condition is met, redistributing the second target storage device and completing data migration operation so as to realize automatic hierarchical storage management of cold and hot data. By acquiring the data access representation and dynamically identifying the popularity state, hierarchical migration of data with different popularity among storage devices with different performances is realized, the resource use efficiency and access performance of a storage system are improved, and the storage cost is reduced.

A device for real-time performance monitoring and optimization in software systems, comprising: a data acquisition module configured to acquire performance measurements from various software core components in real time, wherein the data acquisition module also includes a variety of sensors connected to the software system to acquire measurement data including CPU utilization, memory consumption, I / O operations and network latency; a data processing unit that is operationally connected to the data acquisition module, wherein the data processing unit is configured to receive and preprocess the acquired measurement data, apply filtering mechanisms to remove irrelevant or redundant data, and normalize the data for further analysis; a power analysis unit configured to analyze the processed measurement data in real time, wherein the power analysis unit includes a predictive model to identify potential power bottlenecks and a rule-based logic system that compares current power measurements with predefined thresholds to detect anomalies; an optimization module that dynamically adjusts software system configurations based on the output of the performance analysis unit, wherein the optimization module includes a feedback mechanism that fine-tunes software system parameters by employing automated tuning actions, such as adjusting resource allocation and load balancing; a user interface module that is operationally connected to the optimization module and the data acquisition module, wherein the user interface module is configured to display real-time performance metrics and optimization results to an end user via a graphical interface, including visual representations of the current system status and historical performance trends; a data storage unit configured to log performance metrics, optimization measures, and analysis results for long-term trend analysis, the data storage unit also including a hierarchical storage management system that stores data at different levels of detail depending on the age and relevance of the data.

The invention provides an electric power information fusion system, which comprises a data management module, a mode layer processing module and a data layer processing module, and is characterized in that the data management module is connected with the mode layer processing module, receives electric power multi-mode data from different sources, carries out hierarchical storage management on the electric power multi-mode data, obtains hierarchical storage data, and stores the hierarchical storage data in the mode layer processing module; the mode layer processing module is connected with the data layer processing module, analyzes the hierarchical storage data to obtain electric power professional knowledge information, constructs the electric power professional knowledge information based on a graph database to obtain a knowledge graph mode layer, and performs knowledge calculation on the knowledge graph mode layer to obtain a standardized knowledge graph mode layer; and the data layer processing module carries out data filling on the standardized knowledge graph mode layer based on hierarchical storage data. The data fusion method and device are high in data analysis accuracy, high in calculation efficiency, small in resource consumption and stable in data fusion effect.

The invention discloses a distributed medical image retrieval and hierarchical storage management system and method. The system comprises an interface processing module, a distributed image storage cluster, a routing forwarding module, a metadata management module and a storage management module. The interface processing module receives the image retrieval request, generates a routing identifier for positioning the storage position of a target medical image file and sends the routing identifier to the routing forwarding module; and the routing forwarding module queries a mapping relationship between the medical image file and the storage node and / or the storage hierarchy based on the routing identifier, determines a first storage node and forwards the request, so that the first storage node returns a target medical image file. And the storage management module obtains the access statistical information, determines a target storage level and / or a target storage node, migrates the medical image file, and triggers updating of the mapping relation after migration is completed. Therefore, cross-node accurate positioning and unified retrieval are realized, positioning consistency is kept after file migration, and hierarchical storage management based on access conditions is supported.

The invention provides a unified context and multi-level memory cooperation system and method for an LLM agent, and belongs to the field of artificial intelligence. The system comprises a unified context module which is used for aggregating and structuring all input information required for organizing agent operation, and providing an integrated context view for an LLM agent core; the LLM agent core is used for executing task planning, tool calling, result evaluation and response generation based on the context provided by the unified context module; and the multi-level memory system is connected with the unified context module and the LLM agent core and is used for hierarchically storing, managing and feeding back experience and knowledge of the agents. According to the method, the problems that an existing LLM intelligent agent is limited in context, low in memory efficiency and lack of self-correction capacity are solved, and the intelligent level and reliability of complex task processing are remarkably improved.

The invention provides an intelligence agent adaptive acquisition method and system based on long memory and long-time scheduling, and relates to the technical field of data processing, the method comprises the following steps: 1, receiving an intelligence acquisition task, analyzing task parameters to construct a target object portrait, extracting and standardizing target features, and forming a structured target portrait; 2, based on the structured target portrait, abstracting previous similar tasks into experience triples, performing vectorization coding, storing the experience triples into an experience library, and constructing semantic retrieval indexes to implement hierarchical storage management; similar experience is matched from an experience library through semantic retrieval, and a self-adaptive acquisition strategy is generated in combination with similarity and a previous success rate. According to the invention, continuous and stable execution of long-period intelligence collection tasks is realized, and the collection success rate and recovery efficiency in a dynamic confrontation environment are improved.

The invention relates to the technical field of data processing, and discloses a fresh agricultural product full-link traceability management system, which comprises a data acquisition module used for acquiring original data representing the state of the data acquisition module; the artificial intelligence analysis module is used for processing the original data to generate an analysis result and a credibility score corresponding to the analysis result; an intelligent contract is deployed in the data write-in scheduling module, and the intelligent contract is used for receiving the analysis result and the credibility score; and the hierarchical block chain storage module is used for responding to the write-in instruction generated by the data write-in scheduling module. Through intelligent analysis and hierarchical block chain storage, quantitative evaluation and dynamic hierarchical storage management of source credibility of fresh agricultural product traceability data are realized, a main chain is effectively prevented from being polluted by abnormal data, a data closed-loop recheck mechanism is established, and the overall authenticity and reliability of traceability information are improved.

The invention provides a lightweight storage method and system for time series data of the Internet of Things, and relates to the technical field of data storage and compression of the Internet of Things, and the method comprises the steps: collecting multi-mode time series data and mechanical vibration energy data generated during the operation of equipment of the Internet of Things; extracting vibration frequency characteristics from the mechanical vibration energy data, and generating a time sequence vibration signal based on the vibration frequency characteristics; performing space-time correlation on the time sequence vibration signal and the multi-modal time sequence data to identify a data activeness level; according to the data activeness level, performing sparse representation processing on the multi-modal time series data to obtain sparse representation data; and carrying out dynamic dictionary construction and coding on the sparse representation data by adopting a self-adaptive dictionary coding algorithm to generate compressed coding data, and carrying out hierarchical storage management on the compressed coding data to realize lightweight storage. According to the invention, the storage efficiency and the processing capability of the time series data of the Internet of Things edge device are improved.

Data resiliency in a cloud-based storage system, including: receiving, for storage within a first tier of cloud storage of the cloud-based storage system, one or more segments of data; generating, for each of one or more shards of data of the one or more segments of data, self-describing information for recoverability of the one or more shards of data; and storing, within a second tier of cloud storage of the cloud-based storage system, both the one or more shards of data and the generated self-describing information for recoverability of the one or more shards of data.

The invention relates to the technical field of data storage management, and discloses an Internet of Things mass data hierarchical storage management system and method based on popularity perception. Standard access frequency intervals required by different data popularity types of different industrial Internet of Things application scenes are adaptively matched according to target industrial Internet of Things application scene type identification information in combination with an intelligent search algorithm and different industrial Internet of Things standard data popularity type access frequency intervals based on big data storage; access frequency intervals of three data popularity types of hot, warm and cold are dynamically and scientifically set in different industrial Internet of Things application scenes; based on the real-time data popularity type identification information of the target industrial Internet of Things storage data, the data hierarchical storage scheme of the industrial Internet of Things real-time storage data is accurately analyzed in combination with the intelligent search algorithm and the hierarchical storage scheme data of different data popularity types of standard storage, and the data access response efficiency of the industrial Internet of Things is improved.

The embodiment of the invention provides an eMMC log dynamic hierarchical storage management method, system and device and a storage medium, and belongs to the technical field of computers. According to the scheme, an eMMC storage space is divided into a plurality of log partitions; performing proportional processing according to the residual erasure period and the rated erasure period of each log partition to obtain a corresponding residual life index; based on the partition wear level threshold value, partition level division processing is carried out according to the remaining life index, and the partition wear level is obtained; determining a currently used partition from a plurality of log partitions according to the partition wear level; a log level is set according to the partition wear level corresponding to the currently used partition, and the log level is used for determining the lowest log level allowed to be written into the currently used partition; and performing log storage management according to the log level and the current log level of the log information. According to the invention, the overall service life of the eMMC can be prolonged, and the resource utilization rate is optimized.

The invention provides a hierarchical storage management method based on land observation satellite data value, which belongs to the field of application in a land observation satellite data processing system, and comprises the following steps of: calculating an initial comprehensive weight score during data archiving by creating seven types of core data tables such as a satellite parameter table and a data value parameter table; determining the number of storage days of the online or near-line cluster in combination with a dynamic storage strategy table; obtaining user order update data return times in real time, and recalculating the weight score at regular time to dynamically adjust the storage strategy; and finally, realizing data cleaning in combination with the water and fire wire threshold of the storage device. The problems of low resource utilization efficiency and slow user data return caused by making a storage strategy only based on the number of storage days in the prior art are solved, and the storage resource utilization rate and the user data response efficiency are remarkably improved.

This invention relates to a method for calculating data temperature and performing hierarchical storage management, belonging to the field of computer data processing. Based on data access time, data access frequency, data attributes, and the attributes of the user initiating the access, a data temperature calculation model is constructed. This model manages the dynamic migration of data between a local hot database, a local cold database, and a remote cold database. Newly generated data and cold data with rising local temperatures are stored in the hot database. Periodic time-driven migrations are performed, storing data predicted to be migrated to the hot database in the next cycle and data with currently high temperatures in the hot database. Based on the benefit of storing unit data remotely compared to storing it locally, a decision is made regarding whether cold data needs to be migrated remotely. This invention can significantly reduce costs while ensuring data access performance, predict new data volume, balance hot storage space utilization and data migration overhead, and pre-migrate data by identifying periodic data migration patterns, avoiding time lag issues in migration.

The invention discloses an AI-based information security risk early warning system, and relates to the technical field of network security. The system extracts transmission layer features for the encrypted traffic through an entropy feature quantization unit and generates an entropy turbulence index; calculating, by a state scheduling unit, a resident energy value of the network connection based on a nonlinear thermodynamic model; executing serialized replacement and wake-up operation between the dynamic random access memory and the nonvolatile solid state disk through the hierarchical storage management unit, and executing space-time folding to eliminate a physical time interval; and finally, performing Granger causal verification on the spliced discontinuous historical state sequence through a discontinuous causal verification unit. The technical problem that continuous state tracking cannot be carried out on hidden attack behaviors crossing a long period under the hardware condition that gateway equipment memory resources are limited is solved, high-speed small-capacity memory resources are replaced with a low-speed large-capacity storage medium, and risk early warning is achieved on the premise that context logic integrity is guaranteed.

The application provides an intelligence agent adaptive collection method and system based on long memory and long-time scheduling, and relates to the technical field of data processing.The method comprises the following steps: step 1, receiving an intelligence collection task, analyzing task parameters to construct a target object image, extracting and standardizing target features, and forming a structured target image; step 2, based on the structured target image, abstracting past similar tasks into experience triples and performing vectorization coding to store in an experience library, constructing a semantic retrieval index to implement hierarchical storage management; matching similar experiences from the experience library through semantic retrieval, and combining similarity and past success rate to generate an adaptive collection strategy. The application realizes continuous and stable execution of long-period intelligence collection tasks, and improves the collection success rate and recovery efficiency in a dynamic confrontation environment.

The application relates to the technical field of railway traffic, in particular to a data temperature evaluation method and device for heavy-load railway electric service data, computer equipment and a storage medium. The method comprises the following steps: obtaining initial hotness evaluation values of a plurality of business data objects in heavy-load railway electric service and correlation relationship parameters between any two business data objects; determining a current hotness value of a target business data object based on access information, a historical hotness value and a time decay model of the target business data object; determining hotness conduction increments of other business data objects associated with the target business data object based on the correlation relationship parameters and the current hotness value of the target business data object, and updating the current hotness values of the other business data objects by combining the historical hotness values of the other business data objects and the time decay model; and performing hierarchical storage management on the plurality of business data objects based on the current hotness values of the business data objects.

The invention discloses a system capable of efficiently processing massive aviation bus data, which comprises a multi-bus protocol parallel acquisition module, a heterogeneous data analysis engine, a service calculation and statistical analysis module, a hierarchical storage management module and a resource scheduling and load management module, the multi-bus protocol parallel acquisition module comprises a protocol interface array, a many-core processing core and a dynamic buffer management unit; the heterogeneous data analysis engine comprises a configurable protocol analyzer, a data path selection and frame coding unit and a streaming data processing pipeline; the business calculation and statistical analysis module comprises a distributed real-time calculation unit, a multi-layer statistical analysis unit and a semantic analysis unit; the hierarchical storage management module adopts a three-layer storage architecture, and the resource scheduling and load management module comprises a load prediction unit, a self-adaptive resource allocation unit and a model migration and knowledge sharing unit. The problems that the real-time performance of aviation multi-bus data processing is insufficient, the protocol compatibility is limited, and the system resource utilization rate is low are solved.

The invention relates to the technical field of data storage and computers and the technical field of videos, and provides a hierarchical storage management system, a repeated data deletion method, equipment and a medium. Each storage layer in the quantity management storage system is used for storing reference fingerprint information of a corresponding grade of a reference data block obtained according to a preset variable-length fingerprint structure, and calculating target fingerprint information of a corresponding grade of a target data block according to the variable-length fingerprint structure; and performing incremental updating and storage on the reference fingerprint information based on the position comparison result. According to the quantity management storage system, a variable-length fingerprint structure is adopted, so that each storage layer can dynamically adjust the fingerprint information precision according to the characteristics and storage requirements of the data blocks, and the flexibility and performance of the system are remarkably improved. Meanwhile, through an incremental updating mechanism, the system only needs to compare and store different difference data contents between the target data block and the reference data block, repeated storage of repeated data is reduced, and the storage overhead is reduced.

The application discloses a kind of based on big data's computer software development data management method.The method is obtained from the multi-source heterogeneous development data of demand management, code repository, continuous integration, test management, defect management and deployment operation platform by distributed acquisition node, and standard development data object is generated by field extraction, cleaning and standardization mapping, and is written into distributed storage cluster;According to the association of requirement identification, task identification, code submission identification, build identification, test identification, defect identification and deployment identification, generate development link uniform identification, form development data link set;The integrity check of the development data link set is executed, missing node is automatically supplemented, link heat value and heat level are determined and hierarchical storage management is carried out, and development data retrieval and trace output are executed when receiving query request.The application can improve the correlation, integrity, traceability and management efficiency of development data.

The invention relates to the field of data collaboration, and discloses a DCS cross-domain data collaborative optimization method based on privacy computing, which is used for realizing efficient collaboration and dynamic prediction of cross-domain data on the premise of ensuring data security. Comprising the following steps: extracting topological features from generator set, power grid dispatching and equipment monitoring data, forming an encryption feature set through algebraic coding and homomorphic encryption, constructing a hierarchical encryption architecture based on secure multi-party calculation and hierarchical storage management, supporting cross-system feature alignment and weighted fusion, generating a dynamically optimized global encryption topological graph, and generating a dynamic optimization global encryption topological graph. High-precision state deduction is realized by combining uncertainty quantization, a feature extraction strategy, an encryption parameter and a prediction model are continuously optimized through a closed-loop feedback mechanism, and a privacy protection audit log is generated. The method breaks through the limitation of a traditional data island, improves the cross-domain cooperation efficiency on the premise of guaranteeing privacy security, and is suitable for real-time decision making and intelligent optimization of high-security demand scenes such as electric power and energy.

The invention relates to the technical field of intelligent video processing and cloud computing, and discloses an intelligent video resource scheduling management system and method. The system comprises a resource monitoring module, an intelligent analysis module, a task scheduling module and a storage management module. The method comprises the following steps: monitoring load states of computing nodes, network bandwidth and storage resources in real time; based on the type and priority of the video task requested by the user, predicting and analyzing the resource demand through an intelligent algorithm; dynamically generating an optimal resource scheduling strategy, and distributing the task to the most suitable processing node; and hierarchical storage management is carried out on hot, warm and cold data. According to the method, the problems of non-uniform distribution of video processing resources, high response delay and low hardware resource utilization rate in the prior art are solved, the intelligence and optimization of resource scheduling are realized, and the system throughput and the task processing efficiency are remarkably improved.

The invention discloses a hybrid Katz centrality measurement system based on pre-filtering, electronic equipment and a storage medium. The measurement system comprises six parts of data stream sampling, centrality normalization calculation, Prefilter precise region maintenance, EdgeSketch compression back-end storage, time attenuation and final centrality query. The invention further discloses electronic equipment and a storage medium. And through an incremental updating mechanism of hierarchical storage management and time perception, the contradiction among calculation efficiency, memory occupation and measurement precision is effectively balanced.

The present application relates to the technical field of unmanned aerial vehicle, and specifically provides a multi-dimensional flight data hierarchical storage management method, system, device and storage medium, comprising: firstly, real-time monitoring of hierarchical storage system resource state, when the node meets the downgrade trigger condition, determining the source node to be downgraded; based on the access frequency, value and data classification label of data fragmentation, sensitive level and compliance rules, screening out target downgrade data. Further, planning a downgrade path for it: first, excluding non-compliant paths according to compliance rules to obtain a candidate set, and then using a downgrade decision model, combining the real-time state of source and target nodes, data attributes and migration cost benefit model, determining the optimal downgrade path from the candidate set. Finally, migrating data according to the path and updating the access route, so as to ensure that the downgrade decision can dynamically adapt to the rapid conversion of unmanned aerial vehicle tasks.

The invention discloses a neuron and synapse unified four-level hierarchical storage management method and system in neural simulation. According to the method, neuron entities and synaptic entities are abstracted into unified entity objects, and the unified entity objects are uniquely identified by unified entity identifiers comprising entity types, global unique indexes and brain region identifiers to which the entities belong. Four storage levels including a virtual level, a dormancy level, an active level and a key level are defined, and the two types of entities are managed completely in a unified mode. Comparable importance scores for neurons and synapses are calculated by an importance evaluator, respectively, wherein non-linear enhancement protection is applied to long-term memory synapses in a consolidated state. The unified storage manager automatically performs upgrade and downgrade migration of the storage hierarchy according to the importance score. According to the method, the problem of dual-system coordination is eliminated, dynamic allocation between the neurons and the synapses of the graphic processing unit is realized, and the key memory is ensured not to be expelled under the storage pressure.

The invention discloses an embedded version management system supporting cross-version jump upgrade, and belongs to the technical field of embedded software upgrade management. The system comprises a version modeling module, an upgrade path planner, an upgrade package manager, an upgrade execution engine, a rollback control module and a configuration management module. The version modeling module analyzes a dependency relationship between management software versions and constructs a version relationship graph; the path planner calculates an optimal upgrading path; the upgrade package manager is responsible for resource storage and distribution; the execution engine executes upgrading according to the plan; the rollback module processes the failure condition; and the configuration module manages system parameters. The system adopts a graph theory algorithm for modeling, supports jump upgrade or multi-step path planning, and manages upgrade packages through hierarchical storage. The problem that a traditional system lacks flexible path planning and cross-version upgrading support is solved, and upgrading reliability and flexibility are improved.

The application relates to the technical field of data processing, and discloses a plastic new material research and development data processing method based on a cloud platform, which comprises the following steps: collecting multi-source heterogeneous plastic research and development original data through an edge node, and pre-processing the data; classifying the processed data and performing hierarchical storage management and optimization; analyzing a research and development calculation task submitted by a user, and dynamically scheduling and distributing cloud computing resources according to task attributes to execute the task; mining and analyzing the stored data to construct a knowledge graph, train a performance prediction model and generate a formula recommendation; and visually displaying data processing results, an analysis process and a task state, and providing permission-based data sharing and collaborative research and development support. The application realizes intelligent management of the whole process of plastic research and development data, can significantly accelerate the research and development cycle, and reduce innovation cost.

The application discloses an AI intelligent hierarchical storage management system for mass unstructured data, relates to the field of big data processing, and comprises the following steps: standardization features of text, image and other data are extracted through an AI mixed analysis technology; a data hierarchical strategy is dynamically adjusted based on reinforcement learning, intelligent scheduling of four-layer storage resources is realized in combination with a greedy algorithm, and an asynchronous migration mechanism is adopted to guarantee data cross-layer flow transfer; LRU-K algorithm is used to optimize high-frequency data caching, and response efficiency is improved through preloading and cluster deployment; multiple protections such as encryption, permission control, desensitization and auditing are integrated in the security aspect, and system full-link management and control are realized in cooperation with distributed monitoring and AI anomaly detection.The application has the advantages that through deep integration of a distributed architecture and AI technology, precise feature analysis and dynamic intelligent grading are realized, storage resource scheduling, caching strategy and data migration efficiency are optimized, layered security protection and AI operation and maintenance monitoring are matched, and high intelligence, high efficiency, low cost and high security are achieved.

The application provides a lightweight storage method and system for Internet of Things time series data, and relates to the technical field of Internet of Things data storage and compression, wherein the method comprises: collecting multi-modal time series data and mechanical vibration energy data generated during the operation of an Internet of Things device; extracting vibration frequency features from the mechanical vibration energy data, and generating time series vibration signals based on the vibration frequency features; performing spatio-temporal correlation between the time series vibration signals and the multi-modal time series data to identify a data activity level; performing sparse representation processing on the multi-modal time series data according to the data activity level to obtain sparse representation data; performing dynamic dictionary construction and coding on the sparse representation data using an adaptive dictionary coding algorithm to generate compressed coding data; and performing hierarchical storage management on the compressed coding data to achieve lightweight storage. The application improves the storage efficiency and processing capacity of Internet of Things edge device time series data.

The invention relates to the technical field of data classification and management systems, in particular to a real-time streaming data classification labeling and hierarchical storage management system based on deep learning, which comprises a data acquisition module, a data acquisition module, a deep learning classification module, a labeling module, a hierarchical storage management module and a user interaction interface. Through cooperative use of the data acquisition module and the data acquisition module, the purpose can be achieved, and data can be classified, labeled and analyzed in time; through cooperative use of the labeling module, the hierarchical storage management module and the deep learning classification module, accurate and effective information classification and storage can be realized.

The invention relates to the technical field of data processing, and discloses a plastic new material research and development data processing method based on a cloud platform, which comprises the following steps: collecting multi-source heterogeneous plastic research and development original data through edge nodes, and preprocessing the data; classifying the processed data and performing hierarchical storage management and optimization; analyzing a research and development calculation task submitted by a user, and dynamically scheduling and allocating cloud computing resources according to task attributes to execute the task; performing mining analysis on the stored data to construct a knowledge graph, train a performance prediction model and generate formula recommendation; the data processing result, the analysis process and the task state are visually displayed, and permission-based data sharing and collaborative research and development support is provided. The full-process intelligent management of the plastic research and development data is realized, the research and development period can be obviously accelerated, and the innovation cost is reduced.