Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1962 results about "Data migration" patented technology

Data migration is the process of selecting, preparing, extracting, and transforming data and permanently transferring it from one computer storage system to another. Additionally, the validation of migrated data for completeness and the decommissioning of legacy data storage are considered part of the entire data migration process. Data migration is a key consideration for any system implementation, upgrade, or consolidation, and it is typically performed in such a way as to be as automated as possible, freeing up human resources from tedious tasks. Data migration occurs for a variety of reasons, including server or storage equipment replacements, maintenance or upgrades, application migration, website consolidation, disaster recovery, and data center relocation.

Intelligent computing power and storage scheduling method and system of multi-service system

The invention relates to an intelligent computing power and storage scheduling method and system for a multi-service system, and belongs to the technical field of intelligent computing resource scheduling and storage management, and the method comprises the steps: collecting the real-time monitoring data of each node in the multi-service system, and carrying out the feature extraction; predicting a computing power demand value and a storage demand layering strategy in a future preset time window; dynamically correcting the prediction result, and generating a dynamic resource demand table and a cross-service priority weight matrix; generating a task allocation scheme through a hybrid scheduling algorithm; generating a storage data migration instruction through a storage scheduling engine; executing calculation node capacity expansion, task migration and storage data redistribution operation; and based on the executed latest state snapshot of the resource pool and the scheduling failure case library, updating a weight parameter of the time sequence prediction model and a scheduling strategy rule library to obtain an updated strategy version, and applying the updated strategy version to a next scheduling period. According to the invention, efficient scheduling of resources can be realized in a multi-service system environment.
Owner:GOLDEN TIMES CULTURE COMM

Multi-level cloud storage cluster construction and data storage method

The invention relates to the technical field of data processing, and discloses a multilevel cloud storage cluster construction and data storage method. The method comprises the steps that S1, multiple levels of storage layers are constructed and comprise a cache layer, a performance layer and a capacity layer, and the storage layers are provided with different storage media which are used for storing hot data, warm data and cold data respectively; s2, determining data popularity by monitoring data access frequency, access time interval and data importance; and S3, dynamically adjusting the storage hierarchy of the data according to the data popularity, and triggering hierarchical migration of the data. By constructing a multi-level storage layer and adopting a hierarchical architecture that the cache layer stores high-frequency data, the performance layer stores intermediate-frequency data and the capacity layer stores low-frequency data, high-heat data is preferentially stored in a high-performance medium, access delay is reduced, and data reading and writing efficiency is improved. Meanwhile, in combination with a dynamic data migration strategy, the storage position can be adjusted in real time according to the data access popularity, and the overall performance of the system is further optimized.
Owner:BEIJING HANXINSHENG TECH CO LTD

Memory cache storage system oriented to big data flow

The invention discloses a big data stream-oriented memory cache storage system, relates to the technical field of memory cache storage, and is used for solving the problem of poor memory cache storage management. According to the method, the access frequency, the timeliness and the service importance of the data are evaluated, the priority ranking model is constructed, the cache strategy is optimized based on the ranking result, the data storage and scheduling efficiency is improved, the write-in and read operation of the memory cache is adjusted in real time in combination with the flow prediction model, memory overload is effectively avoided, and the data storage and scheduling efficiency is improved. According to the method, non-emergency data writing is delayed by adopting a delayed writing and lazy loading strategy, data is loaded as required, meanwhile, a cache elimination mechanism is designed according to data priority, access frequency, survival time and size, data migration is carried out when a cache space is insufficient, reasonable resource allocation is ensured, and through a multi-level and dynamic cache management scheme, the cache management efficiency is improved. Therefore, the system performance, the resource utilization rate and the response speed are remarkably improved, the data processing efficiency is optimized, and cache conflicts are reduced.
Owner:SHENZHEN LOVE HEART AUTOMATION TECH CO LTD

Storage hard disk remote diagnosis system and method based on Internet of Things

The invention discloses a storage hard disk remote diagnosis system and method based on the Internet of Things, and relates to the technical field of health management of the Internet of Things and storage equipment. The system comprises a data fusion module, a causal analysis module, a risk analysis module and a map construction module. The data fusion module collects SMART parameters, IO operation time sequence data and environment data through Internet of Things equipment, dynamically distributes multi-source data weights by using an attention mechanism, and extracts hard disk health state features. And the causal analysis module is combined with the multi-scale time sequence convolutional network and the Bayesian causal network to identify periodic abnormal fluctuation and generate a fault root cause analysis report. The risk analysis module matches historical cases through federal incremental learning, calculates a hard disk fault risk index and generates an early warning signal. And the atlas construction module optimizes resource isolation, data migration and response paths according to the risk indexes, generates a visual operation and maintenance atlas, provides fault positioning, risk links and repair priorities, and improves the operation and maintenance management efficiency.
Owner:SHENZHEN SANSHANG SCIENCE & TECHNOLOGY CO LTD

Data processing method and apparatus

PCT designated stage expiredWO2025107540A1Input/output to record carriersData storeStorage type
The present application relates to the field of storage, and discloses a data processing method and apparatus. The method is applied to an intelligent tiering system, the intelligent tiering system is used for carrying out intelligent tiering processing on service data in a storage system, and the storage system comprises at least one storage path. The method comprises: acquiring first service data, wherein the first service data is stored in a target storage path in the storage system, and the storage type of the first service data is an intelligent tiering type; acquiring an access log of the first service data; analyzing the access log of the first service data on the basis of a hot and cold identification algorithm configured by a user for the target storage path, to obtain the access popularity of the first service data; and sending a data migration instruction to the storage system on the basis of the access popularity of the first service data, so that a storage node of the storage system performs data migration on the first service data. The present application is used for reducing the impact on the storage system, improving the operation performance of the storage system.
Owner:HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD

Radioactive measurement data processing method for improving uranium exploration efficiency

The invention discloses a radioactive measurement data processing method for improving uranium exploration efficiency. The method comprises the following steps: standardized data acquisition: integrating a multi-parameter module, carrying out time-space synchronous acquisition of geological geophysical environment parameters, standardizing protocol alignment data, and supporting three-dimensional modeling; a three-dimensional coupling model is used for processing interference in a sub-module mode, LiDAR-DEM is used for correcting gamma attenuation in the terrain, an optical fiber thermopermeability instrument is used for correcting daughter errors in the hydrology, signal attenuation of a borehole is compensated through a drilling fluid chart, and a space-time continuous interference field is formed; performing dynamic equilibrium coefficient inversion: constructing equilibrium coefficient isoparametric mapping, and inputting rock core, logging and geochemical data into a random forest model; the transfer learning is trained by using historical data, and dynamic inversion and updating of a new area are carried out; intelligent data processing three-dimensional visualization: self-adaptive denoising and spectral shape matching reinforcement abnormity are carried out; constructing a three-dimensional model, and carrying out transfer learning to optimize the precision; the WebGL platform integrates multi-parameter display, virtual drilling functions and high-dimensional data intuitive interpretation.
Owner:安徽省核工业勘查技术总院

Data hierarchical storage method and device based on access characterization, equipment and medium

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical treatment and health and the like, and discloses a data hierarchical storage method, device, equipment and medium based on access characterization. Writing the data into a matched first target storage device according to a preset device performance corresponding relation; after the data are stored, continuously monitoring subsequent access characteristics of the data, and dynamically judging whether the heat state is changed or not; and if the popularity state is changed and a specific migration condition is met, redistributing the second target storage device and completing data migration operation so as to realize automatic hierarchical storage management of cold and hot data. By acquiring the data access representation and dynamically identifying the popularity state, hierarchical migration of data with different popularity among storage devices with different performances is realized, the resource use efficiency and access performance of a storage system are improved, and the storage cost is reduced.
Owner:PING AN TECH (SHENZHEN) CO LTD

Laser processing parameter autonomous generation system and method based on digital twinning

The invention discloses a laser processing parameter autonomous generation system and method based on digital twinning, and relates to the field of digital twinning, and the method comprises the steps: collecting and preprocessing processing data in real time through a multi-source sensor; processing and analyzing the task instruction by using a natural language, extracting a constraint condition and forming a structured demand; a processing parameter candidate set is generated through a Transform model in combination with historical data transfer learning, and virtual processing is performed by means of multi-physics field simulation; an improved non-dominated sorting genetic algorithm is adopted to dynamically optimize the parameter weight, and a global optimal parameter combination is obtained through digital twin iteration verification; and after full-process virtual processing verification and task demand comparison, parameters are adaptively corrected, and model parameters are continuously optimized according to physical and simulation data deviation after actual processing. The method has the advantages that the digital twin is used as a core, multi-source real-time data, the AI algorithm and multi-physical field simulation are fused, and autonomous generation, multi-target optimization and virtual-real closed-loop iteration of laser processing parameters are achieved.
Owner:CHENGDU MRJ LASER TECH CO LTD

Management method and management apparatus for hybrid memory system, and device and medium

The present application relates to a management method and management apparatus for a hybrid memory system, and a device and a medium. The method comprises: acquiring a write popularity value and an access popularity value of a currently accessed memory page; comparing the write popularity value with a specified write popularity threshold value, and generating write popularity page migration flag information; comparing the access popularity value with a specified access popularity threshold value, and generating access popularity page migration flag information; grouping hybrid memories in a hybrid memory system, and generating, on the basis of memory page information of all grouped memory banks, memory page management information corresponding to the hybrid memories; and on the basis of the write popularity page migration flag information, the access popularity page migration flag information, and current memory page management information corresponding to the currently accessed memory page, determining whether the current access memory page is a popular data page, and performing data migration according to a preset unpopular and popular data page migration strategy. The technical solution can improve the overall performance of a hybrid memory system having a horizontal architecture.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Market supervision data exchange platform migration and security enhancement method and system based on credential environment

The invention discloses a market supervision data exchange platform migration and security enhancement method and system based on a credential environment, and relates to the technical field of network security. In the data migration process, the consistency and privacy protection of the data are ensured through Hash check and a multi-party security computing technology; a domestic commercial cryptographic algorithm and a quantum resistance encryption technology are introduced, so that the protection capability of the platform to existing security threats is effectively improved, and the resistance to future quantum computing threats is enhanced; an intrusion detection system and a virtual honeypot node are integrated, and a defense strategy is monitored and updated in real time in combination with a deep learning model, so that novel attack behaviors can be detected and coped with in real time; according to an intrusion detection alarm and a block chain audit result, a security policy and resource allocation are dynamically adjusted in combination with a machine learning technology, it is ensured that high-risk nodes are monitored in time, resource allocation is automatically optimized, and the resource utilization efficiency and security of the system are improved.
Owner:江苏省市场监督管理局数据中心

Distributed time sequence library data management method supporting cold and hot data level-to-level management

The invention discloses a distributed time sequence library data management method supporting cold and hot data level-to-level management, and relates to the technical field of computer databases, comprising: receiving a write-in request of time sequence data, and executing preliminary write-in data aggregation and sorting; dynamically identifying data cold and hot attributes based on a multi-dimensional data cold and hot degree calculation model in combination with a write-in behavior and an access behavior of time series data; according to the cold and hot attribute recognition result, in combination with the hierarchy boundary of self-adaptive division, automatic hierarchical storage of the time series data is executed; based on a cold and hot data dynamic migration and scheduling mechanism, according to the access frequency and time decay characteristics of the time series data, dynamically adjusting the storage hierarchy of the time series data, and executing a data migration task; and constructing a hierarchical index system adaptive to different cold and hot attribute time series data, and combining query frequency dynamic identification and index structure automatic upgrading. By adopting a cold and hot data automatic identification and hierarchical storage mechanism, the overall performance and the resource utilization rate of the system are improved.
Owner:GUODIAN NANJING AUTOMATION

Channel model dynamic association method based on multi-source data index

The invention discloses a channel model dynamic association method based on a multi-source data index, and relates to channel engineering digital management. The channel model dynamic association method comprises the steps that component codes of channel engineering are constructed and used for identifying the same physical entity; establishing a three-layer index table structure according to the component codes; constructing a dynamic partition management mechanism of a three-level cache architecture, wherein the three-level cache architecture comprises a first-level cache stored in a memory, a second-level cache stored in a local disk and a third-level cache stored in a distributed manner; according to data access information in the three-layer index table structure, data migration is carried out among all levels of caches; according to the dynamic partition management mechanism, a data version management mechanism based on a timestamp concurrency control protocol is established, the data version management mechanism generates incremental data by comparing original data with updated data, and incremental updating or full updating is selected according to a threshold value. According to the method, the problem that data evolution tracking is difficult due to fragmentation of multi-source heterogeneous data and unbalanced access efficiency in navigation channel engineering is solved.
Owner:JSTI GRP CO LTD

Semantic-based migration and consistency verification method, system and equipment and medium

The invention provides a semantic-based migration and consistency verification method, system and device and a medium, and relates to the technical field of databases. The method comprises the following steps: performing metadata topology scanning analysis by obtaining metadata, including generating an abstract syntax tree and constructing a semantic graph; according to a metadata topology scanning analysis result, performing data mapping and conversion, including loading a YAML rule and generating corresponding type mapping and constraint conversion; data migration is carried out through primary key fragmentation parallel migration, batch writing optimization and real-time double-writing verification; through structure-data-business three-layer verification, simulation business SQL comparison and automatic difference repair, the integrity of migrated data and business compliance are ensured, the semantic gap problem in heterogeneous database migration is solved, high-precision and high-efficiency database migration is realized, the migration efficiency is improved, and the data migration efficiency is improved. The method is suitable for scenes with strict requirements on data consistency and migration efficiency, such as financial, government affair and enterprise-level data centers.
Owner:CHINA YANGTZE POWER

Task scheduling method applicable to large language model (LLM), and computing device, storage medium and computer program product

Provided in the present disclosure are a task scheduling method applicable to a large language model (LLM), and a computing device, a storage medium and a computer program product. The method comprises: when determining that a target task processing node currently executing a target task meets a resource reporting condition, a target second task scheduler sending a target running resource of the target task processing node to a first task scheduler; when determining that the target running resource meets a task migration condition, the first task scheduler determining a migration task processing node from among a plurality of task processing nodes, and sending node information of the migration task processing node to the target second task scheduler; and the target second task scheduler determining the migration task processing node on the basis of the node information, determining the target task currently executed by the target task processing node, as well as task data of the target task, and migrating the target task and the task data in batches to the migration task processing node for execution, so as to obtain a task processing result of the target task.
Owner:CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD

Marine engine lubricating oil system fault diagnosis method and system based on transfer learning from simulation domain to real domain

The invention provides a marine engine lubricating oil system fault diagnosis method and system based on transfer learning from a simulation domain to a real domain, and the method comprises the steps: carrying out the fault simulation of different working conditions based on a built marine engine lubricating oil system simulation model, and obtaining a simulation fault signal response; acquiring real domain fault data of different working conditions of an engine lubricating oil system based on an engine experiment platform; a feature fusion structure based on an encoder-decoder is adopted to obtain simulation domain fault data; an engine lubricating oil system fault diagnosis model based on data migration and algorithm migration dual migration learning is obtained; and based on the engine lubricating oil fault diagnosis model, performing real-time fault diagnosis on the marine engine lubricating oil system to obtain a diagnosis result. According to the method, transfer learning from simulation domain fault features to real domain fault features is realized, so that the problems of high construction cost and insufficient generalization performance of a diagnosis model are solved.
Owner:HARBIN ENG UNIV

Frequency access data storage migration method and device for big data

The invention belongs to the field of data storage, and discloses a frequency access data storage migration method and device for big data, and the method comprises the steps: obtaining a data access log, carrying out the preprocessing of the data access log, and obtaining a time window statistical matrix; carrying out timeliness attenuation weighting calculation on a plurality of data access times in the time window statistical matrix to obtain a weighted access frequency vector; obtaining a multi-dimensional feature matrix based on the weighted access frequency vector; clustering the feature vector of each time window slice in the multi-dimensional feature matrix through a Gaussian mixture model to obtain a clustering tag vector; inputting the clustering label vector into an access frequency prediction model to obtain an access frequency prediction value of the data in a future preset time step; the migration decision matrix is determined based on the access frequency predicted value and the data storage cost parameter, data migration is carried out based on the migration decision matrix, the data migration complexity can be reduced, and the access speed and the storage cost are balanced.
Owner:XIAMEN MEIYA YIAN INFORMATION TECH CO LTD

Memory data migration method and electronic equipment

The invention discloses a memory data migration method and electronic equipment, and relates to the technical field of computers, and the method comprises the steps: determining the access heat of a storage device accepting the access of a computing node based on a memory access task executed by a switching controller and an address mapping table, so as to determine a source device and a candidate data unit, according to the access parameter of the candidate data unit and the state parameter of the storage device, calculating a performance score after the candidate data unit is stored in the storage device so as to determine the data unit to be migrated and the target device, so that cold and hot data identification from the perspective of a computer fast interconnection memory heterogeneous system is realized; the rationality of data migration is improved on the basis of quantitative performance scores; according to the method, the to-be-migrated data unit is controlled to be migrated from the source device to the target device, and the address mapping table is updated, so that the data migration task is prevented from influencing the host work of the computing node, and a cold and hot data migration scheme without being perceived by the host in the computer fast interconnection memory heterogeneous system is realized.
Owner:LANGCHAO ELECTRONIC INFORMATION IND CO LTD

Intelligent water and fertilizer integrated irrigation system based on Internet of Things sensor and large language model

The invention discloses an intelligent water and fertilizer integrated irrigation system based on an Internet of Things sensor and a large language model, and relates to the technical field of intelligent agriculture. The system obtains soil, weather and plant growth state data in real time through Internet of Things sensors, including a soil sensor, a weather sensor and a multispectral sensor, deployed in a farmland; the method comprises the following steps: based on a large language model architecture, integrating preset agricultural field knowledge base data, performing field adaptive fine tuning on model parameters by adopting a data migration technology, and constructing an intelligent irrigation control model with a dynamic decision-making function; the control model analyzes sensor data and crop growth requirements, generates a water and fertilizer supply strategy, calls an internet-of-things control interface of the water and fertilizer all-in-one machine, and automatically adjusts irrigation water quantity and proportional supply of nutrient elements such as nitrogen, phosphorus and potassium. According to the invention, Internet of Things perception and large model decision are fused, and different environments are adapted through fine adjustment, so that closed-loop precise control is realized, and the water and fertilizer utilization rate and the crop yield are remarkably improved. Actual measurement shows that the system can improve the water and fertilizer utilization rate by 30% or above, and the crop yield is increased by 15-20%.
Owner:SICHUAN HEHU TECHNOLOGY CO LTD

Cross-data-platform database migration verification method and system applied to autonomous controllable transformation

The invention relates to the technical field of database migration verification, in particular to a cross-data-platform database migration verification method and system applied to autonomous controllable transformation. The method comprises the following steps: configuring a source database and a target database in a data source management module; newly establishing a comparison task in the task management module; a comparison database is connected through a dynamic data source assembly, different databases are connected through JDBC, and comparison verification is executed after database connection parameters are configured; database comparison is carried out in the task management module, wherein the database comparison comprises data volume verification and data consistency verification; a comparison report is generated and stored in a built-in mode. The problem of data integrity in the cross-platform data migration process is solved, the system fully considers the difference between the MySQL database and the local database in character set, function and technical characteristics, data verification and comparison work after migration of the MySQL database source and various local target databases is supported, consistency verification before and after data migration is achieved, and the verification efficiency is improved. And the risk of data loss caused by data migration is avoided.
Owner:NAT NUCLEAR INFORMATION TECH CO LTD

Improved inter-ERP (Enterprise Resource Planning) system data migration and synchronization method and system

The invention relates to the technical field of data processing, and particularly discloses an improved inter-ERP (Enterprise Resource Planning) system data migration and synchronization method, which comprises the following steps of S1, metadata acquisition and preprocessing; s2, field intelligent matching; S3, machine learning recommendation; s4, mapping confirmation and feedback; s5, converting rule configuration; s6, carrying out total migration; s7, performing increment synchronization; and S8, monitoring and alarming: counting delay, throughput and failure rate of increment synchronization in real time, and automatically sending an alarm notification when an abnormal threshold is triggered. According to the method, a complete technical system which is infinitely infinitely and covers the whole migration and synchronous life cycle is constructed from metadata acquisition to intelligent mapping, rule driving and full / incremental ETL (Extract Transform Load), and then to monitoring alarm and rollback management; the method is suitable for efficient, reliable and extensible data migration and synchronization between heterogeneous ERP systems, and provides solid support for enterprise informatization upgrading, system splitting / merging and big data analysis enabling.
Owner:上海美武信息技术有限公司

SSD wear leveling method and device based on dynamic data migration and medium

The invention relates to the technical field of storage, in particular to an SSD wear leveling method and device based on dynamic data migration and a medium. The method comprises the following steps: monitoring access frequency, recording read-write times of a logic block in real time, and identifying hotspot data; tracking a wear state, counting the erasure times of the physical flash memory block, and evaluating the wear degree of the physical flash memory block; data migration decision: determining whether to migrate data based on the access frequency and the wear frequency; preferentially migrating data with high access frequency and high wear degree according to access frequency and wear program sorting; and data migration execution: migrating the hotspot data from the high-wear area to the low-wear area, and updating the mapping relation. According to the method, the data storage position can be intelligently adjusted according to the real-time data access characteristics and the abrasion state, and therefore the service life of the SSD is more efficiently prolonged.
Owner:SHANDONG SINOCHIP SEMICON CO LTD

Data migration method and device based on task offloading, equipment and medium

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a data migration method, device, equipment and medium based on task offload, which comprises the following steps: scanning a source end directory hierarchy and a target end directory hierarchy, and generating source end file metadata and target end file metadata; files are classified into source-end unique files, target-end unique files and two-end shared files based on file paths, and the files are transmitted to a comparison processing module; generating a source end newly-added migration task, a difference coverage migration task and a target end deletion migration task; and transmitting the migration task to a migration processing module and executing the migration task to generate a migration result. According to the method, source-end unique files, difference files and target-end redundant files are processed in a classified mode, full-amount migration is disassembled into incremental tasks to be executed in a split mode, and the invalid data transmission amount is reduced; a difference coverage task is generated based on double comparison of metadata and content, repeated coverage of homonymous files is avoided, and redundant data is automatically cleaned.
Owner:PING AN TECH (SHENZHEN) CO LTD

NAND Flash bad block management method and system based on artificial intelligence

The invention discloses an NAND Flash bad block management method and system based on artificial intelligence, and belongs to the technical field of solid-state storage, the method comprises the following steps: S1, carrying out multi-dimensional data acquisition and preprocessing on a bad Page based on preset parameters and preset indexes; s2, inputting the preprocessed multi-dimensional data into a space-time diagram convolutional network model, and outputting the failure probability of each Page; and S3, according to the failure probability obtained in the step S2, a hierarchical isolation strategy is determined, and then error correction code dynamic loading and preventive data migration are carried out. Actual measurement shows that more than 85% of normal Pages can be recycled in a Block with a single bad page, and the effective capacity is improved by 10 times compared with that of a traditional scheme; the overall service life of the chip is prolonged by about 30% by reducing the discarding of the whole chip caused by local damaged pages. The method also has a real-time risk management and control function, the prediction accuracy of the AI model to a potential risk area reaches 92%, and the secondary failure rate is reduced by 70% compared with a static isolation scheme.
Owner:成都芯盛集成电路有限公司

Automatic processing method for real-time observation data of ocean station

The invention provides an automatic processing method for real-time observation data of an ocean station, and belongs to the technical field of ocean observation data processing. A wavelet packet decomposition multi-scale noise separation algorithm is established to distinguish environmental noise and real signals, dual-sensor redundancy configuration is combined with Bayesian inference to identify sensor drift, and a calibration coefficient is updated in real time through a recursive least square method. A one-dimensional time sequence is mapped to a high-dimensional phase space by utilizing a phase space reconstruction algorithm to realize high-precision prediction of a chaotic signal, a hierarchical data storage architecture is established, and a data migration strategy is iteratively optimized through a hierarchical correlation degree function; the technical problem that time synchronization signals are difficult to reconstruct accurately according to asynchronous sampling data of multiple sensors of an ocean station is solved.
Owner:STATE OCEANIC ADMINISTRATION EAST CHINA SEA INFORMATION CENTER (STATE OCEANIC ADMINISTRATION EAST CHINA SEA ARCHIVES)

Efficient memory management method and system based on SOC chip

The invention relates to the technical field of SOC chips, and discloses an efficient memory management method and system based on an SOC chip, which are used for constructing a complete topological graph and calculating access delay distribution by comprehensively analyzing an internal physical structure of the chip. According to the method, the memory access behavior of each processing core is monitored in real time, dynamic memory partitioning is executed according to the access characteristics and the physical topological structure, and the optimal access area is distributed for the processing core. And when an access hot spot is detected, triggering a data migration mechanism to copy hot spot data to a relatively close memory area. According to the method, a memory address space is recombined by adopting a topology-aware address mapping algorithm, a multi-level cache collaboration mechanism is established, and the working mode of a memory controller is dynamically adjusted according to an application type. Through load balancing monitoring and periodic memory recombination, the memory access delay is effectively reduced, the bandwidth utilization rate is improved, and the overall memory access performance of the SOC chip is optimized.
Owner:SUZHOU RIGGER MICRO TECH GRP CO LTD

Heterogeneous storage-oriented hot and cold data management method and system

A heterogeneous storage-oriented hot and cold data management method and system, wherein hot and cold data are managed by means of the following steps: at table granularity, storing data from each table into an independent data bucket in the form of object storage, each data bucket comprising a plurality of partitions, the data being randomly distributed in each partition, and the consistency of replicas of the partitioned data being ensured by means of a distributed consensus algorithm; collecting statistics about the access frequencies of the partitions in one data bucket; when the access frequency of one partition reaches a set threshold, adding a new replica to a target storage medium by means of a membership change in the distributed consensus algorithm; and when data synchronization between the new replica and an old replica is completed, detecting the old replica. In this way, the migration of the partitioned data on different storage media is realized, and data cooling and heating are implemented. The method and system can maximize the use efficiency of high-performance storage media, ensure the access efficiency of users, and can reduce the costs of data migration and maintenance.
Owner:HANGZHOU OPENPIE TECH DEV CO LTD

Cross-cluster doris unattended data migration method

The invention discloses a cross-cluster Dores unattended data migration method which comprises the following steps: acquiring metadata of a to-be-migrated table by connecting a source Dores cluster, analyzing and adjusting table structure features incompatible with a target cluster in the metadata, generating a compatible table establishment statement, and establishing a target table in the target cluster; an API of a data migration platform is called through a preset interface, a migration task is automatically configured and submitted, and the migration task comprises a source table, a target table and table association information; polling a task state in real time, and triggering a next migration task or an exception processing flow according to a task execution result; processing of all the to-be-migrated tables is executed in sequence according to a preset rule, and the number of concurrent tasks is limited to maintain cluster stability; and after migration is completed, performing multi-dimensional data consistency verification on the source table and the target table, and triggering a repair process according to a verification result. According to the cross-version Dores database migration method and device, the efficiency and reliability of cross-version Dores database migration are improved, and manual intervention and operation errors are effectively reduced.
Owner:ZHOUPU DATA TECH NANJING CO LTD

Storage space recovery method, electronic equipment, storage medium and program product

The invention discloses a storage space recovery method, electronic equipment, a storage medium and a program product, and relates to the technical field of computers, the influence of various factors on storage space recovery is comprehensively considered, and the recovery score of a data block is determined specifically according to the invalid data rate, the data heat degree and the wear degree; according to the method and the device, the data blocks are selected to be subjected to storage space recovery, the recovery priorities of the data blocks are determined according to the recovery scores, the data blocks are subjected to storage space recovery according to the recovery priorities, and the data blocks with the maximum recovery benefits are preferentially processed, so that the reasonability of the data blocks subjected to storage space recovery is improved; the effective data migration rate of the recovered data blocks can be reduced, so that the recovery efficiency of the storage space is improved, the influence on the performance of the storage system can be reduced by considering the heat data, in addition, the wear degree of the data blocks is considered, and the service life of the storage system is also prolonged to a certain extent.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Intelligent hierarchical storage method and system for cold and hot data of solid state disk

The invention relates to the technical field of data storage, and discloses an intelligent hierarchical storage method and system for cold and hot data of a solid state disk, and the method comprises the steps: collecting spatial position change data and vibration state data, and carrying out the preprocessing to obtain a multi-dimensional feature data set; performing quantification processing on the influence degree of the spatial position change data and the vibration state data to obtain an interference weight value; constructing a joint feature matrix based on the interference weight value and the historical access record, and predicting hotspot data distribution; if the matching degree of the hotspot data distribution and the current storage layout is lower than a preset matching degree threshold value, migrating the high-frequency access data to a low-delay area to obtain an optimized storage layout; when the access response time is larger than a preset time threshold value, log data are generated, and delay load features are extracted; and carrying out secondary adjustment on the optimized storage layout to obtain a final dynamic storage scheme. According to the method, dynamic optimization of the storage performance of the solid state disk can be realized, and delay loss caused by cold and hot data mismatching is reduced.
Owner:SHENZHEN QUANXING TECH CO LTD

Solid state disk data storage method, device and equipment and storage medium

PendingCN120653202AInput/output to record carriersAutomated tiered storageData access
The invention discloses a solid state disk data storage method, device and equipment and a storage medium, and relates to the technical field of hard disk management, and the method comprises the steps: obtaining a life value corresponding to each flash memory block in a target solid state disk, and dividing the life value into a preparatory TLC region and a preparatory SLC region; based on the current space utilization rate of the target solid state disk, the data proportion of the current hot data to the cold data and the access proportion of the cold data write access times to the total write access times, determining the target partition proportion of the TLC area and the SLC area, and based on the target partition proportion, determining the current TLC area and the current SLC area; and migrating and storing hot data in the first current storage data to the current SLC area and migrating and storing cold data in the second current storage data to the current TLC area based on the data access frequencies corresponding to the first current storage data in the current TLC area and the second current storage data in the current SLC area. Therefore, automatic hierarchical storage of the hard disk data can be realized.
Owner:JINAN MAIWEI INTELLIGENT TECHNOLOGY CO LTD