Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

3093 results about "Data migration" patented technology

Data migration is the process of selecting, preparing, extracting, and transforming data and permanently transferring it from one computer storage system to another. Additionally, the validation of migrated data for completeness and the decommissioning of legacy data storage are considered part of the entire data migration process. Data migration is a key consideration for any system implementation, upgrade, or consolidation, and it is typically performed in such a way as to be as automated as possible, freeing up human resources from tedious tasks. Data migration occurs for a variety of reasons, including server or storage equipment replacements, maintenance or upgrades, application migration, website consolidation, disaster recovery, and data center relocation.

Intelligent computing power and storage scheduling method and system of multi-service system

The invention relates to an intelligent computing power and storage scheduling method and system for a multi-service system, and belongs to the technical field of intelligent computing resource scheduling and storage management, and the method comprises the steps: collecting the real-time monitoring data of each node in the multi-service system, and carrying out the feature extraction; predicting a computing power demand value and a storage demand layering strategy in a future preset time window; dynamically correcting the prediction result, and generating a dynamic resource demand table and a cross-service priority weight matrix; generating a task allocation scheme through a hybrid scheduling algorithm; generating a storage data migration instruction through a storage scheduling engine; executing calculation node capacity expansion, task migration and storage data redistribution operation; and based on the executed latest state snapshot of the resource pool and the scheduling failure case library, updating a weight parameter of the time sequence prediction model and a scheduling strategy rule library to obtain an updated strategy version, and applying the updated strategy version to a next scheduling period. According to the invention, efficient scheduling of resources can be realized in a multi-service system environment.
Owner:GOLDEN TIMES CULTURE COMM

Data storage method and system based on legal knowledge service platform and storage medium

The invention relates to the technical field of data storage, in particular to a data storage method and system based on a legal knowledge service platform. The method comprises the following steps: acquiring legal knowledge service data; performing data feature extraction based on data types, use frequencies, importance and timeliness on the legal knowledge service data, and performing clustering analysis to obtain classification result data; constructing a self-adaptive multi-level storage architecture model based on the classification result data; storage schemes of a hot data layer, a warm data layer and a cold data layer are determined according to the multi-level storage architecture model, and a data migration strategy model based on a historical access mode is established, so that an automatic data migration strategy among different levels is obtained. According to the method, the multi-level storage architecture model of the hot data layer, the temperature data layer and the cold data layer is constructed, hierarchical storage is performed according to the use frequency and importance of the data, and the access efficiency of the data and the utilization rate of storage resources are improved.
Owner:无锡中铠信息咨询服务有限公司

Multi-level cloud storage cluster construction and data storage method

The invention relates to the technical field of data processing, and discloses a multilevel cloud storage cluster construction and data storage method. The method comprises the steps that S1, multiple levels of storage layers are constructed and comprise a cache layer, a performance layer and a capacity layer, and the storage layers are provided with different storage media which are used for storing hot data, warm data and cold data respectively; s2, determining data popularity by monitoring data access frequency, access time interval and data importance; and S3, dynamically adjusting the storage hierarchy of the data according to the data popularity, and triggering hierarchical migration of the data. By constructing a multi-level storage layer and adopting a hierarchical architecture that the cache layer stores high-frequency data, the performance layer stores intermediate-frequency data and the capacity layer stores low-frequency data, high-heat data is preferentially stored in a high-performance medium, access delay is reduced, and data reading and writing efficiency is improved. Meanwhile, in combination with a dynamic data migration strategy, the storage position can be adjusted in real time according to the data access popularity, and the overall performance of the system is further optimized.
Owner:BEIJING HANXINSHENG TECH CO LTD

Data distributed storage method and device

The embodiment of the invention discloses a data distributed storage method and device, and the method comprises the steps: carrying out the fragmentation processing of to-be-stored data, and obtaining an initial fragmentation result; utilizing a reinforcement learning model to perform popularity prediction on the data fragments according to the historical access data to obtain the access popularity of the data fragments; determining the number of copies based on the access popularity and importance of the data fragments, generating fragment copies of the data fragments according to the number of the copies, and storing the data fragments and the fragment copies into storage nodes according to the access popularity and node load state scores; when the loads of the storage nodes are unbalanced, data migration is carried out on the data fragments stored in the storage nodes, and when the node health state of a certain storage node is abnormal or the stored data fragments are lost, data reconstruction is carried out on the data fragments stored in the storage node. According to the method, the problems of non-uniform access load, resource waste, data loss and the like in a distributed storage system can be relieved.
Owner:BEIJING HORIN TECH CO LTD

Memory cache storage system oriented to big data flow

The invention discloses a big data stream-oriented memory cache storage system, relates to the technical field of memory cache storage, and is used for solving the problem of poor memory cache storage management. According to the method, the access frequency, the timeliness and the service importance of the data are evaluated, the priority ranking model is constructed, the cache strategy is optimized based on the ranking result, the data storage and scheduling efficiency is improved, the write-in and read operation of the memory cache is adjusted in real time in combination with the flow prediction model, memory overload is effectively avoided, and the data storage and scheduling efficiency is improved. According to the method, non-emergency data writing is delayed by adopting a delayed writing and lazy loading strategy, data is loaded as required, meanwhile, a cache elimination mechanism is designed according to data priority, access frequency, survival time and size, data migration is carried out when a cache space is insufficient, reasonable resource allocation is ensured, and through a multi-level and dynamic cache management scheme, the cache management efficiency is improved. Therefore, the system performance, the resource utilization rate and the response speed are remarkably improved, the data processing efficiency is optimized, and cache conflicts are reduced.
Owner:SHENZHEN LOVE HEART AUTOMATION TECH CO LTD

Data dynamic storage method and device, equipment and storage medium

The invention relates to the field of computers, and discloses a data dynamic storage method and device, equipment and a storage medium. The method comprises the following steps: acquiring a mapping relationship between data and a data storage layer; when the target data is accessed, routing to the corresponding data storage layer to read the target data, and recording the target data into the data historical access log; according to the data historical access log, performing regular analysis to obtain historical access frequency, time dimension trend and periodic characteristics of the stored data; according to the historical access frequency, the time dimension trend and the periodic characteristics of the storage data, predicting the future access heat of the storage data through an integrated deep learning model; and sequencing the future access popularity of the storage data, and migrating the storage data to different data storage layers according to the sequencing of the future access popularity. In the embodiment of the invention, the data storage can be dynamically adjusted, the storage resources are reasonably distributed, and the utilization rate of the storage resources is improved.
Owner:深圳市金政软件技术有限公司

Method and system for realizing data synchronization between heterogeneous databases based on logs

The invention discloses a method and a system for realizing data synchronization between heterogeneous databases based on logs. The method comprises the following steps: acquiring connection information of a source database and a target database; querying a table structure of the source end database and generating an SQL statement to establish a data mapping rule; matching a corresponding rule model from a mapping rule model library; if the matching is not successful, the system abnormally quits and prompts the user to create a new mapping model and add the new mapping model to the mapping rule model base; converting a table structure of the source end database into a JSON character string based on a standard field type; an SQL statement required by the target database is output through the data mapping engine module; the message monitoring program reads the data in the temporary storage area and converts the data into data meeting the requirements of the target database; formatting the converted data into SQL statements supported by a target database; and sending the generated SQL statement to a target database to complete data migration. Accurate and controllable synchronization of the data and the model in a heterogeneous environment is realized.
Owner:FUJIAN FUJITSU COMM SOFTWARE CO LTD

Data interaction method, device, equipment and storage medium for multi-layer stacked memory

The present invention relates to a data interaction method, device, equipment and storage medium of a multi-layer stacked memory, comprising the following steps: based on the access frequency clustering result, the target transmission data is mapped to a storage location to obtain a multi-layer storage allocation strategy table; based on the multi-layer storage allocation strategy table, the target transmission data is interactively planned to obtain an optimal interactive path plan; based on the optimal interactive path plan, the target transmission data is hierarchically migrated and scheduled to obtain a data migration execution queue; and the data migration execution queue is transmitted in parallel through an asynchronous pipeline transmission mechanism, which solves the technical problem that the existing data transmission strategies often lack in-depth analysis of the historical access characteristics of the storage hierarchy, resulting in unreasonable data distribution, and frequent cross-level data migration increases system delay and energy consumption.
Owner:SHENZHEN OSCOO TECH

Data management method and device based on cold data migration, equipment and storage medium

The invention relates to the technical field of data management and the field of financial science and technology, and discloses a data management method based on cold data migration, which comprises the following steps: extracting metadata in a distributed file system, and importing the metadata into a metadata storage library of a distributed data processing platform. And analyzing file information in the metadata storage library through an offline analysis task, identifying cold data, and storing file path information of the cold data in a database. And according to the identified cold data, generating a data migration plan comprising a migration target position, migration priority ranking and migration time arrangement. And migrating the cold data from the original storage position to the target far-end storage, and updating a table partition path in the metadata storage library to point to a new storage position. And after migration is completed, deleting the cold data in the original storage position. According to the method, the cold data and the computing resources are effectively separated, the utilization efficiency of the storage resources is optimized, the overall resource consumption of the system is reduced, and meanwhile the expansibility and the performance of the system are improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Dynamic data distribution storage method and system for RAID (Redundant Array of Independent Disks)

The invention relates to the technical field of data storage, in particular to a dynamic data distribution storage method and system for an RAID (redundant array of independent disks), and the method comprises the following steps: based on the access frequency of data blocks in a disk, recording the access frequency and timestamp of each data block, carrying out heat rating on the data blocks, and determining thresholds of hot data and cold data. According to the method, the dynamic distribution of data storage is effectively optimized by analyzing the access frequency of the data blocks, and the performance and the economic efficiency of a storage system are remarkably improved by migrating the frequently accessed data to the cache disk and migrating the less accessed data to the low-cost storage; according to the method, unnecessary occupation of high-performance storage resources is reduced, the operation cost is also reduced, heat rating is performed on the data blocks, and the storage positions of the data blocks are correspondingly adjusted, so that the data processing speed is higher, meanwhile, the data security is improved, when a hard disk fails, data recovery is faster, and the service continuity is ensured.
Owner:SHANGHAI FAITH INFORMATION TECH CO LTD

End-to-end TCP monitoring during application migration

A method for performing an application migration operation includes initiating the application migration operation to migrate an application from a source device to a target device, where the source device includes a source data migration agent and the target device comprises a target data migration agent. The method also includes initiating an active scan of network traffic at a transmission control protocol layer. The method further includes establishing a secure data path between the source device and the target device. In addition, the method includes beginning migration of the application from the source device to the target device using the secure data path. Moreover, the method includes making a first determination, using the active scan, that a data packet is associated with suspicious activity and rejecting the data packet based on the first determination.
Owner:DELL PROD LP

Alpine region logging data cloud storage system

The invention relates to the technical field of cloud storage, in particular to an alpine region logging data cloud storage system which comprises a network quality monitoring module, a data grading module, a dynamic bandwidth management module and an intelligent data migration module. According to the invention, by collecting the historical data of the network indexes and calculating the score of the current network state, the network condition is effectively evaluated, the data transmission strategy is optimized, the reliability and efficiency of data transmission are enhanced, priority rating is carried out according to the importance and emergency degree of the data, key data can be transmitted preferentially, the network congestion is effectively reduced, and the data transmission efficiency is improved. The data transmission efficiency is improved, the accuracy and the response speed of data management are optimized, the data access speed is improved by predicting the access frequency of the data and dynamically adjusting the storage medium, the storage cost is reduced, the overall data processing capacity and the cost effectiveness are improved, and the method is suitable for popularization and application. And the security and the quick accessibility of the key data in a severe environment are ensured.
Owner:CHINESE ACAD OF GEOLOGICAL SCI +1

Storage hard disk remote diagnosis system and method based on Internet of Things

The invention discloses a storage hard disk remote diagnosis system and method based on the Internet of Things, and relates to the technical field of health management of the Internet of Things and storage equipment. The system comprises a data fusion module, a causal analysis module, a risk analysis module and a map construction module. The data fusion module collects SMART parameters, IO operation time sequence data and environment data through Internet of Things equipment, dynamically distributes multi-source data weights by using an attention mechanism, and extracts hard disk health state features. And the causal analysis module is combined with the multi-scale time sequence convolutional network and the Bayesian causal network to identify periodic abnormal fluctuation and generate a fault root cause analysis report. The risk analysis module matches historical cases through federal incremental learning, calculates a hard disk fault risk index and generates an early warning signal. And the atlas construction module optimizes resource isolation, data migration and response paths according to the risk indexes, generates a visual operation and maintenance atlas, provides fault positioning, risk links and repair priorities, and improves the operation and maintenance management efficiency.
Owner:SHENZHEN SANSHANG SCIENCE & TECHNOLOGY CO LTD

Decision-making large model-oriented multi-level heterogeneous memory collaborative scheduling method

The invention provides a decision-making large model-oriented multi-level heterogeneous memory collaborative scheduling method, which comprises the following steps of: acquiring hardware configuration information and initial state information of a heterogeneous memory system, constructing a burst memory access model based on an Erlang distribution family, and acquiring dynamic memory access characteristics in combination with self-adaptive stratified sampling; representing the state of the memory system by adopting multi-dimensional efficiency measurement and a hierarchical affinity matrix; predicting load characteristics based on a generalized extreme value theory, and constructing an augmented Lagrange optimization model to generate a resource allocation scheme; formulating a data migration strategy through a Pareto optimization algorithm; and dynamic parameter adjustment of Thompson sampling and an exception handling mechanism based on a combined early warning index are introduced. According to the method, memory access features can be accurately described, resource allocation is optimized, migration overhead is reduced, and large decision model training efficiency is improved.
Owner:NANJING ARTIFICIAL INTELLIGENCE CHIPS RES INST OF AUTOMATION CHINESE ACAD OF SCI +1

Data processing method and apparatus

PCT designated stage expiredWO2025107540A1Input/output to record carriersData storeStorage type
The present application relates to the field of storage, and discloses a data processing method and apparatus. The method is applied to an intelligent tiering system, the intelligent tiering system is used for carrying out intelligent tiering processing on service data in a storage system, and the storage system comprises at least one storage path. The method comprises: acquiring first service data, wherein the first service data is stored in a target storage path in the storage system, and the storage type of the first service data is an intelligent tiering type; acquiring an access log of the first service data; analyzing the access log of the first service data on the basis of a hot and cold identification algorithm configured by a user for the target storage path, to obtain the access popularity of the first service data; and sending a data migration instruction to the storage system on the basis of the access popularity of the first service data, so that a storage node of the storage system performs data migration on the first service data. The present application is used for reducing the impact on the storage system, improving the operation performance of the storage system.
Owner:HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD

Radioactive measurement data processing method for improving uranium exploration efficiency

The invention discloses a radioactive measurement data processing method for improving uranium exploration efficiency. The method comprises the following steps: standardized data acquisition: integrating a multi-parameter module, carrying out time-space synchronous acquisition of geological geophysical environment parameters, standardizing protocol alignment data, and supporting three-dimensional modeling; a three-dimensional coupling model is used for processing interference in a sub-module mode, LiDAR-DEM is used for correcting gamma attenuation in the terrain, an optical fiber thermopermeability instrument is used for correcting daughter errors in the hydrology, signal attenuation of a borehole is compensated through a drilling fluid chart, and a space-time continuous interference field is formed; performing dynamic equilibrium coefficient inversion: constructing equilibrium coefficient isoparametric mapping, and inputting rock core, logging and geochemical data into a random forest model; the transfer learning is trained by using historical data, and dynamic inversion and updating of a new area are carried out; intelligent data processing three-dimensional visualization: self-adaptive denoising and spectral shape matching reinforcement abnormity are carried out; constructing a three-dimensional model, and carrying out transfer learning to optimize the precision; the WebGL platform integrates multi-parameter display, virtual drilling functions and high-dimensional data intuitive interpretation.
Owner:安徽省核工业勘查技术总院

Data hierarchical storage method and device based on access characterization, equipment and medium

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical treatment and health and the like, and discloses a data hierarchical storage method, device, equipment and medium based on access characterization. Writing the data into a matched first target storage device according to a preset device performance corresponding relation; after the data are stored, continuously monitoring subsequent access characteristics of the data, and dynamically judging whether the heat state is changed or not; and if the popularity state is changed and a specific migration condition is met, redistributing the second target storage device and completing data migration operation so as to realize automatic hierarchical storage management of cold and hot data. By acquiring the data access representation and dynamically identifying the popularity state, hierarchical migration of data with different popularity among storage devices with different performances is realized, the resource use efficiency and access performance of a storage system are improved, and the storage cost is reduced.
Owner:PING AN TECH (SHENZHEN) CO LTD

Laser processing parameter autonomous generation system and method based on digital twinning

The invention discloses a laser processing parameter autonomous generation system and method based on digital twinning, and relates to the field of digital twinning, and the method comprises the steps: collecting and preprocessing processing data in real time through a multi-source sensor; processing and analyzing the task instruction by using a natural language, extracting a constraint condition and forming a structured demand; a processing parameter candidate set is generated through a Transform model in combination with historical data transfer learning, and virtual processing is performed by means of multi-physics field simulation; an improved non-dominated sorting genetic algorithm is adopted to dynamically optimize the parameter weight, and a global optimal parameter combination is obtained through digital twin iteration verification; and after full-process virtual processing verification and task demand comparison, parameters are adaptively corrected, and model parameters are continuously optimized according to physical and simulation data deviation after actual processing. The method has the advantages that the digital twin is used as a core, multi-source real-time data, the AI algorithm and multi-physical field simulation are fused, and autonomous generation, multi-target optimization and virtual-real closed-loop iteration of laser processing parameters are achieved.
Owner:CHENGDU MRJ LASER TECH CO LTD

Management method and management apparatus for hybrid memory system, and device and medium

The present application relates to a management method and management apparatus for a hybrid memory system, and a device and a medium. The method comprises: acquiring a write popularity value and an access popularity value of a currently accessed memory page; comparing the write popularity value with a specified write popularity threshold value, and generating write popularity page migration flag information; comparing the access popularity value with a specified access popularity threshold value, and generating access popularity page migration flag information; grouping hybrid memories in a hybrid memory system, and generating, on the basis of memory page information of all grouped memory banks, memory page management information corresponding to the hybrid memories; and on the basis of the write popularity page migration flag information, the access popularity page migration flag information, and current memory page management information corresponding to the currently accessed memory page, determining whether the current access memory page is a popular data page, and performing data migration according to a preset unpopular and popular data page migration strategy. The technical solution can improve the overall performance of a hybrid memory system having a horizontal architecture.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Market supervision data exchange platform migration and security enhancement method and system based on credential environment

The invention discloses a market supervision data exchange platform migration and security enhancement method and system based on a credential environment, and relates to the technical field of network security. In the data migration process, the consistency and privacy protection of the data are ensured through Hash check and a multi-party security computing technology; a domestic commercial cryptographic algorithm and a quantum resistance encryption technology are introduced, so that the protection capability of the platform to existing security threats is effectively improved, and the resistance to future quantum computing threats is enhanced; an intrusion detection system and a virtual honeypot node are integrated, and a defense strategy is monitored and updated in real time in combination with a deep learning model, so that novel attack behaviors can be detected and coped with in real time; according to an intrusion detection alarm and a block chain audit result, a security policy and resource allocation are dynamically adjusted in combination with a machine learning technology, it is ensured that high-risk nodes are monitored in time, resource allocation is automatically optimized, and the resource utilization efficiency and security of the system are improved.
Owner:江苏省市场监督管理局数据中心

Distributed time sequence library data management method supporting cold and hot data level-to-level management

The invention discloses a distributed time sequence library data management method supporting cold and hot data level-to-level management, and relates to the technical field of computer databases, comprising: receiving a write-in request of time sequence data, and executing preliminary write-in data aggregation and sorting; dynamically identifying data cold and hot attributes based on a multi-dimensional data cold and hot degree calculation model in combination with a write-in behavior and an access behavior of time series data; according to the cold and hot attribute recognition result, in combination with the hierarchy boundary of self-adaptive division, automatic hierarchical storage of the time series data is executed; based on a cold and hot data dynamic migration and scheduling mechanism, according to the access frequency and time decay characteristics of the time series data, dynamically adjusting the storage hierarchy of the time series data, and executing a data migration task; and constructing a hierarchical index system adaptive to different cold and hot attribute time series data, and combining query frequency dynamic identification and index structure automatic upgrading. By adopting a cold and hot data automatic identification and hierarchical storage mechanism, the overall performance and the resource utilization rate of the system are improved.
Owner:GUODIAN NANJING AUTOMATION

Data migration method and device, equipment and medium

The invention belongs to the field of big data, and relates to a data migration method, which comprises the following steps: obtaining storage cluster pressure test data, and constructing a performance model, the pressure test data being obtained by testing under different loads. And acquiring real-time monitoring data, and evaluating the current pressure state in combination with the performance model. And if the threshold value is exceeded, extracting a historical data change rule, and generating a load prediction model. And calculating a target migration concurrency number based on the real-time performance index and the prediction model. And adjusting the current data migration concurrence number and the bandwidth limiting parameter to target values. And finally, carrying out data migration of the storage cluster according to a preset protocol and the target parameter. The invention further provides a device, equipment and a medium. In addition, the invention also relates to a block chain technology, and the pressure measurement data and the real-time monitoring data can be stored in a block chain. The data migration efficiency can be improved.
Owner:PING AN TECH (SHENZHEN) CO LTD

Data writing method and device of storage system, storage medium and electronic equipment

The embodiment of the invention provides a data writing method and device of a storage system, a computer readable storage medium and electronic equipment. The method comprises the steps of obtaining historical data writing conditions of a plurality of logic address sections of the storage system; according to historical data writing conditions of the plurality of logical address sections, respectively determining writing heat of the plurality of logical address sections; and performing popularity grading on the to-be-written data according to the write popularity of the plurality of logic address sections to determine the popularity level of each to-be-written data. And writing the to-be-written data with the same heat level into the same physical address section of the storage system. According to the method, the to-be-written data with the same heat level is written into the same physical address section, classified writing of the data according to the heat level is realized, the heat of the data in the physical address section is relatively consistent, so that invalid data can be more effectively recovered due to the fact that the data with different heat is classified during data recovery, and the data recovery efficiency is improved. The data migration amount is reduced, and the recovery efficiency is improved.
Owner:INSPUR SUZHOU INTELLIGENT TECH CO LTD

Channel model dynamic association method based on multi-source data index

The invention discloses a channel model dynamic association method based on a multi-source data index, and relates to channel engineering digital management. The channel model dynamic association method comprises the steps that component codes of channel engineering are constructed and used for identifying the same physical entity; establishing a three-layer index table structure according to the component codes; constructing a dynamic partition management mechanism of a three-level cache architecture, wherein the three-level cache architecture comprises a first-level cache stored in a memory, a second-level cache stored in a local disk and a third-level cache stored in a distributed manner; according to data access information in the three-layer index table structure, data migration is carried out among all levels of caches; according to the dynamic partition management mechanism, a data version management mechanism based on a timestamp concurrency control protocol is established, the data version management mechanism generates incremental data by comparing original data with updated data, and incremental updating or full updating is selected according to a threshold value. According to the method, the problem that data evolution tracking is difficult due to fragmentation of multi-source heterogeneous data and unbalanced access efficiency in navigation channel engineering is solved.
Owner:JSTI GRP CO LTD

Digital country data storage system based on cloud computing

The invention discloses a data storage system of a digital country based on cloud computing, which comprises a client, a data transmission module and a cloud computing center, relates to the technical field of data storage, stores hot data on a high-performance hot data storage layer, ensures that frequently accessed data can be quickly responded, improves the system performance, and at the same time, stores the hot data on a high-performance hot data storage layer. The cold data is stored in a low-performance cold data storage layer, the storage cost is reduced, effective balance of performance and cost is achieved, the data is automatically migrated among different storage layers along with change of data access frequency, so that the performance and the cost are balanced, the frequently accessed data is migrated into a cache, the access speed of the data is increased, and the data access efficiency is improved. And establishing a relationship between a time interval and a data type of historical data, predicting data which are possibly accessed in a given time interval, adding the data into the cache, dynamically adjusting the cache, and ensuring the validity of the cache.
Owner:ZHANGJIAGANG BIG DATA CO LTD

Heterogeneous data migration method, migration device, equipment and medium

The invention provides a heterogeneous data migration method and device, equipment and a medium, and the method comprises the steps: determining a reading path according to a source cluster, and obtaining to-be-migrated source data from the source cluster based on the reading path; determining a required target data mapping rule according to the current data migration scene; constructing an intermediate mapping model based on the data type of the source cluster and the data type of the target cluster, and mapping a field of the source data to a mapping field of the intermediate mapping model by using the target data mapping rule to obtain converted source data; and loading the converted source data into the target cluster by adopting a batch loading mode so as to complete migration of the source data. By means of the method and device, data migration in various heterogeneous database environments is supported, and the efficiency of data migration synchronization in the heterogeneous cluster environment is effectively improved.
Owner:CHINA INFORMATION SAFETY RES INST CO LTD +1

Semantic-based migration and consistency verification method, system and equipment and medium

The invention provides a semantic-based migration and consistency verification method, system and device and a medium, and relates to the technical field of databases. The method comprises the following steps: performing metadata topology scanning analysis by obtaining metadata, including generating an abstract syntax tree and constructing a semantic graph; according to a metadata topology scanning analysis result, performing data mapping and conversion, including loading a YAML rule and generating corresponding type mapping and constraint conversion; data migration is carried out through primary key fragmentation parallel migration, batch writing optimization and real-time double-writing verification; through structure-data-business three-layer verification, simulation business SQL comparison and automatic difference repair, the integrity of migrated data and business compliance are ensured, the semantic gap problem in heterogeneous database migration is solved, high-precision and high-efficiency database migration is realized, the migration efficiency is improved, and the data migration efficiency is improved. The method is suitable for scenes with strict requirements on data consistency and migration efficiency, such as financial, government affair and enterprise-level data centers.
Owner:CHINA YANGTZE POWER

Data migration verification method and system for credential and credential platform

The invention provides a credential platform data migration verification method and system, and relates to the technical field of computers.The method comprises the steps that all to-be-migrated source data in a credential platform database are read, basic information of the source data is obtained, and a data label is set for each piece of data; reading fields and contents in the migration data, and constructing a data similarity verification model; selecting a migration path and a target database according to the migration demand, and dynamically evaluating the similarity between the source data and the target data through a data similarity verification model; verifying the migrated source data based on the target database, and performing incremental verification on data of different fields in the table; the data table of the to-be-migrated source data is verified in real time among different migration nodes, data changes on each node are verified step by step through an increment verification mechanism, if data missing or verification inconsistency occurs in the verification process, an alarm mechanism is triggered, and the problem of data missing or inconsistency can be automatically recognized and corrected.
Owner:HANGZHOU ZHONGER NETWORK TECH CO LTD

Data cold and hot hierarchical adaptive storage optimization system and method based on artificial intelligence

The invention discloses a data cold and hot stratification self-adaptive storage optimization system and method based on artificial intelligence. The method comprises the steps that storage time sequence information of each piece of storage data in storage equipment is collected; inputting the storage time sequence information into a cold and hot state prediction model, and predicting a cold and hot state of the storage data in future time; acquiring the access frequency of the stored data in a unit period; establishing an access frequency analysis set for the access frequencies in the unit period, and extracting access frequency characteristics; the access frequency characteristics are input into a stage data recognition model, a stage judgment result is obtained, and the stage judgment result comprises yes or no; judging whether to migrate the storage data or not according to the staged judgment result of the storage data and the predicted cold and hot state of the storage data in the future time, and if not, judging whether to perform backup or not; data migration or non-migration decisions are reasonably made, frequent and unnecessary data migration is avoided to the maximum extent, and therefore the storage operation and maintenance cost is reduced.
Owner:北京雅弘商业管理有限公司

Task scheduling method applicable to large language model (LLM), and computing device, storage medium and computer program product

Provided in the present disclosure are a task scheduling method applicable to a large language model (LLM), and a computing device, a storage medium and a computer program product. The method comprises: when determining that a target task processing node currently executing a target task meets a resource reporting condition, a target second task scheduler sending a target running resource of the target task processing node to a first task scheduler; when determining that the target running resource meets a task migration condition, the first task scheduler determining a migration task processing node from among a plurality of task processing nodes, and sending node information of the migration task processing node to the target second task scheduler; and the target second task scheduler determining the migration task processing node on the basis of the node information, determining the target task currently executed by the target task processing node, as well as task data of the target task, and migrating the target task and the task data in batches to the migration task processing node for execution, so as to obtain a task processing result of the target task.
Owner:CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD