88 results about "Data life cycle" patented technology

The present invention addresses the deficiencies of the related art by providing a tier-based data storage solution. Specifically, the present invention evaluates the data contained in database tables by analyzing several metrics for the data, and creating a valuation score. The valuation score is used to determine the storage tier that would meet the performance and availability requirements of the database with the goal of reducing the costs associated with storing that data. The metrics that are used to evaluate database tables include (among others): table purpose, data retention, disk requirements, data criticality, data relevance, and desired performance. As such, the present invention will develop a data (lifecycle) storage plan for each set of data. This storage plan not only identifies an initial storage tier where the data should be stored, but also identifies, if, when and to where the data should be moved thereafter.

The invention discloses an intelligent networked automobile information security platform based on end-pipe-cloud, and the platform carries out the security protection for a vehicle end, a cloud platform, and an Internet, and a trusted execution environment and a security level for the domain isolation, the deep defense, and the software and hardware integration are built at the vehicle end. Nodeverification, file layer encryption protection and key management service are established at the cloud end, and SSL/TLS secure communication is used between the nodes and between the nodes and the application program. For a communication network, a PKI-based communication encryption mechanism, a real-time network abnormal flow monitoring mechanism, an end-pipe-cloud integrated intrusion detectionmechanism, a national vulnerability perception and early warning mechanism and a multi-stage collaborative emergency response and online upgrading mechanism are established. And a data life cycle management mechanism, a key security storage management mechanism and a key data security storage management mechanism are established for data of the vehicle end and the cloud end. The establishment of the information security platform is the first example in the field of intelligent connected automobiles at present, and lays a foundation for the construction of national security systems.

Data life cycle policies are developed by classifying data into data classes based upon predetermined data attributes. States are then specified in which the data classes may reside. Components are defined that support one or more of the states. Transfer agents support transferring data from one component to another component. A state transition diagram is prepared for each data class, including one or more conditions that are necessary for each transition between states. An algorithm is applied to the state transition diagram which generates policies that generate life cycle actions if the data or file belongs to the class, the present state of the data or file, and if the conditions for the transitions between the states for each data class have been met. The algorithm provides a method and system for developing data life cycle policies.

An enhanced mechanism for the allocation, organization and utilization of high performance block storage metadata provides a stream of data (e.g., in a server system, storage system, DASD, etc.) that includes a sequence of fixed-size blocks which together define a page. Each of the fixed-size blocks includes a data block and a footer. A high performance block storage metadata unit associated with the page is created from a confluence of the footers. Each footer in the confluence of footers has space available for application metadata, which are provided as one or more information units. At least one of the footers includes a Checksum field containing a checksum that covers at least the confluence of footers. This approach is advantageous in that it provides data integrity protection, protects against stale data, and significantly increases the amount of metadata space available for application use.

An enhanced mechanism for the utilization of high performance block storage metadata in applications such as security and data life cycle management. A sequence of fixed-size blocks defines a page, wherein each fixed-size block includes a data block and footer. A current date is compared with an erase date contained in a high performance block storage metadata unit. The metadata unit is created from a confluence of the footers and associated with the page. At least a portion of the page is erased if the erase date is on or before the current date. In one embodiment, the data is erased with a simple overwrite. In another embodiment, the data is erased using a selectable type of secure erase that ensures the erased information cannot possibly be recovered. Preferably, an erase confirmation (e.g., erase_status and/or erase_completed_date) is written into the metadata unit upon successful completion of the erase step.

System control of compression and decompression of data based upon system aging parameters, such that compressed data becomes a system managed resource with a distinct place in the system storage hierarchy. Processor registers are backed by cache, which is backed by main storage, which is backed by decompressed disk storage, which is backed by compressed disk storage then tape, and so forth. Data is moved from decompressed to compressed form and migrated through the storage hierarchy under system control according to a data life cycle based on system aging parameters or, optionally, on demand: data is initially created and stored; the data is compressed at a later time under system control; when the data is accessed, it is decompressed on demand by segment; at some later time, the data is again compressed under system control until next reference. Large data objects are segmented and compression is applied to more infrequently used data. A dedicated compression input/output processor (IOP) is controlled by host system defined data structures which include a bus transport mechanism (BTM) which is DMA loaded to the compression IOP storage. The BTM includes a request response control block (RRCB) comprising a compress or decompress operation command, a data out descriptor (DOD) for providing the address of data in host storage that needs to be compressed or decompressed, and data in descriptor (DID) for providing the address in host storage where the resulting decompressed or compressed data is to be stored.

The present invention improves the performance of a storage device by utilizing the special features of different types of volumes. The control part of the storage device constructs a virtual volume in the memory. The virtual volume is constructed by a low-speed volume and a high-speed volume, and has the same size capacity as the low-speed volume. When the application program of the host requests the updating of data, the write data is respectively written into low-speed and high-speed volumes, and a life tag that indicates the storage period is set beforehand in the virtual volume. When the storage period has elapsed, the data is erased from the high-speed volume. The data in the low-speed volume is maintained “as is”. While the data is held in the high-speed volume, access from the host is processed utilizing the data in the high-speed volume.

The invention provides a channel big data oriented collection system and collection method. The system comprises a smart collection device which is deployed around a channel and is used for collecting data; a configuration management center module which is used for configuring the configuration information of the smart collection device; a processing center module which is used for obtaining and updating the configuration information and carrying out protocol analysis, conversion and forwarding on the collected data according to the configuration information; an instruction issuing center module which is used for carrying out instruction assembly according to protocol information of the smart collection device searched by a service system and issuing an instruction to the smart collection device in a network or short message mode, wherein the instruction assembly is realized in a factory mode; and a data life cycle monitoring center module which is used for collecting log information of a whole data collection and instruction issuing process, analyzing the log information through utilization of a big data processing technology and compressively monitoring the life cycle of the collected data.

The invention relates to a method and a device for tracing the whole link of a distributed system log. The method comprises the following steps: S1, creating a configuration file configuration system,and executing according to the following flow: configuring a cluster server; Configuring cluster dependencies; Configuring data logic relationships; 2, read and parsing that configuration file by anapplication program to obtain the key information; S3, acquiring the life course of the system data according to the key information. The apparatus includes a system configuration unit, an informationacquisition unit, and a cluster traversal unit. Abstracts full-link tracing of data logs into three configuration files, These configuration files can be read and parsed by the corresponding application program to obtain the server information corresponding to each cluster, System upstream and downstream sequence and the logical relationship between the system data these key information can be achieved in no code burial point one-click access to a system data life cycle and positioning online or testing process encountered problems.

The invention discloses a solid-state disk garbage recycling method based on a data life cycle. The solid-state disk garbage recycling method comprises the following steps: dividing a solid-state disk into an area Area_timegc for carrying out garbage recycling operation according to the data life cycle and a normal area Area_normalgc; finely dividing the Area_timegc into N sub-sections, such as S1, S2,...SN, by using progressively increasing time, such as T0, T1,...T<N-1>, wherein the sub-section SN is a section located between T<n-1> and Tn and the SN is a section between the T<N-1> and the T0; when the solid-state disk transmits a writing request, recording a predicated deadline T<deadline> of data to be written by a file system; comparing a T<deadline> sequence with the T0, T1,...T<N-1>; if the T<n-1> and Tn which meet the conditions that the T<deadline>is more than the T<n-1> and is smaller than or equal to Tn exist, writing the data into the sub-section Sn determined by the T<n-1> and the Tn; if the T<deadline> is more than the T<N-1> and is smaller than or equal to kT<N-1>, writing the data into the SN; if the T<deadline> is more than or equal to the kT<N-1>, writing the data into the area Area_normalgc; with regard to the area Area_timegc, carrying out the garbage recycling operation based on time on the sub-sections S1, S2,...SN in sequence along the operation of the system, and circularly utilizing the N sub-sections; and with regard to the Area_normalgc area, carrying out existing recycling operation of other garbage.

The invention discloses an innovative method for management of scientific and technological resources. By analyzing the attributes, characteristics and current situation of scientific and technological resources, it adopts the principles of "data life cycle theory", "requirement hierarchy theory of scientific and technological resources" and "big data processing" for scientific and technological resources. Three research methods, put forward the overall structure of scientific and technological resources innovation management, realize the integration, optimization and sharing of scientific and technological resources, standard system construction, quality management, ecological chain management, performance management and other innovative measures and methods, in order to achieve the maximum utilization rate of scientific and technological resources change. The data life cycle theory includes the generation period, integration optimization period, storage period, service period and decline period; the science and technology resource demand hierarchy theory divides the use demand of science and technology resources into acquisition demand, possession demand, interaction demand and quality demand and sharing requirements; the big data processing divides scientific and technological resources into data layer, processing layer and application layer, in order to achieve hierarchical, integrated and intensified data resources.

An exemplary system includes a plurality of heterogeneous data subsystems maintaining local data and a coordinator subsystem configured to maintain global data mapped from the local data. The coordinator subsystem includes a global data life cycle model defining at least one life cycle for the global data. The coordinator subsystem is configured to use the global data life cycle model to coordinate a data update between the global data and the local data. In certain implementations, the coordinator subsystem and the heterogeneous data subsystems are configured to communicate using at least one data update message representative of the data update, the data update message being defined in accordance with the global data life cycle model. In certain implementations, the data update message includes a life cycle status identifier associated with the data update.

Disclosed herein are systems and methods that integrate distributed ledger (or blockchain) capabilities in a highly distributed computing environment, such as mobile, Internet of Things (IoT). A platform allows for such integration, supporting blockchain-enabled transactions between IoT and mobile devices. Particularly, the platform is a Distributed Ledger System for e-Transactions (DLSeT). The DLSeT platform includes various blockchain features, such as computing, distributed ledgers (storage), cryptocurrency, and cryptography (homomorphic encryption). Furthermore, the DLSeT platform utilizes a Lattice Face Secret Key Infrastructure (LFSKI) feature that is distinct to the platform. By employing LFSKI, the DLSeT platform realizes advantages over blockchain platforms using n public key infrastructure (PKI), being able to execute secure, efficient, and decentralized e-transactions. The DLSeT platform supports additional features, such as a Zero Knowledge Triangle Flow protocol, a Prove of Simple Universal Wallet Address protocol, and a Data Life Cycle security with its unique Data life cycle framework.

The invention provides a data processing method and device. According to the method, after to-be-processed data is obtained, feature data of the to-be-processed data is extracted, wherein the featuredata comprises at least one of features of the to-be-processed data in a time dimension or a data access frequency dimension; and based on the feature data, a pre-trained data processing model is utilized to process the feature data so as to determine the life cycle state of the to-be-processed data. According to the scheme, the pre-trained data processing model is utilized to determine the life cycle state of the to-be-processed data based on at least one of the features of the to-be-processed data in the time dimension or the data access frequency dimension, therefore, when the life cycle state of the to-be-processed data is determined, the law of the corresponding relation between at least one of the features of big data in the time dimension or the data access frequency dimension and the life cycle state of the big data is utilized specifically. Compared with the prior art, labor cost is lowered, and the determined data life cycle state can have high accuracy.

The embodiment of the invention provides an internetwork data security sharing and management method and system, and belongs to the technical field of data security. According to the embodiment of theinvention, by means of a dedicated system, data cross-domain sharing is realized; the efficiency is high; the misjudgment rate is low; the management cost is low; effective control on large-scale andbatch data cross-domain sharing can be carried out; online unified implementation can be realized through an integrated dedicated system; the process is full-automatic; internal docking is carried out between different processes; the security risk in a docking process is low; starting from data security, the data life cycle perspective is introduced; targeted systematic effective protection can be formed for data generation, data transmission, data storage, data using and data destruction processes of specific data; after obtaining the specific data, a data demander uses the specific data only in a security environment, so that the specific data is effectively controlled; an audit mechanism for a data cross-domain sharing process is complete; and thus, the problem in the data cross-domainsharing process can be found effectively.

The invention relates to the field of databases, in particular to a database system based on a cloud teaching platform, which comprises a data acquisition layer for acquiring data of each business end; a storage and calculation layer which is used for hierarchically storing the acquired data according to data granularity and calculating themes, dimensions and indexes according to specific data analysis requirements; a management and control layer is used for collecting and calculating time scheduling of tasks, management and maintenance of metadata and management of a data life cycle; and an access layer which is used for providing an access port and sharing and exchanging data with other data platforms. By using the method and the device, the following effects can be realized: the acquired data is stored in a layered manner according to data granularity; the data structure is clear, when the data is used, the data granularity can be quickly positioned according to the data level, andthe blood relationship analysis of the data is facilitated; and the acquired data is monitored to judge whether the data has a problem or not.

The invention discloses a cabling rack for a college user group data life cycle management system and an installation method of the cabling rack. The cabling rack comprises a mounting base and a hanging plate; a wiring seat is slidably connected to the hanging plate; the wiring seat comprises a U-shaped wiring plate, a cover plate, a fixing assembly and a sliding frame, the cover plate is arrangedon the U-shaped wiring plate, the fixing assembly comprises a supporting plate, a pressing plate, end screws and fastening nuts, the end screws are symmetrically distributed between the supporting plate and the pressing plate, the supporting plate is arranged at the bottom of the U-shaped wiring plate, the fastening nuts press the pressing plate, and the pressing plate presses the sliding frame on the cover plate; and the installation method comprises the following steps: (a) preliminarily assembling the wiring seat, (b) enabling a corner reference to be in place, (c) continuously arranging,(d) arranging wires, (e) capping, and (f) performing reinforcement. The cabling rack is stable in structure, high in installation adaptability and high in bearing strength, cables are arranged in order and convenient to manage, the installation method is low in operation difficulty, simple in positioning, high in installation efficiency and easy to control and adjust, and the installation qualityis guaranteed.

The invention provides a big data quality standard management control method, which includes establishing a quality comprehensive evaluation model, wherein the quality comprehensive evaluation model comprises a coverage rate submodel, a public trust submodel, an omission rate submodel, a fineness submodel, a consistency submodel, an accuracy rate submodel, a repetition rate submodel, a null valuerate submodel and an update frequency submodel; receiving test data, inputting the test data into a quality evaluation model, sequentially evaluating the test data according to each submodel in the quality evaluation model, and generating an evaluation result; and presenting the evaluation result of the test data to a user for viewing in the form of a graph list. The big data quality standard management control method provided by the invention realizes the quality standard management of each circulation link of data production, data testing, data storage and data service in a data life cycle.

The invention provides a service data publishing method and device. The service data publishing method comprises steps of receiving service subscription registration request transmitted by a service subscription terminal, wherein the service subscription registration request is processed by one of data life cycle sensing servers in a data life cycle sensing server cluster in order to generate subscription data, synchronizing the subscription data to the rest of the data life cycle sensing servers in the data life cycle sensing server cluster, receiving service data and heartbeat data which are provided by a service supply terminal, wherein the service data and the heartbeat data are processed by one of the data life cycle sensing servers in the data life cycle sensing server cluster, pushing the service data to a corresponding service subscription terminal according to a subscription data and determining a survival status of the service data according to the heartbeat data. The service data publishing method and device can solve a problem that pressure on the service providing terminal is too big, guarantee high efficiency and delay of data transmission and save performance loss.

The invention provides a big data life cycle setting method and device, a storage medium and a server, and the big data life cycle setting method comprises the steps: periodically collecting metadataof a bottom storage system, and extracting the state information of the metadata; aggregating the metadata according to the state information to generate an aggregated graph of user operation storagedata; and identifying a hot data area according to the dense area of the convergence graph, determining hot data of the big data storage system according to the hot data area, and setting a life cycleof operating the storage data by the user according to the hot data. Effective identification of the hot data of the big data storage system is realized, a user is helped to set the optimal life cycle of operating the storage data by the user, the operation is simple and easy, the accuracy is high, and the storage cost of the big data storage system is reduced.

The invention discloses an internet globally unique identifier generation system and a generation method thereof. The internet globally unique identifier generation system comprises a user identity acquisition module, a globally unique ID distribution module and an application layer module. The internet globally unique identifier generation system is based on business cooperation with each third-party Internet service provider, establishes a globally unique identifier database which is longer in life cycle and across screens and platforms, through a public interface provided by a third-party Internet service provider, and through various recognized unified identification modes such as Cookie, a mobile device ID and a user identifier provided by a third party on the premise of reserving theanonymity of the internet user in the actual network behavior, thus effectively solving the problems of existing similar products in the aspects of data openness, data effectiveness and data stability. Compared with the prior art, the internet globally unique identifier generation system has the advantages of being larger in size, higher in data precision, longer in data life cycle and larger indata openness.

The embodiment of the invention provides a data life cycle management method and device, a medium and electronic equipment, and relates to the technical field of data storage. The method comprises thesteps of obtaining to-be-processed metadata from a cluster database, and determining a life cycle state of the to-be-processed metadata; determining initial metadata to be deleted in a target life state from the to-be-processed metadata ; packaging the initial metadata to be deleted to generate an initial logic deletion task, and performing logic deletion processing on the initial logic deletiontask; determining the initial logic deletion task subjected to logic deletion processing as a logic deletion task, and generating a physical deletion task according to the logic deletion task; and acquiring state parameters of the distributed file storage system, and performing physical deletion processing on the physical deletion task according to the state parameters. The invention provides an implementation scheme of Hive life cycle governance by means of isolation, asynchronization, fine grit and self-adaption.

The embodiment of the invention provides a data life cycle management method and device. The method comprises the following steps that: first equipment calculates the data active duration of a data table based on the access condition of the data table; when the creation duration of the data table is greater than the data active duration, the first equipment determines a target processing strategybased on the type of the data table; and the first equipment performs data life cycle management on the data table according to the target processing strategy. Therefore, the operation of manual intervention is reduced, the processing efficiency of data life cycle management is improved, the security of data archiving or data cleaning is ensured, and the cost pressure of enterprise storage is reduced.

The invention relates to a distributed storage system supporting multi-file system mounting, which comprises: a file space management module for dividing a plurality of file subspaces of different usescenes in a root file space of a file arrangement layer, and recording path mapping of a file directory level; the metadata management module which is used for maintaining files mounted on the file arrangement layer; the label management module which is used for recording and maintaining business meaning labels related to files stored in the distributed file storage system; the data life cycle management module which is used for processing storage strategies of different data life stages of cold and hot temperature data of the files in the file storage system, packaging and compressing the cold data and putting the cold data into the object storage system, putting the temperature data into an original underlying file system where the files are located, and loading the hot data into a memory for preparation at any time, the object storage system is used as a cold data storage system, a unique identifier is allocated to each object of the object storage system, and a server or an end user is allowed to retrieve the objects.

The embodiment of the invention provides a distributed object storage method and system, and the method comprises the steps that an intelligent data life cycle management subsystem obtains data in a standard data storage subsystem; the intelligent data life cycle management subsystem classifies the data according to a preset rule and cold and hot degrees and stores the classified data into the corresponding medium storage systems according to categories: when the data is created, the data is divided into hot data, low-frequency data or archived data; and for accessible data, according to the page view of the data real-time log record, the data is divided into hot data, low-frequency data or archived data. According to the scheme, the data are respectively stored in different storages according to the cold and hot degrees, cold and hot separation of the data and mixed storage of various media are realized, so that the storage efficiency and the data safety are improved, and meanwhile, the energy consumption and the storage cost are also greatly reduced.

The embodiment of the invention provides a distributed file cleaning method, device, and system, and is applied to the technical field of artificial intelligence. The method comprises the following steps: checking objects in a distributed file in a main hot cluster according to a configured data life cycle table, and determining to-be-processed objects; counting the to-be-processed objects; if the proportion of the number of the objects in the total number of the objects is greater than or equal to a preset proportion threshold value, marking the distributed file as a target cleaning file; according to the data life cycle table and a processing mode corresponding to a preset object state, cleaning objects in the target cleaning file according to the processing mode. Thus, HDFS files can be partially deleted, storage resources can be released, cluster storage spaces can be saved, and the storage cost can be reduced.

The invention provides a digital delivery method based on a process pipeline, and the method comprises the steps that a design unit carries out the primary delivery of a digital delivery platform of aconstruction unit, and a construction unit carries out the secondary delivery of construction information according to the data format of the digital delivery platform of the construction unit; the construction unit forms all the data into a multi-dimensional completion model to realize a closed loop of a data life cycle and a digital twinning body of intelligent operation, maintenance and management; the method has the advantages that a pipeline installation drawing, a pipe section drawing and construction data can be obtained in time in the prefabrication process, and prefabrication work before construction can be conducted better; in terms of material management, engineering materials are scientifically and intelligently distributed, and material management disorder is reduced; in terms of construction management, through series tracking management of prefabricated pipe sections and field installation, construction delay and rework and repair are reduced, and the operation cost isreduced; through three-dimensional model management, the project progress and the real-time state are visually mastered, and the project management efficiency is improved.