545results about "File system functions" patented technology

The invention discloses a distributed file management system and method, and belongs to the technical field of computer software, the system adopts a tree hierarchical structure to organize files, and the system comprises an application layer used for receiving file operation requests from a plurality of independent clients, and each request carries a unique application identifier; the file system layer comprises a configuration module, an operation module, a metadata management module, a day module, a timed task module and a gateway module, and the request is routed to a corresponding logic storage partition in the data layer according to an application identifier; the data layer comprises a database cluster and an object storage cluster, and performs persistent storage and management of logic isolation on metadata and file data according to application identifiers; and the infrastructure layer provides an extensible server and network resources. Based on a distributed architecture, load balancing and efficient access are realized through data storage partitioning, a metadata management and index technology and a client load balancing technology, and the expandability, fault tolerance and performance of the system are improved.

The invention relates to the technical field of distributed file management, and discloses a college cross-campus distributed file collaborative management system. The system comprises a file operation sensing module, a collaborative strategy decision module and a distributed execution engine module. The file operation sensing module is deployed at each campus file server node, captures user editing operation and converts the user editing operation into an operation feature vector set with a timestamp; the collaborative strategy decision-making module generates a conflict probability matrix by analyzing the operation feature vector, and generates a transmission path scheme and a real-time synchronization instruction in combination with a network state and load data; and the distributed execution engine module is responsible for analyzing the instruction, adjusting transmission parameters, processing conflicts and generating a global audit tracking report. According to the system, refined perception, conflict pre-judgment, intelligent transmission, dynamic synchronization and comprehensive auditing of cross-campus file operation are realized, the efficiency and reliability of college cross-campus file collaborative management are improved, and the system is suitable for a multi-campus college file collaborative management scene.

The invention provides a complex construction site construction equipment control system and method, and the system constructs a basic state semantic model of each type of equipment, and covers all perception and control instruction data; the comprehensive management and control platform dynamically extracts and issues one or more data packet formats from the basic model according to construction scenes, areas and equipment types, and fine adjustment of sensing information and instruction formats is achieved; the edge computing device is deployed at an equipment end or a cluster end and is responsible for dynamic mapping analysis, redundancy removal, priority screening and unified protocol packaging of a multi-heterogeneous protocol; the equipment end reports a null instruction sensing data packet, the comprehensive management and control platform generates an optimal control instruction, compresses recent sensing data and then issues the compressed recent sensing data in the same data packet format; the equipment end decides whether to execute the instruction or not, if not, problem expression is fed back, and a safe closed loop is formed; in a cluster scene, gathering fusion and instruction distribution of multi-level edge computing devices are supported, and efficient collaboration of heterogeneous clusters is achieved.

The invention relates to an electric power cold and hot data judgment optimization method and system based on multi-dimensional data elements, and the method comprises the following steps: S1, building an automatic baseline of data assets and business attributes, and constructing a unified data asset portrait table and an access portrait table; s2, based on the unified data asset portrait table and the access portrait table, constructing multi-dimensional element features required by cold and hot judgment; s3, constructing a cold and hot judgment model, outputting a cold and hot temperature score for each data object according to the multi-dimensional element features, and performing grading according to the score; s4, according to the grading list obtained in the step S3, under the constraint of the SLA, by taking the minimum total cost and the optimal performance as targets, generating an executable layering and migration plan; and S5, based on the layering and migration plan, layering and migration are executed, and an operation log and an audit record are recorded. According to the invention, strong-adaptability cold and hot data management in a complex power business big data scene is realized.

The invention provides a database audit log analysis method, electronic equipment, a computer readable medium and a product, and the analysis method comprises the steps: carrying out the pre-filtering of an original audit log set based on a preset operation type, and obtaining a to-be-analyzed log set; performing rule screening on the to-be-analyzed log set according to a preset safety threshold to obtain a high-risk log set; performing semantic understanding and risk mode recognition on database operation in the high-risk log set by combining a safety threshold through the fine-tuned database semantic risk analysis model, and generating a structured risk judgment result containing risk types, confidence coefficients and key features; and when the confidence reaches an early warning threshold value, taking the corresponding log as a clue, carrying out multi-dimensional association traceability in the original audit log set, and generating a visual audit risk report fusing an association operation behavior chain, an access resource set and session information. According to the invention, the closed loop of real-time identification, active alarm and rapid traceability is realized, and the response efficiency of database audit is improved.

Methods, systems, and devices for data management are described. A data system may capture a first snapshot of a target computing system at a first time, and a second snapshot at a second time that is later than the first time. The system may compare the second snapshot with the first snapshot to identify whether a set of deleted files includes a quantity of deleted files that satisfies a file deletion threshold. Using metadata associated with the target computing system, the system may identify a subset of the set of deleted files as corresponding to one or more system files. The DMS may then determine whether a ratio between a first value of a deletion metric and a second value of the deletion metric satisfies a threshold ratio. In some examples, the system may refrain from generating an alert based on the ratios satisfying the threshold ratio.

A method for managing log files for recording operations on data stored in a database is provided, wherein a set of log files having an allocated first portion of storage is updated, the allocated first portion of storage is monitored, and a second portion of storage is allocated in dependence on a determination that an available portion of storage is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot in dependence on a determined relative order of log records in a set of log files. A method of replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record comprising data indicating the binary large object. A method for deleting one or more binary large objects is provided, wherein the data indicating one or more binary large objects that have been logically deleted are stored in a snapshot and the one or more binary large objects are deleted using the data indicating the one or more binary large objects after a previous snapshot is deleted.

The application relates to a trajectory data compression and decompression method, device, equipment and medium, and is applied to the field of maps. The method comprises the following steps: acquiring a trajectory point representing an object trajectory; matching the trajectory point to a road network of an electronic map to obtain at least one object trajectory section; the object trajectory section is a section passed by the object in the road network; performing thinning processing on the at least one object trajectory section according to section characteristic information corresponding to each object trajectory section to obtain a key trajectory section; the key trajectory section is a part of the object trajectory section and is used for restoring the object trajectory; and the key trajectory section is compressed and stored to obtain trajectory compression data. The method can save storage resources.

The application discloses a metadata cluster expansion method, device, equipment, medium and program product; the method comprises the following steps: obtaining a system state snapshot of a metadata cluster by performing periodic storage processing on metadata information of the metadata cluster; obtaining an operation log entry, the operation log entry comprising data change operation records sorted by time after the storage of the system state snapshot; determining whether the metadata cluster is expanded based on running data of the metadata cluster; if the metadata cluster is expanded, a first node is determined, the first node being a node to be added to the metadata cluster; and performing information synchronization on the first node based on the system state snapshot and the operation log entry, so that the first node after the information synchronization is consistent with the system state of the metadata cluster; thus, the expansion process is realized.

The embodiment of the invention provides a data processing method. The method comprises the steps that service data and monitoring configuration information of a service system are acquired, the service data indicate the duration consumed by at least one node in the service system to complete the service, and the monitoring configuration information indicates the timeliness constraint for the at least one node; and executing a monitoring task for the at least one node, the monitoring task comprising: based on the service data and the monitoring configuration information, determining whether the duration consumed by the at least one node to complete the service satisfies the corresponding timeliness constraint; and sending a first alarm notification based on alarm configuration information which indicates an object of the first alarm notification and a mode of the first alarm notification in response to the fact that the duration consumed by the at least one node to complete the service does not meet the corresponding timeliness constraint, the first alarm notification indicates that the duration consumed by the at least one node in the service system to complete the service does not meet the corresponding timeliness constraint. Therefore, the time consumption monitoring efficiency of the service node can be improved.

A processing device writes file system data to a first area of a memory zone of a zoned block memory device based on a current position of a write pointer within the memory zone. The file system data comprises data files contained within a file system and file structure metadata describing a file structure of the file system. The processing device detects a write event based on the write pointer advancing past a predetermined memory address within the memory zone that corresponds to a checkpoint. The checkpoint is a second area within the memory zone that is designated for storing write event data. Based on detecting the write event, the processing device writes write event data to the checkpoint, the first write event data indicating a most recent memory address of a root node of the file structure within the memory zone.

The invention belongs to the technical field of rail train driving, and provides a safety assessment method and system for a rail train, terminal equipment and a medium. The method comprises the following steps: acquiring original syslog log data of a plurality of vehicle-mounted devices in a rail train, and clustering the original syslog log data according to event keywords corresponding to each vehicle-mounted device to obtain a plurality of key syslog log data; and for each piece of key syslog log data, extracting data features of the key syslog log data, and inputting the data features into a pre-trained safety evaluation model to obtain a safety evaluation result of the rail train. According to the method, the abnormal event behavior of the vehicle-mounted equipment can be accurately identified, the consumption of operation resources is reduced, and the information safety problem of each vehicle-mounted equipment in the running process of the rail train is solved.

A fingerprint that includes a plurality of hashes is generated for a file. A deduplicated hash index is utilized to determine a similarity between the file and a plurality of files. An action is performed based on the determined similarity.

A fine-grained file system and a file reading and writing method are provided. When one or more first files require subfile reading or writing, a corresponding reading or writing operation is performed on subfiles of each first file by utilizing a double-layer index structure of this first file, implementing concurrent reading and writing functionality for large individual files, resulting in faster speeds and improved efficiency for file reading and writing.

The application discloses a time series data processing method and system, a storage medium and an electronic device. The method comprises the following steps: in response to a client request, obtaining time series data to be processed from a time series database, wherein the time series data is generated by a data source according to a time period and stored in the time series database; storing a sharding result corresponding to the time series data in a message queue, wherein the sharding result is used to enable the message queue to store a corresponding log file; and writing the sharding result read from the message queue into a memory. The application solves the technical problem of low data storage efficiency.

A method in a data processing system and apparatus for organizing files, data items, web site members, or web pages, structured or unstructured, of multiple users stored across one or more server computers into hierarchical file structures on a recordable medium of a data processing system. A user-defined metalabel is assigned to each of the electronic files, data items, web site members, or web pages. The electronic files, data items, web site members, or web pages are organized as a function of the metalabels into the hierarchical file structures. The method includes automatically weighting of individual metalabels as a function of a relevance of the at least one searchable term found in the corresponding one of the electronic files, data items, web site members, or web pages. Searching the plurality of hierarchical structures can then be done with a query including at least one term and a relevance threshold.

This application discloses a method, device, and storage medium for file comparison and synchronization. The file comparison and synchronization method of this application is applied to a comparison device. Upon receiving first information, the comparison device locates at least one target server and establishes a connection with it. Based on the target file information, it retrieves a reference file and the target file from the target server, synchronizes the reference file and the target file, thereby updating the reference file. This method only requires login authentication to the target server and retrieving the file from the target server to the comparison device, enabling simple and efficient file retrieval and comparison to update the reference file. It avoids the complex operations required by high-level server privileges, which are difficult to use, or the need to repeatedly build file libraries to ensure synchronized file updates.

The application relates to the technical field of data processing, and discloses a large file export optimization method and system based on real-time memory monitoring. The method comprises the following steps: monitoring a memory state in real time and calculating a memory growth rate; taking the growth rate as an independent risk dimension, and giving a warning when the rate exceeds a threshold value and the memory usage rate does not reach a threshold value; performing adjustment in response to the warning, including dynamically updating the throughput control of the data shard size, and triggering the degradation mode to reduce the export field to the data precision control of the core field; and attaching an instruction page to the export file during degradation. Through trend prediction and dynamic control, the application reduces the risk of memory overflow and guarantees core data export.

This disclosure generally relate to a method and system for network policy simulation in a distributed computing system. The present technology relates techniques that enable simulation of a new network policy with regard to its effects on the network data flow. By enabling a simulation data flow that is parallel and independent from the regular data flow, the present technology can provide optimized network security management with improved efficiency.

Systems and methods for providing trusted local orchestration of workspaces are described. In some embodiments, an information handling system (IHS) can include: a processor; and a system memory coupled to the processor, the system memory having program instructions stored thereon that, when executed, cause the IHS to: receive, from a workspace orchestration service, orchestration code; use a trusted controller coupled to the processor to record a log, the log including: the orchestration code, and an indication of a sequence of operations performed by a local management agent during instantiation of a workspace; provide, to the workspace orchestration service, a copy of the log; and in response to the workspace orchestration service successfully: (i) authenticating the orchestration code, and (ii) verifying the sequence of operations, establish a connection between the workspace and the workspace orchestration service.

A parallelized data input sub-system of a database system includes a first set of loader nodes of pluralities of computing nodes of a plurality of computing device clusters. The first set of loader nodes includes a plurality of memory devices and a plurality of processing modules. The first set of loader node is operable to ingest at least a portion of a dataset (“data”), a set of the memory devices stores the data, and a set of the processing modules determines whether the data is regarding a query. When it is regarding the query, the set of processing modules provides the data to a query and response sub-system. When it is not regarding the query, the set of processing modules determines long term storage parameters, processes the data in accordance with the parameters to produce formatted data, and provides the formatted data to a store and compute sub-system.

This application relates to the field of train monitoring technology, and discloses a method, device, and system for obtaining display time difference, to solve the technical problem of low timeliness in obtaining display time difference of vehicle arrival information. The method includes: parsing the operation log obtained from the automatic train monitoring system to obtain the transmission time of the vehicle arrival information issued by the automatic train monitoring system and the platform code corresponding to the vehicle arrival information; obtaining the operation log of the passenger service system, and querying the broadcast controller information of the platform broadcast controller corresponding to the platform code based on the passenger service system's operation log, the broadcast controller information including the broadcast controller code; obtaining the operation log corresponding to the platform broadcast controller through the broadcast controller code; querying the display time when the platform broadcast controller pushes the vehicle arrival information to the broadcast controller's interface in the operation log corresponding to the platform broadcast controller; and obtaining the display time difference of the vehicle arrival information based on the transmission time and display time of the vehicle arrival information issued by the automatic train monitoring system.

The embodiment of the invention discloses an application management method and device, electronic equipment and a computer readable storage medium, and belongs to the technical field of application management.According to the embodiment of the invention, a dynamic loading path and a static loading path of a target application are obtained, and the static loading path and the dynamic loading path are fused; according to the embodiment of the invention, the target loading path of the target application is obtained, and the compressed file of the target application can be decompressed according to the target loading path in response to the starting instruction of the target application. Thus, by optimizing the loading path of the target application, the optimized target loading path can better adapt to the actual use scene of the target application, and meanwhile, the compressed file of the target application can be decompressed more efficiently and reasonably, so that the starting delay of the target application is effectively reduced, and the use experience of a user is improved.

System receives data stream, groups sequential data segments associated with data stream until initial sequence of data segments is formed which is larger than minimum size. System groups sequential data segments which are next after initial sequence of data segments until next sequence of data segments is formed which combined with initial sequence of data segments is larger than maximum size. System determines feature value for each data segment in next sequence of data segments. System selects value from feature values, and data segment corresponding to selected value. System divides next sequence of data segments at selected data segment into part of initial group of data segments and part of next group of data segments. System combines part of initial group of data segments with initial sequence of data segments as initial group of data segments. System routes initial group of data segments or group of corresponding fingerprints for deduplication.

The present disclosure relates to a database log processing method, device, electronic equipment and medium, and relates to distributed storage and cloud computing technology, the method comprises: obtaining an attribute value of a delete lock attribute of an operation log in a database; the delete lock attribute is used to describe a backup state of the operation log in the database; according to the attribute value, determining an incremental operation log of the database; performing backup processing on the incremental operation log; if the backup of the incremental operation log is successful, updating the attribute value of the delete lock attribute; according to the attribute value of the delete lock attribute, recycling the operation log storage space. This method can ensure that part of the operation log recycled during storage is already backed up, regardless of how frequently the operation log is updated, and can achieve real-time and complete backup of the operation log of the database, thereby meeting the requirement of data rollback to any time point.

The invention relates to a log processing method based on dynamic triggering and index driving, and relates to the technical field of computer software, and the method comprises the following steps: coding log entries through a dictionary coding rule, generating coded log entries, and writing the coded log entries into a plaintext log file; monitoring a file state of the plaintext log file, and when the file state meets a preset dynamic triggering condition, compressing the plaintext log file to generate a compressed log file; generating an index file associated with the compressed log file, wherein the index file comprises metadata; and in response to a user query request, positioning and compressing a target data block in the log file based on the metadata, decompressing and reversely decoding the target data block, and filtering decoded log contents according to the query mode to output a log query result. By means of the scheme, development complexity is reduced, efficient real-time compression is achieved, full scanning is avoided, consumed time in the index positioning stage is reduced from the minute level to the millisecond level, and query efficiency is improved.

The application discloses a kind of impedance test file generation methods, comprising the following steps: S1: Genesis software retrieves ERP file and TGZ file from database;The impedance information of PCB board is contained in the ERP file, and the layer information of PCB board is contained in the TGZ file;S2: Genesis software identifies ground point according to TGZ file;S3: Genesis software identifies single-ended impedance line and differential impedance line according to ERP file, and calculates its length;S4: the coordinates of the single-ended impedance line and the end point coordinate of ground point in the preset range are positioned, and the two end point coordinates of differential impedance line in the preset range are positioned;Single-ended impedance line, differential impedance line and corresponding coordinates are formed CSV file output;S5: CSV file is decoded as DIF file and is output.The impedance test file generation method provided by the application can automatically output ground point and impedance line, and can output all data at a time, to ensure that impedance test file can be quickly and effectively generated.

The invention relates to the technical field of data visualization, in particular to a security visual monitoring method based on big data. The method comprises the following steps that original data streams of a plurality of service systems are obtained and classified according to source types, a data partition set is formed, and the data partition set comprises a plurality of data partitions; then extracting basic attributes of each data block to generate a data activity feature table, constructing a data sensitivity visual mapping unit, and converting feature weights into a safe visual mapping graph; monitoring an intensity change abnormal area in the mapping graph, and determining a safety risk area; and generating risk warning information according to the risk area, and outputting the risk warning information to the safety visual monitoring terminal to realize real-time risk identification and early warning. Through multi-source data fusion modeling, sensitivity visual mapping, adaptive anomaly aggregation detection and a risk feedback closed loop mechanism, real-time visual perception, accurate identification and dynamic management and control of security risks in a big data environment are realized.

The invention provides a device for automatic metadata management and data knitting treatment, and the device comprises a semantic mounting unit which is used for carrying out the semantic tag mounting of a structured metadata set of a data factory based on a semantic model of a data knitting architecture, so as to generate enhanced semantic metadata and a complete data line record; the rule binding unit is used for carrying out governance rule binding on the enhanced semantic metadata and the complete data line record on the basis of a data knitting federation governance rule so as to generate a metadata governance list bound with the governance rule; the change updating unit is used for synchronously updating a data set state field and a flow path recorded by a corresponding complete data line to a metadata governance list based on a change data capture mechanism when it is monitored that a data factory has a data set change, so as to generate unified metadata caalog; the interface configuration unit is used for configuring a metadata retrieval interface and a governance rule execution engine for the unified metadata caalog so that the unified metadata caalog can be accessed in the data weaving environment.

A method for managing log files for recording operations on data stored in a database is provided, wherein a set of log files having an allocated first portion of storage is updated, the allocated first portion of storage is monitored, and a second portion of storage is allocated in dependence on a determination that an available portion of storage is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot in dependence on a determined relative order of log records in a set of log files. A method of replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record comprising data indicating the binary large object. A method for deleting one or more binary large objects is provided, wherein the data indicating one or more binary large objects that have been logically deleted are stored in a snapshot and the one or more binary large objects are deleted using the data indicating the one or more binary large objects after a previous snapshot is deleted.