1317results about "File system functions" patented technology

The invention relates to the technical field of archive management, in particular to an archive full life cycle management system and method based on cloud computing. The method specifically comprises an intelligent classification module based on a format adaptation degree function and semantic density analysis, a five-stage life cycle management module, and an authority scheduling module fusing RBAC / ABAC authority control and block chain auditing, and based on a matrix decomposition archiving static coding mechanism, document security storage and analysis are realized through an orthogonal matrix and pseudo-inverse operation, and the file security is realized. The data security in a long-term archiving scene is improved; cloud storage resources are dynamically optimized according to the document life cycle state and the access frequency, and the storage cost is reduced in combination with a cold and hot layering strategy; and an AI algorithm is introduced, the archive retention value is automatically evaluated according to laws and regulations and semantic analysis, and a compliant destruction decision is realized. According to the method, through multi-model fusion metadata extraction, dynamic strategy binding and automatic state transition, the whole-process management efficiency and safety are remarkably improved.

The invention relates to the technical field of intelligent manufacturing, discloses a papermaking equipment fault tracing method and system based on a process knowledge graph, and discloses the papermaking equipment fault tracing method and system based on the process knowledge graph. The method and the system comprise data acquisition and preprocessing, papermaking process knowledge graph construction, fault event detection and matching, fault tracing and propagation path reasoning, and maintenance scheme recommendation and optimization. The method overcomes the limitation that the traditional method is difficult to capture the cross-equipment, cross-process and cross-time sequence deep causal association and fault propagation path of the papermaking equipment. By constructing a comprehensive papermaking process knowledge graph, equipment operation state data, process parameter data, production quality data, equipment structure principle, process flow knowledge, fault mode knowledge, maintenance experience and other heterogeneous knowledge are subjected to deep fusion and semantic association, so that the system can exceed the correlation of the data surface; and an internal mechanism and a propagation chain of the fault are deeply excavated.

The invention relates to the technical field of data processing, in particular to a cross-unit data management method, which comprises the following steps of: performing format standardization processing on data in a data pool of each organ and unit, converting the data into a uniform standard format, extracting semantic features to construct a semantic association network, and recording a semantic relationship of data items and a dynamic authorization and authority mapping mechanism. Then, according to the inquirer identity information, the unit attributes, the business requirements and the data sensitivity level, dynamically generating inquiry authorization and establishing an authority mapping table; and when a query condition is received, analyzing by utilizing a natural language processing technology, and generating a query route and optimizing a statement in combination with the semantic association network and the permission mapping table. And finally, calling data from each data pool according to the route, performing fusion processing, displaying the data to a querier in real time, and recording a query log. The invention aims to solve the problems of data semantic isolation, authority management staticization, low query efficiency and insufficient cross-unit data conflict resolution capability in the existing cross-unit data management.

The embodiment of the invention provides a memory recovery method and device, electronic equipment and a storage medium, and relates to the technical field of data storage. Under the condition that both the file system and the storage device support the perceptual garbage collection function, the device state of the storage device is obtained in response to the starting of the perceptual garbage collection function; according to the equipment state, determining whether an execution condition for sensing a garbage collection function is met or not; and if the execution condition of the garbage collection sensing function is met, the file system is linked to perform memory collection on the storage device, so that the file system can know the data blocks moved by the storage device, the file system is prevented from moving the data blocks moved by the storage device again, and the efficiency of the file system is improved. In addition, invalid blocks in the file system and the storage device can be synchronized, the situation that invalid data blocks in the file system are moved when the storage device executes memory recovery is avoided, and therefore the service life expenditure of the storage device and the write-in amplification value of the storage device are reduced.

The invention discloses a transaction processing mechanism optimization method and system for time series data, and belongs to the technical field of databases, and the method realizes a differentiated ACID support mechanism, strengthens atomicity and durability guarantee, and weakens consistency and isolation requirements; comprising the following steps: designing a collaborative architecture of a transaction system and a pre-write log WAL, and establishing a binding relationship between a transaction state and WAL writing; generating a WAL record only for the successfully submitted transaction operation, and performing persistent storage by adopting a batch group submission strategy; transaction scheduling is realized through a lightweight transaction manager, and the life cycle of a transaction request is managed; the functions of efficient writing of transaction granularity, log storage and rapid recovery are realized through a pre-writing log manager; through intelligent Checkpoint background service, based on a periodic incremental check point mechanism, compression merging and space recovery of the WAL file are realized. According to the invention, the performance breakthrough of the time sequence database in a high-concurrency write-in scene can be realized.

The invention discloses a log template post-processing method and system based on semantic screening and similarity clustering. The method comprises the steps that log information is input into LLM to generate a log template; inputting the new log template and the cache log template into a semantic model, and respectively generating semantic embedding vectors; calculating a similarity distance between the new log template and the cache log template, and screening to obtain a candidate template; according to the generation confidence score, calculating the weighted structure similarity between the new log template and each candidate template, and clustering and combining the candidate templates exceeding a threshold value to generate a final template; and updating and replacing the final template into the cache structure. According to the technical scheme, the semantic similar templates can be recognized and combined more reasonably, efficiently and accurately, a semantic enhanced, controllable and stable template post-processing mechanism can be constructed on the premise that LLM calling burden is not increased, and automatic cache cleaning, template number control and structural consistency maintenance are achieved.

The invention discloses an unstructured storage hierarchical strategy optimization method based on machine learning, and relates to the technical field of data storage management, and the method comprises the steps: monitoring a file access event in real time, generating an access log, and extracting a multi-dimensional feature data set; utilizing the trained multi-dimensional decision tree model to distribute a corresponding storage hierarchy for the storage object to obtain a storage hierarchy decision; according to a storage level decision, distributing the storage objects to different storage layers, and carrying out resource configuration and storage operation; monitoring the access condition of the storage object in the new storage hierarchy in real time, and collecting file access performance, storage cost and response time to obtain feedback data; by monitoring the file access event in real time and extracting the multi-dimensional feature data set containing the basic attribute, the access behavior and the context information, efficient response of hotspot data and reasonable utilization of resources are ensured, and the overall performance utilization rate and the cost effectiveness of storage are remarkably improved.

Systems and methods are provided for generating metadata for an improved data structure that enables improvements to generative AI and LLM processing and use of the data. A system may receive a log file generated by an entity associated with a device and generate metadata of the log file that comprises origination data associated with the entity and location of the device. The system may store the metadata and content of the log file as a document object. A data structure may be generated using various data objects. For example, the system can generate a vector embedding object enriched with a temporal embedding, a positional embedding, or a hierarchy of the metadata. The system may initiate an action based on the vector embedding object.

The invention discloses a configuration management method of a network file system, electronic equipment, a medium and a product, and relates to the technical field of network file systems.The method comprises the steps that a configuration file is analyzed, configuration information obtained through analysis is stored according to a preset format, and a first file is obtained; the preset format is used for indicating memory layouts of different configuration information, and the memory layouts of the different configuration information are continuous, so that the configuration file is converted into a pre-compiled first file, and the configuration information in the first file has the continuous memory layouts, and thus when the content of the first file is loaded, the content of the first file is loaded. In this way, the memory for storing the loaded content is continuous, the memory fragmentation rate is reduced, moreover, due to the fact that the first file is obtained by analyzing the configuration file in advance, the configuration file does not need to be analyzed when the configuration loading instruction is received, time consumed for analyzing the configuration file is avoided, the loading efficiency is improved, and therefore the time consumption of analyzing the configuration file is reduced. The technical problems of low configuration file loading efficiency and high memory fragmentation rate can be solved.

The invention discloses a one-time plot rule mining method and device for a process event log, and belongs to the field of plot mining. The distance between two events is limited by adopting a time interval constraint, and repeated use of the events is avoided by adopting a one-time condition so as to avoid mining excessive meaningless plots. Moreover, when a strong one-time plot rule (i.e., a one-time plot rule with confidence greater than or equal to a minimum confidence threshold) is mined, a triple mechanism of candidate plot generation, support degree calculation and rule generation is provided. In a candidate plot generation stage, eliminating redundant candidate plot extension by adopting a plot connection strategy; in the support degree calculation stage, the support degree is calculated based on the position index; in a rule generation stage, firstly, all frequent one-time plots are mined, and then strong one-time plot rules are screened according to a minimum support degree threshold value and a minimum confidence coefficient threshold value. The technical effect of efficiently and accurately mining the strong one-time plot rule is achieved.

The invention discloses a file integrity monitoring method, electronic equipment and a storage medium. The method comprises the steps of sending a verification request to a scheduling node through a user node when a file integrity verification condition is triggered; determining an edge node group through the scheduling node according to the edge node load condition, and distributing an audit task corresponding to the verification request to each edge node in the edge node group; searching corresponding integrity evidence according to the auditing task through the edge node, and sending the integrity evidence to the scheduling node; consistency verification is carried out through the scheduling node according to the integrity evidence fed back by each edge node, and if the consistency verification is passed, the integrity evidence is sent to the user node; and performing local verification through the user node according to the integrity evidence, and determining a corresponding file verification result. By means of the method, the safety, traceability and credibility of file integrity monitoring in the domestic terminal environment are effectively improved, resource occupation is low, and compatibility is high.

The invention discloses a database traffic auditing method, device and equipment and a storage medium, and relates to the field of databases, and the method comprises the following steps: deploying a target extension program in a kernel layer of an operating system, and capturing a target data packet by using a target probe in the target extension program; performing protocol analysis on the target data packet to extract target data, determining execution time of a target SQL statement based on the target data, and performing semantic analysis on the target SQL statement by using a target analyzer and a way of performing analysis based on syntax imported based on a bypass to obtain a corresponding semantic analysis result; the target data comprises a target SQL statement and target metadata; and storing the target data, the execution time and the semantic analysis result to a target log database, so as to audit the flow of the target database based on the data in the target log database by using a pre-configured target audit strategy. According to the method and the device, efficient, safe and real-time auditing of database traffic is realized.

The invention relates to the field of log data processing and analysis, and discloses a log analysis method, system and device and a storage medium, and the method comprises the steps: collecting log data from a multi-source device, and carrying out the cleaning and structural processing of the log data; generating a dynamic topology network in which nodes represent the equipment and edges represent the equipment interaction based on the interaction relationship between the equipment of the log data; extracting topological characteristics based on the dynamic topological network, wherein the topological characteristics comprise connectivity of a topological structure, a ring structure and complexity characteristics of the network; analyzing network topology characteristic changes in different time scales, and identifying an abnormal mode in the log data; and constructing a dynamic causal network between the devices, extracting an abnormal propagation chain, and positioning root cause devices or events which cause anomalies. The method can adapt to various log formats and multi-source data, supports real-time analysis and anomaly detection, improves the intelligence and efficiency of system log analysis, and provides powerful technical support for system troubleshooting and operation and maintenance management.

The method comprises the steps of obtaining a to-be-uploaded file and a corresponding file type, preprocessing the to-be-uploaded file, generating a to-be-audited file, calling a corresponding auditing model in an AI knowledge base for auditing according to the type of the to-be-audited file, and generating a corresponding auditing result; the multi-hop query request of the user is obtained according to the auditing result, a modification suggestion is generated, and finally the modification suggestion is returned to the user and is encrypted and stored, and more intelligent and accurate auditing is achieved by means of the powerful knowledge representation and reasoning ability of the knowledge base. The modification suggestions provided when the auditing is not passed can help the user to quickly understand the problem and perform targeted modification, so that the efficiency of the content creator is improved, the times of repeated submission and auditing are reduced, the auditing cost is reduced, and the quality and efficiency of the whole auditing process are improved.

The invention discloses an intelligent medical big data information management system, and the system comprises a data collection layer which serves as a sensing neural network, employs an Internet of Things gateway to be compatible with a medical equipment protocol, and deploys an edge calculation node for physiological signal preprocessing; the intelligent preprocessing layer is used for cleaning and standardizing the original data transmitted by the data acquisition layer; the informatization service layer is used for realizing fine-grained authority control based on an RBAC (Role Based Access Control) model, realizing inspection process automation by integrating an LIS (Line Information System) interface through a Hyperdger Fabric block chain evidence storage key operation hash, and constructing a zero-footprint DICOM (Digital Imaging and Communications in Medicine) viewer; the intelligent analysis layer is used for carrying out data life cycle management by adopting a mixed storage architecture, carrying out feature engineering on data, carrying out label matching and classified storage by using efficient neighbor search and similarity / distance measurement, quantifying contribution of each feature or feature combination to a single prediction result through an SHAP value, generating an interpretation result, and outputting the interpretation result; and presenting an interpretation result to a clinician or related personnel through a query interface.

The invention relates to the technical field of system application, and discloses an online document processing method and system based on structured storage, and the method comprises the steps: splitting content elements of an online document into blocks, and carrying out structured storage as independent objects; querying basic information of the document, loading data according to a document block pageBlockId list, creating a component according to the loaded data, constructing a component tree, and performing document rendering by using a lazy loading technology; the method comprises the following steps: abstracting documents and document blocks as objects, synchronizing the objects to a graphic database, modeling a relationship between the documents and the document blocks as an edge in the graphic database, and converting the relationship between the documents into a graphic data structure; when the document content changes, the storage interface is called to store the changed document block, and the online document is automatically stored, the document content is structurally stored in the block form, dynamic updating of the online document is achieved, the reference relation between the documents is stored and managed, and the relation between the documents is visually and clearly displayed.

The invention belongs to the technical field of resource management, and particularly discloses a multi-service domain integrated customizable multi-tenant management system and method, and the system comprises a multi-tenant architecture which is configured to be used for defining different service domains as different service systems, and splitting a plurality of tenants according to different standards under each service system, different organizations are divided under each tenant, and each organization corresponds to different users; the main data management module is configured to be used for defining the dimension relation of basic data according to the relation of tenants, organizations and users under each business system within each year range, and distributing corresponding basic data to each business system; and the portal control module is configured to be used for realizing login operation and entrance authority control of the user. Tenant splitting can be performed according to different standards under different business systems, so that customized design of a multi-tenant framework is realized, and tenant splitting requirements of different business systems can be met.

The embodiment of the invention provides a database log processing method and device, a storage medium and electronic device.The method comprises the steps that based on the total number of logs within a specified log range requested to be queried by a current log query request and the number of query threads in a plurality of query threads, the query threads are queried according to the total number of the logs within the specified log range; segmenting a log requested to be queried by the log query request into a plurality of log ranges, and respectively constructing corresponding query statements to obtain a plurality of query statements; executing the corresponding query statements in parallel through a plurality of query threads, and querying log data in a corresponding log range from a log database; receiving at least one batch of log data returned by the log database through the data receiving thread, and writing the log data into the buffer block queue in a circular writing mode; and executing processing operation on the log data written in the buffer block in a circular processing mode through a data processing thread, and summarizing and sorting the processed log data to obtain a log query result corresponding to the log query request.

Methods, systems, and devices for data management are described. A data management system (DMS) may be employed to manage data associated with one or more computing systems. The DMS may generate metadata when capturing snapshots, which may contain information corresponding to files that have been created, deleted, modified, or otherwise changed since a prior snapshot. The DMS may use learning techniques and experimentation to perform data engineering to analyze metadata such as to perform anomaly detection. The DMS may implement techniques for efficiently storing and organizing metadata for use with machine learning processes—e.g., to support efficiently extracting datasets useful for machine learning experimentation. To obtain a dataset, the DMS may read metadata from different partitions of a storage environment in parallel, and accordingly, metadata may be stored in separate partitions based on customers and time ranges (e.g., to support such parallel reading).

The invention discloses a mirror image volume data synchronization method and system, electronic equipment and a storage medium, and relates to the technical field of computers, and the method comprises the following steps: determining a target synchronization algebra value matched with application scene information of a mirror image volume, and positioning recent snapshot data of a corresponding algebra in a main space according to the target synchronization algebra value, according to the method, the latest snapshot data of the corresponding algebra is used as the to-be-synchronized data, and finally the to-be-synchronized data is synchronized to the auxiliary space, so that the main space and the auxiliary space of the mirror image volume keep complete historical snapshot data, even if the main space breaks down, the auxiliary space can provide the historical snapshot data, and the high availability of the mirror image volume is improved.

The present disclosure relates to systems, methods, and non-transitory computer readable storage media for implementing a scalable, secure, efficient, and adaptable distributed digital ledger transaction network. Indeed, the disclosed systems can reduce storage and processing requirements, improve security of implementing computing devices and underlying digital assets, accommodate a wide variety of different digital programs (or “smart contracts”), and scale to accommodate billions of users and associated digital transactions. For example, the disclosed systems can utilize a host of features that improve storage, account / address management, digital transaction execution, consensus, and synchronization processes. The disclosed systems can also utilize a new programming language that improves efficiency and security of the distributed digital ledger transaction network.

The invention provides a multi-source log association fusion method, system and device based on safety behavior semantic analysis and a medium, and the method comprises the steps: taking an obtained original multi-source host log as the input of a pre-constructed BERT basic model, and obtaining an attribute corresponding to each log text; constructing a preliminary incidence relation graph corresponding to the original multi-source host log text; performing mapping processing on the original multi-source host log text by using the custom log semantic descriptive attribute to obtain a deep semantic attribute space; constructing a deep semantic information association graph by using the obtained preliminary association graph and the deep semantic attribute space; performing clustering analysis on the obtained deep semantic information association graph to obtain a plurality of sub-communities; performing semantic feature fusion on the obtained multiple sub-communities to obtain a security event description corresponding to the original multi-source host log; according to the method, the problems of information loss and difficulty in semantic understanding in the multi-source heterogeneous log are effectively solved, and the association analysis capability of the log is remarkably improved.

The invention discloses a distributed file management system and method, and belongs to the technical field of computer software, the system adopts a tree hierarchical structure to organize files, and the system comprises an application layer used for receiving file operation requests from a plurality of independent clients, and each request carries a unique application identifier; the file system layer comprises a configuration module, an operation module, a metadata management module, a day module, a timed task module and a gateway module, and the request is routed to a corresponding logic storage partition in the data layer according to an application identifier; the data layer comprises a database cluster and an object storage cluster, and performs persistent storage and management of logic isolation on metadata and file data according to application identifiers; and the infrastructure layer provides an extensible server and network resources. Based on a distributed architecture, load balancing and efficient access are realized through data storage partitioning, a metadata management and index technology and a client load balancing technology, and the expandability, fault tolerance and performance of the system are improved.

A resource dependency system displays two dynamically interactive interfaces in a resource dependency user interface, a hierarchical resource repository and a dependency graph user interface. User interactions on each interface can dynamically update either interface. For example, a selection of a particular resource in the dependency graph user interface causes the system to update the dependency graph user interface to indicate the selection and also updates the hierarchical resource repository to navigate to the appropriate folder corresponding to the stored location of the selected resource. In another example, a selection of a particular resource in the hierarchical resource repository causes the system to update the hierarchical resource repository to indicate the selection and also updates the dependency graph user interface to display an updated graph, indicate the selection and, in some embodiments, focus on the selected resource by zooming into a portion of the graph.

The invention relates to the technical field of file storage management, and discloses a Lustre architecture-based high-performance file storage and processing method and system, the method comprises the following steps: obtaining spatio-temporal characteristics of file access; predicting a popularity value of the file based on the spatial-temporal characteristics; determining a storage strategy of the file based on the popularity value of the file; carrying out distributed storage on the file based on the storage strategy, and continuously detecting a node state; if it is detected that the node load is unstable, adaptive data migration is carried out on the node with the unstable load; if the node is detected to have a fault, generating a data recovery path based on the three-dimensional topological coordinates of the node, and migrating a file stored in the node having the fault based on the data recovery path; according to the method, the storage strategy is determined through the file popularity value, and the abnormal condition of the node is efficiently processed, so that the file storage and processing performance and efficiency are comprehensively improved.

The invention discloses an on-line anomaly detection method and device for an autonomous controllable power business system and a medium, and relates to the technical field of log data anomaly detection.The method comprises the steps that original log data of a power system is obtained, the original log data is preprocessed, and sample log data is obtained; based on the sample log data, performing fine tuning on the pre-trained large language model in a self-supervised learning mode to obtain a to-be-used large language model; based on the sample log data, training the to-be-used large language model in a supervised learning mode to obtain a target large language model; and performing anomaly detection on the newly added log data in the power system through the target large language model to obtain a target anomaly detection result. The problems that a traditional anomaly detection method is high in data requirement and labeling cost, low in mobility, poor in expandability and insufficient in generalization ability to an abnormal mode are solved, dependence on labeled data is reduced, mobility and expandability are improved, and the model generalization ability is enhanced.

The invention relates to an AI processing method for NAS equipment and an NAS system, the method is used for an NAS control unit in the NAS equipment, the NAS control unit realizes butt joint with an AI model in a mode of communication connection with an AI processing module, and the method comprises the following steps: the NAS control unit calls a user operation layer in a first form, generates a first AI demand, and sends the first AI demand to the AI model; the AI model receives a first AI demand, converts the first AI demand into an AI instruction, and feeds back the AI instruction to the user operation layer in the first form; the user operation layer in the first form is converted into a user operation layer in a second form based on an AI instruction; and the NAS control unit calls the user operation layer in the second form as an AI processing result. According to the scheme, a super-large-scale model or data set can be processed without being limited by local hardware configuration, and the functions of data security and privacy protection can be achieved in a multi-user access environment at low cost.

The embodiment of the invention discloses a log data processing method and device, electronic equipment and a readable storage medium. According to the embodiment of the invention, through the buffer area group comprising the at least one working buffer area and the at least one idle buffer area, it can be ensured that data competition conflicts do not exist in the data during access, lock synchronization overhead is avoided, and the performance during log collection is improved. Moreover, when the predetermined event occurs, the log data in the time period when the predetermined event occurs can be stored in the file system, and most of the log data in other time periods are filtered, so that the proportion of effective data is increased, and the storage data volume of the log data is reduced. Therefore, the system analysis efficiency can be effectively improved through the embodiment of the invention.

The invention provides a log monitoring method which can be applied to the technical field of artificial intelligence and the technical field of distribution. The method comprises the following steps: based on a stream processing architecture, processing log data acquired in real time in a distributed manner through micro-service; wherein the micro-service comprises a preprocessing micro-service, a feature extraction micro-service and a real-time monitoring micro-service. Standardizing the log data through the preprocessing micro-service to obtain standard semantic data; processing the standard semantic data based on a deep semantic extraction model through a feature extraction micro-service to obtain deep semantic feature data; wherein the deep semantic extraction model is constructed based on an attention mechanism model, an auto-encoder and a graph neural network; and monitoring the dynamic change of the log data in real time based on an online learning mechanism by monitoring the micro-service in real time, and positioning the abnormal log based on the deep semantic feature data. The invention further provides a log monitoring device and equipment, a storage medium and a program product.

The invention provides a real-time processing method and system for sales data of an online shop, and relates to the technical field of data processing, and the method comprises the steps: receiving the sales data from a web server of the online shop, and storing the sales data in a database; real-time changes of a database are monitored in real time through an ERP system, database updating is captured through a binlog, and updated sales data are sent to a message queue system; receiving the updated sales data from the message queue system, and distributing the updated sales data to a plurality of processing modules to obtain a plurality of processing results, the processing modules including a real-time processing module, a full-amount processing module and a special processing module; and dynamically updating and displaying the processing result according to a user demand. By implementing the method, real-time processing and dynamic updating of the data can be realized, and merchants are helped to know and respond to market changes in time, so that decision-making efficiency and accuracy are improved, and customer satisfaction and market competitiveness are enhanced.