495 results about "Log management" patented technology

The present invention relates to the technical field of computers, and provides a log processing method and system, a log management platform, and an electronic device. The method comprises: acquiring real-time log data of each service system, and storing the log data in a message queue; according to a preset time window, writing the log data in the message queue into a target file of a specified storage directory; and in response to completion of data writing of a current time window, on the basis of the storage directory of the target file, calling a log analysis interface of a log analysis service to perform log analysis on the target file. The log processing method in the present invention reduces the risk incident rate of service systems, and improves the localization accuracy of an anomaly condition of a service log.

The invention relates to the technical field of power system data processing and network security, in particular to a system log data anomaly detection and management system and method. The system comprises an atlas establishment module, an anomaly identification module and a feedback adjustment module. According to the method, the global knowledge graph is constructed through the graph establishment module, the deviation degree is calculated in combination with historical baselines, direct association anomalies are judged, the potential influence degree between nodes is calculated based on a graph neural network propagation mechanism through the anomaly recognition module, and non-direct association anomalies are recognized through feature vector cosine similarity. A multi-dimensional feature matching rule base is combined to determine an anomaly type and execute response measures, a feedback adjustment module collects execution data through buried points, quantifies and evaluates effectiveness, optimizes a scheme through feature backtracking and root cause positioning and relies on digital twinborn simulation and gray scale landing to realize scheme iteration, and a feedback adjustment module performs feedback adjustment. The accuracy of anomaly detection and the dynamic adaptation capability of response measures are improved, and the method is suitable for log management and risk prevention and control of a complex system.

The embodiment of the invention discloses a log management method and device, equipment and a storage medium. The method comprises the steps that multiple pieces of to-be-stored log information generated by the electronic rearview mirror are acquired, and scenes corresponding to the multiple pieces of to-be-stored log information are determined; determining storage priorities corresponding to the multiple pieces of to-be-stored log information based on the scenes corresponding to the multiple pieces of to-be-stored log information and preset scene priorities; if the storage priorities of the multiple pieces of to-be-stored log information are the same, obtaining data sizes and receiving time corresponding to the multiple pieces of to-be-stored log information; and determining a storage sequence of the to-be-stored log information based on the data size and the receiving time, and managing and recording the to-be-stored log information based on the storage sequence. According to the embodiment of the invention, the log information in an important scene can be ensured to be stored preferentially, the importance of the data is considered, the timeliness of the data is also considered, the overall processing speed of the log can be improved, and delay or resource waste caused by an improper processing sequence is reduced.

The invention discloses a transaction processing mechanism optimization method and system for time series data, and belongs to the technical field of databases, and the method realizes a differentiated ACID support mechanism, strengthens atomicity and durability guarantee, and weakens consistency and isolation requirements; comprising the following steps: designing a collaborative architecture of a transaction system and a pre-write log WAL, and establishing a binding relationship between a transaction state and WAL writing; generating a WAL record only for the successfully submitted transaction operation, and performing persistent storage by adopting a batch group submission strategy; transaction scheduling is realized through a lightweight transaction manager, and the life cycle of a transaction request is managed; the functions of efficient writing of transaction granularity, log storage and rapid recovery are realized through a pre-writing log manager; through intelligent Checkpoint background service, based on a periodic incremental check point mechanism, compression merging and space recovery of the WAL file are realized. According to the invention, the performance breakthrough of the time sequence database in a high-concurrency write-in scene can be realized.

The invention relates to the technical field of Internet of Things terminals, and discloses an Internet of Things equipment security access method based on a mobile communication network, which specifically comprises the steps of Internet of Things equipment identity authentication, Internet of Things equipment permission setting, encryption of data transmitted by Internet of Things equipment, and encryption of the process of data transmission of the Internet of Things equipment. A data backup and recovery system is established through a safety auditing system and auditing log management of the system, and the operation state, network communication quality and the like of the Internet of Things equipment are monitored and analyzed in real time through a performance monitoring system and then optimized. The method can enhance the security of equipment identity authentication, improve the confidentiality and integrity of data transmission, reduce the security risk and vulnerability utilization, improve the flexibility and expandability of the system, enhance the defense capability and emergency response speed of the system, and improve the user experience and data reliability. And the security and the reliability of the Internet of Things equipment when the Internet of Things equipment accesses the mobile communication network are ensured.

The invention provides a multi-core log management architecture and method based on a power gap operating system and a computer program product, which are suitable for the technical field of computers, and comprise a master core, a slave core and a shared memory area, the slave core further comprises a log generation module, a buffer module and a log uploading module; the main core is responsible for operating a power gap operating system; the slave core is responsible for running a naked running program or a real-time operating system; the log uploading module writes target log data in the buffer module into a shared memory area when the data cached in the buffer module reaches a preset condition; under the condition that the master core monitors that the slave core uploads the target log data to the shared memory area in the shared memory area, the target log data in the shared memory area is read, and integration, classification and storage operation is conducted on the target log data in sequence. Therefore, the efficiency problem of log collection, storage and management in the multi-core heterogeneous architecture can be solved, the characteristics of the master core and the slave core are fully utilized, and the log processing capacity of the system is improved.

The invention relates to the technical field of charging pile monitoring management, and discloses a charging operation management platform based on data analysis, and the platform comprises a data collection module which obtains and uploads the multi-source data of a corresponding charging pile in real time; the data preprocessing module is used for carrying out normalization processing on the multi-source data and filling missing values; the feature extraction and analysis module is used for extracting and acquiring statistical features of the multi-source data according to a time sequence based on a sliding window algorithm; the AI analysis module is used for outputting a fault probability and a residual life prediction value according to statistical characteristics by adopting a multi-task deep learning model optimized by transfer learning; the early warning analysis module is used for triggering graded early warning according to the fault probability, and is linked with the digital twin platform to generate a processing plan; and the log management module is used for recording an alarm event, a processing state and an equipment historical health degree curve, multi-dimensional data is subjected to cross validation through a multi-task deep learning model, potential faults of the charging pile can be recognized in real time, active early warning can be performed, and the operation and maintenance efficiency and safety are improved.

The invention discloses a voice platform full-link log management analysis method and system, and relates to the technical field of information, and the method comprises the steps: obtaining log data; storing the log data by using a log storage engine; analyzing the log data through a machine learning algorithm; visually displaying the log data; and log security control. According to the method, a distributed architecture is adopted, and a collector and a storage engine are deployed at a log generation source, so that comprehensive collection and storage of full-link log data are realized, real-time monitoring and analysis are ensured, and data collection efficiency and data comprehensiveness are improved; by utilizing the efficient storage engine, the system can process a large amount of log data, and the query efficiency is improved; by introducing a machine learning algorithm, intelligent analysis and mining of log data are realized, and potential problems and optimization points in the log data are found.

The invention provides a large model training system and method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a configuration file inputted by a user, and analyzing the configuration file; based on the analyzed configuration file, configuring the target plug-in; scheduling a data set preprocessing module, a training module and a log management module based on the configured target plug-in, and training a target large model based on the called module; the data set preprocessing module is used for preprocessing the training data set according to a preset standardized data processing flow specification; the training module is used for training a to-be-trained target large model based on a standardized training data set according to a preset standardized model training process specification; and the log management module is used for processing log data of the large model training system according to a preset standardized log processing specification. The method is high in flexibility and expandability, customized model training can be achieved, and user experience is improved.

The invention relates to the technical field of temperature control valve fault detection based on big data analysis, and discloses a temperature control valve fault detection method based on big data analysis. Acquiring and cleaning temperature data according to a set sampling interval; calculating a temperature difference in combination with a set temperature and verifying a baseline period sample; extracting a maximum temperature difference, a minimum temperature difference and an envelope width by using a sliding window in the baseline period, and determining a compensation constant; calculating a baseline oscillation coefficient and statistical characteristics based on window analysis; continuously extracting real-time oscillation coefficients during operation; dynamically generating a time-varying threshold value according to historical characteristics and real-time parameters; and comparing the real-time coefficient with a threshold value to judge a fault and generate a structured log, and finally uploading the structured log to a big data platform. Through systematic data preprocessing, multi-scale feature extraction and dynamic threshold adjustment, the method overcomes missing report and false report under the influence of a fixed threshold and noise, and realizes online closed-loop automatic fault early warning and log management.

The invention discloses a text log management method and device based on an embedded system, equipment and a medium. According to the text log management method based on the embedded system, a storage medium is provided, the storage medium is divided into storage areas, and the storage areas are divided according to partitions; the management method comprises a writing method, a quick retrieval method and a data reading method. The method can be conveniently applied to an embedded system with relatively few hardware resources, the contradiction between embedded system development and large-capacity log data storage requirements is solved, meanwhile, the requirement that field maintenance is simple and easy to use can be met, and the method can be suitable for various storage media, occupies few system hardware resources, can be rapidly deployed, and has high practicability. And the method also has the advantages of quick retrieval, sequential writing, data reading, automatic coverage, few occupied resources and the like.

The invention provides a log management system which can be applied to the technical field of data processing. The system comprises a management controller of a server, and the management controller is used for executing the following operations: running a log acquisition engine, and the log acquisition engine is used for acquiring log information generated by each thread of an application process in the management controller in a business request processing process; the log information is transmitted to an identifier allocation engine, and the identifier allocation engine allocates a corresponding storage identifier for each piece of log information through a shared variable by using a lock-free operation mechanism shared by multiple threads in an application process; the log information with the storage identification is input into a storage engine, and the storage engine writes a function call chain of the log information into a corresponding storage position in a memory space pre-allocated to the application process when the log information and the function call relation change according to the storage identification; and managing the log information based on the function call chain in the storage engine through a log management engine.

The invention belongs to the technical field of computer system software, and particularly relates to a fault monitoring system based on an embedded Linux operating system, which is used for visually monitoring the embedded Linux operating system. The fault monitoring system comprises a monitoring data acquisition module, a fault diagnosis and processing module, a log management module and a visualization module. According to the method, the change of the system state can be captured and fed back in time, rapid fault judgment and processing are realized, and a visual graphical interface is provided, so that the usability is improved.

The invention provides a distributed storage log management method and device, electronic equipment and a storage medium, and relates to the technical field of distributed storage. Dynamic configuration information of a specified to-be-collected log type, collection granularity, a collection time range and a log data output mode is received; after the log collection task is generated, the task is scheduled to the target edge node to be executed according to the comprehensive load score of the main node and the edge node, and collection parameters are dynamically adjusted in the collection process according to the running state fed back by the target edge node and the system resource condition. The problems that in the prior art, log collection requirements in different scenes cannot be flexibly adapted due to dependence on static configuration, node load is too high and response delay is caused by adoption of a centralized architecture, and log collection task execution abnormity or resource waste is likely to occur due to lack of a dynamic adjustment mechanism based on node running states and system resources can be solved.

The invention provides a Mosquitto log management system based on an ARM processor, and the system comprises a log generation end which is used for generating log data and controlling a sending behavior through a condition triggering mechanism; the log storage end is used for storing the log data generated by the log generation end; the user control end is used for sending a log viewing request and a storage level adjusting instruction; the log transmission module is used for transmitting log data to the log storage end or the user control end and transmitting a control instruction to the log generation end; the ARM processor is used for executing log data processing and analysis and coordinating operation of each module; and the log generation end only sends key log data to the log storage end when not receiving a request of the user control end, and sends full log data to the user control end when receiving a log viewing request of the user control end. According to the method, the flexibility and the resource utilization rate of the log system can be improved, the storage cost and the system load are reduced, and the expandability and the stability of the system are enhanced.

The invention relates to an extensible system for credible management of academic / educational credentials, and belongs to the field of block chain technology and credible management of academic / educational credentials. The system comprises a user layer, an application layer, a contract layer, an under-chain storage layer, a block chain bottom layer and a management layer. The user layer supports a user to perform user registration, identity verification, certificate service and identity management; the application layer provides a certificate management operation function, an intelligent contract operation function, role and authority management and an API interface; the block chain bottom layer is based on an FISCO-BCOS platform, an alliance chain architecture is adopted, and a z-PBFT consensus mechanism is used; the management layer relates to monitoring and management of nodes in the block chain network, dynamic adding or exiting of the nodes and log management. According to the method, the block chain storage cost is effectively reduced, the expansibility and performance of the system are improved, the credibility and integrity of the data are considered, and the method is suitable for a large-scale distributed academic / educational certificate credibility management scene.

The invention relates to the technical field of computers, in particular to a method and system for achieving large-scale node monitoring based on the ELK Stack technology, and the method comprises the following steps: data collection, message queue buffering, data processing, data storage, and visual display and alarm. The method has the advantages that large-scale node monitoring is achieved based on the ELK Stack technology, the problem of pain points of a traditional log management mode is solved, and a comprehensive operation and maintenance monitoring solution is provided for enterprises through the efficient data processing capacity, the flexible expansibility and the powerful visualization function of the method. The core value of the method is to improve the stability of the system, reduce the operation and maintenance cost, accelerate troubleshooting and support data-driven decisions under various business scenes.

The invention discloses a confidential evaluation compliance document automatic generation system and method based on strategic data management and control and full-link verification, and aims at solving the problems that traditional confidential evaluation document generation is low in efficiency, insufficient in safety protection, poor in expansibility and the like. According to the method, hard coding logic is replaced by a strategy engine center, a scene-template-strategy association mechanism is constructed, and full-process automatic generation of secret evaluation documents is achieved through scene configuration, intelligent template matching, parameter analysis, multi-dimensional data cleaning and desensitization, national secret standard automatic signing and full-process log management and control. The generation efficiency is remarkably improved, a sensitive data'identification-processing-protection 'safety closed loop is formed, the tamper-proof and auditing requirements of secret evaluation are met, a newly-added scene can be quickly accessed through strategy configuration, and the expansibility and reusability are excellent.

The invention relates to an inventory data synchronization and consistency verification system supporting network disconnection fault tolerance, and discloses an inventory data synchronization and consistency verification system which realizes continuous recording of inventory operation during a network disconnection period through network state detection and local cache, performs batch transmission after network recovery through a synchronous uploading and verification mechanism, and ensures data integrity. According to the inventory data synchronization and consistency verification system supporting network disconnection fault tolerance, data conflicts are automatically solved through a conflict detection and strategy processing module, a difference report is generated through consistency verification, and secondary synchronization is executed, so that safe synchronization and final consistency guarantee of inventory data are achieved, and it is ensured that the service process is continuous and stable. The system is characterized by comprising a network state detection module, a local cache module, a transaction log management module, a synchronous uploading module, a conflict detection module, a conflict resolution strategy module, a consistency verification module and an abnormity alarm and manual intervention interface module.

The invention discloses an intelligent control method and system for dynamic balance of three floating bodies for crude oil transfer in deep and far sea. The method comprises the following steps: (1) data acquisition and state vector construction; (2) constructing a dynamic response prediction model: constructing a floating body historical motion database based on the initial motion data set, fusing hydrodynamic interference coefficient matrix calculation and multi-body motion mixed constraint analysis, and designing a space-time prediction algorithm based on a graph neural network and a recurrent nerve process to obtain a dynamic response prediction model; establishing a dynamic response prediction model under close-range coupling interference of the three floating bodies; (3) evaluating and correcting data confidence; (4) self-adaptive thrust control and early warning are carried out; and (5) visual monitoring and log management. The problems that in the prior art, positioning precision is low, response lags behind, the abnormal rate of sensor data is high, hydrodynamic coupling interference is not fully considered, and visual monitoring and intelligent early warning are lacked are solved, and the safety, stability and operation efficiency of the deep and far sea crude oil transfer process are effectively improved.

The invention discloses a log processing method and device, a medium and a product, relates to the technical field of distributed system log management, and can be applied to the field of financial science and technology. The method comprises the following steps: sending a receiving node query request to a policy service, so that the policy service determines a target receiving node according to the receiving node query request; obtaining a target receiving node fed back by the policy service; based on the local system portrait model, processing the current system index data to obtain a business trough period window; determining an estimated transmission time length and a recommended transmission time period according to the data volume and the current available bandwidth of the to-be-transmitted log file and the service trough period window; sending a transmission decision request to a policy service; obtaining a target transmission time period and a fragment sequence fed back by the policy service; and transmitting a fragment sequence to the server through the target receiving node based on the target transmission time period, the fragment sequence comprising at least one log fragment. Through the technical scheme, the log processing efficiency can be improved.

The invention discloses an OTA upgrade process customizing method and system adaptable to multiple vehicle types, and relates to the technical field of OTA upgrade.The OTA upgrade process customizing system comprises an OTA upgrade process customizing layer, is compatible with an existing OTA network architecture, and mainly relates to upgrade strategy management, dependency management, task scheduling and exception handling technologies; a user can customize upgrading paths of different vehicle types through the upgrading strategy management module; dynamically distributing upgrading tasks through an upgrading task scheduling module; analyzing the dependency relationship of the software component through a dependency relationship management module; the user-defined upgrading process is executed through an OTA upgrading execution module; and a log recording and exception handling mechanism is provided through the exception handling and log management module. Therefore, by the adoption of the OTA upgrading process self-defining method and system capable of adapting to the multiple vehicle types, OTA upgrading optimization of the different vehicle types can be achieved, the success rate is increased, and the maintenance cost is reduced.

The invention discloses a log management method and device for a solid state disk, equipment and a medium, and relates to the technical field of computers. The method is applied to a device side and comprises the steps that in the firmware compiling generation stage of the solid state disk, a log mapping table is generated through a preset script; in the running process of the solid state disk, persistently storing the generated log data into a nonvolatile storage medium of the solid state disk to obtain an original log text; when log export needs to be carried out, obtaining a log export command issued by the host side through the out-of-band channel, reading the original log text from the nonvolatile storage medium according to the log export command, and converting the original log text into target data containing numerical information; and sending the target data to the host side through the out-of-band channel, so that the host side restores the target data into the original log text according to the log mapping table. According to the technical scheme, the problem that the file transmission time is long in a traditional mode can be solved.

The invention relates to the technical field of message management, in particular to a log management tracking method based on message middleware. The method comprises the following steps: constructing a message sending log recording mechanism so as to write a message generation log table when a message producer sends a message to message middleware; constructing a message consumption log recording mechanism at the message consumer node to perform repeatability verification when the message is transmitted to the consumer node through the message middleware, and transmitting the message to the actual service processing logic; updating the message state in the message consumption table after the execution of the business processing logic is completed, and generating an error processing task record at the same time; constructing a distributed log aggregation index system to establish an associated index, and forming a message full-link tracking view; and constructing a visual log management interface based on the message full-link tracking view to provide log retrieval, log data export and abnormal message retransmission control functions. According to the invention, efficient management and accurate tracking of the message middleware log can be realized.

The invention discloses a commission tracking and accurate settlement system of a private domain ecological multi-level recommendation network, and relates to the technical field of application software development, and the system comprises a multi-level recommendation network construction module which supports 1-5 levels of recommendation relationships, is bound through invitation codes and the like, activates authority through identity authentication, and stores the relationships through a MySQL cluster; the recommendation behavior full-link tracking module is used for distributing a unique UUID to track full-process data and monitoring abnormal behaviors; the dynamic commission rule configuration module supports fixed or proportional commission and realizes differentiated configuration by using a rule engine; a commission accurate settlement module; the visual income display and detail query module is used for displaying data and supporting screening and exporting; and the system interaction and log management module is used for pushing notifications and recording operation logs. According to the invention, accurate commission tracking and reasonable distribution are realized, the system stability and the operation efficiency are improved, the cost and the risk are reduced, and a private domain multi-level recommendation scene is adapted.

The invention provides an application log management method and system based on a cloud native scene. According to the method, firstly, log classification standards are set based on the requirements of different application programs deployed by a Kubernetes cluster, then an output path is configured based on the classification standards, and different types of logs are recorded into respective corresponding files respectively, so that subsequent management and analysis of the application logs are facilitated, and the efficiency of the application logs is improved. And searching in a large number of mixed logs is not needed. The method comprises the following steps of: using a local shared storage volume EmptyDir mounted on a temporary directory of each application Pod to share a log directory between a main application container and a log agent container Sedar, running a Logging Agent through the log agent container Sedar, monitoring a log file in the local shared storage volume, and sending the log file to a remote storage so as to be convenient to retrieve. According to the method, complicated middleware and configuration are not needed, so that the management efficiency of the application logs can be improved.

The invention provides a micro-grid monitoring-oriented intelligent industrial Internet of Things gateway system, which comprises an application layer, a core framework layer and a driving layer, and each layer performs decoupling cooperation through an adapter and a unified data interface; the application layer is in bridge connection with a cloud end and a terminal through an application adapter to realize remote operation and maintenance, and realizes data instruction interaction with the core framework layer through the application adapter; the core framework layer is used for realizing data circulation, task scheduling, protocol adaptation, data storage and log management by scheduling an internal task converted by an application adapter through an information manager, and supporting multi-protocol unified processing and task scheduling; and the driving layer is used for accessing multi-protocol industrial equipment, and cooperating with the core framework layer to realize equipment data acquisition, state monitoring and remote control. According to the invention, equipment access of various industrial protocols can be supported, and unified acquisition and management requirements of multi-source heterogeneous data in complex industrial scenes such as micro-grids and the like are met.

The invention discloses an alarm platform multi-level processing system and method based on data flow, and relates to the technical field of log management, and the system comprises a data collection layer which is used for obtaining log data generated in the system operation process from various data sources and a plurality of platforms; the data processing layer is used for receiving the data of the data acquisition layer and performing preprocessing operations of cleaning, format unification and log separation; the data storage layer is used for storing the data output by the data processing layer in a specified database for subsequent analysis and query; the data analysis layer is used for analyzing the data in the data storage layer in real time so as to find abnormal and potential problems; and the data alarm layer is used for generating alarm information according to a preset rule and strategy when the data analysis layer finds an exception or a potential problem. The method is used for improving system operation and maintenance efficiency and troubleshooting speed.

The invention provides a language resource library system based on data security control, and the system comprises an acquisition module which collects language resources in a manner of manual input and crawler program directional capture, obtains a language data set, and stores the language data set in a database; the processing module develops a software program according to a preset metadata specification, and processes and manually marks the language data set to obtain a standard data set; the management module is used for performing data management, user management and log management in combination with the data access control model and the network security monitoring model; the presentation module is used for providing data retrieval and statistical analysis functions according to the data type and carrying out visualized display of the data; and the product library module is used for developing a language resource platform. By establishing a perfect security mechanism, unauthorized access and data leakage can be effectively prevented, the security of language resources in cross-border transmission and use is improved, and efficient and stable operation of the system is ensured.

A correlation analysis platform and a correlation analysis method for rail transit equipment logs are characterized in that a log management module analyzes uploaded logs and stores the analyzed logs in a database, and a data interaction module accesses a data processing engine through a data processing engine interface; the data processing engine compiles a data query statement to perform query and interactive analysis on logs in the database, an analysis result is visually displayed, a log analysis module configures different analysis rules through the analysis rule configuration interface, the logs in the database are automatically analyzed according to the analysis rules, and the analysis result is displayed according to the analysis rules. And generating a log analysis result. According to the invention, more efficient and more-dimensional log analysis means and method are provided, the log utilization rate and analysis efficiency are improved, and the purposes of more efficient equipment guarantee and maintenance and more stable equipment operation are achieved.