Log processing method, log processing device, computer device, and storage medium
By classifying, extracting information, and analyzing features from user-end log data, target computing models are selected, target log datasets are generated, and stored in the event monitoring system. This solves the problem of exploiting undisclosed vulnerabilities, enables rapid detection and prevention, and improves internet security.
Patent Information
- Application Number
- CN202210689197.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-17
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2042-06-17
AI Technical Summary
Existing internet security measures are insufficient to effectively prevent attacks exploiting undisclosed vulnerabilities, especially when vendors fail to release upgrade packages in a timely manner or administrators fail to update programs promptly, making it impossible to detect and prevent attacks in a timely manner.
By classifying, extracting information, extracting features, and performing calculations on the initial log dataset generated by the user terminal access message system, the target calculation model is selected, the target log dataset is generated and stored in the event monitoring system, and user terminal behavior is monitored to quickly investigate suspicious behavior.
It enables rapid detection and prevention of attacks on internet platforms, improves the ability to detect and defend against network threats, allows for timely protective measures, and enhances internet security.
Smart Images

Figure CN115033876B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a log processing method, a log processing device, a computer device and a storage medium. BACKGROUND
[0002] At present, there are great hidden dangers in the security of the Internet, and the means of attacking the Internet are also increasing. The main security protection measure for the Internet is to extract the vulnerability features and analyze the vulnerability features to investigate attack behaviors. However, when an undisclosed vulnerability is exploited by hackers, a manufacturer does not timely release an upgrade package, or an administrator does not timely update a program, it is difficult to prevent attacks on the Internet. SUMMARY
[0003] The main purpose of the embodiments of the present application is to propose a log processing method, a log processing device, a computer device and a storage medium, which aims to analyze and investigate attack behaviors according to logs, so as to improve the attack prevention of the Internet.
[0004] To achieve the above-mentioned purpose, a first aspect of the embodiments of the present application proposes a log processing method, which comprises:
[0005] obtaining an initial log data set from a message system; wherein the initial log data set is generated by a user terminal accessing the message system, and the initial log data set comprises at least two log data;
[0006] performing classification processing on the initial log data set to obtain a log type of each log data;
[0007] performing information extraction on the initial log data set according to the log type to obtain data information of each log data;
[0008] performing feature extraction on the data information to obtain data features;
[0009] selecting a target calculation model from a plurality of pre-designed calculation models according to the data features;
[0010] performing calculation on the log data according to the target calculation model to obtain a calculation value;
[0011] obtaining the log data whose calculation value is greater than a preset threshold to obtain a target log data set;
[0012] storing the target log data set into an event monitoring system; wherein the target log data set can enable the event monitoring system to monitor behaviors of the user terminal.
[0013] In some embodiments, the classification processing on the initial log data set to obtain a log type comprises:
[0014] acquiring a data type of an initial log data set;
[0015] performing classification processing on the initial log data set according to the data type, to obtain the log type.
[0016] In some embodiments, the data information of each log data includes one of the following: business data information, interface data information, buried point data information, and security data information. Correspondingly, the information extraction on the initial log data set according to the log type to obtain the data information of each log data includes one of the following steps:
[0017] if the log type is a global business type, the log data is extracted in a preset data format to obtain the business data information;
[0018] if the log type is an interface access type, the log data is extracted in a preset extraction instruction to obtain the interface data information;
[0019] if the log type is a business log buried point type, the log data is extracted according to a preset field to obtain the buried point data information;
[0020] if the log type is a security device type, the log data is extracted in a preset regular expression to obtain the security data information.
[0021] In some embodiments, the filtering of the target computing model from a plurality of preset computing models according to the data feature includes:
[0022] acquiring a model category label of each preset computing model;
[0023] performing matching processing on the data feature and the model category label by calling a preset matching engine to obtain a target category label set; wherein the target category label set includes at least one model category label;
[0024] filtering the target computing model from a plurality of preset computing models according to the target category label set.
[0025] In some embodiments, if the target category label set includes at least two model category labels, the filtering of the target computing model from a plurality of preset computing models according to the target category label set includes:
[0026] filtering at least two initial computing models from a plurality of preset computing models according to at least two model category labels; wherein the number of model category labels is equal to the number of initial computing models.
[0027] ranking the at least two model category labels to obtain a ranking result;
[0028] combining the at least two initial computing models according to the ranking result to obtain the target computing model.
[0029] In some embodiments, the storing the target log data set into the event monitoring system comprises:
[0030] accessing a storage database of the event monitoring system; wherein the storage database comprises a storage type;
[0031] obtaining a log category of the target log data set; wherein the log category is one of the log types;
[0032] filtering a target database from the storage database according to the storage type and the log type;
[0033] storing the log data into the target database.
[0034] In some embodiments, after the storing the target log data set into the event monitoring system, the method further comprises:
[0035] visualizing the log data, specifically comprising:
[0036] receiving a query request sent by the user end;
[0037] extracting a target keyword from the query request;
[0038] extracting a target log from the event monitoring system according to the target keyword; wherein the target log is the log data;
[0039] processing the target log to obtain a log analysis view.
[0040] To achieve the above object, a second aspect of the embodiment of the present application provides a log processing device, which comprises:
[0041] a log obtaining module, configured to obtain an initial log data set from a message system; wherein the initial log data set is generated by a user end accessing the message system, and the initial log data set comprises at least two log data;
[0042] a classification module, configured to perform classification processing on the initial log data set to obtain a log type of each log data;
[0043] an information extraction module, configured to perform information extraction on the initial log data set according to the log type, to obtain data information of each log data;
[0044] a feature extraction module, configured to perform feature extraction on the data information, to obtain data features;
[0045] a screening module, configured to screen a target calculation model from a plurality of pre-designed calculation models according to the data features;
[0046] a calculation module, configured to perform calculation on the log data according to the target calculation model, to obtain calculation values;
[0047] a data acquisition module, configured to acquire the log data with the calculation values greater than a preset threshold, to obtain a target log data set;
[0048] a storage module, configured to store the target log data set into an event monitoring system; wherein the target log data set can enable the event monitoring system to monitor behaviors of the user end.
[0049] To achieve the above object, a third aspect of the embodiments of the present application provides a computer device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the method of the first aspect.
[0050] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer readable storage medium, for computer readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the method of the first aspect.
[0051] The log processing method, log processing device, computer device and storage medium provided by the present application analyze the initial log data set generated by the user end accessing the message system, perform calculation according to the log type by using the corresponding target calculation model, monitor the user end behaviors by the event monitoring system according to the log data, so as to quickly find out the suspicious user behaviors, check out the attack behaviors in time, make corresponding protection measures, and improve the security of the Internet. BRIEF DESCRIPTION OF DRAWINGS
[0052] Figure 1 is a flowchart of the log processing method provided by the embodiments of the present application;
[0053] Figure 2 is Figure 1 the flowchart of step S102 in
[0054] Figure 3 is a flowchart of step S103 in Figure 1
[0055] Figure 4 is a flowchart of step S105 in Figure 1
[0056] Figure 5 is a flowchart of step S403 in Figure 4
[0057] Figure 6 is a flowchart of step S108 in Figure 1
[0058] Figure 7 is a flowchart of a log processing method provided by another embodiment of the present application;
[0059] Figure 8 is a structural schematic diagram of a log processing apparatus provided by an embodiment of the present application;
[0060] Figure 9 is a hardware structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.
[0062] It should be noted that although the functional modules are divided in the apparatus schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the apparatus or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.
[0063] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.
[0064] First, several terms involved in the present application are analyzed:
[0065] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.
[0066] Flume NG: Flume NG is a distributed, reliable, and available system that efficiently collects, aggregates, and moves massive amounts of log data from different data sources, ultimately storing it in a centralized data storage system. The architecture has been restructured from the original Flume OG to the current Flume NG, and the NG version is completely incompatible with the original OG version. After the architectural restructuring, Flume NG is more like a lightweight tool, very simple, easily adaptable to various log collection methods, and supports failover and load balancing.
[0067] Log formatting: Log formatting formats log messages so that they can be used by various log handlers. A handler can be configured using a log formatter that knows how to format log records. Events (represented by log record objects) are passed from the handler to the appropriate formatter. The formatter returns formatted output to the handler, which then writes the output to an output device.
[0068] Log tracking: Tracking refers to embedding SDKs (tool code) into the business system program to collect event data and perform various event collection. Log tracking is the main data source for data warehouses. There are two types of tracking: ① Tracking code can be embedded in the backend program of the business system (such as Java, PHP, etc.); ② Tracking code can also be embedded in the frontend program of the business system (native app, page JS, WeChat mini program).
[0069] Nginx: Nginx is a high-performance HTTP and reverse proxy web server, additionally also providing IMAP / POP3 / SMTP services. Its source code is published under a BSD-like license. It is known for its stability, rich feature set, simple configuration file and low system resource consumption. Nginx is a lightweight Web / Reverse proxy server and e-mail proxy server, released under the BSD-like license. Its features are low memory usage, high concurrency, and in fact, Nginx has better concurrency performance than other web servers of the same type.
[0070] log_format: The Nginx server log related instructions mainly have two: one is log_format, which is used to set the log format; the other is access_log, which is used to specify the storage path, format and cache size of the log file, and is generally configured in the Nginx configuration file.
[0071] Kafka: Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data of consumers in a website. This action (web page browsing, search and other user actions) is a key factor in many social functions on modern networks. This data is usually solved by processing logs and log aggregation due to the requirements of throughput. For log data and offline analysis systems like Hadoop, but with the limitation of real-time processing requirements, this is a viable solution. The purpose of Kafka is to unify online and offline message processing through Hadoop's parallel loading mechanism, and also to provide real-time messages through clusters.
[0072] Regular expression: Regular expression, also known as regular expression, is a concept of computer science. Regular expressions are usually used to retrieve and replace text that matches a certain pattern (rule). Regular expressions are a logical formula for string operations, that is, to use some specific characters and combinations of these specific characters to form a "rule string", which is used to express a filtering logic for strings.
[0073] Flink: Apache Flink is a framework and distributed processing engine that enables stateful computations over unbounded and bounded data streams. Flink can run in all common cluster environments and perform computations at memory speed and at any scale. Apache Flink is powerful and supports the development and execution of a wide variety of different kinds of applications. Its main features include: batch and streaming integration, precise state management, event time support, and exactly-once state consistency guarantees. Flink can run on multiple resource management frameworks, including YARN, Mesos, and Kubernetes, and can also be deployed independently on bare-metal clusters. With high-availability options enabled, it does not have a single point of failure. Flink has proven to scale to thousands of cores with TBs of state and still maintain high throughput and low latency. There are many demanding stream processing applications running on Flink all over the world.
[0074] Webshell: A webshell is a code execution environment in the form of an asp, php, jsp, or cgi web page file, mainly used for website management, server management, and permission management. It is simple to use, and only needs to upload a code file and access it through a web address to perform many daily operations, greatly facilitating the management of websites and servers. For this reason, some people modify the code and use it as a backdoor program to control the website server.
[0075] Relational database: A relational database is a database that uses a relational model to organize data. It stores data in the form of rows and columns to facilitate user understanding. The series of rows and columns in a relational database are called tables, and a group of tables forms a database. Users retrieve data in the database through queries, which are execution codes used to limit certain areas in the database. The relational model can be simply understood as a two-dimensional table model, and a relational database is a data organization composed of two-dimensional tables and their relationships.
[0076] With the rapid development of the Internet, more and more network security incidents have made Internet security face unprecedented challenges. Attackers have also developed from single attack behavior to organized, targeted, and long-lasting attacks, so APT attacks have become more and more familiar to people. In order to obtain important information of an organization or even a country, attackers will use various attack methods, and even many attacks use vulnerabilities that have not been publicly disclosed. During the attack process, various techniques will be used, and long-term lurking in the system will be used to collect various information to achieve the purpose.
[0077] Although a large number of security protection devices for a certain type of threat have been deployed in the Internet field, many security protection devices still rely on the way of extracting vulnerability features to detect attacks, and if an undisclosed vulnerability is exploited by hackers, the manufacturer does not timely release an upgrade package or the administrator does not timely update the program, etc., the security protection device cannot detect and prevent attacks.
[0078] Based on this, the embodiments of the present application provide a log processing method, a log processing device, a computer device and a storage medium, by analyzing the initial log data set generated by the user end access message system, to calculate according to the corresponding target calculation model according to the log type, to monitor the user end behavior according to the log data through the event monitoring system, in order to quickly investigate the suspicious user behavior, check the attack behavior in time to facilitate the corresponding protection measures, and improve the security of the Internet.
[0079] The log processing method, log processing device, computer device and storage medium provided by the embodiments of the present application are specifically explained by the following embodiments, first, the log processing method in the embodiments of the present application is described.
[0080] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.
[0081] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.
[0082] The log processing method provided in the embodiments of the present application relates to the technical field of artificial intelligence. The log processing method provided in the embodiments of the present application can be applied to a terminal, can also be applied to a server end, and can further be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like; the server end can be configured as a stand-alone physical server, can also be configured as a server cluster or a distributed system formed by a plurality of physical servers, and can further be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms; and the software can be an application that implements the log processing method, but is not limited to the above forms.
[0083] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.
[0084] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the user, such as user information, user behavior data, user history data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards in relevant countries and regions. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to normally operate will be obtained.
[0085] Figure 1 is an optional flowchart of the log processing method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to steps S101 to S108.
[0086] Step S101, obtaining an initial log data set from a message system; wherein the initial log data set is generated by a user terminal accessing the message system, and the initial log data set includes at least two log data;
[0087] Step S102, performing classification processing on the initial log data set to obtain a log type of each log data;
[0088] Step S103, performing information extraction on the initial log data set according to the log type to obtain data information of each log data;
[0089] Step S104, performing feature extraction on the data information to obtain data features;
[0090] Step S105, screening a target calculation model from a plurality of pre-designed calculation models according to the data features;
[0091] Step S106, performing calculation on the log data according to the target calculation model to obtain a calculation value;
[0092] Step S107, obtaining log data with a calculation value greater than a preset threshold to obtain a target log data set;
[0093] Step S108, storing the target log data set into an event monitoring system; wherein the target log data set enables the event monitoring system to monitor behaviors of the user terminal.
[0094] The steps S101 to S108 shown in the embodiments of the present application obtain an initial log data set generated by a user terminal accessing a message system, and the initial log data set includes at least two log data, then perform classification processing on the log data in the initial log data set to obtain a log type of the log data. According to the log type of the log data, perform information extraction on the initial log data set to extract data information of the log data of the same log type by using a corresponding information extraction mode. Perform feature extraction on the data information to obtain data features, and screen a target calculation model from a plurality of pre-designed calculation models according to the data features, i.e., determine the target calculation model corresponding to each log data. Then, perform calculation on the log data according to the target calculation model to obtain a calculation value, and collect log data with a calculation value greater than a preset threshold into a target log data set. Store the target log data set into an event monitoring system to monitor behaviors of the user terminal according to the target log data set, analyze suspicious behaviors by monitoring the behaviors of the user terminal, further trace abnormal reasons, and timely perform corresponding protection measures on abnormal problems.
[0095] In step S101 of some embodiments, by loading the message system on the Internet platform, when the external user terminal accesses the Internet platform, the message system is first accessed to monitor the user terminal accessing the Internet platform through the message system. At the same time, the message system generates log data according to the user access, and stores the log data into the initial log data set. Therefore, by obtaining the initial log data set from the message system, the access behavior of the user terminal is monitored according to the initial log data set.
[0096] It should be noted that the message system in the present embodiment is a Flume NG system, and the storage system of the Flume NG system is a Kafka message queue. The Flume NG system collects, aggregates and moves the log data of the user terminal accessing the Internet platform, and finally stores the log data into the Kafka message queue. Therefore, by setting the Flume NG system to collect log data when the user terminal accesses the Internet platform, the initial log data set is obtained, and the access to the Internet platform is detected and attacked according to the initial log data set, so as to improve the network threat detection and defense capability of the Internet.
[0097] In step S102 of some embodiments, since the log data generated by the user terminal accessing the message system has different log types, and different information extraction methods are used according to different log types. Therefore, by classifying the initial log data set, the log type of each log data in the initial log data set is obtained by classifying the initial log data set, so as to determine the log formatting method of each log data according to the log type.
[0098] In step S103 of some embodiments, by extracting information from the initial log data set according to the log type, that is, extracting information from the log data according to the same information extraction method according to the same log type. Wherein, according to the log type, the initial log data set is extracted, that is, the initial log data set is formatted, so as to format the log data of the same log type to obtain the data information of the log data. Specifically, the data information includes at least one of: data source information, data request parameter, request agent, request response value and return data length. Therefore, the log data is formatted, and the log data of the same log type is output in a unified format, so as to facilitate subsequent log data analysis.
[0099] In step S104 of some embodiments, after the log data is formatted, data information is obtained, data features are obtained by feature extraction on the data information, and the data features are used to represent the features of the data information, so as to determine the type of the log data through the data features, so as to match the data features with the plurality of pre-designed calculation models. For example, if the data information is the information of a mobile phone number, the data features obtained by feature extraction on the data information are the number length, the place of origin, etc. Therefore, the data features are obtained by feature extraction on the data information, so as to determine the feature identifier of the log data through the data features.
[0100] In step S105 of some embodiments, a plurality of pre-designed calculation models are set in advance according to the models required for log data analysis, and the pre-designed calculation models matched by different data features are different, so as to match the corresponding pre-designed calculation models according to different data features. Therefore, the target calculation model is screened out from the plurality of pre-designed calculation models through the data features, so as to screen out the corresponding target calculation model for log data calculation, so as to realize corresponding analysis of the log data.
[0101] In step S106 of some embodiments, a calculation value is obtained by calculating the log data through the target calculation model, and the calculation value is used to represent the abnormality degree of the log data, and the calculation value is proportional to the abnormality degree. Therefore, the calculation value of the log data is calculated, so as to quickly analyze the log data with abnormality, so as to analyze the suspicious behavior of the user end according to the abnormal log data, and to timely investigate external attacks, thereby improving the network threat detection and defense capability.
[0102] In step S107 of some embodiments, the log data with a calculation value greater than a preset threshold is collected to obtain a target log data set, and the log data in the target log data set is suspicious log data, so as to further analyze the abnormal log data through the target log data set, without monitoring the behavior of the user end according to all the log data, so as to reduce the resources consumed by monitoring and improve the detection efficiency of abnormal log data.
[0103] In step S108 of some embodiments, the target log data set is stored in an event monitoring system, so as to analyze the behavior of the user end through the event monitoring system according to the target log data set, so as to quickly detect whether the Internet platform is attacked, and to analyze which application is affected after the Internet platform is attacked, thereby improving the network threat detection capability. Specifically, the storage database in the event monitoring system is a relational database, and the log data in the relational database is stored in the form of rows and columns, so that the user can retrieve the log data in the relational database through a query request, so as to further detect whether there is suspicious feature in the log data through manual confirmation, so as to take corresponding preventive measures according to the suspicious feature, thereby improving the security of the Internet platform.
[0104] Referring to Figure 2 In some embodiments, step S102 can include, but is not limited to, steps S201-S202:
[0105] In step S201, the data type of the initial log data set is obtained.
[0106] In step S202, the initial log data set is classified according to the data type, and the log type is obtained.
[0107] In step S201 of some embodiments, the data type of the log data in the initial log data set is obtained to determine the log type of each log data in the initial log data set.
[0108] In step S202 of some embodiments, the initial log data set is classified according to the log type of each log data, that is, log data of the same data type is classified into a category to obtain the log type of each log data. Therefore, by classifying the log data of the initial log data set according to the data type to determine the log type, the log data classification process is simple.
[0109] Referring to Figure 3 In some embodiments, the data information of each log data includes one of the following: business data information, interface data information, buried point data information, and security data information. Correspondingly, step S103 can include one of the following steps S301, S302, S303, and S304:
[0110] In step S301, if the log type is a global business type, the log data is extracted in a preset data format to obtain business data information.
[0111] In step S302, if the log type is an interface access type, the log data is extracted in a preset extraction instruction to obtain interface data information.
[0112] In step S303, if the log type is a business log buried point type, the log data is extracted according to a preset field to obtain buried point data information.
[0113] In step S304, if the log type is a security device type, the log data is extracted in a preset regular expression to obtain security data information.
[0114] It should be noted that the log data generated by the message system mainly includes global service log data, interface access log data, service log point data and security device log data, and the log data of the same log type needs to be changed into data information in a unified format, and then information extraction is performed on the log data according to different information extraction methods to obtain data information.
[0115] In step S301 of some embodiments, if the log type is a global service type, the log data is information extracted in a preset data format, so that the global service log data is unified in a data format, and the service data information is fixed and segmented.
[0116] If the log type is a global service type, the global service log data is obtained through a distributed log link, a tracking is allocated to each access thread, and the whole process of the user accessing the message system is tracked through the tracking. Specifically, a logical decoding unit is provided, and the logical decoding unit is a log-unit project. The user accesses the message system through the log-unit project, all modules of the message system depend on the project, and the log-unit project provides a unified log-back.xml file to store the collected global service log data in the log-back.xml file, and the collection of the global service log data is realized.
[0117] In step S302 of some embodiments, the log data is information extracted according to a preset extraction instruction, that is, the log data is log formatted, so as to realize format unification. If the log type is an interface access type, the log data is information extracted through a preset extraction instruction to obtain interface data information, that is, the log data is log formatted to realize format unification of the interface access log data.
[0118] In this embodiment, the preset extraction instruction is a log_format instruction of Nginx, and the log data format is set through the log_format instruction. The information in the log data that meets the log data format is extracted according to the log data format corresponding to the log_format instruction, so as to obtain the interface data information. Specifically, the log_format instruction is used to set the data format of the log, and the syntax is "log_format name format{format…}". In which, name represents the defined format name, and format represents the defined format style. Therefore, the log data is field segmented through the log_format instruction, and is combined according to the log data format corresponding to the log_format instruction to obtain the interface data information in a unified data format.
[0119] In step S303 of some embodiments, if the log type is a business log point type, the log data is business log point data. The business log point data records data of a specific online business key point and collects business information in different scenarios. Therefore, information extraction is performed on the log data according to the preset field, that is, key field information in the log data that meets the preset field is extracted to obtain point data information. For example, if the preset field includes a customer number, a case number, and a serial number, the key field information of the customer number, the case number, and the serial number in the log data is extracted to obtain the point data information, so that the point data information is used for correlation detection processing to determine whether the business key point is at risk and improve the Internet risk detection capability.
[0120] In step S304 of some embodiments, if the log type is a security device type, information extraction is performed on the security device log data. The information extraction is log formatting, which mainly performs log formatting on the security data information according to a preset regular expression to output the security data information in a unified data format. The regular expression uses some specific characters and combinations of the specific characters to form a “rule string”, which is used to express a filtering logic for a string. Therefore, the security data information is represented by the “rule string” to facilitate subsequent feature extraction of the security data information.
[0121] It should be noted that the data information of each log data is obtained by performing information extraction on the log data. The business data information corresponding to the global business log data, the interface data information corresponding to the interface access log data, the point data information corresponding to the business log point data, and the security data information corresponding to the security device log data. Therefore, the corresponding information extraction is used according to different log types to obtain the data information of the log data, so that the information extraction is performed on the log data in a targeted manner.
[0122] Please refer to Figure 4 In some embodiments, step S105 can include but is not limited to steps S401 to S403:
[0123] Step S401: obtaining a model category label of each preset calculation model;
[0124] Step S402: calling a preset matching engine to perform matching processing on the data feature and the model category label to obtain a target category label set. The target category label set includes at least one model category label.
[0125] Step S403: filtering a target calculation model from the plurality of preset calculation models according to the target category label set.
[0126] In step S401 of some embodiments, a pre-designed computing model is used for computing log data, and different data features match different pre-designed computing models, so the model parameters of the original computing model are adjusted according to different data features to obtain a pre-designed computing model, and the pre-designed computing model is set with a corresponding model category label to distinguish each pre-designed computing model through the model category label. Therefore, the model category label of each pre-designed computing model is obtained, and the model category label is matched with the data feature.
[0127] In step S402 of some embodiments, the data feature and the model category label are matched by calling a preset matching engine. The preset matching engine is a search engine, and in this embodiment, the search engine is a Flink engine. The Flink engine searches in a Key-Value manner, and the Flink engine sets the Key as the data feature and the Value as the model category label. Therefore, the corresponding model category label is quickly found according to the data feature to obtain a target category label set. For example, if the data feature is the number length and the place of origin, the matched model category label is a mobile phone number analysis label, and the pre-designed computing model corresponding to the model category label is used for analyzing and processing the mobile phone number to obtain a mobile phone number analysis result. The mobile phone number analysis result is a calculation value, and the calculation value represents the abnormality degree of the mobile phone number.
[0128] In step S403 of some embodiments, after obtaining the target category label set, the model category label in the target category label set is used to filter a target computing model from a plurality of pre-designed computing models. The target computing model can be a pre-designed computing model or a computing model chain formed by combining a plurality of pre-designed computing models. For example, if the target computing model is a mobile phone number analysis model, the length and place of origin of the mobile phone number in the log data are analyzed through the mobile phone number analysis model to determine whether the length of the mobile phone number meets the standard and whether the place of origin is correct to obtain a mobile phone number analysis result. The mobile phone number analysis result is a calculation value, and if the calculation value is greater than a preset threshold, it indicates that the mobile phone number is abnormal. Therefore, the log data corresponding to the mobile phone number abnormality is recorded to facilitate the event monitoring system to monitor the behavior of the user end to determine whether the mobile phone number abnormality is caused by external attacks, thereby improving the abnormal detection capability of the Internet platform.
[0129] Please refer to Figure 5 In some embodiments, if the target category label set includes at least two model category labels, step S403 can include but is not limited to steps S501 to S503:
[0130] Step S501, screening at least two initial computing models from the plurality of pre-designed computing models according to at least two model category labels; wherein the number of model category labels is equal to the number of initial computing models;
[0131] Step S502, sorting the at least two model category labels to obtain a sorting result;
[0132] Step S503, combining the at least two initial computing models according to the sorting result to obtain a target computing model.
[0133] In step S501 of some embodiments, if the target category label set includes at least two model category labels, it indicates that the target computing model is obtained by combining a plurality of pre-designed computing models. In order to ensure the accuracy of the log data calculation, at least two initial computing models need to be screened from the plurality of pre-designed computing models according to the at least two model category labels, that is, the pre-designed computing models corresponding to the model category labels are extracted to obtain the at least two initial computing models.
[0134] In step S502 of some embodiments, the model category labels are sorted to obtain a sorting result, and the sorting result is the sorting order corresponding to each model category label. For example, if there are three model category labels, which are analysis label, statistical label and matching label, and it is determined that the sorting result of the analysis label is third, the sorting result of the statistical label is second, and the sorting result of the matching label is first. Therefore, the sorting result of each model category label is determined by sorting the model category labels, so as to construct the target computing model according to the sorting result.
[0135] In step S503 of some embodiments, the ranking result of the model category label is determined, that is, the ranking result of the initial calculation model corresponding to the model category label is determined, and then at least two initial calculation models are combined according to the ranking result to obtain a target calculation model, and the target calculation model constructed is used to analyze the log data to obtain more accurate calculation values. For example, if the ranking result of the analysis label is third, the ranking result of the statistics label is second, and the ranking result of the matching label is first, then the matching model, the statistics model and the analysis model are combined to form a target calculation model according to the ranking result, and then the log data is processed according to the target calculation model. First, the matching processing is performed, then the statistical processing is performed, and finally the analysis processing is performed to obtain the corresponding calculation value. For example, the target calculation model is used to calculate whether the number of occurrences of the "B field" is abnormal, then the field in the log data is matched with the "B field", the number of occurrences of the B field is counted, and finally the number of occurrences is analyzed to determine the abnormal result, and the abnormal result is represented by a calculation value. Therefore, the initial calculation models are combined to obtain the target calculation model according to the ranking result, and more accurate calculation values can be obtained by calculating the log data according to the target calculation model.
[0136] Referring to Figure 6 In some embodiments, step S108 includes but is not limited to steps S601 to S604:
[0137] Step S601, accessing a storage database of an event monitoring system; wherein the storage database includes a storage type;
[0138] Step S602, obtaining a log category of a target log data set; wherein the log category is one of the log types;
[0139] Step S603, filtering a target database from the storage database according to the storage type and the log type;
[0140] Step S604, storing the log data to the target database.
[0141] In step S601 of some embodiments, a plurality of storage databases are provided on the event monitoring system, so that by accessing the storage database of the event monitoring system, the storage type of the storage database can be obtained, and the log data can be classified and stored in the corresponding storage database to realize the classified storage of the log data.
[0142] In step S602 of some embodiments, the log category of the target log data set is obtained, that is, the log type of each log data in the target log data set is obtained to obtain the log category, and then the log data in the target log data set can be classified and stored according to the log category.
[0143] In step S603 of some embodiments, the storage type and the log type are matched to obtain a target database as a storage database matched in storage type and log type. For example, if the log type is a global service type, a storage database with a global service type in storage type is obtained as the target database.
[0144] In step S604 of some embodiments, after the target database is screened out, the log data corresponding to the log type is stored in the target database, so as to store the log data in the target log data set in the corresponding storage database, realize the classified storage of the log data, and facilitate the quick acquisition of the log data when the log data is queried, so as to make the log data extraction more rapid.
[0145] Please refer to Figure 7 In some embodiments, after step S108, the log processing method further includes the step of: visualizing and displaying the log data.
[0146] It should be noted that in order to facilitate the user to quickly analyze the log data, the log data is visualized and displayed, so that the user can quickly find out the abnormal data through the visualized log data to detect the attack behavior of the user end.
[0147] The visualization and display of the log data can include, but is not limited to, steps S701 to S704:
[0148] Step S701, receiving a query request sent by a user end;
[0149] Step S702, keyword extraction is performed on the query request to obtain a target keyword;
[0150] Step S703, target log is extracted from the event monitoring system according to the target keyword; wherein the target log is log data;
[0151] Step S704, visualizing and processing the target log to obtain a log analysis view.
[0152] In step S701 of some embodiments, when the user needs to query the log data for analysis to determine whether a webshell exists in the log data, a query request sent by the user terminal is received, and the query request is used to query specified log data. The webshell is a backdoor program discovered by hackers to find system vulnerabilities of an Internet platform, and the webshell is uploaded to the Internet platform using the system vulnerability to obtain upload permission. The webshell is a backdoor program written in a scripting language and can run on the Internet platform. Attackers can execute system commands through a web page to read system text. Therefore, from the perspective of user access behavior, the webshell is a command execution environment that exists when an attacker intrudes a website. Therefore, by detecting the webshell, it can be determined whether the user terminal of the attacker has accessed the network. Therefore, after obtaining the abnormal log data, the log data is output according to the query execution of the user terminal, so as to detect whether the webshell exists in the log data by manual analysis, thereby quickly determining the attack on the Internet platform and improving the network threat detection capability of the Internet platform.
[0153] In step S702 of some embodiments, the query request includes a keyword, so the keyword is extracted from the query request to obtain a target keyword. The log data is extracted according to the target keyword, so that the log data extraction is more accurate.
[0154] In step S703 of some embodiments, the target log is extracted from the event monitoring system according to the target keyword. The storage type is determined according to the target keyword, and then the target database is determined according to the storage type. The log data is extracted from the target database according to the target keyword to obtain the target log, so that the target log extraction is more accurate.
[0155] In step S704 of some embodiments, in order to facilitate the user of the user terminal to quickly detect the webshell, the target log is visualized to obtain a log analysis view. Whether the webshell exists can be determined through the target analysis view, so as to determine the attack on the Internet platform in time according to the webshell, and take corresponding defense measures in time, thereby improving the network defense capability of the Internet platform. The log analysis view can be displayed on a system-specific page according to the query request, so that the user can analyze which page has an attack behavior, and take defense measures in time.
[0156] The embodiment of the present application generates log data after the user terminal accesses the message system, and collects the log data into an initial log data set, and then obtains the data type of each log data in the initial log data set, so as to classify and process the initial log data according to the data type to obtain a log type. If the log type is a global service type, the log data is extracted in a preset data format to obtain service data information; if the log type is an interface access type, the log data is extracted in a preset extraction instruction to obtain interface data information; if the log type is a service log burying point type, the field information in the log data that meets a preset field is extracted to obtain burying point data information; and if the log type is a security device type, the log data is set in a preset regular expression to obtain security data information. The log data is formatted to obtain corresponding data information, and the data information is feature extracted to obtain data features, and then a preset matching engine is called to match the data features and model category labels to obtain a target category label set, and the initial calculation model is selected from a plurality of preset calculation models according to the target category label set. The model category labels are sorted to obtain a sorting result, the initial calculation model is combined to obtain a target calculation model according to the sorting result, and then the log data is calculated according to the target calculation model to obtain a calculation value. The log data whose calculation value is greater than a preset threshold is obtained to obtain a target log data set, and the target log data set is stored in an event monitoring system, so that the event monitoring system monitors the behavior corresponding to the user terminal according to the target log data, to check whether there is abnormal behavior, thereby improving the network threat detection and defense capability of the Internet platform.
[0157] Please refer to Figure 8 The embodiment of the present application also provides a log processing device, which can implement the log processing method described above, and the device comprises:
[0158] The log acquisition module 801 is configured to acquire an initial log data set from a message system; wherein the initial log data set is generated by a user terminal accessing the message system, and the initial log data set comprises at least two log data;
[0159] The classification module 802 is configured to classify and process the initial log data set to obtain the log type of each log data;
[0160] The information extraction module 803 is configured to extract information from the initial log data set according to the log type to obtain data information of each log data;
[0161] The feature extraction module 804 is configured to extract features from the data information to obtain data features;
[0162] The screening module 805 is configured to screen a target calculation model from a plurality of preset calculation models according to the data features;
[0163] The computing module 806 is configured to perform calculation on the log data according to a target calculation model to obtain a calculation value.
[0164] The data acquisition module 807 is configured to acquire log data with a calculation value greater than a preset threshold to obtain a target log data set.
[0165] The storage module 808 is configured to store the target log data set into the event monitoring system, wherein the target log data set enables the event monitoring system to monitor the behavior of the user terminal.
[0166] The specific implementation of the log processing apparatus is basically the same as the specific implementation of the log processing method, and thus will not be described herein.
[0167] The embodiments of the present application further provide a computer device, which comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing the connection and communication between the processor and the memory, and the program is executed by the processor to realize the log processing method. The computer device can be any smart terminal, such as a tablet computer or a vehicle-mounted computer.
[0168] Please refer to Figure 9 , Figure 9 The hardware structure of the computer device of another embodiment is illustrated, which comprises:
[0169] The processor 901 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to realize the technical solutions provided by the embodiments of the present application.
[0170] The memory 902 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 902 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to realize the log processing method of the embodiments of the present application.
[0171] The input / output interface 903 is configured to realize information input and output.
[0172] The communication interface 904 is configured to realize the communication interaction between the device and other devices, and can realize the communication through a wired manner (for example, a USB, a network cable, or the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like).
[0173] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.
[0174] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are communicatively connected to each other within the device through the bus 905.
[0175] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium, is used for computer readable storage, and stores one or more programs. The one or more programs can be executed by one or more processors to implement the log processing method.
[0176] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0177] The log processing method, the log processing device, the computer device, and the storage medium provided by the embodiment of the present application can obtain an initial log data set generated by a user terminal according to a message system, perform classification processing on the initial log data set to obtain a log type of each log data, then perform information extraction on the initial log data set according to the log type to obtain data information of each log data, perform feature extraction on the data information to obtain data features, select a target calculation model from a plurality of pre-designed calculation models according to the data features, perform calculation on the log data through the target calculation model to obtain a calculation value, obtain a target log data set with the calculation value greater than a preset value, and store the target log data set in an event monitoring system, so that the event monitoring system detects the behavior of the user terminal according to the target log data set. Therefore, the abnormal behavior of the user terminal access is controlled through the log analysis manner, so as to detect whether the message system is attacked, and timely make corresponding defense measures, thereby improving the network threat detection and defense capability.
[0178] The embodiments described in the specification of the present application are intended to more clearly illustrate the technical solutions of the present application, and do not constitute a limitation on the technical solutions provided by the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the present application are also applicable to similar technical problems.
[0179] Those skilled in the art can understand that, Figures 1-7 The technical solutions shown in the above description do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown, or combine certain steps, or different steps.
[0180] The device embodiments described above are merely illustrative, and units described as separate components can or can not be physically separated, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0181] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.
[0182] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above description are used to distinguish similar objects, and do not necessarily indicate a specific order or chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0183] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.
[0184] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.
[0185] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.
[0186] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.
[0187] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.
[0188] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.
Claims
1. A log processing method, characterized in that, The method includes: Obtain the initial log dataset from the messaging system; wherein the initial log dataset is generated by the user accessing the messaging system, the initial log dataset includes at least two log data sets, and the messaging system is loaded on an Internet platform; The initial log dataset is classified to obtain the log type for each log data. Information is extracted from the initial log dataset according to the log type to obtain the data information of each log data; Feature extraction is performed on the data information to obtain data features; The target computing model is selected from multiple preset computing models based on the data characteristics. The log data is calculated according to the target calculation model to obtain the calculated value; Obtain the log data whose calculated value is greater than a preset threshold to obtain the target log dataset; The target log dataset is stored in an event monitoring system so that the event monitoring system can analyze the user's behavior based on the target log dataset and detect whether the Internet platform is under attack. The data information of each log data includes one of the following: business data information, interface data information, event tracking data information, and security data information. Correspondingly, the step of extracting information from the initial log dataset according to the log type to obtain the data information of each log data includes one of the following steps: If the log type is a global business type, the log data will be extracted in a preset data format to obtain the business data information; If the log type is an interface access type, the log data will be extracted using a preset extraction instruction to obtain the interface data information; If the log type is a business log tracking type, then information is extracted from the log data according to the preset fields to obtain the tracking data information; If the log type is a security device type, the log data will be extracted using a preset regular expression to obtain the security data information.
2. The method according to claim 1, characterized in that, The initial log dataset is classified to obtain log types, including: Obtain the data type of the initial log dataset; The initial log dataset is classified according to the data type to obtain the log type.
3. The method according to claim 1 or 2, characterized in that, The step of selecting the target computing model from multiple preset computing models based on the data features includes: Obtain the model category label for each of the preset computational models; A preset matching engine is invoked to perform matching processing on the data features and the model category labels to obtain a target category label set; wherein, the target category label set includes at least one of the model category labels; The target computing model is selected from multiple preset computing models based on the target category label set.
4. The method according to claim 3, characterized in that, If the target category label set includes at least two model category labels, the step of selecting the target computation model from multiple preset computation models based on the target category label set includes: At least two initial computational models are selected from a plurality of preset computational models based on at least two model category labels; wherein the number of model category labels is equal to the number of initial computational models; Sort at least two of the model category labels to obtain the sorting results; Based on the sorting results, at least two of the initial computational models are combined to obtain the target computational model.
5. The method according to claim 1 or 2, characterized in that, The step of storing the target log dataset into the event monitoring system includes: Access the storage database of the event monitoring system; wherein, the storage database includes storage types; Obtain the log categories of the target log dataset; wherein, the log category is one of the log types; Target databases are selected from the storage databases based on the storage type and the log type; The log data is stored in the target database.
6. The method according to claim 1 or 2, characterized in that, After storing the target log dataset into the event monitoring system, the method further includes: Visualizing the log data specifically includes: Receive the query request sent by the user terminal; Keyword extraction is performed on the query request to obtain the target keywords; Target logs are extracted from the event monitoring system based on the target keywords; wherein the target logs are the log data. The target log is then visualized to obtain a log analysis view.
7. A log processing device, characterized in that, The device includes: The log acquisition module is used to acquire an initial log dataset from the messaging system; wherein the initial log dataset is generated by the user accessing the messaging system, the initial log dataset includes at least two log data sets, and the messaging system is loaded on an Internet platform; The classification module is used to classify the initial log dataset to obtain the log type of each log data. The information extraction module is used to extract information from the initial log dataset according to the log type to obtain data information for each log data. The data information for each log data includes one of the following: business data information, interface data information, event tracking data information, and security data information. Correspondingly, the step of extracting information from the initial log dataset according to the log type to obtain data information for each log data includes one of the following steps: if the log type is a global business type, then the log data is extracted using a preset data format to obtain the business data information; if the log type is an interface access type, then the log data is extracted using a preset extraction instruction to obtain the interface data information; if the log type is a business log event tracking type, then the log data is extracted according to a preset field to obtain the event tracking data information; if the log type is a security device type, then the log data is extracted using a preset regular expression to obtain the security data information. The feature extraction module is used to extract features from the data information to obtain data features; The filtering module is used to filter out the target computing model from multiple preset computing models based on the data characteristics; The calculation module is used to calculate the log data according to the target calculation model to obtain the calculated value; The data acquisition module is used to acquire the log data whose calculated value is greater than a preset threshold, and obtain the target log dataset; The storage module is used to store the target log dataset into the event monitoring system, so that the event monitoring system can analyze the behavior of the user terminal based on the target log dataset and detect whether the Internet platform is under attack.
8. A computer device, characterized in that, The computer device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.
9. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Log analysis method and equipment and computer readable storage medium
CN107391746A
Log processing method and device, electronic device and storage medium
CN114138720A