A method and apparatus for log processing

Abstract

An embodiment of the present invention provides a method and apparatus for log processing, the method comprising: obtaining target log data corresponding to a target business type; determining process configuration information corresponding to the target business type; for each process item in the process configuration information, determining whether the target log data is missing first key information corresponding to the process item; and when it is determined that the target log data is not missing the first key information corresponding to the process item, generating first process result information corresponding to the process item based on the target log data. The embodiment of the present invention implements analysis of the process items in the target log data, so that when a business fails, the business problem can be quickly located through the first process result information, thereby improving the efficiency of locating the business problem.

Description

Technical Field

[0001] The present invention relates to the technical field of visual networking, and in particular to a method and device for log processing. Background Art

[0002] In the visual Internet of Things, when problems arise in visual Internet services, testers often do not know where the problem occurs immediately because the visual Internet of Things services may involve multiple component products (such as terminals, conference management, core servers, etc.). They can only find the corresponding R&D personnel to locate the problem based on surface phenomena, resulting in inefficient problem location.

[0003] For example, when the conference manager fails to upload a meeting, the tester will first find the conference manager R&D personnel to locate the problem. After locating the problem, the conference manager R&D personnel find that the result returned by the core server is wrong. The problem is transferred to the relevant R&D personnel of the core server. After the core server R&D personnel locate the problem, they finally find that it is caused by an error in the terminal business logic. In this process, due to the inability to accurately find the link where the problem occurs, a lot of time is wasted in locating the problem, resulting in inefficient location of business problems. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a log processing method and apparatus that overcomes the above problems or at least partially solves the above problems, including:

[0005] A log processing method, the method comprising:

[0006] Obtain target log data corresponding to the target business type;

[0007] Determine process configuration information corresponding to the target business type;

[0008] For each process item in the process configuration information, determining whether the target log data is missing first key information corresponding to the process item;

[0009] When it is determined that the target log data is not missing the first key information corresponding to the process item, first process result information corresponding to the process item is generated according to the target log data.

[0010] A log processing device, comprising:

[0011] Target log data acquisition module, used to obtain target log data corresponding to the target business type;

[0012] A process configuration information acquisition module is used to determine the process configuration information corresponding to the target business type;

[0013] a judgment module, configured to judge, for each process item in the process configuration information, whether the target log data lacks first key information corresponding to the process item;

[0014] The first process result information generating module is configured to generate the first process result information corresponding to the process item according to the target log data when it is determined that the target log data does not lack the first key information corresponding to the process item.

[0015] The embodiments of the present invention have the following advantages:

[0016] The embodiment of the present invention obtains target log data corresponding to the target business type, determines the process configuration information corresponding to the target business type, and for each process item in the process configuration information, determines whether the target log data is missing the first key information corresponding to the process item. When it is determined that the target log data is not missing the first key information corresponding to the process item, the first process result information corresponding to the process item is generated according to the target log data, thereby realizing analysis of the process items of the target log data, so that when a business fails, the business problem can be quickly located through the first process result information, thereby improving the efficiency of locating the business problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic diagram of a visual network provided by one embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the hardware structure of a node server provided by one embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the hardware structure of an access switch provided by one embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the hardware structure of an Ethernet protocol conversion gateway provided by one embodiment of the present invention;

[0022] Figure 5 This is a flow chart of the steps of a log processing method provided by one embodiment of the present invention;

[0023] Figure 6 This is a flowchart of another log processing method provided by one embodiment of the present invention;

[0024] Figure 7 This is a log processing framework diagram provided by an embodiment of the present invention;

[0025] Figure 8 This is a flow chart of a log analysis tool processing logs provided by an embodiment of the present invention;

[0026] Figure 9 This is a structural block diagram of a log processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0028] The visual Internet is an important milestone in the development of the Internet. It is a real-time network that can realize real-time transmission of high-definition video, pushing many Internet applications towards high-definition video and high-definition face-to-face communication.

[0029] The visual network adopts real-time high-definition video exchange technology, which can integrate the required services such as high-definition video conferencing, video monitoring, intelligent monitoring analysis, emergency command, digital radio and television, delayed TV, online teaching, live broadcast, VOD on demand, TV mail, personalized recording (PVR), intranet (self-operated) channels, intelligent video broadcast control, information release and dozens of other video, voice, image, text, communication, data and other services into one system platform, and realize high-definition quality video playback through TV or computer.

[0030] To help those skilled in the art better understand the embodiments of the present invention, the following describes the visual networking:

[0031] Some of the technologies used in the visual Internet of Things are as follows:

[0032] Network Technology

[0033] Visual networking's network technology innovations improve upon traditional Ethernet to handle the potentially massive video traffic on the network. Unlike simple packet switching or circuit switching, visual networking utilizes packet switching to meet streaming requirements. Visual networking combines the flexibility, simplicity, and cost of packet switching with the quality and security of circuit switching, enabling seamless connectivity across the entire network, including switched virtual circuits and data formats.

[0034] Switching Technology

[0035] Visual Networking leverages the advantages of Ethernet's asynchronous nature and packet switching, eliminating Ethernet's shortcomings while maintaining full compatibility. It provides seamless end-to-end connectivity across the entire network, directly connecting to user terminals and carrying IP data packets. User data requires no format conversion across the network. Visual Networking is a more advanced form of Ethernet, a real-time switching platform capable of achieving large-scale, real-time transmission of high-definition video across the entire network, a feat currently unattainable on the Internet. This will drive the high-definition and unified nature of many network video applications.

[0036] Server Technology

[0037] The server technology used in the Visual Network and Unified Video Platform differs from traditional servers. Its streaming media transmission is based on a connection-oriented approach. Its data processing capabilities are independent of traffic volume or communication time, and a single network layer can handle both signaling and data transmission. For voice and video services, streaming media processing on the Visual Network and Unified Video Platform is significantly simpler than data processing, and its efficiency is over a hundred times greater than that of traditional servers.

[0038] Storage Technology

[0039] To accommodate the ultra-large capacity and high-volume media content, the unified video platform's ultra-high-speed storage technology utilizes a state-of-the-art real-time operating system. This technology maps program information from server commands to specific hard drive space, eliminating the need for server-side media delivery and delivering it directly to the user's terminal instantly. User wait times are typically less than 0.2 seconds. Optimized sector distribution significantly reduces the mechanical motion of the hard drive head during seeks, consuming only 20% of the resources of comparable IP-based internet solutions while generating three times the concurrent traffic of traditional hard drive arrays, resulting in over tenfold increases in overall efficiency.

[0040] Network Security Technology

[0041] The structural design of the visual network completely eradicates the network security issues that plague the Internet through separate licensing for each service and complete isolation of equipment and user data. Generally, antivirus programs and firewalls are not required, which prevents attacks from hackers and viruses and provides users with a structurally worry-free and secure network.

[0042] Service Innovation Technology

[0043] The unified video platform integrates services and transmission. Whether it's a single user, a private network user, or a network aggregate, it's all just one automatic connection. User terminals, set-top boxes, or PCs can connect directly to the unified video platform to access a rich variety of multimedia video services. The unified video platform uses a "recipe-based" table configuration model to replace traditional complex application programming. Complex applications can be implemented with minimal code, enabling "unlimited" new service innovation.

[0044] The networking of the visual network is as follows:

[0045] The visual network is a centrally controlled network structure. The network can be a tree network, star network, ring network, etc., but on this basis, a centralized control node is required in the network to control the entire network.

[0046] like Figure 1 As shown in the figure, the visual network is divided into two parts: access network and metropolitan area network.

[0047] Access network equipment can be divided into three categories: node servers, access switches, and terminals (including various set-top boxes, encoding boards, storage devices, etc.). Node servers are connected to access switches, which can be connected to multiple terminals and Ethernet.

[0048] A node server is a centralized control node in the access network that controls access switches and terminals. A node server can be directly connected to an access switch or a terminal.

[0049] Similarly, the devices in the metropolitan area network can be divided into three categories: metropolitan area servers, node switches, and node servers. The metropolitan area servers are connected to the node switches, and the node switches can be connected to multiple node servers.

[0050] The node server is a node server of the access network part, that is, the node server belongs to both the access network part and the metropolitan area network part.

[0051] A metropolitan area server (MAN) is a node in a metropolitan area network (MAN) that provides centralized control and can control both node switches and node servers. A MAN can be directly connected to either a node switch or a node server.

[0052] It can be seen from this that the entire visual network is a hierarchical and centrally controlled network structure, and the network controlled by the node server and the metropolitan area server can be various structures such as tree, star, and ring.

[0053] Figuratively speaking, the access network part can form a unified video platform (the part in the dotted circle), and multiple unified video platforms can form a visual network; each unified video platform can be interconnected through the metropolitan area and wide area visual networks.

[0054] 1. Classification of visual networking devices

[0055] 1.1 The devices in the visual network of the present invention can be divided into three main categories: servers, switches (including Ethernet protocol conversion gateways), and terminals (including various set-top boxes, encoding boards, storage devices, etc.). The visual network as a whole can be divided into metropolitan area networks (or national networks, global networks, etc.) and access networks.

[0056] 1.2 The equipment in the access network can be mainly divided into three categories: node servers, access switches (including Ethernet protocol conversion gateways), and terminals (including various set-top boxes, encoding boards, storage, etc.).

[0057] The specific hardware structure of each access network device is as follows:

[0058] Node server:

[0059] like Figure 2 As shown, it mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;

[0060] Packets from the network interface module 201, CPU module 203, and disk array module 204 all enter the switching engine module 202. The switching engine module 202 queries the address table 205 for incoming packets to obtain the packet's routing information. Based on the packet's routing information, the packet is stored in the corresponding packet buffer 206 queue. If the packet buffer 206 queue is nearly full, the packet is discarded. The switching engine module 202 polls all packet buffer queues and forwards packets if the following conditions are met: 1) the port's send buffer is not full; 2) the queue's packet counter is greater than zero. The disk array module 204 primarily controls the hard disk, including initializing, reading, and writing to the hard disk. The CPU module 203 is primarily responsible for protocol processing with the access switch and terminals (not shown), configuring the address table 205 (including the downstream protocol packet address table, upstream protocol packet address table, and data packet address table), and configuring the disk array module 204.

[0061] Access switch:

[0062] like Figure 3 As shown, it mainly includes a network interface module (downlink network interface module 301, uplink network interface module 302), a switching engine module 303 and a CPU module 304;

[0063] Among them, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the destination address (DA), source address (SA), data packet type and packet length of the packet meet the requirements. If they meet the requirements, the corresponding stream identifier (stream-id) is assigned and enters the switching engine module 303, otherwise it is discarded; the packet (downlink data) coming from the uplink network interface module 302 enters the switching engine module 303; the data packet coming from the CPU module 304 enters the switching engine module 303; the switching engine module 303 processes the incoming packet The address table 306 is looked up to obtain the packet's guidance information. If the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in combination with the stream identifier (stream-id). If the queue of the packet buffer 307 is nearly full, the packet is discarded. If the packet entering the switching engine module 303 is not from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 according to the packet's guidance information. If the queue of the packet buffer 307 is nearly full, the packet is discarded.

[0064] The switching engine module 303 polls all packet buffer queues. In the embodiment of the present invention, there are two situations:

[0065] If the queue is from the downlink network interface to the uplink network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero; 3) a token generated by the rate control module is obtained;

[0066] If the queue is not going from the downlink network interface to the uplink network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero.

[0067] The code rate control module 308 is configured by the CPU module 304 to generate tokens for all packet buffer queues from the downstream network interface to the upstream network interface within a programmable interval to control the code rate of the upstream forwarding.

[0068] The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306 , and configuration of the bit rate control module 308 .

[0069] Ethernet protocol conversion gateway :

[0070] like Figure 4 As shown, it mainly includes a network interface module (downlink network interface module 401, uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a rate control module 408, an address table 406, a packet buffer 407 and a MAC adding module 409, and a MAC deleting module 410.

[0071] The data packet received by the downlink network interface module 401 enters the packet detection module 405. The packet detection module 405 checks whether the Ethernet MAC DA, Ethernet MAC SA, Ethernet length or frame type, visual network destination address DA, visual network source address SA, visual network data packet type, and packet length of the data packet meet the requirements. If they meet the requirements, the corresponding stream identifier (stream-ID) is assigned. Then, the MAC deletion module 410 subtracts the MAC DA, MAC SA, and length or frame type (2 bytes) and enters the corresponding receive buffer. Otherwise, it is discarded.

[0072] The downlink network interface module 401 detects the sending buffer of the port. If there is a packet, it obtains the Ethernet MAC DA of the corresponding terminal based on the visual network destination address DA of the packet, adds the Ethernet MAC DA of the terminal, the MACSA of the Ethernet protocol conversion gateway, and the Ethernet length or frame type, and sends it.

[0073] The functions of other modules in the Ethernet protocol conversion gateway are similar to those of the access switch.

[0074] terminal:

[0075] It mainly includes network interface module, business processing module and CPU module; for example, the set-top box mainly includes network interface module, audio and video encoding and decoding engine module, CPU module; the encoding board mainly includes network interface module, audio and video encoding engine module, CPU module; the memory mainly includes network interface module, CPU module and disk array module.

[0076] 1.3 MAN equipment can be divided into two categories: node servers, node switches, and MAN servers. Node switches primarily consist of network interface modules, switching engine modules, and CPU modules; MAN servers primarily consist of network interface modules, switching engine modules, and CPU modules.

[0077] 2. Definition of Visual Network Data Packet

[0078] 2.1 Access Network Data Packet Definition

[0079] The data packet of the access network mainly includes the following parts: destination address (DA), source address (SA), reserved bytes, payload (PDU), and CRC.

[0080] As shown in the following table, the data packet of the access network mainly includes the following parts:

[0081] DA SA Reserved Payload CRC

[0082] in:

[0083] The destination address (DA) consists of 8 bytes. The first byte indicates the type of data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), with a maximum of 256 possible types. The second to sixth bytes are the metropolitan area network address, and the seventh and eighth bytes are the access network address.

[0084] The source address (SA) is also composed of 8 bytes and has the same definition as the destination address (DA).

[0085] The reserved bytes consist of 2 bytes;

[0086] The payload part has different lengths depending on the type of datagram. If it is a packet of various protocols, it is 64 bytes. If it is a unicast or multicast data packet, it is 32+1024=1056 bytes. Of course, it is not limited to the above two types.

[0087] The CRC consists of 4 bytes and its calculation method follows the standard Ethernet CRC algorithm.

[0088] 2.2 Definition of Metropolitan Area Network Data Packet

[0089] The topology of a metropolitan area network (MAN) is a graph. Two devices may have two or more connections. For example, there may be more than two connections between a node switch and a node server, between node switches, or between node switches and node servers. However, the MAN address of a MAN device is unique. To accurately describe the connection relationship between MAN devices, this embodiment of the present invention introduces a parameter: a label, which uniquely describes a MAN device.

[0090] The definition of labels in this manual is similar to that of MPLS (Multi-Protocol Label Switch). Assuming there are two connections between device A and device B, the data packet from device A to device B will have two labels, and the data packet from device B to device A will also have two labels. Labels are divided into input labels and output labels. Assuming the label of the data packet entering device A (input label) is 0x0000, the label of the data packet when leaving device A (output label) may become 0x0001. The access process of the metropolitan area network is a centralized access process, which means that the address allocation and label allocation of the metropolitan area network are dominated by the metropolitan area server. The node switches and node servers are passively executed. This is different from the label allocation of MPLS, which is the result of negotiation between the switch and the server.

[0091] As shown in the following table, the data packets of the metropolitan area network mainly include the following parts:

[0092] DA SA Reserved Label Payload CRC

[0093] The label format is defined as follows: the label is 32 bits, with the upper 16 bits reserved and only the lower 16 bits used. It is located between the reserved bytes and the payload of the data packet.

[0094] Reference Figure 5 , which shows a flow chart of a log processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0095] Step 501: Obtain target log data corresponding to the target business type;

[0096] When performing various business operations in the visual network (such as uploading conference services, starting conference services, etc.), the business system server (such as terminals, conference management, core servers, etc.) can generate corresponding log data for various businesses. When the visual network fails to perform a certain business operation, it can obtain the target log data corresponding to the business type based on the business type of the target business, so that the business problem can be further located based on the target log data.

[0097] For example, the target log data for uploading a conference service and starting a conference service can be represented as follows (where id is a tracking identifier, which can be a global identifier for the visual network service or other identifier used to track the process. When the business system prints key logs, it can indicate the business operation corresponding to each key log data. When multiple operations are performed on the same business, the tracking identifier corresponding to each business operation is different):

[0098] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting start,id=1

[0099] 2020-07-27 11:42:33,521INFO meeting(1406): ....Other non-critical logs printed

[0100] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add start,id=1,user_list=...

[0101] 2020-07-27 11:42:33,521INFO meeting(1406): ....Other non-critical logs printed

[0102] 2020-07-27 11:42:33,003INFO meeting(1406):upload_meeting_responsestart,id=1

[0103] 2020-07-27 11:42:33,299INFO meeting(1406):#####Meeting id=1Number of meeting members=100###

[0104] 2020-07-27 11:42:33,521INFO meeting(1406): ....Other non-critical logs printed

[0105] 2020-07-27 11:42:33,656INFO meeting(1406):upload_meeting_responseend,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are duplicated'}

[0106] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add end,id=1,result=true

[0107] 2020-07-27 11:42:33,521INFO meeting(1406): ....Other non-critical logs printed

[0108] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting end,id=1,result=true

[0109] 2020-07-27 11:42:33,521INFO meeting(1407):start_meeting start,id=2

[0110] 2020-07-27 11:42:33,521INFO meeting(1407): ....Other non-critical logs printed

[0111] 2020-07-27 11:42:33,003INFO meeting(1407):start_meeting_responsestart,id=2

[0112] 2020-07-27 11:42:33,521INFO meeting(1407): ....Other non-critical logs printed

[0113] 2020-07-27 11:43:25,656INFO meeting(1407):start_meeting_response end,id=2,result={'code':0,'message':'Meeting started successfully'}

[0114] 2020-07-27 11:43:33,521INFO meeting(1407): ....Other non-critical logs printed

[0115] 2020-07-27 11:43:33,521INFO meeting(1407):start_meeting end,id=2,result=true

[0116] Step 502: Determine the process configuration information corresponding to the target service type;

[0117] After obtaining the target log data, the log data printing of the visual network-related business systems is relatively standardized. The start of each process has a corresponding end, and each process has a corresponding tracking identifier. The corresponding process configuration information can be set for different log data according to the business type, so that the process configuration information corresponding to the target business type can be selected from the pre-set multiple process configuration information.

[0118] For example, in the operations of uploading a meeting and starting a meeting:

[0119] (1) The process configuration information corresponding to the upload conference service is as follows:

[0120] upload_meeting start: upload meeting start

[0121] meet_add start: Before uploading the meeting to the core server, other business processes begin, such as adding meeting information to save locally, etc.

[0122] upload_meeting_response start: Start uploading the meeting list to the core server upload_meeting_response end: End uploading the meeting list to the core server and print the core server results

[0123] meet_add end: Add meeting information to the local end

[0124] upload_meeting end: upload meeting end

[0125] (2) The process configuration information corresponding to the business of starting a meeting is as follows:

[0126] start_meeting start: Meeting starts

[0127] start_meeting_response start: command sent to the core server to start

[0128] start_meeting_response end: The command is sent to the core server and the result returned by the core server is printed.

[0129] start_meeting end: Meeting start and end

[0130] Step 503: for each process item in the process configuration information, determine whether the target log data is missing the first key information corresponding to the process item;

[0131] After the process configuration information is determined, for each process item in the process configuration information, a full log search may be performed to determine whether the first key information corresponding to the process item is missing in the target log data.

[0132] For example, in the business of starting a meeting, as in the above example, there may be four pieces of process configuration information. For the first piece of process configuration information "start_meeting start: meeting start", the corresponding first key information may be "start_meeting start"; for the second piece of process configuration information "

[0133] start_meeting_response start: command sent to the core server to start", the corresponding first key information is "start_meeting_response start", and other process configuration information and its corresponding first key information can also be determined as above.

[0134] For each process item in the process configuration information, to determine whether the process is missing, you can use the following steps to determine:

[0135] (1) First, obtain the key information of the first process (start_meeting start), and perform a full log search in the target log data. If the first corresponding log data is retrieved in the target log data, the process tracking identifier of the log data (i.e., the id in the log) can be obtained and the next process search can be performed. If no process is retrieved in the entire log, it means that the business has not been initiated.

[0136] (2) Based on the key information of the second process (start_meeting_response start + process tracking identifier), perform a full log search in the target log data. If the target is not found in the entire log, it means that the business system itself has failed in business processing.

[0137] (3) Perform a full log search of the target log data based on the key information corresponding to the third process (upload_meeting_response end + process tracking identifier). If the code in the result is not equal to 0, it means that the core server has failed to process the business.

[0138] (4) Perform a full log search of the target log data based on the key information corresponding to the fourth process (start_meeting end + process tracking identifier). If the result is false, but the result in the third result is true, it means that the business system itself is wrong. Otherwise, it means that the core server has failed to process the business.

[0139] Step 504 : When it is determined that the target log data does not lack the first key information corresponding to the process item, first process result information corresponding to the process item is generated according to the target log data.

[0140] After determining whether the target log data is missing the first key information corresponding to the process item, if it is determined that the target log data is not missing the first key information corresponding to the process item, first process result information corresponding to the process item can be further generated based on the target log data.

[0141] For example, when all process items for starting a meeting are complete, all process items in the target log data corresponding to the process of starting the meeting can be used as first process result information and stored in a database.

[0142] Start a meeting:

[0143] 2020-07-27 11:42:33,521INFO meeting(1407):start_meeting start,id=1

[0144] 2020-07-27 11:43:12,003INFO meeting(1407):start_meeting_responsestart,id=1

[0145] 2020-07-27 11:43:25,656 INFOmeeting(1407):start_meeting_with_responseend,id=1,result={'code':0,'message':'The meeting started successfully'}

[0146] 2020-07-27 11:42:33,521INFO meeting(1407):start_meeting end,id=1,result=true

[0147] In one embodiment of the present invention, the method further includes:

[0148] When it is determined that the target log data lacks the first key information corresponding to the process item, second process result information corresponding to the process item is generated.

[0149] According to the determination that the target log data is missing the first key information corresponding to the process item, it can be determined that the process of the target log data is missing, and a second process result corresponding to the process item can be generated.

[0150] For example: when searching for process items in the target log data during the start of a meeting, the process item "start_meeting end: meeting start and end" of the start of the meeting process, the corresponding first key information "start_meetingend" is not retrieved in the target log data, indicating that the target log data lacks the meeting end process item, so that the second process result information can be generated based on the business name, the corresponding process item in the target log data, and the analysis results.

[0151] The target log data for uploading the meeting is as follows:

[0152] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting start,id=1

[0153] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add start,id=1,user_list=...

[0154] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add end,id=1,result=true

[0155] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting end,id=1,result=true

[0156] According to the process configuration information corresponding to the business type of uploading the meeting, it can be determined that the key process items are missing, "upload_meeting_response start: start uploading the meeting list to the core server" and "upload_meeting_response end: end uploading the meeting list to the core server, print the core server results", which means that an error occurred when the business system processed the business in the meet_add link, resulting in the failure to send the upload meeting list command to the core server.

[0157] The corresponding second process result information can be generated as follows:

[0158] Upload meeting:

[0159] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting start,id=1

[0160] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add start,id=1,user_list=...

[0161] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add end,id=1,result=true

[0162] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting end,id=1,result=true

[0163] Analysis results: The process items "upload_meeting_response start: start uploading the meeting list to the core server" and "upload_meeting_response end: end uploading the meeting list to the core server and printing the core server results" are missing, and an error occurred when processing the business in the meet_add link.

[0164] In one embodiment of the present invention, the method further includes:

[0165] The first process result information and the second process result information are displayed.

[0166] After the first process result information and / or the second process result information are generated, the first process result information and / or the second process result information may be displayed to relevant personnel so that the relevant personnel can quickly locate problems with the business system.

[0167] In an embodiment of the present invention, by obtaining target log data corresponding to a target business type, the process configuration information corresponding to the target business type is determined, and for each process item in the process configuration information, it is determined whether the target log data is missing the first key information corresponding to the process item. When it is determined that the target log data is not missing the first key information corresponding to the process item, the first process result information corresponding to the process item is generated based on the target log data, thereby realizing analysis of the process items of the target log data, so that when a business fails, the business problem can be quickly located through the first process result information, thereby improving the efficiency of locating the business problem.

[0168] Reference Figure 6 , shows a flowchart of another log processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0169] Step 601: Obtain target log data corresponding to the target business type;

[0170] Step 602: Determine the process configuration information corresponding to the target service type;

[0171] Step 603: for each process item in the process configuration information, determine whether the target log data is missing the first key information corresponding to the process item;

[0172] Step 604: When it is determined that the target log data is not missing the first key information corresponding to the process item, determine the data type corresponding to the process item, and determine the second key information corresponding to the process item from the target log data according to the data type;

[0173] After determining whether the target log data is missing the first key information corresponding to the process item, when it is determined that the target log data is not missing the first key information corresponding to the process item, the process execution result of the process item can be determined, thereby determining the data type of the process execution result corresponding to the process item, such as string data, dictionary data, Boolean data and other data types, and according to different data types, the second key information corresponding to the process item can be determined in the target log data.

[0174] For example: (1) A single process item in the target log data is:

[0175] upload_meeting_response end,id=1,result="-1"

[0176] You can configure the judgment basis for a single process result:

[0177] The result judgment keyword is: result

[0178] Value type: string data

[0179] Judgment result value: "0" success, other failures

[0180] After the current business process is completed, the data type of the result field is determined according to the process in the target log data information. When the result is string data, its value is equal to "0" (the second key information) for success, otherwise it fails.

[0181] (2) A single process item of the target log data is:

[0182] upload_meeting_response end,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are duplicated'}

[0183] You can configure the judgment basis for a single process result:

[0184] The result judgment keyword is: result

[0185] Value type: dictionary data

[0186] The key for the judgment result is: code (code=0 success, other failures)

[0187] After the current business process is completed, the data type of the result field is determined according to the process item in the target log data information. When the result is dictionary data, the code value (the second key information) in the dictionary is taken. When the code value is equal to 0, it is successful, otherwise it fails.

[0188] Step 605: Generate first process result information corresponding to the process item based on the second key information.

[0189] After determining the second key information, the process result of the single process item can be judged based on the second key information. When the process result fails, the first process result information corresponding to the process item can be generated according to the business name, the corresponding process item in the target log data, and the analysis result. When the process result is successful, it means that there is no problem with the process, and the first process result information corresponding to the process item can be generated according to the business name and the corresponding process item in the target log data.

[0190] For example: (1) In the meeting upload business, "2020-07-27 11:43:25,656INFO meeting(1406):upload_meeting_response end,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are repeated'}", the corresponding core server returns the result, where the data type is dictionary data, the "code" value of the first key information is 1, indicating that the process execution failed, and the corresponding first process result information is:

[0191] Upload meeting:

[0192] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting start,id=1

[0193] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add start,id=1,user_list=...

[0194] 2020-07-27 11:43:12,003INFO meeting(1406):upload_meeting_responsestart,id=1

[0195] 2020-07-27 11:43:25,656INFO meeting(1406):upload_meeting_responseend,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are duplicated'}

[0196] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add end,id=1,result=true

[0197] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting end,id=1,result=true

[0198] Analysis result: The core server returns a failure

[0199] (2) In the business of starting a meeting, "2020-07-27 11:43:25,656INFO meeting(1407):start_meeting_with_response end,id=1,result={'code':0,'message':'The meeting started successfully'}", the data type is a dictionary, and the "code" value of the second key information is 1, indicating that the execution result of the process is successful. The corresponding first process result information is:

[0200] Start a meeting:

[0201] 2020-07-27 11:42:33,521INFO meeting(1407):start_meeting start,id=1

[0202] 2020-07-27 11:43:12,003INFO meeting(1407):start_meeting_responsestart,id=1

[0203] 2020-07-27 11:43:25,656INFO meeting(1407):start_meeting_with_responseend,id=1,result={'code':0,'message':'Meeting started successfully'}

[0204] In one embodiment of the present invention, generating the first process result information corresponding to the process item according to the second key information includes:

[0205] When the second key information is non-specified information, first position information corresponding to the process item is determined in the target log data; and first process result information corresponding to the process item is generated according to the second key information and the first position information.

[0206] In actual applications, the second key information can be specified information or unspecified information. When the second key information is specified information, it can indicate that the execution result of the single process is successful. When the second key information is unspecified information, it can indicate that the execution result of the single process fails. The problem of the corresponding business system can be located in the process, so that the first position information corresponding to the process item can be determined in the target log data, so that the first process result information corresponding to the process item can be generated based on the second key information and the first position information.

[0207] For example, in the meeting upload service, the first position information of "2020-07-27 11:43:25,656INFO meeting(1406):upload_meeting_response end,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are duplicated'}" in the target log data can be represented by the line number. When the line number of this process item is [10,21,33], the corresponding first process result information can be represented as:

[0208] Upload meeting:

[0209] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting start,id=1

[0210] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add start,id=1,user_list=...

[0211] 2020-07-27 11:43:12,003INFO meeting(1406):upload_meeting_responsestart,id=1

[0212] 2020-07-27 11:43:25,656INFO meeting(1406):upload_meeting_responseend,id=1,result={'code':1,'message':'Meeting upload failed, meeting list members are duplicated'}

[0213] 2020-07-27 11:42:33,521INFO meeting(1406):meet_add end,id=1,result=true

[0214] 2020-07-27 11:42:33,521INFO meeting(1406):upload_meeting end,id=1,result=true

[0215] Analysis result: The core server returns a failure

[0216] The error occurred in the original log file line number: [10,21,33]

[0217] In an embodiment of the present invention, target log data corresponding to a target business type is obtained to determine process configuration information corresponding to the target business type. For each process item in the process configuration information, it is determined whether the target log data is missing the first key information corresponding to the process item. When it is determined that the target log data is not missing the first key information corresponding to the process item, the data type corresponding to the process item is determined. According to the data type, the second key information corresponding to the process item is determined from the target log data. Based on the second key information, the first process result information corresponding to the process item is generated, thereby determining the first position information of the process item with the problem. Therefore, when a business fails, the first process result information containing the first position information can be generated by analyzing the log, thereby quickly locating the business problem and improving the efficiency of locating the business problem.

[0218] The following combination Figure 7-8 The above embodiments of the present invention are exemplarily described as follows:

[0219] like Figure 7 As shown, in the visual network, a business system server and a log analysis tool can be set up. Among them, the business system server can include servers such as conference management servers, network management servers, terminal servers, and core servers, which are used to process various types of visual network services and generate corresponding log data; the log analysis tool can include an interface module (for displaying log analysis results), a log analysis module (for performing full log retrieval and error analysis for each business), for setting corresponding process configuration information for each business, and a database module (for storing log analysis results).

[0220] Log analysis tools can be used to:

[0221] 1) The log analysis tool configures the log analysis process for each business;

[0222] 2) The log analysis tool connects to the business system server to obtain analysis log files, or imports local log files;

[0223] 3) Select the business to be analyzed (single or multiple);

[0224] 4) Perform full log retrieval and error analysis for each business according to the log analysis process configured for the business

[0225] 5) Save the analysis result data and perform conditional filtering and query on the result data.

[0226] exist Figure 8 In the log analysis process, the log analysis process is described in detail based on the functions of the above log analysis tools:

[0227] 1) Open the log anomaly analysis tool and configure the log analysis process for each business (set the process configuration information corresponding to various business types). After configuration, you can perform data analysis and packaging into the database in the log analysis module, and save the configured log analysis process (process configuration information) in the database module.

[0228] 2) After the log analysis tool completes the log analysis process configuration for the business (after the configuration is completed, other users do not need to configure it again after logging into the log analysis tool), the log analysis tool can connect to the business system server via FTP (File Transfer Protocol) based on the IP, port, user name, password and log path of the business system entered by the user, download the log to be analyzed (target log data), or import local logs into the log analysis tool.

[0229] 3) Select the business to be analyzed for log analysis. The log analysis tool can support single business analysis and simultaneous analysis of multiple businesses.

[0230] 4) The log analysis module can parse the data to obtain the business to be analyzed, and send messages to the database to obtain the log analysis process configuration corresponding to the business. The database can return the log analysis process configuration, so that after the user clicks on log analysis, the log analysis tool can perform full log retrieval one by one according to the log analysis process configuration of the business.

[0231] The entire log process can be retrieved and analyzed through the log analysis process + process tracking identifier (first keyword).

[0232] After the process retrieval is completed, if the business log analysis process is lost or the result is not equal to true and the code is not equal to 0 at the end of a process, it means that the business execution has failed. The retrieved process and analysis results can be recorded and stored in the database.

[0233] After the process retrieval is completed, the business log analysis process is not missing, and you can further analyze whether the resule in the process failed. If it fails, the single process execution result fails, and the reason for the failure is analyzed. If result = true and code = 0 (the second key information) at the end of each process, it means that the business execution is successful.

[0234] 5) After the log analysis tool analyzes the complete process of a business, the process log is extracted from the original log file and saved in the database.

[0235] When the business execution is successful, the saving format is: business name + process log information; when the business execution fails, the saving format is: business name + process log information + analysis results + line number of the original log file where the error occurred (first position information).

[0236] Based on the line number where the error occurred in the original log file, you can quickly open the original log file and locate the corresponding line number in the original log, making it easier for testers or developers to view more log information and locate the problem.

[0237] 6) After all business analyses are completed, the analysis results can be obtained from the data and displayed to relevant personnel through the interface.

[0238] The log analysis tool also supports combined filtering and querying of analysis results. The filtering conditions can be: business, business + success, business + failure, business + process missing, etc. This log analysis tool is not limited to visual network business log analysis, but also applies to other business systems with log printing specifications.

[0239] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0240] Reference Figure 9 , shows a structural block diagram of a log processing device provided by an embodiment of the present invention, which may specifically include the following modules:

[0241] Target log data acquisition module 901, used to acquire target log data corresponding to the target business type;

[0242] A process configuration information acquisition module 902 is used to determine the process configuration information corresponding to the target business type;

[0243] The judgment module 903 is used to judge, for each process item in the process configuration information, whether the target log data lacks the first key information corresponding to the process item;

[0244] The first process result information generating module 904 is configured to generate first process result information corresponding to the process item according to the target log data when it is determined that the target log data does not lack key information corresponding to the process item.

[0245] In one embodiment of the present invention, the apparatus may further include:

[0246] The second process result information generating module is configured to generate second process result information corresponding to the process item when it is determined that the target log data lacks the first key information corresponding to the process item.

[0247] In one embodiment of the present invention, the apparatus may further include:

[0248] The process result information display module is used to display the first process result information and the second process result information.

[0249] In one embodiment of the present invention, the first process result information generating module 904 includes:

[0250] A second key information determination submodule is configured to determine a data type corresponding to the process item, and determine the second key information corresponding to the process item from the target log data according to the data type;

[0251] The first process result information generating submodule is configured to generate the first process result information corresponding to the process item according to the second key information.

[0252] In one embodiment of the present invention, the first process result information generating submodule includes:

[0253] a first location information determining unit, configured to determine, in the target log data, first location information corresponding to the process item when the second key information is non-specified information;

[0254] The first process result information generating unit is configured to generate first process result information corresponding to the process item according to the second key information and the first location information.

[0255] In an embodiment of the present invention, by obtaining target log data corresponding to a target business type, the process configuration information corresponding to the target business type is determined, and for each process item in the process configuration information, it is determined whether the target log data is missing the first key information corresponding to the process item. When it is determined that the target log data is not missing the first key information corresponding to the process item, the first process result information corresponding to the process item is generated based on the target log data, thereby realizing analysis of the process items of the target log data, so that when a business fails, the business problem can be quickly located through the first process result information, thereby improving the efficiency of locating the business problem.

[0256] An embodiment of the present invention further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the above log processing method is implemented.

[0257] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned log processing method is implemented.

[0258] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0259] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0260] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0261] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0262] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0263] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0264] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0265] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0266] The above is a detailed introduction to the log processing method and device provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A log processing method, characterized in that: A log analysis tool connected to a business system server is used to set corresponding process configuration information for each business, perform full log retrieval and error analysis on each business, save log analysis results, and display log analysis results; the method includes: Obtain target log data corresponding to a target service type; wherein the target service is a conference service in a visual network, and the target service type is an upload conference service or a start conference service; Determine the process configuration information corresponding to the target business type from a plurality of pre-set process configuration information; wherein the plurality of process configuration information includes: business process configuration information for uploading a meeting, and business process configuration information for starting a meeting; For each process item in the process configuration information, determining whether the target log data is missing the first key information and the process tracking identifier corresponding to the process item; When it is determined that the target log data is not missing the first key information and the process tracking identifier corresponding to the process item, generating first process result information corresponding to the process item according to the target log data; wherein the first key information is used to determine whether the process is missing; When it is determined that the target log data lacks the first key information corresponding to the process item, generating second process result information corresponding to the process item; The generating, according to the target log data, first process result information corresponding to the process item includes: Determine a data type corresponding to the process item, and determine second key information corresponding to the process item from the target log data according to the data type; wherein the second key information is used to determine a process result of a single process item; Generate first process result information corresponding to the process item based on the second key information.

2. The method according to claim 1, characterized in that Also includes: The first process result information and the second process result information are displayed.

3. The method according to claim 1, characterized in that The generating, based on the second key information, first process result information corresponding to the process item includes: When the second key information is non-specified information, determining first position information corresponding to the process item in the target log data; First process result information corresponding to the process item is generated according to the second key information and the first location information.

4. A log processing device, characterized in that: A log analysis tool connected to a business system server, the log analysis tool is used to set corresponding process configuration information for each business, perform full log retrieval and error analysis for each business, save log analysis results, and display log analysis results; the device includes: A target log data acquisition module is used to acquire target log data corresponding to a target service type; wherein the target service is a conference service in a visual network, and the target service type is an upload conference service or a start conference service; A process configuration information acquisition module is used to determine the process configuration information corresponding to the target business type from a plurality of pre-set process configuration information; wherein the plurality of process configuration information includes: business process configuration information for uploading a meeting, and business process configuration information for starting a meeting; a judgment module, configured to judge, for each process item in the process configuration information, whether the target log data lacks the first key information and the process tracking identifier corresponding to the process item; A first process result information generating module is configured to generate first process result information corresponding to the process item based on the target log data when it is determined that the target log data does not lack first key information corresponding to the process item and the process tracking identifier; wherein the first key information is used to determine whether a process is missing; A second process result information generating module is configured to generate second process result information corresponding to the process item when it is determined that the target log data lacks the first key information corresponding to the process item; The first process result information generation module includes: a second key information determination submodule, configured to determine a data type corresponding to the process item, and determine, based on the data type, second key information corresponding to the process item from the target log data; wherein the second key information is used to determine a process result of a single process item; The first process result information generating submodule is configured to generate the first process result information corresponding to the process item according to the second key information.

5. The device according to claim 4, characterized in that Also includes: The process result information display module is used to display the first process result information and the second process result information.

6. The device according to claim 4, characterized in that The first process result information generation submodule includes: a first location information determining unit, configured to determine, in the target log data, first location information corresponding to the process item when the second key information is non-specified information; The first process result information generating unit is configured to generate first process result information corresponding to the process item according to the second key information and the first location information.

Images