Network element log extraction method, device, electronic device and storage medium

Abstract

The present disclosure provides a network element log extraction method, device, electronic device and storage medium, which relate to the field of communication technology. The scheme is: receiving a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries the storage address of a log file segment; generating an orderly increasing message identifier for the log report message; based on a load balancing strategy, selecting a target server from a server cluster, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network; based on the message identifier, the storage address of the log file segment and the IP address of the target server, a log processing request is generated and broadcasted in the multicast network. The present disclosure implements dynamic cluster deployment of log services based on log proxy servers, solves the problem of disordered messages between cluster services, and greatly improves the efficiency of network element log extraction.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a network element log extraction method, device, electronic device and storage medium. Background Art

[0002] In the related art, in order to solve the problem of disordered messages between cluster services and the problem of scattered storage of file fragments caused by multiple File Transfer Protocol (FTP) addresses, the log service can only be deployed at a single point. In the case of single-point deployment, the performance upper limit of the log service is low, and it cannot be dynamically expanded horizontally, which greatly affects the efficiency of extracting network element logs. Moreover, if a single point of the log service fails, all log business functions of the network management will be unavailable. Therefore, how to achieve cluster deployment of the log service is something that needs to be urgently solved now. Summary of the invention

[0003] The present invention provides a network element log extraction method, device, electronic device and storage medium.

[0004] According to one aspect of the present disclosure, a network element log extraction method is provided, comprising:

[0005] Receive a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries a storage address of a log file segment;

[0006] Generate sequentially increasing message identifiers for log reporting messages;

[0007] Based on the load balancing strategy, a target server is selected from the server cluster, where a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network;

[0008] Based on the message identifier, the storage address of the log file fragment and the IP address of the target server, a log processing request is generated and broadcasted in the multicast network.

[0009] In the disclosed embodiment, the reverse proxy, multicast message and semaphore orchestration technology based on the log proxy server realizes the dynamic cluster deployment of the log service, solves the problem of message disorder between cluster services, greatly improves the efficiency of network element log extraction, and avoids the uneven busy and idle state of the server through the load balancing strategy, thereby improving the utilization rate of server resources.

[0010] According to another aspect of the present disclosure, a network element log extraction device is provided, comprising:

[0011] A receiving module, used for receiving a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries a storage address of a log file segment;

[0012] A message identification module, used to generate orderly increasing message identifications for log reporting messages;

[0013] A selection module is used to select a target server from a server cluster based on a load balancing strategy, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network;

[0014] The broadcast module is used to generate a log processing request based on the message identifier, the storage address of the log file fragment and the IP address of the target server and broadcast it in the multicast network.

[0015] According to another aspect of the present disclosure, there is provided an electronic device, including a memory and a processor;

[0016] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the network element log extraction method of the embodiment of the first aspect of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the network element log extraction method of the embodiment of the first aspect of the present disclosure is implemented.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the network element log extraction method of the embodiment of the first aspect of the present disclosure is implemented.

[0019] The reverse proxy, multicast message and semaphore orchestration technology based on the log proxy server in the present invention realizes the dynamic cluster deployment of log services, solves the problem of message disorder between cluster services, greatly improves the efficiency of network element log extraction, and avoids the uneven busy and idle state of servers through load balancing strategies, thereby improving server resource utilization.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0022] Figure 2 This is the log server cluster architecture diagram;

[0023] Figure 3 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0024] Figure 4 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0025] Figure 5 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0026] Figure 6 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0027] Figure 7 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure;

[0028] Figure 8 is a structural diagram of a network element log extraction device according to an embodiment of the present disclosure;

[0029] Fig. 9 is a structural diagram of a network element log extraction device according to an embodiment of the present disclosure;

[0030] Fig.10 is a block diagram of an electronic device for implementing the network element log extraction method of the embodiment of the present disclosure;

[0031] Fig.11 It is a block diagram of an electronic device used to implement the network element log extraction method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0033] The following describes the network element log extraction method, device, electronic device and storage medium disclosed in the present invention with reference to the accompanying drawings.

[0034] Figure 1 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Figure 1 As shown, the method is applicable to a target proxy server and includes the following steps:

[0035] S101, receiving a log reporting message of a target network element sent by a network element adapter NEA, wherein the log reporting message carries a storage address of a log file segment.

[0036] NE logs are the primary means for wireless network devices to record their configuration parameters, operating status, and various indicators. They are one of the main bases for network maintenance personnel to troubleshoot, locate, and solve NE problems and optimize the network.

[0037] The user can initiate the extraction of network element logs through the client user interface. After receiving the network element log extraction instruction, the target network element uploads the log file segments to the file system of the FTP service and sends a log reporting message to the network element adapter (NEA). Each log file segment corresponds to a log reporting message.

[0038] NEA adapts and forwards the log report message to the target proxy server. The log report message carries the storage address of the log file segment, that is, the FTP address where the segment is stored. Based on the storage address, the corresponding log file segment can be found in the FTP service.

[0039] S102, generating orderly increasing message identifiers for log reporting messages.

[0040] In the present disclosure, multiple servers process messages simultaneously. In order to prevent message disorder between cluster services, it is necessary to sort the log report messages, generate ordered and increasing message identifiers, and cache the message identifiers of the currently generated log report messages in sequence in the identifier cache of the target network element.

[0041] A hash sorting algorithm is used to assign an ordered and increasing message identifier to different network element log messages. Optionally, a sorting algorithm such as radix sorting, counting sorting, or a counter may be used to sort the log report messages.

[0042] Hash is a hash table, also called a hash table. The dictionaries and sets we commonly use belong to hash tables. Hash tables use hash functions when storing messages, which can map messages of any length to a shorter and fixed memory address, and then return them to us in the form of an index, and then save the value to the corresponding memory.

[0043] S103, based on the load balancing strategy, selecting a target server from the server cluster, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network.

[0044] Load balancing strategy is a technology that expands the bandwidth of existing network devices and servers, increases throughput, strengthens network data processing capabilities, and improves network flexibility and availability. The main algorithms used to implement load balancing strategy are weighted polling algorithm and weighted least connection algorithm.

[0045] The target server for processing log files is selected according to the weighted polling algorithm or the weighted least connection algorithm, so that the processing volume of each target server is roughly the same, improving the utilization rate of server resources, avoiding uneven busy and idle time, and reducing the probability of problems occurring under high load conditions.

[0046] The weighted round-robin algorithm assigns a weight to each log server, which represents the server's ability to handle log services relative to other servers. When a server has a weight of n, the load balancing strategy allocates n new connections to the server before allocating traffic to the next server.

[0047] The weighted least connections algorithm assigns new connections to the real server with the fewest active connections. Each real server is assigned a weight m, and the server's ability to handle active connections is equal to m divided by the sum of all server weights. The load balancing strategy assigns new connections to real servers with far fewer active connections than their capacity.

[0048] The server cluster includes multiple servers, each of which is deployed with a log service. Optionally, an FTP server can also be deployed on each server. In the log extraction process, the log proxy server can select a suitable server from the server cluster as the target server based on the load balancing strategy. The server cluster and the target proxy server form a multicast network, such as Figure 2 As shown. Messages can be broadcasted between servers in a multicast network. For example, a log proxy server can broadcast messages to servers in a server cluster. Servers in a server cluster can also broadcast messages to other servers and log proxy servers within the cluster.

[0049] The log processing progress and shared message identifier can be broadcasted in the multicast network, so that each server in the server cluster and the target proxy server can understand the current progress of network element log extraction, and then perform the next step of processing according to the progress.

[0050] S104, based on the message identifier, the storage address of the log file segment and the Internet Protocol (IP) address of the target server, a log processing request is generated and broadcasted in the multicast network.

[0051] Generate a log processing request, which includes a message identifier of the log reporting information, an FTP storage address of the log file fragment, and an IP address of a target server for executing the service.

[0052] The log processing request is broadcasted in the multicast network so that each log server in the server cluster can receive the log processing request. Accordingly, after receiving the log processing request, the target server extracts the log file fragments according to the storage address of the log file fragments in the request and merges them.

[0053] In the disclosed embodiment, a log report message of a target network element sent by a network element adapter NEA is received, wherein the log report message carries the storage address of a log file segment, an orderly increasing message identifier is generated for the log report message, and a target server is selected from a server cluster based on a load balancing strategy, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network, and a log processing request is generated based on the message identifier, the storage address of the log file segment and the IP address of the target server and broadcasted in the multicast network. In the disclosed embodiment, the reverse proxy, multicast message and message identifier orchestration technology based on the log proxy server realizes the dynamic cluster deployment of the log service, solves the problem of disordered messages between cluster services, greatly improves the efficiency of network element log extraction, and avoids the situation of uneven busy and idle servers through the load balancing strategy, thereby improving the utilization rate of server resources.

[0054] Figure 3 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Based on the above embodiment, further combined with Figure 3 , explaining the process of generating sequentially increasing message identifiers for log report messages, including the following steps:

[0055] S301: Obtain a network element identifier of a target network element.

[0056] In the embodiment of the present disclosure, each network element is provided with a corresponding network element identifier, and a target network element for sending a log reporting message can be searched based on the network element identifier.

[0057] S302: Generate a message identifier according to the network element identifier and the reporting order of the log reporting message in the target network element.

[0058] A message identifier is generated according to the network element identifier and the reporting order of the log report message in the target network element. Based on the message identifier, it can be identified which network element reported the log report message and which message it is, thereby ensuring the order of messages within the network element.

[0059] Optionally, the message identifier may be formed by concatenating the network element identifier and the reporting sequence, or the reporting sequence may be used as a subscript of the network element identifier to generate the message identifier.

[0060] In the disclosed embodiment, the network element identifier of the target network element is obtained, and the message identifier is generated according to the network element identifier and the reporting order of the log report message in the target network element. In the disclosed embodiment, the log report message in each network element is sorted to ensure the order of the messages inside the network element.

[0061] Figure 4is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Based on the above embodiment, a log processing request is generated and broadcasted in a multicast network. Figure 4 As shown, it also includes:

[0062] S401, receiving a first log upload completion message sent by the NEA to the target network element.

[0063] After the log file upload is completed, the target network element sends a first log upload completion message to the NEA, and the NEA forwards the message to the target proxy server after receiving the first log upload completion message. Accordingly, the target proxy server can receive the first log upload completion message sent by the NEA to the target network element, and based on the first log upload completion message, it can be determined that the target network element has uploaded all log file segments to the FTP address of the log server.

[0064] S402, receiving a clear request message broadcasted by a target server, wherein the clear request message carries a message identifier of a request for processing logs to be cleared.

[0065] When the target server processes the corresponding log file fragment according to the log report message, it will broadcast a request to clear information in the multicast network. The request to clear message contains the message identifier of the log processing request to be cleared and the storage address of the corresponding log file fragment.

[0066] Since the log file fragment corresponding to the log processing request to be cleared has been processed, in order to prevent the target server from reprocessing the processed log file fragment according to the message identifier in the identifier cache, the message identifier of the log processing request to be cleared needs to be cleared from the identifier cache of the target network element.

[0067] After receiving the clear request message broadcasted by the target server, the target proxy server clears the message identifier of the log processing request to be cleared from the identifier cache of the target network element according to the message identifier in the clear request message.

[0068] S403: When the message identifier of the log processing request to be cleared is the maximum message identifier corresponding to the target network element, it is determined that the log file upload of the target network element is completed.

[0069] Since the target server will give priority to the log processing request corresponding to the smallest message ID, when the message ID of the received log processing request to be cleared is the largest message ID corresponding to the target network element, it proves that all log file segments of the target network element have been uploaded and processed.

[0070] The maximum message identifier corresponding to the target network element is the maximum message identifier read from the identifier cache, and optionally, may also be the last message identifier in the identifier cache.

[0071] S404: Generate a second log upload completion message according to the IP address of the target server, and broadcast the second log upload completion message in the multicast network.

[0072] A second log upload completion message is generated according to the IP address of the target server. The second log upload completion message represents that the log server has merged all the log file fragments uploaded by the target network element and generated a log file that can be directly downloaded and viewed by the client. The generated second log upload completion message is broadcast in the multicast network to notify each server that the network element log upload is complete.

[0073] In the disclosed embodiment, the target proxy server receives the first log upload completion message sent by the NEA to the target network element, and receives the request to clear message broadcasted by the target server, wherein the request to clear message carries the message identifier of the request to clear the log processing. When the message identifier of the request to clear the log processing is the maximum message identifier corresponding to the target network element, it is determined that the log file upload of the target network element is completed, and a second log upload completion message is generated according to the IP address of the target server, and the second log upload completion message is broadcasted in the multicast network. In the disclosed embodiment, it is determined whether the log upload is completed based on the log upload completion message and the message identifier of the request to clear the log processing, and the completion message is broadcasted in the multicast network after completion, so that each server can understand the processing progress of the network element log, and thus implement different message processing according to the progress.

[0074] Figure 5 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Figure 5 As shown, the method is applicable to any server in a server cluster, and includes the following steps:

[0075] S501, receiving a clearing request message, wherein the clearing request message includes a first storage address of a first log file corresponding to a log processing request to be cleared.

[0076] refer to Figure 2 Each server in the server cluster can receive the log processing request broadcasted by the log proxy server, wherein the log processing request is generated by the log proxy server according to the storage address of the log file shard, the message identifier and the IP address of the target server. The message identifier is incremented in order according to the reporting order of the log file shard in the network element.

[0077] The server receives a clear request message broadcasted in a multicast network, which includes a message identifier of the log processing request to be cleared and a first storage address of a first log file, and clears the log processing request to be cleared cached in the message cache according to the message identifier.

[0078] S502, sequentially obtain a target log processing request corresponding to an unprocessed minimum message identifier from a message cache, wherein the target log processing request includes a second storage address of a corresponding second log file and a first IP address of a corresponding target server.

[0079] Since message IDs increase in sequence according to the upload time, in order to ensure the orderliness of message processing, data uploaded earlier should be processed first, that is, log processing requests with smaller message IDs.

[0080] A log processing request corresponding to an unprocessed minimum message identifier is obtained based on the message identifier in the log processing request, wherein the target log processing request further includes a second storage address of a corresponding second log file and a first IP address of a corresponding target server.

[0081] S503, when the second IP address of any server is consistent with the first IP address, download the first log file and the second log file according to the first storage address and the second storage address respectively, merge the two log files to obtain a third log file, and store it to the target storage address.

[0082] When the second IP address of the server itself is consistent with the first IP address in the target log processing request, the current server is the target server and is responsible for processing the corresponding log segment file.

[0083] The first log file and the second log file are downloaded according to the first storage address and the second storage address, the two log files are merged to obtain a third log file, and the third log file is stored in the target storage address.

[0084] Optionally, the target storage address may be the second storage address or another new storage address.

[0085] After the files are merged and stored, a request clear message for the target log processing request is generated and broadcasted in the multicast network, where the multicast network includes a server cluster and a target proxy server. Accordingly, the role of each server in the server cluster is the same, so the server can not only broadcast the request clear message externally, but also receive the request clear message broadcasted by other servers.

[0086] After receiving the clear request message, any server in the multicast network clears the log processing request to be cleared cached in the message cache.

[0087] In the disclosed embodiment, a request to clear message is received, wherein the request to clear message includes the first storage address of the first log file corresponding to the log processing request to be cleared, and the log processing request corresponding to the unprocessed minimum message identifier is obtained from the message cache in sequence, wherein the target log processing request includes the second storage address of the corresponding second log file and the first IP address of the corresponding target server, and when the second IP address of any server itself is consistent with the first IP address, the first log file and the second log file are downloaded according to the first storage address and the second storage address respectively, and the two log files are merged to obtain a third log file, and stored in the target storage address. In the disclosed embodiment, two log file fragments uploaded by a network element are merged, and all log file fragments uploaded by a network element in succession can be merged together through repeated operations, which is convenient for the client to view.

[0088] Figure 6 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Based on the above embodiment, Figure 6 As shown, it also includes:

[0089] S601, receiving a log processing request, wherein the log processing request includes a message identifier of a log reporting message from a network element and a storage address of a corresponding log file segment, and an IP address of a target server of the log processing request.

[0090] refer to Figure 2 Each server in the server cluster can receive the log processing request broadcasted by the log proxy server, wherein the log processing request is generated by the log proxy server according to the storage address of the log file shard, the message identifier and the IP address of the target server. The message identifier is incremented in order according to the reporting order of the log file shard in the network element.

[0091] S602: After receiving the log processing request, the received log processing request is cached in order into a message cache according to the message identifier of the log processing request.

[0092] After receiving the log processing request, the server caches the log processing request in the message cache in ascending order of the message identifier, so that when the target server obtains the log processing request, it gives priority to the log processing request corresponding to the smallest unprocessed message identifier.

[0093] S603, after receiving the request clear message, sequentially obtain the target log processing request corresponding to the unprocessed minimum message identifier from the message cache.

[0094] The clearing request message includes a first storage address of a first log file corresponding to the log processing request to be cleared.

[0095] The target log processing request includes the second storage address of the corresponding second log file and the first IP address of the corresponding target server.

[0096] S604, when the second IP address of any server is consistent with the first IP address, download the first log file and the second log file according to the first storage address and the second storage address respectively, merge the two log files to obtain a third log file, and store it to the target storage address.

[0097] For the detailed description of steps 603 to 604, please refer to the relevant contents in the above embodiment, which will not be repeated here.

[0098] It should be noted that, in the disclosed embodiment, the first log file downloaded for the Nth time is the third log file obtained by merging in the N-1th time, which is a cumulative merging process. For example, in this merging, the log file fragments corresponding to the message identifiers 1 and 2 are merged, and in the next merging, the merged file of 1 and 2 is merged with the log file fragment corresponding to the message identifier 3.

[0099] In the embodiment of the present disclosure, two log file sub-packages and fragments uploaded by a network element are merged. By repeated operations, all log file sub-packages and fragments successively uploaded by a network element can be merged together, which is convenient for the client to view.

[0100] In the disclosed embodiment, a log processing request is received. After receiving the log processing request, the received log processing request is sequentially cached in the message cache according to the message identifier of the log processing request. After receiving the request to clear the message, the target log processing request corresponding to the smallest unprocessed message identifier is sequentially obtained from the message cache. When the second IP address of any server itself is consistent with the first IP address, the first log file and the second log file are downloaded according to the first storage address and the second storage address respectively, and the two log files are merged to obtain a third log file, and stored in the target storage address. In the disclosed embodiment, two log file sub-packets and fragments uploaded by a network element are merged. Through repeated operations, all log file sub-packets and fragments uploaded by a network element can be merged together, which is convenient for the client to view.

[0101] Figure 7 is a flow chart of a network element log extraction method according to an embodiment of the present disclosure. Figure 7 As shown, the method comprises the following steps:

[0102] S701, receiving a second log upload completion message broadcasted by a target proxy server in a multicast network, wherein the second log upload completion message carries a first IP address of a last server for executing a log processing request corresponding to a maximum message identifier.

[0103] The server receives the second log upload completion message, which proves that the file upload merging is now complete, and can clear the log processing request of the network element in the message cache.

[0104] The second log upload completion message carries the first IP address of the last server used to execute the log processing request corresponding to the maximum message identifier.

[0105] S702: When the first IP address and the second IP address of the last server are consistent, a download message is generated according to the target storage address corresponding to the last server and sent to the client.

[0106] Compare and respond to whether the first IP address of the last server is consistent with the second IP address of the current server itself. If they are consistent, generate a download message based on the target storage address corresponding to the last server and send it to the client. The client can download the log file from the corresponding address based on the download message.

[0107] In the disclosed embodiment, a second log upload completion message broadcasted by a target proxy server in a multicast network is received, wherein the second log upload completion message carries the first IP address of the last server for executing the log processing request corresponding to the maximum message identifier, and when the first IP address of the last server is consistent with the second IP address, a download message is generated and sent to the client according to the target storage address corresponding to the last server. In the disclosed embodiment, a notification is sent to the client after the log file upload is completed, prompting the download, so that the user can know the extraction progress of the network element log in time, thereby improving the user experience.

[0108] Figure 8 is a structural diagram of a network element log extraction device according to an embodiment of the present disclosure, such as Figure 8 As shown, the network element log extraction device 800 is applicable to the target proxy server and includes:

[0109] The receiving module 810 is used to receive a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries a storage address of a log file segment;

[0110] A message identification module 820, used to generate orderly increasing message identifications for log reporting messages;

[0111] A selection module 830 is used to select a target server from a server cluster based on a load balancing strategy, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network;

[0112] The broadcast module 840 is used to generate a log processing request based on the message identifier, the storage address of the log file and the IP address of the target server and broadcast it in the multicast network.

[0113] It should be noted that the above explanation of the network element log extraction method embodiment is also applicable to the network element log extraction device of this embodiment, and will not be repeated here.

[0114] Furthermore, in a possible implementation of the embodiment of the present disclosure, the message identification module 820 is also used to: obtain the network element identification of the target network element; and generate a message identification according to the network element identification and the reporting order of the log reporting message in the target network element.

[0115] Furthermore, in a possible implementation of the embodiment of the present disclosure, the message identification module 820 is also used to: after generating orderly increasing message identifiers for the log reporting message, cache the message identifiers of the currently generated log reporting message in sequence into the identification cache of the target network element.

[0116] Furthermore, in a possible implementation of the embodiment of the present disclosure, the broadcast module 840 is also used to: after generating a log processing request and broadcasting it in the multicast network, receive a first log upload completion message sent by the NEA to the target network element; receive a request to clear message broadcasted by the target server, wherein the request to clear message carries the message identifier of the log processing request to be cleared; when the message identifier of the log processing request to be cleared is the maximum message identifier corresponding to the target network element, it is determined that the log file upload of the target network element is completed; generate a second log upload completion message according to the IP address of the target server, and broadcast the second log upload completion message in the multicast network.

[0117] Furthermore, in a possible implementation of the embodiment of the present disclosure, the broadcast module 840 is also used to: after receiving the request clear message broadcasted by the target server, clear the message identifier of the log processing request to be cleared from the identifier cache of the target network element.

[0118] Fig. 9 is a structural diagram of a network element log extraction device according to an embodiment of the present disclosure, such as Fig. 9 As shown, the network element log extraction device 900 is applicable to any server in a server cluster and includes:

[0119] A first receiving module 910 is configured to receive a clearing request message, wherein the clearing request message includes a first storage address of a first log file corresponding to the log processing request to be cleared;

[0120] The acquisition module 920 is used to sequentially acquire the target log processing request corresponding to the unprocessed minimum message identifier from the message cache, wherein the target log processing request includes the second storage address of the corresponding second log file and the first IP address of the corresponding target server;

[0121] The merging module 930 is used to download the first log file and the second log file according to the first storage address and the second storage address respectively when the second IP address of any server itself is consistent with the first IP address, merge the two log files to obtain a third log file, and store it to the target storage address.

[0122] Furthermore, in a possible implementation of the embodiment of the present disclosure, the network element log extraction device 900 also includes: a second receiving module 940, which is used to receive the log processing request broadcast by the log proxy server, and each time a log processing request is received, the received log processing request is cached in sequence into the message cache according to the message identifier of the log processing request; wherein the log processing request includes the message identifier of the log reporting message from the network element and the storage address of the corresponding log file, as well as the IP address of the target server of the log processing request.

[0123] Furthermore, in a possible implementation of the embodiment of the present disclosure, the first receiving module 910 is further configured to: after receiving the clear request message, clear the log processing request to be cleared cached in the message cache.

[0124] Furthermore, in a possible implementation of the embodiment of the present disclosure, the merging module 930 is also used to: after storing the third log file to the target storage address, generate a request clear message for the target log processing request, and broadcast it in the multicast network, wherein the multicast network includes a server cluster and a target proxy server.

[0125] Furthermore, in a possible implementation of the embodiment of the present disclosure, the network element log extraction device 900 further includes:

[0126] The third receiving module 950 is used to receive a second log upload completion message broadcasted by the target proxy server in the multicast network, wherein the second log upload completion message carries the first IP address of the last server for executing the log processing request corresponding to the maximum message identifier;

[0127] The generating module 960 is used to generate a download message and send it to the client according to the target storage address corresponding to the last server when the first IP address and the second IP address of the last server are consistent.

[0128] Furthermore, in a possible implementation of the embodiment of the present disclosure, the third receiving module 950 is further configured to: after receiving the second log upload completion message broadcasted by the target proxy server in the multicast network, clear the log processing request in the message cache.

[0129] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0130] Fig.10 A schematic block diagram of an example electronic device 100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0131] like Fig.10 As shown, it includes a memory 101, a processor 102, and a computer program stored in the memory 101 and executable on the processor 102. When the processor 102 executes the program, the aforementioned network element log extraction method executed by the target proxy server is implemented.

[0132] Fig.11 A schematic block diagram of an example electronic device 110 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Fig.11 As shown, it includes a memory 111, a processor 112, and a computer program stored in the memory 111 and executable on the processor 112. When the processor 112 executes the program, the aforementioned network element log extraction method executed by any server in the server cluster is implemented.

[0134] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0136] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0139] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0140] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0141] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0142] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A network element log extraction method, characterized in that: Applicable to a target proxy server, the method comprises: Receiving a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries a storage address of a log file segment; Obtaining a network element identifier of the target network element; generating an orderly increasing message identifier for the log reporting message according to the network element identifier and the reporting order of the log reporting message in the target network element; Based on the load balancing strategy, a target server is selected from the server cluster, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network; Based on the message identifier, the storage address of the log file fragment and the IP address of the target server, a log processing request is generated and the log processing progress and the shared message identifier are broadcast in the multicast network; Receiving a first log upload completion message of the target network element sent by the NEA; Receiving a clear request message broadcasted by the target server, wherein the clear request message carries a message identifier of a log processing request to be cleared; When the message identifier of the log processing request to be cleared is the maximum message identifier corresponding to the target network element, it is determined that the log file upload of the target network element is completed; A second log upload completion message is generated according to the IP address of the target server, and the second log upload completion message is broadcasted in the multicast network.

2. The method according to claim 1, characterized in that After the orderly increasing message identifier is generated for the log reporting message, the method further includes: The message identifiers of the currently generated log reporting messages are cached in sequence in the identifier cache of the target network element.

3. The method according to claim 1, characterized in that After receiving the request clear message broadcasted by the target server, the method further includes: The message identifier of the log processing request to be cleared is cleared from the identifier cache of the target network element.

4. A network element log extraction method, characterized in that: Applicable to any server in a server cluster, the method comprises: Receive a request to clear message, wherein the request to clear message includes a first storage address of a first log file corresponding to the log processing request to be cleared; wherein the log processing request is generated by the log proxy server according to the storage address of the log file fragment, the message identifier and the IP address of the target server, and the message identifier is sequentially increased according to the reporting order of the log file fragment in the network element; Sequentially obtain a target log processing request corresponding to an unprocessed minimum message identifier from the message cache, wherein the target log processing request includes a second storage address of a corresponding second log file and a first IP address of a corresponding target server; When the second IP address of any server is consistent with the first IP address, download the first log file and the second log file according to the first storage address and the second storage address respectively, merge the two log files to obtain a third log file, and store it to the target storage address; Generate a request clear message for the target log processing request, and broadcast the log processing progress and the shared message identifier in a multicast network, wherein the multicast network includes the server cluster and the target proxy server; The method further comprises: Receiving a second log upload completion message broadcasted by the target proxy server in the multicast network, wherein the second log upload completion message carries a first IP address of a last server for executing a log processing request corresponding to a maximum message identifier; When the first IP address of the last server is consistent with the second IP address, a download message is generated according to the target storage address corresponding to the last server and sent to the client.

5. The method according to claim 4, characterized in that The method further comprises: Receiving a log processing request broadcasted by a log proxy server, and each time the log processing request is received, caching the received log processing request in order into the message cache according to the message identifier of the log processing request; The log processing request includes a message identifier of a log reporting message from a network element and a storage address of a corresponding log file segment, and an IP address of the target server of the log processing request.

6. The method according to claim 4, characterized in that After receiving the request to clear the message, the method further includes: The log processing request to be cleared cached in the message cache is cleared.

7. The method according to claim 4, characterized in that After receiving the second log upload completion message broadcasted by the target proxy server in the multicast network, the method further includes: Clear the log processing request in the message buffer.

8. A network element log extraction device, characterized in that: Applicable target proxy servers include: A receiving module, configured to receive a log report message of a target network element sent by a network element adapter NEA, wherein the log report message carries a storage address of a log file segment; A message identification module, used to obtain the network element identification of the target network element; generate an orderly increasing message identification for the log reporting message according to the network element identification and the reporting order of the log reporting message in the target network element; A selection module is used to select a target server from a server cluster based on a load balancing strategy, wherein a log service is deployed in the target server, and the server cluster and the target proxy server form a multicast network; A broadcast module, for generating a log processing request based on the message identifier, the storage address of the log file fragment and the IP address of the target server, and broadcasting the log processing progress and the shared message identifier in the multicast network; The broadcast module is further used for: After generating the log processing request and broadcasting it in the multicast network, receiving a first log upload completion message of the target network element sent by the NEA; Receiving a clear request message broadcasted by the target server, wherein the clear request message carries a message identifier of a log processing request to be cleared; When the message identifier of the log processing request to be cleared is the maximum message identifier corresponding to the target network element, it is determined that the log file upload of the target network element is completed; A second log upload completion message is generated according to the IP address of the target server, and the second log upload completion message is broadcasted in the multicast network.

9. The device according to claim 8, characterized in that The message identification module is also used for: After sequentially increasing message identifiers are generated for the log report messages, the message identifiers of the currently generated log report messages are cached in sequence in the identifier cache of the target network element.

10. The device according to claim 8, characterized in that The broadcast module is further used for: After receiving the clear request message broadcasted by the target server, the message identifier of the log processing request to be cleared is cleared from the identifier cache of the target network element.

11. A network element log extraction device, characterized in that: Applicable to any server in the server cluster, including: A first receiving module is used to receive a request to clear message, wherein the request to clear message includes a first storage address of a first log file corresponding to the log processing request to be cleared; wherein the log processing request is generated by the log proxy server according to the storage address of the log file fragment, the message identifier and the IP address of the target server, and the message identifier is sequentially increased according to the reporting order of the log file fragment in the network element; An acquisition module, used for sequentially acquiring a target log processing request corresponding to an unprocessed minimum message identifier from a message cache, wherein the target log processing request includes a second storage address of a corresponding second log file and a first IP address of a corresponding target server; a merging module, configured to, when the second IP address of any server is consistent with the first IP address, download the first log file and the second log file according to the first storage address and the second storage address respectively, merge the two log files to obtain a third log file, and store the third log file at a target storage address; Generate a request clear message for the target log processing request, and broadcast the log processing progress and the shared message identifier in a multicast network, wherein the multicast network includes the server cluster and the target proxy server; The network element log extraction device also includes: A third receiving module is used to receive a second log upload completion message broadcasted by the target proxy server in the multicast network, wherein the second log upload completion message carries a first IP address of a last server for executing a log processing request corresponding to a maximum message identifier; A generation module is used to generate a download message and send it to the client according to the target storage address corresponding to the last server when the first IP address of the last server is consistent with the second IP address.

12. The device according to claim 11, characterized in that The network element log extraction device also includes: A second receiving module is used to receive the log processing request broadcasted by the log proxy server, and each time after receiving the log processing request, cache the received log processing request in order into the message cache according to the message identifier of the log processing request; The log processing request includes a message identifier of a log reporting message from a network element and a storage address of a corresponding log file segment, and an IP address of the target server of the log processing request.

13. The device according to claim 11, characterized in that The first receiving module is further used for: After receiving the request to clear message, the log processing request to be cleared cached in the message cache is cleared.

14. The device according to claim 11, characterized in that The third receiving module is further used for: After receiving the second log upload completion message broadcasted by the target proxy server in the multicast network, the log processing request in the message buffer is cleared.

15. An electronic device, characterized in that: Including memory and processor; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method according to any one of claims 1 to 3.

16. An electronic device, characterized in that: Including memory and processor; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the method as claimed in any one of claims 4 to 7.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 4 to 7 is implemented.

Images