A log file collection information generation method, device, equipment and medium
By automatically comparing and updating the collection strategy table, the problems of low efficiency and low accuracy in existing log collection technologies are solved, and efficient and accurate log file collection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-21
- Publication Date
- 2026-03-31
AI Technical Summary
Existing log collection technologies require manual maintenance, have low collection efficiency and low accuracy, and are prone to missing log information.
The system retrieves a list of log files periodically reported by multiple data acquisition terminals from the server, determines whether the currently maintained data acquisition strategy table is empty, compares the log file list with the strategies recorded in the data acquisition strategy table, identifies the missed acquisition strategies of the target data acquisition terminal and the unacquired strategies of all data acquisition terminals, and automatically updates the data acquisition strategy table.
It improves the efficiency and accuracy of log collection and reduces the occurrence of missed log information.
Smart Images

Figure CN117093558B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of log collection, and in particular to a method, apparatus, device, and medium for generating log file collection information. Background Technology
[0002] With the widespread adoption of smartphones and mobile internet devices, and the arrival of the 3G (3rd Generation) and 4G (4th Generation) eras, mobile internet has become an indispensable part of our daily work and life. People have become accustomed to using apps to access the internet, obtain information, and interact. To keep up with this trend, more and more companies and individuals are investing their efforts in application development. Correspondingly, log collection technology serving application development has also emerged.
[0003] Log collection technology primarily helps application vendors collect data on the application's operating environment and process. This data is used to analyze user attributes and behaviors, understand the application's promotion and operation status, and provide data support for vendors to optimize application product design and formulate application operation and promotion strategies.
[0004] Log collection technology typically uses statistical platforms to obtain logs generated by applications during terminal operation, and then aggregates the data in the logs to understand the application's status. During the log collection process, application vendors, in addition to obtaining user-related statistical data such as the number of new users, active users, and cumulative users, also need to track user operations and system behavior, that is, to perform statistical analysis on events within the application. Existing log collection technologies require manual configuration of log file collection parameters, including the log file path, filename, application system, and collection strategy. When an enterprise operates a large number of hosts, existing log collection methods require manual maintenance, which is time-consuming and labor-intensive, and lacks accuracy, frequently resulting in missed data collection. Summary of the Invention
[0005] This invention provides a method, apparatus, device, and medium for generating log file collection information, which can solve the problems of existing log collection technologies requiring manual maintenance, having low collection efficiency, low collection accuracy, and being prone to missing log information.
[0006] In a first aspect, embodiments of the present invention provide a method for generating log file collection information, executed by a server, the method comprising:
[0007] Obtain a list of log files periodically reported by multiple data collection terminals;
[0008] Determine if the current collection strategy table maintained by the server is empty. If not, compare each log file list with each collection strategy recorded in the collection strategy table.
[0009] If the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, then the first collection strategy is determined to be the missed collection strategy of the target collection terminal.
[0010] If none of the acquisition terminals hit the second acquisition strategy, then the second acquisition strategy will be determined as the unaccepted strategy for all acquisition terminals.
[0011] Secondly, embodiments of the present invention provide a log file collection information generation apparatus, executed by a server, the apparatus comprising:
[0012] The log acquisition module is used to obtain a list of log files periodically reported by multiple collection terminals;
[0013] The strategy table judgment module is used to determine whether the collection strategy table currently maintained by the server is empty. If not, it compares each log file list with each collection strategy recorded in the collection strategy table.
[0014] The first collection strategy judgment module is used to determine the first collection strategy as the missed collection strategy of the target collection terminal if the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy.
[0015] The second acquisition strategy judgment module is used to determine the second acquisition strategy as the unacquired strategy of all acquisition terminals if none of the acquisition terminals have matched the second acquisition strategy.
[0016] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:
[0017] At least one processor; and
[0018] A memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the log file collection information generation method according to any embodiment of the present invention.
[0020] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement the log file collection information generation method described in any embodiment of the present invention.
[0021] The technical solution of this invention first obtains a list of log files periodically reported by multiple collection terminals, then determines whether the collection strategy table currently maintained by the server is empty. If not, each log file list is compared with each collection strategy recorded in the collection strategy table. If the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, then the first collection strategy is determined as the missed collection strategy of the target collection terminal. If all collection terminals do not match the second collection strategy, then the second collection strategy is determined as the uncollected strategy of all collection terminals. This solves the problems of existing log collection technologies requiring manual maintenance, low collection efficiency, low collection accuracy, and easy log omissions, and achieves the effect of improving the generation efficiency and accuracy of log collection information.
[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a flowchart of a method for generating log file collection information according to Embodiment 1 of the present invention;
[0025] Figure 2 This is a flowchart of a method for generating log file collection information according to Embodiment 2 of the present invention;
[0026] Figure 3 This is a schematic diagram illustrating the specific data content of a conventional file obtained by the method provided in Embodiment 2 of the present invention;
[0027] Figure 4 This is a schematic diagram illustrating the specific contents of a list of log files obtained according to the method provided in Embodiment 2 of the present invention;
[0028] Figure 5 This is a schematic diagram of a blank acquisition strategy table obtained according to the method provided in Embodiment 2 of the present invention;
[0029] Figure 6 This is a schematic diagram of a data acquisition strategy table obtained according to the method provided in Embodiment 2 of the present invention;
[0030] Figure 7 This is a schematic diagram of a log file collection information generation device according to Embodiment 3 of the present invention;
[0031] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the log file collection information generation method of this invention. Detailed Implementation
[0032] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0034] Example 1
[0035] Figure 1 This is a flowchart illustrating a method for generating log file collection information according to Embodiment 1 of the present invention. The method is executed by a server and is applicable to situations involving the generation of log file collection information. This method can be executed by a log file collection information generation device, which can be implemented in hardware and / or software. This device can be configured in a terminal or server with log file collection information generation functionality. Figure 1 As shown, the method includes:
[0036] S110. Obtain a list of log files periodically reported by multiple data collection terminals.
[0037] In this embodiment, the method can periodically report log file lists according to a preset collection period. Furthermore, the preset collection period can be manually set and adjusted according to actual log collection needs. In this embodiment, the preset period can be set to one week. It should be noted that in this embodiment, the log file lists reported by the collection terminal are automatically filtered and reported based on system-set filtering conditions, without manual filtering. Therefore, the types and quantities of log file lists reported by the collection terminal may be insufficient, resulting in low collection efficiency and accuracy when automatically reporting log file lists, easily leading to missed log entries. Further confirmation and inspection of all reported log lists are required. The method described in this embodiment can solve the problem of existing automatic log file list reporting by the collection terminal requiring manual maintenance, resulting in low collection efficiency, low accuracy, and easy omissions of log information.
[0038] Before obtaining the list of log files periodically reported by multiple collection terminals, the collection terminals can further be used for:
[0039] A preset log collection program is executed at a preset cycle to collect host process information matching the collection terminal; wherein, the host process information includes: process name, process ID, and host information; a list of running files matching the current collection terminal is obtained according to the process ID; each file in the list of running files is filtered sequentially according to preset log filtering rules to obtain at least one regular file that meets the preset filtering rules; each regular file and the process name, process ID, file name, and host information matching each regular file are associated to generate a list of log files matching each regular file.
[0040] Furthermore, based on the above steps, the filtering rules are as follows: Each process file is filtered based on a preset first filtering condition to obtain each pre-process file containing the first target field; each pre-process file is filtered based on a preset second filtering condition to obtain each candidate log file containing the second target field; the current candidate log file is sequentially obtained from each candidate log file and parsed to obtain the filename, file size, storage address, and number of lines of the current candidate log file; the filename, storage address, and the last digit of the current candidate log file are obtained. Tail data is used as feature data for the current candidate log file, and the hash value of the current candidate log file is calculated based on the feature data. The comparison file set is queried to see if there is a target comparison file whose file name matches that of the current candidate log file. If so, the difference between the file size and the number of lines in the current candidate log file and the target comparison file is used to determine whether the growth rate of the current candidate log file is positive. If the growth rate is positive, and the hash value of the current candidate log file is different from the hash value of the target comparison file, the current candidate log file is determined to be a regular file that meets the preset filtering rules.
[0041] In this embodiment, the host information specifically includes: the host name matching the regular file, the host IP address of the host, and the file path and file name matching the regular file obtained based on the specific data of the regular file.
[0042] S120. Determine whether the data collection strategy table currently maintained by the server is empty.
[0043] If not, compare each log file list with each collection strategy recorded in the collection strategy table.
[0044] In this embodiment, after comparing each log file list with each collection strategy recorded in the collection strategy table, the method further includes: if each log file list contains a target log file list that does not match any collection strategy, then a new collection strategy is generated based on the target log file list, and the new collection strategy is added to the collection strategy table.
[0045] For example, if the server-acquired list of all log files contains a list of log files that does not match any of the current collection strategies, then the current log file list is extracted and added to the collection strategy table as a new collection strategy. Through the method described in this embodiment, the collection strategy table can be periodically updated. This supplementation and improvement further ensures the comprehensiveness of the collection strategy table, thereby guaranteeing the collection accuracy of the log collection work.
[0046] Specifically, the log file list includes: each log file collected by the collection terminal based on preset collection rules, and file description information matching each log file, which also includes the process name and file name.
[0047] As those skilled in the art will know, the file description information matching the log file can be directly extracted from the specific data of the log file based on the log file format, which is existing technology and will not be described here.
[0048] Optionally, after determining whether the data collection strategy table currently maintained by the server is empty, the following steps are also included:
[0049] If so, the current collection strategy table will be updated based on the list of each log file.
[0050] For example, if the collection strategy table is empty when the server runs this method for the first time, that is, it does not contain any collection strategy, then all the log file lists are treated as uncollected strategies, and an uncollected strategy list is generated and output to the display device. Relevant personnel determine whether each log file list can be used as a collection strategy by checking the boxes, and add the log file lists checked by the personnel as the current collection rules to the collection strategy table, thus completing the update of the blank collection strategy table.
[0051] S130. If the log file list of the target acquisition terminal does not match the first acquisition strategy, and at least one other acquisition terminal matches the first acquisition strategy, then the first acquisition strategy is determined as the missed acquisition strategy of the target acquisition terminal.
[0052] The omission strategy is a collection strategy that includes log files that were not collected by the target collection terminal, but were found by other collection terminals.
[0053] For example, if there is a first collection strategy for collection terminal 1 in the collection strategy table, that is, there is at least one collection strategy A in the collection strategy table, and there is no matching log file list in the log file list uploaded by collection terminal 1, then collection strategy A is marked as the first collection strategy. If at this time, there is a log file list in the log files uploaded by collection terminal 2 that matches the first collection strategy, that is, collection terminal 2 has collected a log file list that matches the first collection strategy, then the first collection strategy is determined as the missed collection strategy of collection terminal A.
[0054] Specifically, in this embodiment, after determining the first collection strategy as the missed collection strategy of the target collection terminal if the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, the method further includes: aggregating and displaying the missed collection strategies of each collection terminal to generate a missed collection strategy list, pushing it to the display device for display, and generating a missed collection prompt; wherein, the missed collection prompt includes a missed collection color prompt and a missed collection sound prompt.
[0055] S140. If none of the acquisition terminals hit the second acquisition strategy, then the second acquisition strategy will be determined as the unacquired strategy of all acquisition terminals.
[0056] The omission strategy is a collection strategy that is not collected by any of the collection terminals and exists in the collection strategy table.
[0057] Optionally, after determining the second acquisition strategy as the unacquired strategy for all acquisition terminals if none of them are matched, the method further includes: aggregating and displaying all unacquired strategies from all acquisition terminals to generate a list of unacquired strategies and pushing it to the display device for display.
[0058] For example, if there is a collection strategy B in the collection strategy table, and there is no matching log file list in the list of all log files uploaded by all collection terminals, then the collection strategy B is marked as the second collection strategy. Furthermore, the second collection strategy is marked as the uncollected strategy of all collection terminals.
[0059] In this embodiment, it should be noted that S130 and S140 do not have a specific order in actual operation. If the log file list of the target collection terminal does not match the first collection strategy, S130 is executed; if none of the collection terminals match the second collection strategy, S140 is executed directly, without needing to perform S130 first and then S140.
[0060] The technical solution of this invention first obtains a list of log files periodically reported by multiple collection terminals, then determines whether the collection strategy table currently maintained by the server is empty. If not, each log file list is compared with each collection strategy recorded in the collection strategy table. If the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, then the first collection strategy is determined as the missed collection strategy of the target collection terminal. If all collection terminals do not match the second collection strategy, then the second collection strategy is determined as the uncollected strategy of all collection terminals. This achieves the effect of improving the generation efficiency of log collection information and the accuracy of log file collection information.
[0061] Example 2
[0062] Figure 2 This is a flowchart of a method for generating log file collection information according to Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiment. Specifically, this embodiment refines the method of comparing each log file list with each collection strategy recorded in the collection strategy table.
[0063] like Figure 2 As shown, the method includes:
[0064] S210. Obtain a list of log files periodically reported by multiple data collection terminals.
[0065] S220. If the current collection strategy table maintained by the server is not empty, obtain the target collection end from all collection ends and obtain the current collection strategy in the collection strategy table in turn.
[0066] S230. Determine whether there is a log file list in the target acquisition terminal that has the same process name and file name as the current acquisition strategy.
[0067] If so, then execute S240;
[0068] If not, then execute S250.
[0069] The process name and file name refer to the file name and process name information contained in the current log file list.
[0070] S240. Determine that the current acquisition strategy is the target acquisition strategy within the current period, and execute S250.
[0071] The collected strategy is one that exists in the current collection strategy table and has a process name and file name that are the same as the current collection strategy in the current collection terminal.
[0072] S250. Continue to check if there are any log files in the log file list of other collection terminals that have the same process name and file name as the current collection strategy.
[0073] If any other acquisition terminal's log file list contains a log file with the same process name and file name as the current acquisition strategy, execute S260.
[0074] If none of the log files in the log file lists of all other acquisition terminals have the same process name and file name as the current acquisition strategy, execute S270.
[0075] S260. Determine the current acquisition strategy as the first acquisition strategy for the target acquisition end, and execute S280.
[0076] S270. Determine the current acquisition strategy as the second acquisition strategy for all acquisition terminals, and execute S290.
[0077] S280. If the log file list of the target acquisition terminal does not match the first acquisition strategy, and at least one other acquisition terminal matches the first acquisition strategy, then the first acquisition strategy is determined as the missed acquisition strategy of the target acquisition terminal.
[0078] S290. If none of the acquisition terminals hit the second acquisition strategy, then the second acquisition strategy is determined as the unacquired strategy of all acquisition terminals.
[0079] The technical solution of this invention first obtains a list of log files periodically reported by multiple collection terminals, then determines whether the collection strategy table currently maintained by the server is empty. If not, the target collection terminal is obtained from all collection terminals, and the current collection strategy is obtained sequentially from the collection strategy table. Then, it is determined whether there is a log file list in the target collection terminal's log file list that has the same process name and file name as the current collection strategy. If so, it is determined that the current collection strategy is the collected strategy of the target collection terminal in the current period. If not, it continues to check whether there are log files in the log file lists of other collection terminals that have the same process name and file name as the current collection strategy. If any other acquisition terminal's log file list contains a log file with the same process name and file name as the current acquisition strategy, then the current acquisition strategy is determined as the first acquisition strategy for the target acquisition terminal. If none of the log file lists of all other acquisition terminals contain a log file with the same process name and file name as the current acquisition strategy, then the current acquisition strategy is determined as the second acquisition strategy for all acquisition terminals. If the target acquisition terminal's log file list does not match the first acquisition strategy, and at least one other acquisition terminal matches the first acquisition strategy, then the first acquisition strategy is determined as the missed acquisition strategy for the target acquisition terminal. If none of the acquisition terminals match the second acquisition strategy, then the second acquisition strategy is determined as the unacquired strategy for all acquisition terminals. This achieves the effect of improving the efficiency of log acquisition information generation and the accuracy of log file acquisition information.
[0080] Specific implementation scenarios
[0081] To more clearly illustrate the technical solutions provided by the embodiments of the present invention, this embodiment will briefly introduce a specific implementation scenario obtained according to this embodiment.
[0082] Step 1: Execute a preset log collection program based on a preset period to collect host process information that matches the collection terminal; wherein, the host process information includes: process name, process ID and host information.
[0083] Step 2: Obtain a list of running files that match the current acquisition terminal based on the process ID.
[0084] Step 3: Based on preset log filtering rules, filter each file in the running file list sequentially to obtain at least one regular file that meets the preset filtering rules. In this implementation scenario, the specific data content of a specific regular file is provided as follows: Figure 3 As shown.
[0085] Step 4: Associate each regular file with the process name, process ID, file name and host information that match each regular file to generate a list of log files that match each regular file.
[0086] Step 5: The server obtains a list of log files periodically reported by multiple data collection terminals.
[0087] Step 6: Each server responds to the server's request by uploading a list of log files.
[0088] In this implementation scenario, the specific content of the list of log files uploaded by the data collection terminal A and received by the server can be as follows: Figure 4 As shown.
[0089] Step 7: Determine if the current collection strategy table maintained by the server is empty. If not, compare each log file list with each collection strategy recorded in the collection strategy table.
[0090] Specifically, the target acquisition terminal is obtained from all acquisition terminals, and the current acquisition strategy is sequentially retrieved from the acquisition strategy table. It is then determined whether the log file list of the target acquisition terminal contains a log file with the same process name and file name as the current acquisition strategy. If so, the current acquisition strategy is determined to be the acquisition strategy for the target acquisition terminal in the current period. If not, the system continues to check whether the log file lists of other acquisition terminals contain log files with the same process name and file name as the current acquisition strategy. If any other acquisition terminal contains a log file with the same process name and file name as the current acquisition strategy, the current acquisition strategy is determined as the first acquisition strategy for the target acquisition terminal. If none of the log file lists of all other acquisition terminals contain a log file with the same process name and file name as the current acquisition strategy, the current acquisition strategy is determined as the second acquisition strategy for all acquisition terminals.
[0091] Optionally, when the collection strategy table is empty, the collection strategy table shown is as follows: Figure 5 As shown; after updating the current collection strategy table based on the list of each log file, the updated collection strategy table is as follows. Figure 6 As shown.
[0092] Step 8: If the log file list of the target acquisition terminal does not match the first acquisition strategy, and at least one other acquisition terminal matches the first acquisition strategy, then the first acquisition strategy is determined as the missed acquisition strategy of the target acquisition terminal.
[0093] If, in the collection strategy table of collection terminal 1, there exists a first collection strategy for collection terminal 1, that is, there is at least one collection strategy A in the collection strategy table, and there is no matching log file list in the log file list uploaded by collection terminal 1, then collection strategy A is marked as the first collection strategy. If, at this time, there is a log file list in the log files uploaded by collection terminal 2 that matches the first collection strategy, that is, collection terminal 2 has collected a log file list that matches the first collection strategy, then the first collection strategy is determined as the missed collection strategy of collection terminal A.
[0094] Step 9: If none of the acquisition terminals hit the second acquisition strategy, then the second acquisition strategy is determined as the unacquired strategy of all acquisition terminals.
[0095] If a collection strategy B exists in the collection strategy table, and there is no matching log file list in the list of all log files uploaded by all collection terminals, then the collection strategy B is marked as the second collection strategy. Furthermore, the second collection strategy is marked as the uncollected strategy of all collection terminals.
[0096] Example 3
[0097] Figure 7 This is a schematic diagram of a log file collection information generation device provided in Embodiment 3 of the present invention.
[0098] like Figure 7 As shown, the device includes:
[0099] The log acquisition module 710 is used to acquire a list of log files periodically reported by multiple acquisition terminals;
[0100] The strategy table judgment module 720 is used to determine whether the collection strategy table currently maintained by the server is empty. If not, it compares each log file list with each collection strategy recorded in the collection strategy table.
[0101] The first collection strategy judgment module 730 is used to determine the first collection strategy as the missed collection strategy of the target collection terminal if the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy.
[0102] The second acquisition strategy judgment module 740 is used to determine the second acquisition strategy as the unacquired strategy of all acquisition terminals if none of the acquisition terminals have matched the second acquisition strategy.
[0103] The technical solution of this invention first obtains a list of log files periodically reported by multiple collection terminals, then determines whether the collection strategy table currently maintained by the server is empty. If not, each log file list is compared with each collection strategy recorded in the collection strategy table. If the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, then the first collection strategy is determined as the missed collection strategy of the target collection terminal. If all collection terminals do not match the second collection strategy, then the second collection strategy is determined as the uncollected strategy of all collection terminals. This achieves the effect of improving the generation efficiency of log collection information and the accuracy of log file collection information.
[0104] Based on the above embodiments, the log file collection information generation device further includes:
[0105] The strategy table initialization module is used to generate a new collection strategy based on the target log file list if any collection strategy is not matched in the log file list, and add the new collection strategy to the collection strategy table.
[0106] Based on the above embodiments, the strategy table judgment module 720 is further configured to:
[0107] After determining whether the current collection strategy table maintained by the server is empty, if so, the current collection strategy table is updated based on the list of each log file.
[0108] Based on the above embodiments, the strategy table judgment module 720 includes:
[0109] The current acquisition strategy acquisition unit is used to acquire the target acquisition end from all acquisition ends and sequentially acquire the current acquisition strategy in the acquisition strategy table;
[0110] The log judgment unit is used to determine whether there is a log file list in the target collection terminal that has the same process name and file name as the current collection strategy;
[0111] The collected strategy determination unit is used to determine, if so, the current collection strategy as the collected strategy of the target collection end in the current period;
[0112] The continued detection unit is used to check, if not, whether there is a log file in the log file list of other collection terminals that has the same process name and file name as the current collection strategy;
[0113] The first collection strategy determination unit is used to determine the current collection strategy as the first collection strategy for the target collection terminal if there is a log file in the log file list of any other collection terminal with the same process name and file name as the current collection strategy.
[0114] The second collection strategy confirmation unit is used to determine the current collection strategy as the second collection strategy for all collection terminals if there are no log files with the same process name and file name as the current collection strategy in the log file list of all other collection terminals.
[0115] Based on the above embodiments, the first acquisition strategy judgment module 730 is further configured to:
[0116] If the log file list of the target acquisition terminal does not match the first acquisition strategy, and at least one other acquisition terminal matches the first acquisition strategy, then after determining the first acquisition strategy as the missed acquisition strategy of the target acquisition terminal, the missed acquisition strategies of each acquisition terminal are aggregated and displayed to generate a missed acquisition strategy list, which is pushed to the display device for display, and a missed acquisition prompt is generated; wherein, the missed acquisition prompt includes a missed acquisition color prompt and a missed acquisition sound prompt.
[0117] Based on the above embodiments, the second acquisition strategy judgment module 740 is further configured to:
[0118] If none of the acquisition terminals match the second acquisition strategy, then the second acquisition strategy is determined as the unacquired strategy for all acquisition terminals. The unacquired strategies from all acquisition terminals are then aggregated and displayed to generate a list of unacquired strategies, which is then pushed to the display device for display.
[0119] The log file collection information generation device provided in the embodiments of the present invention can execute the log file collection information generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0120] Example 4
[0121] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.
[0122] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0123] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0124] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method for generating log file collection information.
[0125] Accordingly, the method includes:
[0126] Obtain a list of log files periodically reported by multiple data collection terminals;
[0127] Determine if the current collection strategy table maintained by the server is empty. If not, compare each log file list with each collection strategy recorded in the collection strategy table.
[0128] If the log file list of the target collection terminal does not match the first collection strategy, and at least one other collection terminal matches the first collection strategy, then the first collection strategy is determined to be the missed collection strategy of the target collection terminal.
[0129] If none of the acquisition terminals hit the second acquisition strategy, then the second acquisition strategy will be determined as the unaccepted strategy for all acquisition terminals.
[0130] In some embodiments, the method for generating log file collection information can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the log file collection information generation method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the log file collection information generation method by any other suitable means (e.g., by means of firmware).
[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0132] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0133] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0136] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0137] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
Claims
1. A method of generating log file collection information, executed by a server, characterized by, The method comprises the following steps: acquiring a plurality of log file lists periodically reported by collection terminals; determining whether a collection strategy table currently maintained by a server is empty, and if not, comparing each log file list with each collection strategy recorded in the collection strategy table; if the log file list of a target collection terminal does not hit a first collection strategy, and at least one other collection terminal hits the first collection strategy, determining the first collection strategy as a missed collection strategy of the target collection terminal; if none of the collection terminals hits a second collection strategy, determining the second collection strategy as an uncollected strategy of all the collection terminals; after determining whether the collection strategy table currently maintained by the server is empty, the method further comprises the following steps: if yes, updating the current collection strategy table based on each log file list; the step of comparing each log file list with each collection strategy recorded in the collection strategy table comprises the following steps: acquiring a target collection terminal from all the collection terminals, and sequentially acquiring a current collection strategy from the collection strategy table; determining whether there is a log file list in the log file list of the target collection terminal which has the same process name and file name as the current collection strategy; if yes, determining that the current collection strategy is a collected strategy of the target collection terminal in the current period; if no, continuing to detect whether there is a log file list in the log file list of other collection terminals which has the same process name and file name as the current collection strategy; wherein the process name and the file name are file name and process name information contained in the current log file list; if there is a log file list in the log file list of any other collection terminal which has the same process name and file name as the current collection strategy, determining the current collection strategy as the first collection strategy for the target collection terminal; if there is no log file list in the log file list of all the other collection terminals which has the same process name and file name as the current collection strategy, determining the current collection strategy as the second collection strategy for all the collection terminals.
2. The method of claim 1, wherein, after comparing each log file list with each collection strategy recorded in the collection strategy table, the method further comprises the following steps: if there is a target log file list which does not hit any collection strategy in each log file list, generating a new collection strategy according to the target log file list, and adding the new collection strategy to the collection strategy table.
3. The method of claim 1, wherein, The log file list specifically comprises each log file collected by a collection terminal based on a preset collection rule, and file description information matched with each log file, wherein the file description information further comprises a process name and a file name.
4. The method of claim 1, wherein, after determining that the log file list of the target collection terminal does not hit the first collection strategy, and at least one other collection terminal hits the first collection strategy, the method further comprises the following steps: aggregating and displaying the missed collection strategies of each collection terminal to generate a missed collection strategy list, pushing the missed collection strategy list to a display device for display, and generating a missed collection prompt; wherein the missed collection prompt comprises a missed collection color prompt and a missed collection sound prompt.
5. The method of claim 1, wherein, After determining the second collection strategy as the non-collection strategy of all collection ends if none of the collection ends hits the second collection strategy, the method further comprises: Aggregating and displaying each non-collection strategy of all collection ends to generate a non-collection strategy list and pushing the non-collection strategy list to a display device for display.
6. A log file collection information generation apparatus executed by a server, characterized by comprising: The method comprises: a log acquisition module configured to acquire log file lists periodically reported by a plurality of collection ends; a strategy table judgment module configured to judge whether a collection strategy table currently maintained by a server is empty, and if not, compare each log file list with each collection strategy recorded in the collection strategy table; a first collection strategy judgment module configured to determine the first collection strategy as a missed collection strategy of a target collection end if the log file list of the target collection end does not hit the first collection strategy and at least one other collection end hits the first collection strategy; a second collection strategy judgment module configured to determine the second collection strategy as a non-collection strategy of all collection ends if none of the collection ends hits the second collection strategy; the strategy table judgment module is further configured to: after judging whether the collection strategy table currently maintained by the server is empty, if yes, update the current collection strategy table based on each log file list; the strategy table judgment module comprises: a current collection strategy acquisition unit configured to acquire a target collection end among all collection ends and acquire current collection strategies in the collection strategy table in sequence; a log judgment unit configured to judge whether there is a log file list with the same process name and file name as the current collection strategy in a log file list of the target collection end; wherein the process name and the file name are file name and process name information contained in the current log file list; a collected strategy determination unit configured to determine that the current collection strategy is a collected strategy of the target collection end in a current period if yes; a continuous detection unit configured to continue to detect whether there is a log file with the same process name and file name as the current collection strategy in a log file list of another collection end if no; a first collection strategy determination unit configured to determine the current collection strategy as the first collection strategy for the target collection end if there is a log file with the same process name and file name as the current collection strategy in a log file list of any other collection end; a second collection strategy confirmation unit configured to determine the current collection strategy as the second collection strategy for all collection ends if there is no log file with the same process name and file name as the current collection strategy in a log file list of all other collection ends.
7. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for generating log file collection information according to any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the method for generating log file collection information according to any one of claims 1-5 when executed.
Citation Information
Patent Citations
Method and system for losslessly acquiring logs in real time in distributed environment
CN106991177A
Log processing method and device, storage medium and electronic equipment
CN116541362A