Business log screening method, device and equipment and storage medium
Patent Information
- Application Number
- CN202211643853.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-20
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2042-12-20
AI Technical Summary
[0003]在传统的实现方式中,由于运维人员在排查问题时,需要根据完整的业务受理链路跟踪日志查看问题所在,为了提供完整的业务受理链路跟踪日志,业务服务平台需要采集所有服务系统的全部日志数据,但在业务高峰中会因业务日志数据爆增导致业务日志数据采集和消费延时,进而在业务处理出现问题时不能及时的提供业务日志全文检索服务,导致运维人员无法及时排查问题
[0034] The memory stores computer instructions that can be executed by at least one processor to enable the at least one processor to perform the methods involved in the first aspect above.
Smart Images

Figure CN116089365B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data log technology, and in particular to a business log filtering method, apparatus, device, and storage medium. Background Technology
[0002] As we all know, business systems generate a large amount of business logs during operation. Furthermore, a single business transaction request often requires processing through dozens of service systems to complete the final transaction. Each service system outputs business logs recording the tracking information and processing results of the currently processed transactions, facilitating troubleshooting in case of anomalies. However, with the continuous expansion of the business scope and the increasing volume of user data, the number of calls between service systems is also growing, leading to a dramatic expansion of log data and a massive increase in the amount of business logs that need to be collected, especially during peak sales periods when the volume of business log data increases exponentially. For example, the peak traffic at midnight on Singles' Day reaches tens of billions of business log data points per minute. Collecting such a large amount of business log data consumes significant machine resources and utilizes numerous network interfaces. The business log processing server system also requires substantial machine resources for real-time indexing and storage to complete data collection and provide full-text search services.
[0003] In traditional implementations, maintenance personnel need to traceive complete business processing logs to pinpoint issues. To provide this complete log, the business service platform must collect all log data from all service systems. However, during peak business periods, the surge in log data can cause delays in log collection and consumption. Consequently, when problems arise in business processing, full-text log search services cannot be provided promptly, hindering timely troubleshooting by maintenance personnel. This results in reduced reliability of business system services, negatively impacted user experience, and lowered the trustworthiness of the full-text log search service system. Summary of the Invention
[0004] Based on this, this application provides a business log filtering method, apparatus, device, and storage medium. By filtering the output business logs before completing the subsequent collection, the amount of business logs collected can be reduced, the reliability and stability of business system services can be improved, and user satisfaction can be enhanced.
[0005] Firstly, a business log filtering method is provided, which includes:
[0006] Monitor business link tracing logs in real time and generate business log index files based on the business link tracing logs;
[0007] Send the business link tracing log to the business log service platform and obtain the business acceptance result obtained by the business log service platform based on the business link tracing log;
[0008] The business log tracing logs are filtered based on the business log index file and business acceptance results to generate a dedicated file for retrieval and query.
[0009] According to one achievable method in the embodiments of this application, there is at least one business link tracing log, and a business log index file is generated based on the business link tracing log, including:
[0010] Obtain the business link encoding information from the tracing logs of each business link;
[0011] A business log index file is generated based on the business link coding information. The business log index file is a collection of multiple business link coding information and the corresponding business log information for each business link coding information.
[0012] According to one possible implementation method in this application embodiment, the business link tracing log includes at least one business monitoring log; sending the business link tracing log to the business log service platform includes:
[0013] Monitor the monitoring logs of each business in real time and collect specific information from each business monitoring log;
[0014] Send the specific information of each business monitoring log to the business log service platform.
[0015] According to one achievable method in this application embodiment, the specific information in the business monitoring log includes business acceptance response code information and other information. Obtaining the business acceptance result from the business log service platform based on the business link tracing log includes:
[0016] Obtain the first acceptance status from the business log service platform based on the business acceptance response code information;
[0017] Obtain the second acceptance status from the business log service platform based on other information;
[0018] The business acceptance result is obtained based on the first acceptance status and the second acceptance status.
[0019] According to one achievable method in this application embodiment, the service acceptance result includes service acceptance failure; the service link tracing log is filtered based on the service log index file and the service acceptance result to generate a dedicated file for retrieval and query, including:
[0020] When the service acceptance result is service acceptance failure, obtain the service link encoding information from the corresponding service link tracing log;
[0021] The business link tracing logs are filtered in the business log index file based on the business link encoding information to generate a dedicated file for retrieval and query.
[0022] According to one achievable method in an embodiment of this application, the dedicated file includes a filter file. The business link tracing logs are filtered in the business log index file based on business link encoding information to generate a dedicated file for retrieval and querying, including:
[0023] Query the corresponding business log information in the business log index file based on the business link encoding information;
[0024] Based on the business link encoding information and the corresponding business log information, a filter file for retrieval and query is generated.
[0025] According to one achievable method in this application embodiment, the service acceptance result includes successful service acceptance, and the dedicated file includes a storage file; the service link tracing log is filtered based on the service log index file and the service acceptance result to generate a dedicated file for retrieval and query, including:
[0026] When the business processing result is successful, the filtering of the business link tracing log is ignored, and a storage file for retrieval and query is directly generated based on the business link tracing log.
[0027] Secondly, a business log filtering device is provided, the device comprising:
[0028] The first generation module is used to monitor business link tracing logs in real time and generate business log index files based on the business link tracing logs.
[0029] The business processing module is used to send business link tracing logs to the business log service platform and obtain the business acceptance results obtained by the business log service platform based on the business link tracing logs.
[0030] The second generation module is used to filter the business link tracing logs based on the business log index file and the business acceptance results to generate a dedicated file for retrieval and query.
[0031] Thirdly, a computer device is provided, comprising:
[0032] At least one processor; and
[0033] A memory that is communicatively connected to at least one processor; wherein,
[0034] The memory stores computer instructions that can be executed by at least one processor to enable the at least one processor to perform the methods involved in the first aspect above.
[0035] Fourthly, a computer-readable storage medium is provided, having stored thereon computer instructions, characterized in that the computer instructions are used to cause a computer to perform the methods involved in the first aspect above.
[0036] According to the technical content provided in the embodiments of this application, business link tracing logs are monitored in real time, and a business log index file is generated based on the business link tracing logs. The business link tracing logs are then sent to a business log service platform, and the business processing results obtained by the business log service platform based on the business link tracing logs are obtained. The business link tracing logs are then filtered according to the business log index file and the business processing results to generate a dedicated file for retrieval and querying. The above operations, by filtering the output business link tracing logs to obtain a dedicated file for retrieval and querying, allow maintenance personnel to directly perform business queries based on this dedicated file, thereby reducing the amount of subsequent business log collection, improving the reliability and stability of business system services, and enhancing user satisfaction. Attached Figure Description
[0037] Figure 1 This is an application environment diagram of a business log filtering method in one embodiment;
[0038] Figure 2 This is a flowchart illustrating a business log filtering method in one embodiment;
[0039] Figure 3 This is a schematic diagram of a preferred process for a business log filtering method in one embodiment;
[0040] Figure 4 This is a structural block diagram of a business log filtering device in one embodiment;
[0041] Figure 5 This is a schematic structural diagram of a computer device in one embodiment. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0043] This application provides a business log filtering method that can be applied to, for example... Figure 1In the application environment shown, client 102 communicates with server 104 via a network. Client 102 includes an edge computing client and a log collection client, while server 104 includes a log service platform. Specifically, the edge computing client monitors business link tracing logs in real time and generates a business log index file based on the logs; the log collection client sends the business link tracing logs to the log service platform and obtains the business processing results obtained by the log service platform based on the business link tracing logs; the edge computing client filters the business link tracing logs according to the business log index file and the business processing results to generate a filter file for retrieval and query. Client 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices, and server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.
[0044] In one embodiment, such as Figure 2 As shown, a business log filtering method is provided, which can be applied to... Figure 1 Taking the client in [the document] as an example, the steps are as follows:
[0045] Step 201: Monitor the business link tracing log in real time and generate a business log index file based on the business link tracing log.
[0046] The business link tracing log includes status-related logs for each step of the business process, and the log lines in the business link tracing log are separated by newline characters.
[0047] Specifically, the business service system is equipped with a distributed tracing component. When outputting business logs, the business service system can add distributed tracing information to obtain business distributed tracing logs, with one log output corresponding to each business request chain. The edge computing client monitors the distributed tracing logs output by the business service system in real time and generates a business log index file based on these logs. The distributed tracing component can be the open-source SkyWalking component.
[0048] Step 203: Send the business link tracing log to the business log service platform and obtain the business acceptance result obtained by the business log service platform based on the business link tracing log.
[0049] The log collection client collects business link tracing logs and sends them to the log service platform. After receiving the business link tracing logs, the log service platform can parse them to obtain the business acceptance result of the corresponding link, and then send the business acceptance result to the edge computing client. The edge computing client then obtains the business acceptance result.
[0050] Step 205: Filter the business link tracing logs based on the business log index file and business acceptance results to generate a dedicated file for retrieval and query.
[0051] Specifically, since business processing results include both normal and abnormal processing, a dedicated file for retrieval and query can be obtained regardless of whether the processing is normal or abnormal. However, this application mainly focuses on the case of abnormal processing, and generates a dedicated file for retrieval and query by filtering the business link tracking logs of abnormal processing results.
[0052] As can be seen, this embodiment of the application monitors business link tracing logs in real time, generates a business log index file based on the business link tracing logs, sends the business link tracing logs to a business log service platform, and obtains the business acceptance results obtained by the business log service platform based on the business link tracing logs. The business link tracing logs are then filtered according to the business log index file and the business acceptance results to generate a dedicated file for retrieval and querying. The above operations, by filtering the output business link tracing logs to obtain a dedicated file for retrieval and querying, allow maintenance personnel to directly perform business queries based on this dedicated file, thereby reducing the amount of subsequent business log collection, improving the reliability and stability of business system services, and enhancing user satisfaction.
[0053] The following is a detailed description of each step in the above method flow. First, step 201, namely "real-time monitoring of business link tracing logs and generation of business log index files based on business link tracing logs," will be described in detail with reference to the embodiment.
[0054] Obtain the business link encoding information from the tracing logs of each business link; generate a business log index file based on the business link encoding information.
[0055] Among them, at least one business link tracing log is required.
[0056] Here, the business link tracing log contains business link coding information, which runs throughout the entire business acceptance link until the business acceptance link is completed. Furthermore, all business link tracing logs output by the business service system carry this business link coding information.
[0057] The edge computing client monitors the status of the tracing logs for each business link in real time and generates a business log index file based on the business link codes of each business link tracing log. The business log index file can be a collection of multiple business link codes and the corresponding business log information; in other words, the business log index file is a summary file of the business link tracing logs.
[0058] The above operations will aggregate all business link tracing logs into a single file, making it easier for staff to query and filter these logs later.
[0059] The following describes in detail step 203, namely "sending the business link tracing log to the business log service platform and obtaining the business acceptance status obtained by the business log service platform based on the business link tracing log", with reference to the embodiments.
[0060] Monitor the logs of each business in real time and collect the specific information of each business monitoring log; send the specific information of each business monitoring log to the business log service platform; obtain the first acceptance status obtained by the business log service platform based on the business acceptance response code information; obtain the second acceptance status obtained by the business log service platform based on other information; obtain the business acceptance result based on the first acceptance status and the second acceptance status.
[0061] Each business link tracking log includes at least one business monitoring log; the specific information in the business monitoring log includes business acceptance response code information and other information, including but not limited to business log time information, the identification information corresponding to the current business log, and business requirement rule information.
[0062] Here, the business service system receives business service requests, processes them, and outputs two types of logs: a business tracing log and a business monitoring log. The business monitoring log is also included in the business tracing log.
[0063] Specifically, the log collection client has a log collection component installed. The client monitors the status of various business monitoring logs in real time and collects specific information from each log through the component. Simultaneously, it sends this information to the business log service platform, which receives and parses the logs. The log monitoring component can be a Flume component.
[0064] Since the business monitoring logs contain business acceptance response codes, which can be used to determine whether a business application was successfully accepted, the business log service platform can determine the first acceptance status based on these codes. Generally, a six-digit zero response code indicates successful acceptance, while other codes indicate failure. Of course, the business log service platform can also determine the second acceptance status based on other information.
[0065] The log collection client obtains the first acceptance status from the business log service platform based on the business acceptance response code information and the second acceptance status from the business log service platform based on other information; based on the first acceptance status, the second acceptance status, and preset rules, it obtains the business acceptance result. The preset rules are not time-limited and are mainly based on the current business needs of the customer.
[0066] By taking into account both the first and second acceptance states, the business acceptance result is obtained, resulting in a more accurate and better customer-satisfied outcome.
[0067] The above step 205, namely "filtering the business link tracing logs according to the business log index file and the business acceptance result to generate a dedicated file for retrieval and query", will be described in detail with reference to the embodiments.
[0068] In one feasible approach, when the business acceptance result is business acceptance failure, the business link encoding information of the corresponding business link tracing log is obtained; the business link tracing log is filtered in the business log index file based on the business link encoding information to generate a dedicated file for retrieval and query.
[0069] Among them, the business processing result includes business processing failure; the exclusive document includes the filtered document.
[0070] Specifically, when a service application fails, the edge computing client obtains the service link encoding information from the corresponding service link tracing log in real time, and then queries the corresponding service log information in the service log index file based on the service link encoding information. The service log information includes the name and line number of the service log corresponding to the service application failure.
[0071] A filter file for retrieval and querying is generated based on the business link coding information and corresponding business log information. This filter file contains all business link tracing logs of failed business processing, effectively filtering out these logs. This allows staff to directly query and retrieve information based on the filter file to find the cause of the failure.
[0072] The above operations enable the filtering of business process tracking logs when the business processing result is business processing failure, and generate a filter file for retrieval and query, so that staff can directly perform queries and retrievals based on the filter file.
[0073] In another possible approach, when the business acceptance result is successful, a storage file for retrieval and query is generated based on the business link tracing log.
[0074] Among them, dedicated files include storage files.
[0075] Specifically, when the service processing result is successful, the service trace log is directly retrieved, and a storage file for retrieval and querying is generated based on the service trace log. This storage file is mainly for backup purposes, so that retrieval and querying can be performed through this storage file in the event of data loss.
[0076] Through the above operations, when the business acceptance result is successful, a storage file for retrieval and query is generated based on the business link tracking log, so that staff can use the storage file for retrieval and query in case of data loss.
[0077] Based on the implementation methods in the above embodiments, the following will be combined with... Figure 3 A preferred method flow provided in an embodiment of this application will be described by way of example. For instance... Figure 3 As shown, the method may include the following steps:
[0078] Step 301: Monitor the business link tracing log in real time.
[0079] Step 302: Obtain the business link encoding information from the tracing logs of each business link.
[0080] Step 303: Generate a business log index file based on the business link encoding information.
[0081] Step 304: Monitor the monitoring logs of each service in real time and collect the specific information of each service monitoring log. The specific information of the service monitoring log includes service acceptance response code information and other information.
[0082] Step 305: Send the specific information of each business monitoring log to the business log service platform.
[0083] Step 306: Obtain the first acceptance status from the business log service platform based on the business acceptance response code information.
[0084] Step 307: Obtain the second acceptance status from the business log service platform based on other information.
[0085] Step 308: Obtain the business acceptance result based on the first acceptance status and the second acceptance status; if the business acceptance result is business acceptance failure, execute step 309; if the business acceptance result is business acceptance success, execute step 312.
[0086] Step 309: Obtain the business link encoding information of the corresponding business link tracing log.
[0087] Step 310: Query the corresponding business log information in the business log index file based on the business link encoding information.
[0088] Step 311: Generate a filter file for retrieval and query based on the business link encoding information and the corresponding business log information.
[0089] Step 312 generates a storage file for retrieval and querying based on the business link tracing log.
[0090] It should be understood that, although Figures 2-3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this application, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Furthermore, Figures 2-3 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0091] The above method embodiments can be applied to various application scenarios, including but not limited to the following:
[0092] The edge computing client monitors business link tracing logs in real time and generates a business log index file based on these logs. Since the business link tracing logs include business monitoring logs, the business monitoring logs can be sent to the business log service platform, and the platform can retrieve the business processing result based on the monitoring logs. When the processing result is a failure, the client retrieves the business link encoding information from the corresponding business link tracing log and searches for the corresponding business log information in the business log index file based on this encoding information. Finally, a filter file for retrieval and querying is generated based on the business link encoding information and the corresponding business log information.
[0093] Figure 4 This is a schematic diagram of a business log filtering device provided in an embodiment of this application. The device can be installed in... Figure 1The client in the application environment shown is used to execute, for example... Figures 2-3 The method flow is shown below. Figure 4 As shown, the device may include: a first generation module 401, a service processing module 403, and a second generation module 405. The main functions of each component module are as follows:
[0094] The first generation module 401 is used to monitor the business link tracing log in real time and generate a business log index file based on the business link tracing log.
[0095] The business processing module 403 is used to send the business link tracing log to the business log service platform and obtain the business acceptance result obtained by the business log service platform based on the business link tracing log.
[0096] The second generation module 405 is used to filter the business link tracing logs based on the business log index file and the business acceptance results to generate a dedicated file for retrieval and query.
[0097] In one embodiment, where there is at least one business link tracing log, the first generation module 401 is further configured to:
[0098] Obtain the business link encoding information from the tracing logs of each business link;
[0099] A business log index file is generated based on the business link coding information. The business log index file is a collection of multiple business link coding information and the corresponding business log information for each business link coding information.
[0100] In one embodiment, the service tracing log includes at least one service monitoring log, and the service processing module 403 is further used for:
[0101] Monitor the monitoring logs of each business in real time and collect specific information from each business monitoring log;
[0102] Send the specific information of each business monitoring log to the business log service platform.
[0103] In one embodiment, the specific information in the business monitoring log includes business acceptance response code information and other information. The business processing module 403 is also used for:
[0104] Obtain the first acceptance status from the business log service platform based on the business acceptance response code information;
[0105] Obtain the second acceptance status from the business log service platform based on other information;
[0106] The business acceptance result is obtained based on the first acceptance status and the second acceptance status.
[0107] In one embodiment, the service acceptance result includes service acceptance failure, and the second generation module 405 is further configured to:
[0108] When the service acceptance result is service acceptance failure, obtain the service link encoding information from the corresponding service link tracing log;
[0109] The business link tracing logs are filtered in the business log index file based on the business link encoding information to generate a dedicated file for retrieval and query.
[0110] In one embodiment, the second generation module 405 is further configured to:
[0111] Query the corresponding business log information in the business log index file based on the business link encoding information;
[0112] Based on the business link encoding information and the corresponding business log information, a filter file for retrieval and query is generated.
[0113] In one embodiment, the service acceptance result includes successful service acceptance, and the dedicated file includes a stored file; the second generation module 405 is further configured to:
[0114] When the business processing result is successful, the filtering of the business link tracing log is ignored, and a storage file for retrieval and query is directly generated based on the business link tracing log.
[0115] According to the specific embodiments provided in this application, the technical solution provided in this application can have the following advantages:
[0116] 1) The amount of business log data collected was reduced, saving network I / O traffic.
[0117] 2) By filtering the business link tracing logs, a large amount of business log data can be reduced, thereby saving a lot of computing and storage resources in the final business log query and indexing.
[0118] 3) During peak business system operation, the problem of not being able to query business logs in real time due to high peak traffic will no longer exist, thus ensuring the stability of real-time query of business log data during peak periods.
[0119] It is understood that implementing any method or product of this application does not necessarily require achieving all of the above advantages simultaneously.
[0120] The same or similar parts among the above embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.
[0121] It should be noted that the embodiments of this application may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., explicit consent from the user, actual notification to the user, explicit authorization from the user, etc.).
[0122] According to embodiments of this application, this application also provides a computer device and a computer-readable storage medium.
[0123] like Figure 5 The diagram shown is a block diagram of a computer device according to an embodiment of this application. The term "computer device" is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smartphone, a wearable device, etc.
[0124] like Figure 5 As shown, device 500 includes a computing unit 501, a ROM 502, a RAM 503, a bus 504, and an input / output (I / O) interface 505. The computing unit 501, ROM 502, and RAM 503 are interconnected via the bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.
[0125] The computing unit 501 can execute various processes in the method embodiments of this application according to computer instructions stored in the read-only memory (ROM) 502 or computer instructions loaded from the storage unit 508 into the random access memory (RAM) 503. The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The computing unit 501 can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. In some embodiments, the methods provided in the embodiments of this application can be implemented as computer software programs, which are tangibly contained in a computer-readable storage medium, such as the storage unit 508.
[0126] RAM 503 can also store various programs and data required for the operation of device 500. Part or all of the computer program can be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509.
[0127] The input unit 506, output unit 507, storage unit 508, and communication unit 509 in device 500 can be connected to I / O interface 505. The input unit 506 can be, for example, a keyboard, mouse, touchscreen, or microphone; the output unit 507 can be, for example, a display, speaker, or indicator light. Device 500 can exchange information and data with other devices through the communication unit 509.
[0128] It should be noted that the device may also include other components necessary for normal operation. It may also include only the components necessary for implementing the solution of this application, without necessarily including all the components shown in the figures.
[0129] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof.
[0130] The computer instructions used to implement the methods of this application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 501 such that when executed by the computing unit 501, such as a processor, the computer instructions cause the execution of the steps involved in the embodiments of the methods of this application.
[0131] The computer-readable storage medium provided in this application can be a tangible medium that can contain or store computer instructions for performing the steps involved in the method embodiments of this application. The computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.
[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for filtering business logs, characterized in that, The method includes: Real-time monitoring of business link tracing logs; generation of business log index files based on business link tracing logs; business link tracing logs include at least one business monitoring log; business monitoring logs include business acceptance response code information and other information; the other information includes any one or more of the following: business log time information, identification information corresponding to the current business log, and business requirement rule information. Send the business acceptance response code information and the other information to the business log service platform, and obtain the first acceptance status obtained by the business log service platform based on the business acceptance response code information; The second acceptance status obtained by the business log service platform based on the other information is obtained, and the business acceptance result is obtained based on the first acceptance status and the second acceptance status. The business acceptance result includes business acceptance failure and business acceptance success. When the service acceptance result is service acceptance failure, obtain the service link encoding information of the corresponding service link tracing log; The business link tracking logs are filtered in the business log index file based on the business link encoding information to generate a dedicated file for retrieval and query. When the service acceptance result is successful, the filtering of the service link tracing log is ignored, and a storage file for retrieval and query is directly generated based on the service link tracing log.
2. The method according to claim 1, characterized in that, The business link tracing logs are at least one, and the step of generating a business log index file based on the business link tracing logs includes: Obtain the business link encoding information from the trace logs of each business link; A business log index file is generated based on the business link encoding information. The business log index file is a collection of multiple business link encoding information and the business log information corresponding to each business link encoding information.
3. The method according to claim 2, characterized in that, The dedicated file includes a filter file. The step of filtering the business link tracing logs in the business log index file based on the business link encoding information to generate a dedicated file for retrieval and querying includes: Based on the business link encoding information, query the corresponding business log information in the business log index file; A filter file for retrieval and query is generated based on the business link encoding information and the corresponding business log information.
4. A business log filtering device, characterized in that, The device includes: The first generation module is used to monitor the business link tracing log in real time and generate a business log index file based on the business link tracing log. The business link tracing log includes at least one business monitoring log. The business monitoring log includes business acceptance response code information and other information. The other information includes any one or more of the following: business log time information, identification information corresponding to the current business log, and business requirement rule information. The business processing module is used to send the business acceptance response code information and the other information to the business log service platform, and to obtain the first acceptance status obtained by the business log service platform based on the business acceptance response code information; to obtain the second acceptance status obtained by the business log service platform based on the other information, and to obtain the business acceptance result based on the first acceptance status and the second acceptance status, wherein the business acceptance result includes business acceptance failure and business acceptance success. The second generation module is used to obtain the business link encoding information of the corresponding business link tracing log when the business acceptance result is business acceptance failure; filter the business link tracing log in the business log index file according to the business link encoding information to generate a dedicated file for retrieval and query; when the business acceptance result is business acceptance success, ignore the filtering of the business link tracing log and directly generate a storage file for retrieval and query based on the business link tracing log.
5. A computer device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores computer instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-3.
6. A computer-readable storage medium storing computer instructions thereon, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3.
Citation Information
Patent Citations
Log data processing method and system, storage medium and electronic equipment
CN113722276A