Log processing method and system, electronic device, and storage medium

Abstract

The present disclosure provides a log processing method and system, electronic equipment and storage medium, relates to the technical field of computers, and particularly relates to the fields of log management, big data, etc.The specific implementation scheme is as follows: a log processing request from a client is received, the log processing request comprising a log type; a to-be-processed log conforming to the log type is acquired; and the to-be-processed log is subjected to step-by-step funnel aggregation statistics to obtain a processing result.The log collection and processing work can be automatically completed by the present disclosure, and the log processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer technology, in particular to the fields of log management, big data, etc. BACKGROUND

[0002] At present, various application programs can be installed in mobile phones, tablets and the like. The log analysis process in the application program relies on repeated communication and confirmation of developers and operation, product and the like personnel, the process is complex, and the time cost and labor cost are relatively high. SUMMARY

[0003] The present disclosure provides a log processing method, system, electronic device, storage medium and computer program product, which can reduce the time cost and labor cost without the participation of developers and operation, product and the like personnel and repeated communication and confirmation to complete the collection and processing of logs.

[0004] According to a first aspect of the present disclosure, a log processing method is provided, comprising: receiving a log processing request from a client, the log processing request comprising a log type; obtaining a to-be-processed log conforming to the log type; and performing step-by-step funnel aggregation statistics on the to-be-processed log to obtain a processing result.

[0005] According to a second aspect of the present disclosure, a log processing method is provided, comprising: sending a log processing request to a server, the log processing request comprising a log type; and receiving a funnel analysis processing result obtained by performing step-by-step funnel aggregation statistics on a to-be-processed log by the server, the to-be-processed log being a log conforming to the log type.

[0006] According to a third aspect of the present disclosure, a log processing device is provided, comprising: a receiving request module configured to receive a log processing request from a client, the log processing request comprising a log type; a log obtaining module configured to obtain a to-be-processed log conforming to the log type; and a log processing module configured to perform step-by-step funnel aggregation statistics on the to-be-processed log to obtain a processing result.

[0007] According to a fourth aspect of the present disclosure, a log processing device is provided, comprising: a sending request module configured to send a log processing request to a server, the log processing request comprising a log type; and a receiving result module configured to receive a processing result obtained by performing step-by-step funnel aggregation statistics on a to-be-processed log by the server, the to-be-processed log being a log conforming to the log type.

[0008] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the foregoing method.

[0009] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to cause a computer to perform the foregoing method.

[0010] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the foregoing method.

[0011] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

[0012] The scheme provided by the embodiment can automatically complete the collection and processing of logs, and improve the log processing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings are used to better understand the scheme, and do not constitute a limitation on the present disclosure. Among them:

[0014] Figure 1 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 1 ;

[0015] Figure 2 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 2 ;

[0016] Figure 3 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 3 ;

[0017] Figure 4 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 4 ;

[0018] Figure 5 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 5 ;

[0019] Figure 6 is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 6 ;

[0020] Figure 7is a schematic diagram of a log processing method according to an embodiment of the present disclosure Figure 7 ;

[0021] Figure 8 is a schematic diagram of a log management and analysis method according to the related art

[0022] Figure 9 is a schematic diagram of a log processing method according to an embodiment of the present disclosure

[0023] Figure 10 is a schematic diagram of a log processing system according to an embodiment of the present disclosure

[0024] Figure 11a is a schematic diagram of a data warehouse model according to an embodiment of the present disclosure

[0025] Figure 11b is an example diagram of a processing result according to an embodiment of the present disclosure

[0026] Figure 12 is a schematic diagram of a log processing apparatus according to an embodiment of the present disclosure Figure 1 ;

[0027] Figure 13 is a schematic diagram of a log processing apparatus according to an embodiment of the present disclosure Figure 2 ;

[0028] Figure 14 is a block diagram of an electronic device for implementing a log processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the present disclosure by a person of ordinary skill in the art, and should not be construed as limiting the present disclosure to particular embodiments. Accordingly, those of ordinary skill in the art will recognize that there are various modifications and changes that can be made thereto without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0030] The present disclosure provides a log processing method, which can be applied to a server. As shown in Figure 1 , the log processing method comprises:

[0031] S101, receiving a log processing request from a client, the log processing request comprising a log type;

[0032] S102, obtaining a to-be-processed log conforming to the log type;

[0033] S103, performing step-by-step funnel aggregation statistics on the to-be-processed log to obtain a processing result.

[0034] In the embodiments of the present disclosure, the log processing method can be implemented through the interaction of the client and the server. The client can include an application installed in an electronic device, such as a mobile phone, a tablet, and the like. For example, a user can check the feature log of interest on the client, so as to obtain the log type of the feature log. Then, a log processing request is generated according to the log type of the feature log, and the request is sent to the server, so that the processing result is automatically generated.

[0035] In the embodiments of the present disclosure, the collection and processing of logs can be automatically completed, and the log processing efficiency is improved. For example, the development personnel, operation personnel, and product personnel do not need to participate repeatedly, and the whole process can be self-helped, that is, the collection and processing of logs can be completed by the user, and the log processing efficiency is improved. Further, the time cost and the labor cost are reduced, and the product personnel and the operation personnel can make operation decisions better, and the development personnel can release the repeated development labor, and the overall research and development efficiency is improved.

[0036] The present disclosure also provides a log processing method, which can be applied to a server. The method of the embodiment can include one or more features of the method of the above-mentioned embodiments. As shown in the following, in some embodiments, the log processing method comprises: Figure 2 The log processing method comprises:

[0037] S201, performing a funnel aggregation statistics on the to-be-processed logs according to log attributes and a hierarchical use order of the log attributes. The hierarchical use order of the log attributes can include log attributes required to be used at each level. The log attributes used at each level can include one or more. By performing the funnel aggregation statistics on the log attributes and the hierarchical use order of the log attributes, a funnel analysis result including multiple levels can be obtained, which is beneficial to be applied to various analysis scenarios.

[0038] In some embodiments, the log attributes can include at least one of a device type, an application type, an action type, a log type, and a log generation time.

[0039] For example, the action type can be data content in a JSON (JavaScript Object Notation, JS object notation) format. The action type can record specific behaviors of a user, such as clicking, viewing, commenting, liking, forwarding, purchasing, refunding, and the like.

[0040] For example, the log type can be a specific type meaning of various log rules configured, such as an article type, a short video type, a live broadcast type, and the like.

[0041] For example, the device type can include the device and operating system setting type of the client used by the user, such as mobile terminal-Android version, mobile terminal-IOS version, computer terminal-PC version, and the like.

[0042] For example, the application type can include the specific application program used on the client. This attribute can conveniently support subsequent application extension.

[0043] In the embodiments of the present disclosure, the user can perform configuration, selection, combination, and the like on various log attributes. For example, the number of users watching a live program (based on the log type), the number of users using application program A through a mobile terminal (based on the device type and the application type).

[0044] In the embodiments of the present disclosure, multi-level funnel aggregation statistics can be performed through various log attributes, which is beneficial to various analysis scenarios.

[0045] In addition, the user can configure the execution order of the log attributes at each level. For example, the device type is used at the first level, the application type is used at the second level, and the action type is used at the third level. For another example, the application type is used at the first level, the device type is used at the second level, and the log type is used at the third level. For another example, the application type is used at the first level, the log type is used at the second level, the log generation time is used at the third level, and the action type is used at the fourth level. The number of specific levels and the log attributes used at each level can be flexibly selected according to actual needs, and are not limited herein.

[0046] In some embodiments, according to the log attributes and the execution order of the log attributes at each level, the funnel aggregation statistics of the to-be-processed log at each level further includes: according to the Nth log attribute and the Nth object, the number of (N+1)th objects is counted from the Nth object in the to-be-processed log, and N is a positive integer.

[0047] In the embodiments of the present disclosure, the number of (N+1)th objects is determined from the Nth object, which can gradually reduce the number of objects and obtain a funnel analysis result that is more suitable for various scene requirements.

[0048] In some embodiments, the number of first-level objects is counted from the to-be-processed log according to at least one log attribute.

[0049] In the embodiments of the present disclosure, the number of first-level objects can be counted from the to-be-processed log according to one or more log attributes; or the to-be-processed log can be directly used as the first-level object. Specifically, according to the execution order of the user at each level, the number of (N+1)th objects is counted from the Nth object according to the Nth log attribute. In the embodiments of the present disclosure, the number of first-level objects can be flexibly determined, and then the number of objects is gradually reduced, thereby serving as the basis of the funnel analysis result.

[0050] For example, the number of first-level objects is counted from the to-be-processed logs according to the device type and the application type. Then, the number of second-level objects is counted from the first-level objects according to a certain action type. By analogy, the number of N+1-level objects can be counted from N-level objects according to an Nth action type.

[0051] For another example, all the to-be-processed logs are taken as first-level objects. The number of second-level objects is counted from the to-be-processed logs according to the device type. The number of third-level objects is counted from the second-level objects according to the application type. The number of fourth-level objects is counted from the third-level objects according to the log type.

[0052] In some embodiments, as shown in Figure 3 The method further includes:

[0053] S301, receiving a login request from the client, the login request including an application program identifier and a first user identifier requesting login;

[0054] S302, obtaining, according to the first user identifier and the application program identifier, operation logs corresponding to the first user identifier on the application program;

[0055] S303, storing the operation logs, the first user identifier and the application program identifier in association.

[0056] In the embodiments of the present disclosure, after the client sends a login request and successfully logs in, the click log generated in a certain application program is uploaded to the background in real time for storage. In the embodiments, the click log can be stored in a storage module such as ElasticSearch, and each log corresponds to the user's terminal ID. The user identifier requesting login can include the ID account, registered mobile phone number and other identifier information of the user in the application program. ElasticSearch is a distributed storage engine that can support full-text search and quickly respond to query requests.

[0057] In the embodiments of the present disclosure, the log can be stored in association with the user identifier, application program identifier and the like, so as to facilitate subsequent searching and use of the stored log. Further, real-time uploading of the log can be realized, so as to achieve no omission of data and more convenient management of log data.

[0058] In some embodiments, as shown in Figure 4 The method further includes:

[0059] S401, verifying the operation logs according to a log preset format in a click log specification;

[0060] S402, marking the operation log which does not conform to the preset format of the log.

[0061] For example, the preset format of the log in the dot-logging specification set by the user can be a JSON parsing format. The non-standard dot-logging is identified according to the JSON parsing format, and the non-standard logs are marked with a highlight label. This process can be a process of illegal log rendering. When the user selects a log type, the user can select from the non-highlighted logs. By marking the operation log that does not conform to the preset format, the legal and illegal logs can be distinguished, and the subsequent selection of the appropriate log type for viewing is facilitated.

[0062] In the embodiments of the present disclosure, the logs can also be sorted in descending order according to the time stamp of the log generation, so that the latest operation log of the user can be displayed first on the page, facilitating dynamic verification in priority.

[0063] In some embodiments, according to the log business mapping relationship, the operation log is parsed to obtain the attribute of the operation log. For example, the log business mapping relationship configured by the user can include a log business meaning mapping rule, which can be recorded in a log dictionary in a mysql (a relational database management system), mainly storing log type identifiers and corresponding business meanings. The dictionary meaning is called to render the log, which can parse the log and mark the business meaning of each log. Through the log business mapping relationship, the log attribute can be obtained, and further dynamic verification can be performed.

[0064] In some embodiments, the method further includes: calling a first log corresponding to the first user identifier; and sending the first log to the client. In this way, log query can be automatically implemented based on the user identifier, the query efficiency is high, and the query result is accurate.

[0065] For example, after the client logs in, the page can automatically jump to the log display page, and the original log content generated by the user operation is queried in real time according to the end ID reported by the user. This query process automatically filters the end ID of the user as a condition, so as to query only the behavior log generated by the current user operation. By using the above scheme to automatically display various information of the log of the current ID to the user, unified management of various log business meanings, log categories and the like by the user can be realized, facilitating subsequent maintenance and expansion.

[0066] In some embodiments, the method further includes:

[0067] The second log corresponding to the log type is called, and the log type is a log type selected from the logs in the first log which are not marked;

[0068] acquire a second user identifier corresponding to the second log;

[0069] retrieve a to-be-processed log corresponding to the second user identifier.

[0070] For example, if the type of the target log is browsing commodity A and purchasing commodity A, all logs of the same type can be retrieved according to the above type to form a user group. The operation logs of the users in the user group are analyzed as to-be-analyzed logs, that is, all user logs of the type circled by the user are funnel analyzed according to the type of the log circled by the user. More extensive log data can be obtained through the logs that the user is interested in, so that more abundant analysis results can be obtained. In addition, the behavior habits of more extensive users can be further understood, and more reasonable operation decisions can be made.

[0071] The present disclosure provides a log processing method, which can be applied to a client. As shown in the method, the log processing method comprises: Figure 5

[0072] S501, sending a log processing request to a server, the log processing request comprising a log type;

[0073] S502, receiving a funnel analysis processing result obtained by performing step-by-step funnel aggregation statistics on to-be-processed logs of the server, the to-be-processed logs being logs of the log type.

[0074] In the embodiments of the present disclosure, the collection and processing of logs can be automatically completed, and the log processing efficiency is improved. Further, the time and labor costs are reduced.

[0075] In some embodiments, as shown in the method, the method further comprises: Figure 6

[0076] S601, sending a login request by using a code scanning function of an application program, the login request comprising an application program identifier and a first user identifier requesting to log in to the application program;

[0077] S602, receiving and displaying a first log corresponding to the first user identifier.

[0078] For example, a user (a product or an operation, or a user member in other roles) accesses a system, and uploads a unique identifier (an end ID) on a mobile phone to the system by scanning a two-dimensional code by using the mobile phone. The user can conveniently log in to the system and view information in each link of log management.

[0079] In some embodiments, as shown in the method, the method further comprises: Figure 7

[0080] ​​​S701, select a log type in the log that is not marked in the first log;

[0081] S702, generate the log processing request according to the log type.

[0082] In the embodiments of the present disclosure, automation and user self-service of the entire process can be realized, the user can select the type of processing log according to his own needs, and obtain personalized processing results. Further, it can help professionals to make better operation decisions, and can help developers to release repetitive development manpower, and improve the overall R&D efficiency.

[0083] As Figure 8 shown, log verification often needs R&D to design log dotting according to the specific needs of product and operation description, then manual testing and network packet capture are performed, and then log extraction is performed on the captured data. The extracted log is extracted according to a specific specification, and finally the result details are generated by script tasks and other technical means, and communicated with product, operation and other personnel for confirmation. The whole process not only has huge communication cost, but also requires a lot of R&D manpower. In the long run, a lot of unnecessary communication cost and manpower cost is wasted. The logs captured by various tools at present mainly include raw logs, without extracting corresponding meta information. For example, the person in charge of the log, the timing and business meaning of the log reporting, the access volume brought by the unit independent visitor, and whether the forwarding recommendation strategy and other information. Therefore, the operation personnel need to check the log management system to know the details. In addition, whether the same event is repeatedly reported, it is currently impossible to directly analyze from the debug log.

[0084] For product and operation personnel, it is often necessary to know the specific details of the user behavior funnel result to further understand the user behavior habits and make more reasonable decisions.

[0085] The embodiments of the present disclosure can realize the automation of the entire log processing process, support the product and operation personnel to self-service obtain various result details, and improve the overall R&D efficiency.

[0086] When troubleshooting online problems or designing dotting, professionals need to use the APP they are responsible for to check the behavior log details reported. In the related art, it is necessary to install the Debug package of the App, set up the mobile phone and Mac agent, use the packet capture tool to check the log details, or check on the test environment of the log management system (also need to install the Debug package), which is high in cost and low in efficiency.

[0087] As Figure 9As shown in the embodiments of the present disclosure, a log processing system is provided. The system can realize a whole set of platform services through big data technology, and automatically transform the reporting, checking, funnel analysis, log meta information hosting and other links of the end log, thereby saving labor costs and improving research and development efficiency. The log processing flow of the system can be based on real-time checking and processing of the end log based on dynamic code scanning, and can specifically include the following steps:

[0088] 1. User members in roles such as product (PM, project management) or operation, or other roles, access the platform through a terminal device such as a mobile phone, and upload the unique identification code (such as an end ID) on the mobile phone to the system by scanning a two-dimensional code on the mobile phone.

[0089] 2. Automatic log flow. The log generated by the user operating the mobile phone application is uploaded to the system background storage in real time.

[0090] The front end of the log processing system queries the log content generated by the user operation in real time through the end ID reported by the user scanning the code, and supports parsing and displaying the business meaning of the log. For logs that do not meet the specifications, a highlight prompt is given.

[0091] 3. One-key behavior analysis. When professional personnel such as operation and PM view the results through the application, they can select the feature logs of interest, and then click the analysis button such as the 'one-key funnel analysis' button to perform behavior analysis, thereby automatically generating an analysis report such as a funnel analysis report. The feature logs here can support funnel analysis of all end user logs of the same category. For example, for the purchase behavior category log of the user browsing the goods, the following funnel analysis can be performed on the user group: which users only browse the goods, which users browse and then purchase, and which users purchase and then initiate a refund.

[0092] Referring to Figure 10 The log processing flow can be described in detail in combination with the system architecture.

[0093] First, the user end can configure the log specifications through the page entry (such as the entry link) of the platform front-end interaction layer. The configuration can involve log business mapping relationship, dotting log specification setting, etc. The log meaning mapping rule will be recorded in the log dictionary in the mysql, and can store the log type identifier and the corresponding business meaning. The dotting log specification can store the agreed JSON structure, which is used to check whether the actual generated log meets the expectation. The log specification setting can use the function of the rule generator of the service response layer. The rule generator can also support query condition splicing, timestamp sorting, illegal log rendering, native log query, dictionary meaning rendering, log specification checking, etc.

[0094] For example, the log query page jumps, establishes a query session according to the terminal ID, splices the query conditions in the rule generator, and displays the log query results in real time.

[0095] Secondly, the user accesses the system code scanning entrance, scans the two-dimensional code with the mobile phone, and opens the test mode through the platform button. The user's operation behavior, such as operating the mobile phone application, such as Baidu APP, generates the dotting log (for example, the behavior log), which can be uploaded to the storage engine, such as ElasticSearch, in the background data storage layer in real time. Each log can correspond to the terminal ID of the user. ElasticSearch is a distributed storage engine that can well support full-text search and quickly respond to query requests.

[0096] In addition, after the user scans the code, the platform page will automatically jump to the log display page (i.e. the log query page jump), and the logs are displayed in real time. At this time, the platform will establish a query session according to the terminal ID reported by the user when scanning the code, and query the native log content generated by the user's operation in real time. This query process can automatically filter the terminal ID of the user as a condition, and can also use the query condition splicing function, so as to query only the behavior log generated by the current user's operation.

[0097] Then, the system will render the native log according to the log specification set by the user and the corresponding dictionary meaning (the function of dictionary meaning rendering), and mark the business meaning of each log. In addition, it can distinguish the non-standard dotting log according to the JSON parsing format (the function of log specification verification), and mark the non-standard log with a highlight label (the function of illegal log rendering). Finally, the logs are sorted in descending order according to the time stamp of the log generation (the function of time stamp sorting), so that the latest operation log of the user can be displayed first, which is convenient for dynamic verification.

[0098] Finally, the user can select the business meaning log of interest according to the displayed log content. After selecting the feature log, the user can click the analysis button, such as the "one-key funnel analysis" button, to analyze the user's operation behavior, and the system can give a detailed funnel analysis report. The service response layer pushes the results, and the front-end interaction layer supports viewing the results.

[0099] For example, as shown in FIG. 6, the user scans the code, and the platform page automatically jumps to the log display page, and the logs are displayed in real time. Figure 11aAs shown, in the construction of the bottom-layer data warehouse model, a user behavior log detail table in a data warehouse such as a Hive data table can be used to record historical inventory behavior detail data of a user on a client. For example, attribute detail data in the detail table can include core attributes such as a log type, a log time, a device type, an application type, and behavior content of the log. Hive is a distributed offline analytical data warehouse that relies on a distributed file system and can flexibly support storage and analysis of a large-scale data set. These data can be periodically and in batches synchronized from a search server such as ElasticSearch that stores log data to the detail table. When a user selects a corresponding type of log and clicks a funnel analysis button such as "one-key funnel analysis", the system starts a computing engine such as a Spark computing analysis task. Spark is an in-memory-based distributed computing engine that has good horizontal expansion capability and can support fast offline computing and analysis of a large-scale data set and can well support interaction with a bottom-layer data warehouse such as Hive.

[0100] An example of the processing logic of the analysis task is as follows: all log detail data conforming to the type of log input by the user is obtained from the user inventory behavior log detail table of Hive according to the type of log. Through, for example, Spark SQL, funnel aggregation statistics are performed at each level in combination with the device type, the application type, and the behavior content of the log.

[0101] An example of the process of funnel aggregation statistics can include:

[0102] (1) the number D1 of users using an application program B through a mobile phone;

[0103] (2) the number D2 of users in the D1 number of users who watched a live broadcast type program;

[0104] (3) the number D3 of users in the D2 number of users who clicked a hanging product (obtained by analyzing the behavior content) in the live broadcast program;

[0105] (4) the number D4 of users in the D3 number of users who finally purchased the hanging product in the live broadcast program;

[0106] (5) the number D5 of users in the finally purchased user group who initiated a refund.

[0107] Finally, the analysis result can be pushed to a front-end page. After receiving the data result, the front end can render a behavior funnel analysis graph as shown in Figure 11b .

[0108] The above is an example of processing logs on a mobile phone. In addition to this, logs on other devices such as computers can also be processed. In this way, users such as product managers and marketers can more intuitively see which application types can more effectively promote the sale of a certain product, and thus carry out more rapid and accurate marketing activities or product strategies.

[0109] In the embodiments of the present disclosure, a user can configure log rules through a platform page, and can check the reported logs in real time by scanning a code, without manual intervention, which can greatly improve the efficiency of PM design and the efficiency of RD verification of correct dotting. It also greatly improves the efficiency of daily troubleshooting of online problems. A user can directly check the interested logs on the page, and then perform funnel analysis by one key, and the system can automatically generate an analysis report, which can significantly improve the R&D efficiency. The platform uniformly manages log meta information, including log business meaning and dotting mapping rules, which can facilitate subsequent use and support subsequent expansion. Scanning a code can access a test environment to perform a series of verification and analysis processes, and the complicated and lengthy setting and interaction process is eliminated, and unnecessary communication costs are saved.

[0110] The embodiments of the present disclosure also provide a log processing device, as shown in Figure 12 The device can be arranged on a server and includes a receiving request module 1201, a log obtaining module 1202, and a log processing module 1203. The receiving request module 1201 is configured to receive a log processing request from a client, and the log processing request includes a log type.

[0111] The log obtaining module 1202 is configured to obtain a to-be-processed log conforming to the log type.

[0112] The log processing module 1203 is configured to perform a funnel aggregation and statistics on the to-be-processed log in a level-by-level manner to obtain a processing result.

[0113] It can be seen that the device of the embodiment can realize self-help of the entire log processing process without repeated communication and confirmation of a developer and an operator and a product, that is, the device can realize self-collection and processing of logs, which can reduce time cost and labor cost, and thus helps product and operation personnel to make operation decisions better.

[0114] The embodiments of the present disclosure also provide a log processing device, and in some embodiments, the log processing module is further configured to perform a funnel aggregation and statistics on the to-be-processed log in a level-by-level manner according to a log attribute and a level-by-level use order of the log attribute.

[0115] In some embodiments, the log processing module is further configured to count the number of objects at the N+1 level from the objects at the N level according to the Nth log attribute and the Nth log attribute of the objects at the N level, N being a positive integer.

[0116] In some embodiments, the log processing module is further configured to count the number of objects at the first level according to at least one log attribute from the logs to be processed.

[0117] In some embodiments, the apparatus further comprises a log storage module configured to receive a login request from the client, the login request comprising an application identifier and a first user identifier requesting login; obtain operation logs corresponding to the first user identifier on the application according to the first user identifier and the application identifier; and store the operation logs, the first user identifier and the application identifier in association.

[0118] In some embodiments, the apparatus further comprises a log verification module configured to verify the operation logs according to a log preset format in a dotting log specification; and mark operation logs that do not conform to the log preset format.

[0119] In some embodiments, the apparatus further comprises a log analysis module configured to analyze the operation logs according to a log business mapping relationship to obtain attributes of the operation logs.

[0120] In some embodiments, the apparatus further comprises a log sending module configured to retrieve first logs corresponding to the first user identifier; and send the first logs to the client.

[0121] In some embodiments, the log obtaining module is further configured to retrieve second logs corresponding to a log type, the log type being selected from the logs that are not marked in the first logs; obtain a second user identifier corresponding to the second logs; and retrieve logs to be processed corresponding to the second user identifier.

[0122] The embodiments of the present disclosure further provide a log processing apparatus, as shown in the following table. Figure 13 The apparatus can be arranged on a client. The apparatus can comprise a sending request module 1301 and a receiving result module 1302. The sending request module 1301 is configured to send a log processing request to a server, the log processing request comprising a log type. The receiving result module 1302 is configured to receive a processing result obtained by the server through step-by-step funnel aggregation statistics on logs to be processed, the logs to be processed being logs conforming to the log type. In the embodiments of the present disclosure, a user can complete log obtaining and processing by himself / herself, thereby reducing time and labor costs.

[0123] In some embodiments, the apparatus further includes a log display module configured to, using the code scanning function of the application, send a login request, the login request including an application identifier and a first user identifier requesting to log in to the application; and receive and display a first log corresponding to the first user identifier. The user can be conveniently logged into the system and view information at each link in the log management.

[0124] In some embodiments, the apparatus further includes a request generation module configured to, in the logs in the first log that are not marked, select a log type; and generate the log processing request according to the log type.

[0125] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0126] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0127] Figure 14 A schematic block diagram of an example electronic device 1400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0128] As shown in Figure 14 The electronic device 1400 includes a computing unit 1401 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1402 or a computer program loaded from a storage unit 1408 into a random access memory (RAM) 1403. Various programs and data required for the operation of the electronic device 1400 can also be stored in the RAM 1403. The computing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0129] A plurality of components in the electronic device 1400 are connected to the I / O interface 1405, including: an input unit 1406, such as a keyboard, a mouse, etc.; an output unit 1407, such as various types of displays, speakers, etc.; a storage unit 1408, such as a magnetic disk, an optical disk, etc.; and a communication unit 1409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1409 allows the electronic device 1400 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0130] The computing unit 1401 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1401 performs various methods and processes described above. For example, in some embodiments, the various methods described above can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 1408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1400 via the ROM 1402 and / or the communication unit 1409. When the computer program is loaded onto the RAM 1403 and executed by the computing unit 1401, one or more steps of the various methods described above can be performed. Alternatively, in other embodiments, the computing unit 1401 can be configured to perform the various methods described above by other any appropriate means, such as by means of firmware.

[0131] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0132] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0135] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] The computer system can include clients and servers. This relationship can be. The servers are typically remote from the clients with the interactions between them occurring over a communication network. The relationship between a client and a server is one of client-server. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0137] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, and the present disclosure is not limited herein.

[0138] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above.

Claims

1. A log processing method, comprising: receiving a log processing request from a client, the log processing request comprising a log type, wherein the log processing request is generated in response to a selection operation of a user on the client on the log type; obtaining logs to be processed conforming to the log type; performing a step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result, wherein the step-by-step funnel aggregation statistics is performed based on log attributes configured by a user and a step-by-step use order of the log attributes, the log attributes comprising a device type, an application type, an action type, a log type and a log generation time.

2. The method of claim 1, wherein, According to the log attributes and the step-by-step use order of the log attributes, performing the step-by-step funnel aggregation statistics on the logs to be processed further comprises: According to an Nth object and an Nth log attribute, obtaining a number of an N+1th object from the Nth object from the logs to be processed, N being a positive integer.

3. The method of claim 1, wherein, The number of the first-level object is obtained from the logs to be processed according to at least one log attribute. 4.The method of claim 1, further comprising: receiving a login request from the client, the login request comprising an application identifier and a first user identifier requesting login; obtaining operation logs corresponding to the first user identifier on the application according to the first user identifier and the application identifier; storing the operation logs, the first user identifier and the application identifier in association. 5.The method of claim 4, further comprising: verifying the operation logs according to a log preset format in a dotting log specification; labeling operation logs not conforming to the log preset format. 6.The method of claim 4, further comprising: parsing the operation logs according to a log business mapping relationship to obtain attributes of the operation logs. 7.The method of any one of claims 4 to 6, further comprising: calling a first log corresponding to the first user identifier; sending the first log to the client.

8. The method of claim 7, wherein, Obtaining logs to be processed conforming to the log type comprises: calling a second log corresponding to the log type, the log type being a log type selected from logs in the first log that are not labeled; obtaining a second user identifier corresponding to the second log; calling logs to be processed corresponding to the second user identifier. 9.A log processing method, comprising: sending a log processing request to a server, the log processing request comprising a log type, wherein the log processing request is generated in response to a selection operation of a user on a client on the log type; receiving a processing result obtained by performing a step-by-step funnel aggregation statistics on logs to be processed by the server, the logs to be processed being logs conforming to the log type, wherein the step-by-step funnel aggregation statistics is performed based on log attributes configured by a user and a step-by-step use order of the log attributes, the log attributes comprising a device type, an application type, an action type, a log type and a log generation time. 10.The method of claim 9, further comprising: The application program is used to scan a code, and a login request is sent, the login request including an application program identifier and a first user identifier requesting to log in to the application program; The first log corresponding to the first user identifier is received and displayed.

11. The method of claim 10, further comprising: selecting a log type from the logs in the first log that are not marked; generating the log processing request according to the log type.

12. A log processing apparatus, comprising: a receiving request module configured to receive a log processing request from a client, the log processing request including a log type, wherein the log processing request is generated in response to a selection operation of a user on the client on the log type; a log obtaining module configured to obtain logs to be processed that conform to the log type; a log processing module configured to perform a funnel aggregation and statistics on the logs to be processed level by level to obtain a processing result, and configured to perform the funnel aggregation and statistics on the logs to be processed level by level according to log attributes and a level-by-level use order of the log attributes, wherein the log attributes include a device type, an application type, an action type, a log type, and a log generation time.

13. The apparatus of claim 12, wherein, The log processing module is further configured to obtain a quantity of an N+1 level object from the N level object from the logs to be processed according to an N level object and an N type of log attribute, N being a positive integer.

14. The apparatus of claim 12, wherein, The log processing module is further configured to obtain the quantity of the first level object from the logs to be processed according to at least one log attribute.

15. The apparatus of claim 12, further comprising a log storage module configured to receive a login request from the client, the login request including an application program identifier and a first user identifier requesting to log in, obtain operation logs corresponding to the first user identifier on the application program according to the first user identifier and the application program identifier, and store the operation logs, the first user identifier, and the application program identifier in association.

16. The apparatus of claim 15, further comprising a log checking module configured to check the operation logs according to a log preset format in a dotting log specification, and mark operation logs that do not conform to the log preset format.

17. The apparatus of claim 15, further comprising a log analysis module configured to analyze the operation logs according to a log service mapping relationship to obtain attributes of the operation logs.

18. The apparatus of any one of claims 15 to 17, further comprising a log sending module configured to retrieve a first log corresponding to the first user identifier, and send the first log to the client.

19. The apparatus of claim 18, wherein, The log obtaining module is further configured to retrieve a second log corresponding to a log type, the log type being selected from logs in the first log that are not marked, retrieve a second user identifier corresponding to the second log, and retrieve logs to be processed corresponding to the second user identifier.

20. A log processing apparatus, comprising: The sending request module is configured to send a log processing request to a server, the log processing request comprising a log type; wherein the log processing request is generated in response to a selection operation of a user on a client on the log type; The receiving result module is configured to receive a processing result obtained by performing a funnel aggregation and statistics on the server on a log to be processed, the log to be processed being a log conforming to the log type; wherein the funnel aggregation and statistics are performed based on log attributes configured by a user and a use order of the log attributes; and the log attributes comprise a device type, an application type, an action type, a log type, and a log generation time.

21. The apparatus of claim 20, further comprising a log display module configured to: send a login request using a code scanning function of an application program, the login request comprising an application program identifier and a first user identifier requesting to log in to the application program; and receive and display a first log corresponding to the first user identifier.

22. The apparatus of claim 21, further comprising a request generation module configured to: select a log type in a log that is not marked in the first log; and generate the log processing request according to the log type.

23. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or 9-11.

24. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-8 or 9-11.

25. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-8 or 9-11.

Images