Log data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a log data processing method and device, electronic equipment and a storage medium, and the method comprises the steps: generating an event log after an application client triggers a target event; according to the usage demand scene of the event log, a corresponding coding mode is determined, compression coding is carried out on the event log based on the coding mode, log data are generated, and resource overhead during compression coding of the event log is related to the coding mode; and uploading the log data to an application server under the target triggering condition. The corresponding coding mode is determined according to the use demand scene of the event log, then the log data is generated based on the coding mode, dynamic compression of the log data is achieved, then the log data is uploaded to the server side, collection of the event log is achieved, the situation that the load of terminal equipment is increased due to real-time collection of the event log is avoided, and the user experience is improved. And the running stability and fluency of the application program are improved.

Description

Technical Field

[0001] This disclosure relates to the field of Internet technology, and in particular to a log data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, event logs for abnormal events in applications (APPs) are often used to trace the source of abnormal events, thereby optimizing defects in the application, improving the application's performance and stability, and so on.

[0003] In existing technologies, after event logs are generated on the terminal device side for various reasons, they are typically sent to the application's server side in real time to collect event logs. However, the above-mentioned solutions in existing technologies increase the resource overhead of the terminal device and reduce the stability and smoothness of the application. Summary of the Invention

[0004] This disclosure provides a log data processing method, apparatus, electronic device, and storage medium to overcome problems affecting the stability and smoothness of application operation.

[0005] In a first aspect, embodiments of this disclosure provide a log data processing method, including:

[0006] After the target event is triggered on the application client, an event log is generated; based on the usage scenario of the event log, the corresponding encoding method is determined, and the event log is compressed and encoded based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method; under the target trigger condition, the log data is uploaded to the application server.

[0007] Secondly, embodiments of this disclosure provide a log data processing apparatus, comprising:

[0008] The generation module is used to generate event logs after a target event is triggered by the application client;

[0009] The determination module is used to determine the corresponding encoding method according to the usage scenario of the event log, and to compress and encode the event log based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method.

[0010] The upload module is used to upload the log data to the application server when the target is triggered.

[0011] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0012] The memory stores computer-executed instructions;

[0013] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the log data processing method as described in the first aspect and various possible designs of the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the log data processing method described in the first aspect and various possible designs of the first aspect.

[0015] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the log data processing method described in the first aspect and various possible designs of the first aspect.

[0016] The log data processing method, apparatus, electronic device, and storage medium provided in this embodiment generate an event log after a target event is triggered on the application client; determine the corresponding encoding method based on the usage scenario of the event log, and compress and encode the event log based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method; and upload the log data to the application server under the target trigger condition. By determining the corresponding encoding method based on the usage scenario of the event log and then generating log data based on that encoding method, dynamic compression of log data is achieved. The log data is then uploaded to the server, realizing the collection of event logs. This avoids the increased load on terminal devices caused by real-time event log collection, improving the stability and smoothness of application operation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An application scenario diagram of the log data processing method provided in the embodiments of this disclosure;

[0019] Figure 2 Flowchart of the log data processing method provided in the embodiments of this disclosure Figure 1 ;

[0020] Figure 3 for Figure 2 A flowchart illustrating the specific implementation of step S102 in the illustrated embodiment;

[0021] Figure 4 Flowchart of the log data processing method provided in the embodiments of this disclosure Figure 2 ;

[0022] Figure 5 for Figure 4 A flowchart illustrating the specific implementation of step S204 in the illustrated embodiment;

[0023] Figure 6 for Figure 4 A flowchart of a specific implementation of step S205 in the illustrated embodiment;

[0024] Figure 7 for Figure 4 A flowchart of another specific implementation of step S205 in the illustrated embodiment;

[0025] Figure 8 for Figure 4 A flowchart of another specific implementation of step S205 in the illustrated embodiment;

[0026] Figure 9 This is a schematic diagram illustrating a process for uploading event logs, provided in an embodiment of this disclosure.

[0027] Figure 10 A structural block diagram of the log data processing apparatus provided in the embodiments of this disclosure;

[0028] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0029] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0032] The application scenarios of the embodiments of this disclosure are explained below:

[0033] Figure 1 This diagram illustrates an application scenario of the log data processing method provided in this embodiment. The log data processing method can be applied to applications (APPs) with log uploading functionality, and more specifically, to application scenarios involving the collection of event logs after an application experiences an abnormal event. The executing entity in this embodiment can be a terminal device running the aforementioned application with log uploading functionality, or other electronic devices performing similar functions.

[0034] In some embodiments, terminal devices or similar electronic devices can implement the log data processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be program-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be local applications, i.e., programs that need to be installed in the operating system to run, or small programs embedded in any app, i.e., programs that run in a browser environment. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin; the specific implementation can be configured as needed.

[0035] refer to Figure 1 As shown, taking a terminal device as an example, when a specific event occurs during the running of a target application, such as the application becoming unresponsive or exiting abnormally, the operating system or client generates a corresponding event log. This event log records relevant operational information of the terminal device at the time of the event, such as the terminal device's memory, processes, and threads. The terminal device then sends this event log back to the server hosting the application. Subsequently, the application developers use the content of the event log to trace the source of problems and optimize the program.

[0036] In existing technologies, after generating event logs, the terminal device can use two methods: real-time uploading and offline uploading. The generated event logs are sent from the client to the server for storage in real time. This method offers good real-time performance, allowing the server to collect event logs for specific events promptly and make timely adjustments. However, this approach incurs significant network resource overhead on the terminal device. Another method is offline uploading, where the generated event logs are first stored locally on the terminal device and then uploaded at specific intervals, such as once a week. This approach has less impact on the terminal device's network resources, but because it requires storing memory data on disk, it incurs additional storage and computing resource overhead. In this case, when the terminal device has limited available resources, it can lead to application lag and stuttering.

[0037] This disclosure provides a log data processing method to solve the above-mentioned problems.

[0038] refer to Figure 2 , Figure 2 Flowchart of the log data processing method provided in the embodiments of this disclosure Figure 1 The method of this embodiment can be applied in a terminal device. This log data processing method includes:

[0039] Step S101: After the target event is triggered in the application client, an event log is generated.

[0040] Step S102: Determine the corresponding encoding method based on the usage scenario of the event log, and compress and encode the event log based on the encoding method to generate log data. The resource overhead of compressing and encoding the event log is related to the encoding method.

[0041] For example, refer to Figure 1The illustrated application scenario diagram illustrates a scenario where, in this embodiment, a terminal device is used as the execution subject of the method provided. Specifically, the terminal device is, for example, a smartphone, running an application client with log upload functionality. Examples include short video application clients and social application clients. When the target event is triggered during the operation of the application client, the terminal device generates a corresponding event record based on the logic set in the client program. This record records relevant information about the terminal device and the application client at the time the target event occurs. For example, the occurrence time of the target event and the description of the exception thrown. The target event can be any event pre-set by the application client, including but not limited to events such as access exceptions, detection of slow functions, unresponsive external services, and events corresponding to specific user operations, such as launching an application or entering a specific application page. The specific content of the target event is not described here; it can be set as needed. Furthermore, the specific implementation method of the application generating the corresponding log after detecting the target event is prior art known to those skilled in the art and will not be elaborated upon here.

[0042] Subsequently, unlike existing technologies, the terminal device dynamically detects the event log and determines the corresponding encoding method to compress and encode the event log, storing it locally on the terminal device to generate log data. The purpose of compression encoding is to reduce the amount of data after the event log is stored locally on the terminal device, i.e., after log data generation, thereby improving disk storage efficiency. The encoding method characterizes the execution steps for compressing and encoding the event log. In this embodiment, the encoding method is a dynamic encoding method. For example, as the number and intensity of execution steps increase, the compression rate of the generated log data increases. Simultaneously, the resource overhead consumed by executing this encoding method to compress and encode the event log also increases.

[0043] Specifically, the terminal device first obtains the usage scenario corresponding to the target event described in the event log. The usage scenario refers to the purpose of analyzing the event log, such as adjusting the application's security policy, recommendation policy, or optimizing the smoothness of application use. Furthermore, different encoding methods are adopted when the usage scenario is different. For example, when the usage scenario is to adjust the smoothness of application use, since the real-time requirements of the event log are not high, the event log can be compressed and encoded using an encoding method that can achieve lower real-time requirements, thereby reducing resource consumption. On the other hand, when the usage scenario is to adjust the recommendation policy, since the real-time requirements of the event log are higher, the event log can be compressed and encoded using an encoding method that can achieve higher real-time requirements.

[0044] Furthermore, usage scenarios can be represented as data in the form of scenario identifiers, feature vectors, or feature matrices, and these scenarios can be preset values ​​determined based on configuration files. The encoding method can be a script, program, or other directly or indirectly executable file. In one possible implementation, there is a preset mapping relationship between scenario identifiers and filenames corresponding to encoding methods. Based on this mapping relationship, the encoding method corresponding to the usage scenario of the event log is determined, thereby achieving compressed encoding of the event log. In another possible implementation, such as... Figure 3 As shown, the specific implementation steps of step S102 include:

[0045] Step S1021: Obtain the device running status of the terminal device running the application client.

[0046] Step S1022: Determine the corresponding encoding method based on the usage scenarios of the event log and the device operating status.

[0047] For example, in this embodiment, the encoding method is determined by combining the device operating status and usage scenario of the terminal device. The device operating status, for example, includes at least one of the following: CPU load, input / output (I / O) interface load, disk load, and network bandwidth load. This is because the smoothness of the terminal device's operation is closely related to its current operating status. When the terminal device's resource consumption is high, such as high CPU load or high I / O interface data traffic, using an encoding method that achieves a high compression ratio will cause system lag. Conversely, when the terminal device's resource consumption is low, an encoding method that achieves a high compression ratio can be used, thereby improving data storage efficiency.

[0048] Specifically, one possible implementation is to use a specific mapping table to determine the encoding method corresponding to the two dimensions of event log usage scenarios and device operating status; while another possible implementation is to use a pre-trained strategy model to process event log usage scenarios and device operating status to obtain the encoding method.

[0049] Further, exemplarily, prior to step S1022, the method further includes:

[0050] Step S1022-1: Obtain the functional scenario when the application client triggers the target event;

[0051] Step S1022-2: Based on the functional scenarios and the requirements for analyzing event logs within those functional scenarios, obtain the usage requirement scenarios.

[0052] For example, in this embodiment, the technical features of functional scenarios are introduced. A functional scenario represents the specific function performed by the application client, such as playing video, live streaming, taking photos, purchasing goods, browsing web pages, etc. Before determining the encoding method, the functional scenario when the application client triggers the target event is first obtained. This functional scenario is used as another reference information to deduce the usage requirement scenario. Specifically, it is based on the functional scenario and the requirement information for analyzing event logs under the functional scenario. For example, in a live video streaming scenario (functional scenario), the requirement to adjust the live streaming room recommendation strategy (requirement information) is determined as a usage requirement scenario A. As another example, in browsing a text and image page (functional scenario), the requirement to adjust the cache release strategy within the application client (requirement information) is determined as another usage requirement scenario B.

[0053] In this embodiment, by combining the functional scenario when the application client triggers the target event and the requirement information for analyzing the event log under the functional scenario, the resulting usage requirement scenario can more precisely distinguish the scenario characteristics of the usage requirements for the event log, thereby improving the accuracy of the encoding method determined based on the usage requirement scenario and improving the encoding effect of the encoding method.

[0054] Accordingly, after performing the above steps, the specific implementation of step S1022 includes:

[0055] Step S1022-3: Input the scenario information representing the usage demand scenario and the state information representing the device operating status into the pre-trained policy model to obtain the encoding method.

[0056] For example, after obtaining the usage requirement scenario based on the functional scenario, the scenario information representing the usage requirement scenario and the state information representing the device operating state are input as input parameters to the pre-trained policy model. The scenario information and state information can be a scenario identifier representing the usage requirement scenario and a state identifier representing the device operating state, respectively; or they can be a scenario vector or matrix representing the usage requirement scenario and a state vector or matrix representing the device operating state. Then, the above scenario information and state information are input into the policy model, for example, the policy model obtains an encoding method that matches the above scenario information and state information.

[0057] The strategy model is a model trained using sample data. Based on this, and depending on the specific training and construction method of the strategy model, it can also include other input parameters, such as the device's performance level (e.g., high-end, mid-range, low-end), network status (e.g., weak network connection, Wi-Fi connection, 3G connection, 4G connection, 5G connection), and user habits. This improves the prediction accuracy of the strategy model, making its output encoding more closely match the current state of the terminal device and the user's usage habits.

[0058] Step S1023: Based on the encoding method, perform at least one compression encoding step on the event log to generate log data, wherein the number of compression encoding steps and / or the execution strength of each compression encoding step are determined by the encoding method.

[0059] For example, further, after obtaining the encoding method, at least one compression encoding step is used to generate log data. This compression encoding step includes word segmentation encoding, compression, and aggregation. Word segmentation encoding refers to splitting the log text content at a certain granularity, including word-level, character-level, and mixed segmentation; the specific implementation methods are not detailed here. Then, the segmented objects are encoded, for example, encoding "123456" as "1_6", thereby achieving data compression. Compression refers to further compressing the data using special compression algorithms, such as Huffman compression. Aggregation refers to the aggregation of duplicate field values ​​when there is local similarity between different event logs, i.e., the field values ​​of different event logs are repeated. In this case, the duplicate field values ​​can be aggregated to form aggregated events, thereby reducing the log volume. The word segmentation encoding, compression, and aggregation mentioned above are common steps for achieving data compression, and will not be elaborated on here.

[0060] Specifically, depending on the specific implementation of the encoding method, at least one corresponding encoding step can be obtained, and log data can be generated by executing this encoding step. The higher the data compression rate achievable by the encoding method, the greater the number of compression encoding steps, and / or the greater the execution intensity of each compression encoding step. Conversely, the lower the data compression rate achievable by the encoding method, the smaller the number of compression encoding steps, and / or the smaller the execution intensity of each compression encoding step. Through this method, dynamic encoding of event logs based on the encoding method can be achieved. That is, when strong compression of event logs is required, an encoding method with more compression encoding steps and higher execution intensity is used to improve the compression rate of log data; while when slight compression of event logs is required, an encoding method with fewer compression encoding steps and lower execution intensity is used to reduce system resource overhead and improve the stability and smoothness of terminal devices and application clients.

[0061] Step S103: Under the target trigger condition, upload the log data to the application server.

[0062] Furthermore, after generating the log data, when the target trigger condition is met, the compressed log data is uploaded to the application server. The application server then decodes the received log data using the same encoding method (i.e., the encoding method used by the terminal device in the previous steps) to reconstruct the corresponding event log. This allows developers to optimize the application client based on the event log. The target trigger condition can be a fixed trigger period, where the terminal device periodically sends the log data to the application server, or it can be a specific trigger condition corresponding to a target event. For example, depending on the specific content of the target event, the log data can be uploaded in real-time or when the terminal device is idle.

[0063] In this embodiment, after a target event is triggered on the application client, an event log is generated. Based on the usage scenario of the event log, a corresponding encoding method is determined, and the event log is compressed and encoded using this method to generate log data. The resource overhead of compressing and encoding the event log is related to the encoding method. Under the target trigger condition, the log data is uploaded to the application server. By determining the corresponding encoding method based on the usage scenario of the event log and then generating log data based on that encoding method, dynamic compression of the log data is achieved. The log data is then uploaded to the server, realizing the collection of event logs. This avoids the increased load on terminal devices caused by real-time event log collection, improving the stability and smoothness of the application.

[0064] refer to Figure 4 , Figure 4Flowchart of the log data processing method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, step S102 is further refined. The encoding method may include first execution information, second execution information, and third execution information. This log data processing method includes:

[0065] Step S201: After the target event is triggered in the application client, an event log is generated.

[0066] Step S202: Based on the usage scenario of the event log, obtain the target log template, which has template parameters corresponding to the usage scenario.

[0067] Step S203: Based on the template parameters of the target log template, retrieve the corresponding target content text from the event log.

[0068] For example, in this embodiment, the terminal device first obtains log templates corresponding to different event logs from the application server through the application client. These target log templates have template parameters corresponding to the usage scenario, which characterize the content obtained from the event logs. For instance, based on the event log event_1 and its corresponding usage scenario situation_1, a corresponding log template Mod_1 can be determined. This log template Mod_1 is issued by the application server. The log template Mod_1 contains three template parameters: Para_1, Para_2, and Para_3, which are used to obtain three types of information about the target event, such as the occurrence of the event, the event description, and the event level. Then, the content text in the event log event_1 that matches the aforementioned template parameters is extracted; this is the target content text.

[0069] Step S204: Based on the first execution information, segment the target content text to obtain a segmentation set containing at least two segmentation objects.

[0070] For example, the first execution information in the encoding method is then obtained. This first execution information represents the rules for segmenting the content text within the event log. The target content text is then segmented using this first execution information; for example, the text sentence [word_1, word_2, word_3] is segmented into a set of words [word_1], [word_2], and [word_3], i.e., a segmentation set. More specifically, there are various ways to segment the target content text based on the first execution information. For example, the segmentation and encoding methods can be context-dependent, meaning that the segmentation rules represented by the first execution information can differ for different datasets and different usage scenarios to obtain the optimal encoding result.

[0071] For example, such as Figure 5 As shown, the specific implementation of step S204 includes:

[0072] Step S2041: Obtain the corresponding target word segmentation granularity based on the first execution information.

[0073] Step S2042: Segment the target content text based on the target segmentation granularity to obtain a segmentation set containing at least two segmentation objects.

[0074] For example, the target segmentation granularity, when splitting content text into segmented objects, determines the granularity of the segmented objects based on the first execution information. This target granularity is first determined, such as a single Chinese character or a single word. Then, the target content text is split based on this target granularity to obtain the segmented structure. The target granularity is determined based on the first execution information, specifically the usage scenario of the event log. Different usage scenarios for the event log can result in different target granularities, thereby improving segmentation accuracy.

[0075] Step S205: Based on the second execution information, encode at least one word segmentation object in the word segmentation set to generate log data.

[0076] For example, after word segmentation is completed, at least one word segmentation object in the resulting word segmentation set is encoded. Specifically, the second execution information is used to characterize the rules for encoding the target word segmentation object. The word segmentation set may contain a certain number of target word segmentation objects, i.e., word segmentation objects that can be encoded, and a certain number of non-target word segmentation objects, i.e., word segmentation objects that cannot be encoded. Target word segmentation objects can be understood as word segmentation objects that have certain patterns and recur. Encoding the above target word segmentation objects can achieve the effect of data compression.

[0077] One possible implementation is, such as Figure 6 As shown, the specific implementation of step S205 includes:

[0078] Step S205A-1: Based on the second execution information, obtain the word segmentation encoding dictionary corresponding to the target event. The word segmentation encoding dictionary is used to represent the mapping relationship between the target word segmentation object and the corresponding encoding object.

[0079] Step S205A-2: Based on the word segmentation encoding dictionary, map the target word segmentation objects in the word segmentation set to the corresponding encoding objects to generate the encoded word segmentation set.

[0080] Step S205A-3: Generate log data based on the encoded word segmentation set.

[0081] For example, firstly, based on the second execution information, a word segmentation encoding dictionary corresponding to the target event is obtained. This dictionary represents the mapping relationship between the target word segmentation object and its corresponding encoding object. This dictionary can be sent to the terminal device via the application server. After obtaining the dictionary from the second execution information, the terminal device uses the encoding relationship described by the dictionary to map the target word segmentation object in the word segmentation set to its corresponding encoding object. Non-target word segmentation objects in the word segmentation set are left unprocessed, thus generating an encoded word segmentation set. Afterwards, the encoded word segmentation set is compressed and written to a file, among other processing steps, to obtain the log data.

[0082] It's important to note that the word segmentation encoding dictionary is determined based on the second execution information, specifically the usage scenarios represented by the event logs. The word segmentation rules represented by the second execution information are automatically generated or adjusted based on user usage. For example, if the event logs generate a large number of timestamp fields within a certain period, such as 1707295215, 1707295216, and 1707295300, and the terminal device recognizes that "1707295" appears frequently or predicts that it will continue to appear in the future, it will generate a dictionary record and encoding for "1707295" in the word segmentation encoding dictionary. Furthermore, when an abnormal event occurs in the application client, it continuously triggers the generation of abnormal event logs. After the terminal device detects this through the application client program, it will optimize event logs of the same type, allowing the same statement to be optimized into a single key (word segmentation object) in the word segmentation encoding dictionary, thereby reducing data duplication and improving encoding efficiency.

[0083] In another possible implementation, such as Figure 7 As shown, the specific implementation of step S205 includes:

[0084] Step S205B-1: Based on the third execution information, obtain a sub-segmentation set from the segmentation set with the target window length;

[0085] Step S205B-2: Encode the segmented objects in the sub-segmentation set sequentially to generate an encoded sub-segmentation set;

[0086] Step S205B-3: Combine the word segmentation sets of each encoding sub-code to generate log data.

[0087] For example, in another possible implementation, to address the issue of excessively long word segmentation sets, a target window length is determined using third-party execution information. A portion of the word segmentation set, i.e., a sub-segmentation set, is then processed in a sliding window manner, thereby reducing the amount of data to be processed and further lowering resource overhead. Similarly, the target window length is determined based on third-party execution information, i.e., based on the usage scenario and event logs. Therefore, as the usage scenario and other factors change, the target window length will change accordingly. For instance, if the terminal device's operating status indicates high resource overhead, the target window length will be appropriately reduced, thereby decreasing data computation and improving system stability.

[0088] Furthermore, the above can be... Figure 6 and Figure 7 Combining the steps shown, we obtain another implementation of step S205, specifically, as follows: Figure 8 As shown, the specific implementation of step S205 includes:

[0089] Step S2051: Based on the third execution information, obtain a sub-segmentation set from the segmentation set with the target window length.

[0090] Step S2052: Based on the second execution information, obtain the word segmentation encoding dictionary corresponding to the target event. The word segmentation encoding dictionary is used to represent the mapping relationship between the target word segmentation object and the corresponding encoding object.

[0091] Step S2053: Based on the word segmentation encoding dictionary, map the target word segmentation objects in each sub-segmentation set to the corresponding encoding objects in turn, and generate the encoded sub-segmentation set.

[0092] Step S2054: Combine the word segmentation sets of each encoding sub-sub ...

[0093] In this embodiment, the above-mentioned Figure 6 and Figure 7 The steps shown are combined as follows: first, a dynamic target window length is determined based on the third execution information; then, multiple sub-segmentation sets are sequentially obtained using a sliding window approach; simultaneously, the corresponding segmentation encoding dictionary is obtained based on the second execution information; and each sub-segmentation set is processed based on the segmentation encoding dictionary to generate an encoded sub-segmentation set. Finally, the encoded sub-segmentation sets are combined to generate log data. Each step in this implementation has been described in previous embodiments and will not be repeated here.

[0094] Optionally, after step S205, the method further includes:

[0095] Step S206: Obtain compression parameters according to the encoding method. The compression parameters are used to characterize the compression strength and / or compression range of the log data.

[0096] Step S207: Compress the log data according to the compression parameters to generate compressed log data.

[0097] For example, after step S205, compression parameters are first obtained according to the encoding method, based on specific needs. Then, based on the compression parameters, the log data is further compressed to generate compressed log data with a smaller data volume. The compression parameters characterize the compression intensity and / or compression range of the log data. For example, compression parameters include p1 and p2. p1 indicates the compression range, with a value range of [0,1]. The larger p1 is, the larger the range of log data is. For example, p1 = 0.5 indicates that 50% of the log data is compressed; p1 = 1 indicates that all data in the log data is compressed. p2 indicates the compression intensity; similarly, the larger p2 is, the higher the compression intensity and the higher the data compression rate. p2 can also indicate the compression algorithm. For example, p2 = 1 indicates compression algorithm A, and p2 = 2 indicates compression algorithm B, thus achieving different compression intensities for the log data. Afterward, the compressed log data is generated and uploaded to the application server, completing the log upload process.

[0098] Optionally, before generating compressed log data, the method further includes: performing an inverse transformation on the log data according to the encoding method to generate restored log data; correspondingly, the specific implementation of step S207 includes: compressing the restored log data according to compression parameters to generate compressed log data. The inverse transformation is the process of restoring the compressed and encoded log data back to the corresponding event log. Its implementation involves performing the inverse transformation process of the above encoding method, thereby re-reading the data into memory to prepare for subsequent data upload. The specific implementation method will not be elaborated further.

[0099] In this embodiment, by compressing the log data according to the encoding method, the data volume can be further reduced. Since the encoding method is determined based on factors such as usage scenarios, it can match more different usage scenarios, improve its applicability, and enable the terminal device to further release performance and compress log data while ensuring the stable operation of the system and client, thereby reducing the network resource overhead during data upload.

[0100] Figure 9 This is a schematic diagram of an event log uploading process provided in an embodiment of this disclosure. The following is in conjunction with... Figure 9 For a more detailed description of the above embodiments, please refer to [link / reference]. Figure 9As illustrated, exemplarily, firstly, for application clients running on different terminal devices, the original event logs are obtained through the log triggering unit set within the application client. After determining the encoding method, the corresponding log template and word segmentation encoding dictionary are used for word segmentation encoding to generate log data, which is then compressed accordingly. Next, the log data is written to disk through the log receiving unit, i.e., stored locally. Then, after the target triggering condition is met, the log data is retrieved, decompressed, and subjected to an inverse transformation based on the log template and word segmentation encoding dictionary. The event logs are then reread into memory and compressed. Finally, the data is uploaded to the application server, completing the event log upload process.

[0101] Step S208: Under the target trigger condition, upload the compressed log data to the application server.

[0102] In this embodiment, the implementation methods of steps S201 and S208 are the same as those in this disclosure. Figure 2 The implementation methods of steps S101 and S103 in the illustrated embodiment are the same, and will not be described in detail here.

[0103] Corresponding to the log data processing method in the above embodiments, Figure 10 This is a structural block diagram of a log data processing apparatus provided in an embodiment of this disclosure. The method described in the above embodiments can be executed by this log data processing apparatus, which can be implemented by software and / or hardware, and can be integrated into an electronic device with certain data processing capabilities. The electronic device may include, but is not limited to, mobile terminals with big data processing capabilities, as well as fixed terminals with big data processing capabilities such as desktop computers and supercomputers.

[0104] For ease of explanation, only the parts relevant to embodiments of this disclosure are shown. (Refer to...) Figure 10 The log data processing device 3 includes:

[0105] The generation module 31 is used to generate an event log after the target event is triggered by the application client;

[0106] The determination module 32 is used to determine the corresponding encoding method according to the usage requirements of the event log, and to compress and encode the event log based on the encoding method to generate log data. The resource overhead of compressing and encoding the event log is related to the encoding method.

[0107] Upload module 33 is used to upload log data to the application server when the target is triggered.

[0108] According to one or more embodiments of this disclosure, the encoding method includes first execution information, which is used to characterize the rules for segmenting the content text in the event log. When the determining module 32 compresses and encodes the event log based on the encoding method to generate log data, it is specifically used to: obtain a target log template according to the usage scenario of the event log, the target log template having template parameters corresponding to the usage scenario; obtain the corresponding target content text from the event log according to the template parameters of the target log template; segment the target content text based on the first execution information to obtain a segmentation set containing at least two segmentation objects, and encode at least one segmentation object in the segmentation set to generate log data.

[0109] According to one or more embodiments of this disclosure, the encoding method further includes second execution information, which is used to characterize the rules for encoding the target word segmentation object; when the determining module 32 encodes at least one word segmentation object in the word segmentation set to generate log data, it is specifically used to: obtain the word segmentation encoding dictionary corresponding to the target event according to the second execution information, which is used to characterize the mapping relationship between the target word segmentation object and the corresponding encoding object; map the target word segmentation object in the word segmentation set to the corresponding encoding object according to the word segmentation encoding dictionary to generate an encoded word segmentation set; and generate log data according to the encoded word segmentation set.

[0110] According to one or more embodiments of this disclosure, the determining module 32, in performing word segmentation on the target content text based on the first execution information to obtain a word segmentation set containing at least two word segmentation objects, is specifically used to: obtain the corresponding target word segmentation granularity based on the first execution information; and split the target content text based on the target word segmentation granularity to obtain a word segmentation set containing at least two word segmentation objects.

[0111] According to one or more embodiments of this disclosure, the encoding method includes third execution information, which is used to characterize the target window length of the sliding window; when the determining module 32 encodes at least one segmented object in the segmented set to generate log data, it is specifically used to: obtain a sub-segmented set in the segmented set based on the third execution information and the target window length; encode the segmented objects in the sub-segmented set in sequence to generate an encoded sub-segmented set; and combine the encoded sub-segmented sets to generate log data.

[0112] According to one or more embodiments of this disclosure, the determining module 32 is further configured to: obtain compression parameters according to the encoding method, wherein the compression parameters are used to characterize the compression strength and / or compression range of the log data; compress the log data according to the compression parameters to generate compressed log data; and the uploading module 33 is specifically configured to: upload the compressed log data to the application server under the target triggering condition.

[0113] According to one or more embodiments of this disclosure, before generating compressed log data, the determining module 32 is further configured to: perform an inverse transformation on the log data according to the encoding method to generate restored log data; when the determining module 32 compresses the log data according to the compression parameters to generate compressed log data, it is specifically configured to: compress the restored log data according to the compression parameters to generate compressed log data.

[0114] According to one or more embodiments of this disclosure, when determining the event log by compression encoding based on the encoding method to generate log data, the determining module 32 is specifically used to: perform at least one compression encoding step on the event log based on the encoding method to generate log data, wherein the number of compression encoding steps and / or the execution intensity of each compression encoding step is determined by the encoding method; the compression encoding step includes at least one of the following: word segmentation encoding, compression, and aggregation.

[0115] According to one or more embodiments of this disclosure, the determining module 32 is further configured to: obtain the device operating status of the terminal device running the application client; when the determining module 32 determines the corresponding encoding method according to the usage scenario of the event log, it is specifically configured to: determine the corresponding encoding method according to the usage scenario of the event log and the device operating status.

[0116] According to one or more embodiments of this disclosure, the device operating state includes at least one of the following: CPU load, input / output interface load, disk load, and network bandwidth load.

[0117] According to one or more embodiments of this disclosure, the determining module 32 is further configured to: obtain the functional scenario when the application client triggers the target event; obtain the usage requirement scenario based on the functional scenario and the requirement information for analyzing the event log under the functional scenario; when the determining module 32 determines the corresponding encoding method based on the usage requirement scenario of the event log and the device operating status, it is specifically configured to: input the scenario information representing the usage requirement scenario and the status information representing the device operating status into the pre-trained strategy model to obtain the encoding method.

[0118] The generation module 31, the determination module 32, and the upload module 33 are connected sequentially. The log data processing device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0119] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 11 As shown, the electronic device 4 includes:

[0120] Processor 41, and memory 42 communicatively connected to processor 41;

[0121] Memory 42 stores instructions executed by the computer;

[0122] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-9 The log data processing method in the illustrated embodiment.

[0123] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0124] For relevant instructions, please refer to the corresponding text. Figures 2-9 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0125] This disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement this disclosure. Figures 2-9 The log data processing method provided in any of the corresponding embodiments.

[0126] This disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements this disclosure. Figures 2-9 The log data processing method provided in any of the corresponding embodiments.

[0127] To implement the above embodiments, this disclosure also provides an electronic device.

[0128] refer to Figure 12 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0129] like Figure 12As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0130] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 12 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0131] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0132] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0134] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0135] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units or modules do not necessarily limit the specific unit itself.

[0138] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0139] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] In a first aspect, according to one or more embodiments of this disclosure, a log data processing method is provided, comprising:

[0141] After the target event is triggered on the application client, an event log is generated; based on the usage scenario of the event log, the corresponding encoding method is determined, and the event log is compressed and encoded based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method; under the target trigger condition, the log data is uploaded to the application server.

[0142] According to one or more embodiments of this disclosure, the encoding method includes first execution information, which is used to characterize the rules for segmenting the content text in the event log. The step of compressing and encoding the event log based on the encoding method to generate log data includes: obtaining a target log template according to the usage scenario of the event log, the target log template having template parameters corresponding to the usage scenario; obtaining the corresponding target content text from the event log according to the template parameters of the target log template; segmenting the target content text based on the first execution information to obtain a segmentation set containing at least two segmentation objects, and encoding at least one segmentation object in the segmentation set to generate the log data.

[0143] According to one or more embodiments of this disclosure, the encoding method further includes second execution information, which is used to characterize the rules for encoding the target word segmentation object; the step of encoding at least one word segmentation object in the word segmentation set to generate the log data includes: obtaining a word segmentation encoding dictionary corresponding to the target event according to the second execution information, the word segmentation encoding dictionary being used to characterize the mapping relationship between the target word segmentation object and the corresponding encoding object; mapping the target word segmentation object in the word segmentation set to the corresponding encoding object according to the word segmentation encoding dictionary to generate an encoded word segmentation set; and generating log data according to the encoded word segmentation set.

[0144] According to one or more embodiments of this disclosure, the step of segmenting the target content text based on the first execution information to obtain a segmentation set containing at least two segmentation objects includes: obtaining a corresponding target segmentation granularity based on the first execution information; and splitting the target content text based on the target segmentation granularity to obtain a segmentation set containing at least two segmentation objects.

[0145] According to one or more embodiments of this disclosure, the encoding method includes third execution information, which is used to characterize the target window length of the sliding window; the step of encoding at least one segmentation object in the segmentation set to generate the log data includes: obtaining a sub-segmentation set in the segmentation set based on the third execution information and the target window length; sequentially encoding the segmentation objects in the sub-segmentation set to generate an encoded sub-segmentation set; and combining each encoded sub-segmentation set to generate the log data.

[0146] According to one or more embodiments of this disclosure, the method further includes: obtaining compression parameters according to the encoding method, the compression parameters being used to characterize the compression intensity and / or compression range of the log data; compressing the log data according to the compression parameters to generate compressed log data; and uploading the log data to the application server under the target triggering condition, which includes: uploading the compressed log data to the application server under the target triggering condition.

[0147] According to one or more embodiments of this disclosure, before generating compressed log data, the method further includes: performing an inverse transformation on the log data according to the encoding method to generate restored log data; the step of compressing the log data according to the compression parameters to generate compressed log data includes: compressing the restored log data according to the compression parameters to generate compressed log data.

[0148] According to one or more embodiments of this disclosure, the step of compressing and encoding the event log based on the encoding method to generate log data includes: performing at least one compression and encoding step on the event log based on the encoding method to generate the log data, wherein the number of compression and encoding steps and / or the execution intensity of each compression and encoding step is determined by the encoding method; the compression and encoding step includes at least one of the following: word segmentation encoding, compression, and aggregation.

[0149] According to one or more embodiments of this disclosure, the method further includes: obtaining the device operating status of the terminal device running the application client; determining the corresponding encoding method according to the usage scenario of the event log includes: determining the corresponding encoding method according to the usage scenario of the event log and the device operating status.

[0150] According to one or more embodiments of this disclosure, the device operating state includes at least one of the following: CPU load, input / output interface load, disk load, and network bandwidth load.

[0151] According to one or more embodiments of this disclosure, the method further includes: obtaining the functional scenario when the application client triggers the target event; obtaining the usage requirement scenario based on the functional scenario and the requirement information for analyzing the event log under the functional scenario; determining the corresponding encoding method based on the usage requirement scenario of the event log and the device operating state, including: inputting the scenario information representing the usage requirement scenario and the state information representing the device operating state into a pre-trained strategy model to obtain the encoding method.

[0152] Secondly, according to one or more embodiments of this disclosure, a log data processing apparatus is provided, comprising:

[0153] The generation module is used to generate event logs after a target event is triggered by the application client;

[0154] The determination module is used to determine the corresponding encoding method according to the usage scenario of the event log, and to compress and encode the event log based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method.

[0155] The upload module is used to upload the log data to the application server when the target is triggered.

[0156] According to one or more embodiments of this disclosure, the encoding method includes first execution information, which is used to characterize the rules for segmenting the content text in the event log. When the determining module compresses and encodes the event log based on the encoding method to generate log data, it is specifically used to: obtain a target log template according to the usage scenario of the event log, the target log template having template parameters corresponding to the usage scenario; obtain the corresponding target content text from the event log according to the template parameters of the target log template; segment the target content text based on the first execution information to obtain a segmentation set containing at least two segmentation objects, and encode at least one segmentation object in the segmentation set to generate the log data.

[0157] According to one or more embodiments of this disclosure, the encoding method further includes second execution information, which is used to characterize the rules for encoding the target word segmentation object; when the determining module encodes at least one word segmentation object in the word segmentation set to generate the log data, it is specifically used to: obtain a word segmentation encoding dictionary corresponding to the target event according to the second execution information, which is used to characterize the mapping relationship between the target word segmentation object and the corresponding encoding object; map the target word segmentation object in the word segmentation set to the corresponding encoding object according to the word segmentation encoding dictionary to generate an encoded word segmentation set; and generate log data according to the encoded word segmentation set.

[0158] According to one or more embodiments of this disclosure, the determining module, in performing word segmentation on the target content text based on the first execution information to obtain a word segmentation set containing at least two word segmentation objects, is specifically configured to: obtain the corresponding target word segmentation granularity based on the first execution information; and split the target content text based on the target word segmentation granularity to obtain a word segmentation set containing at least two word segmentation objects.

[0159] According to one or more embodiments of this disclosure, the encoding method includes third execution information, which is used to characterize the target window length of the sliding window; when the determining module encodes at least one segmentation object in the segmentation set to generate the log data, it is specifically used to: obtain a sub-segmentation set in the segmentation set based on the third execution information and the target window length; encode the segmentation objects in the sub-segmentation set in sequence to generate an encoded sub-segmentation set; and combine each encoded sub-segmentation set to generate the log data.

[0160] According to one or more embodiments of this disclosure, the determining module is further configured to: obtain compression parameters according to the encoding method, the compression parameters being used to characterize the compression intensity and / or compression range of the log data; compress the log data according to the compression parameters to generate compressed log data; the uploading module is specifically configured to: upload the compressed log data to the application server under the target triggering condition.

[0161] According to one or more embodiments of this disclosure, before generating compressed log data, the determining module is further configured to: perform an inverse transformation on the log data according to the encoding method to generate restored log data; when the determining module compresses the log data according to the compression parameters to generate compressed log data, it is specifically configured to: compress the restored log data according to the compression parameters to generate compressed log data.

[0162] According to one or more embodiments of this disclosure, when the determining module compresses and encodes the event log based on the encoding method to generate log data, it is specifically used to: perform at least one compression encoding step on the event log based on the encoding method to generate the log data, wherein the number of compression encoding steps and / or the execution intensity of each compression encoding step is determined by the encoding method; the compression encoding step includes at least one of the following: word segmentation encoding, compression, and aggregation.

[0163] According to one or more embodiments of this disclosure, the determining module is further configured to: obtain the device operating status of the terminal device running the application client; when the determining module determines the corresponding encoding method according to the usage scenario of the event log, it is specifically configured to: determine the corresponding encoding method according to the usage scenario of the event log and the device operating status.

[0164] According to one or more embodiments of this disclosure, the device operating state includes at least one of the following: CPU load, input / output interface load, disk load, and network bandwidth load.

[0165] According to one or more embodiments of this disclosure, the determining module is further configured to: obtain the functional scenario when the application client triggers the target event; obtain the usage requirement scenario based on the functional scenario and the requirement information for analyzing the event log under the functional scenario; when the determining module determines the corresponding encoding method based on the usage requirement scenario of the event log and the device operating state, it is specifically configured to: input the scenario information representing the usage requirement scenario and the state information representing the device operating state into a pre-trained strategy model to obtain the encoding method.

[0166] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0167] The memory stores computer-executed instructions;

[0168] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the log data processing method as described in the first aspect and various possible designs of the first aspect.

[0169] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the log data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0170] Fifthly, according to one or more embodiments of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the log data processing method as described in the first aspect and various possible designs of the first aspect.

[0171] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0172] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0173] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A log data processing method, characterized in that, include: After the target event is triggered by the application client, an event log is generated; Based on the usage scenario of the event log, the corresponding encoding method is determined, and the event log is compressed and encoded based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method. Under the target trigger condition, the log data is uploaded to the application server.

2. The method according to claim 1, characterized in that, The encoding method includes first execution information, which is used to characterize the rules for segmenting the content text in the event log. The process of compressing and encoding the event log based on the aforementioned encoding method to generate log data includes: Based on the usage scenario of the event log, a target log template is obtained, and the target log template has template parameters corresponding to the usage scenario. Based on the template parameters of the target log template, obtain the corresponding target content text from the event log; Based on the first execution information, the target content text is segmented to obtain a segmentation set containing at least two segmentation objects, and at least one segmentation object in the segmentation set is encoded to generate the log data.

3. The method according to claim 2, characterized in that, The encoding method further includes second execution information, which is used to characterize the rules for encoding the target word segmentation object; The step of encoding at least one segmented object within the segmentation set to generate the log data includes: Based on the second execution information, a word segmentation encoding dictionary corresponding to the target event is obtained. The word segmentation encoding dictionary is used to represent the mapping relationship between the target word segmentation object and the corresponding encoding object. Based on the word segmentation encoding dictionary, the target word segmentation objects in the word segmentation set are mapped to the corresponding encoding objects to generate an encoded word segmentation set; Log data is generated based on the encoded word segmentation set.

4. The method according to claim 2, characterized in that, The step of segmenting the target content text based on the first execution information to obtain a segmentation set containing at least two segmentation objects includes: Based on the first execution information, the corresponding target word segmentation granularity is obtained; The target content text is split based on the target word segmentation granularity to obtain a word segmentation set containing at least two word segmentation objects.

5. The method according to claim 2, characterized in that, The encoding method includes third execution information, which is used to characterize the target window length of the sliding window; the encoding of at least one segmentation object within the segmentation set to generate the log data includes: Based on the third execution information, a sub-segmentation set is obtained from the segmentation set using the target window length; The segmented objects within the sub-segmentation set are encoded sequentially to generate an encoded sub-segmentation set; The log data is generated by combining the sets of each encoded sub-word segmentation.

6. The method according to claim 1, characterized in that, The method further includes: Based on the encoding method, compression parameters are obtained, which are used to characterize the compression strength and / or compression range of the log data. The log data is compressed according to the compression parameters to generate compressed log data; The step of uploading the log data to the application server under the target trigger condition includes: Under the target triggering condition, the compressed log data is uploaded to the application server.

7. The method according to claim 6, characterized in that, Before generating the compressed log data, the method further includes: Based on the encoding method, the log data is inversely transformed to generate restored log data; The step of compressing the log data according to the compression parameters to generate compressed log data includes: The restored log data is compressed according to the compression parameters to generate compressed log data.

8. The method according to claim 1, characterized in that, The process of compressing and encoding the event log based on the aforementioned encoding method to generate log data includes: Based on the encoding method, at least one compression encoding step is performed on the event log to generate the log data, wherein the number of compression encoding steps and / or the execution intensity of each compression encoding step are determined by the encoding method; The compression encoding step includes at least one of the following: Word segmentation encoding, compression, and aggregation.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the device operating status of the terminal device running the application client; The step of determining the corresponding encoding method based on the usage scenario of the event log includes: The corresponding encoding method is determined based on the usage scenarios of the event log and the operating status of the device.

10. The method according to claim 9, characterized in that, The device operating status includes at least one of the following: CPU load, input / output interface load, disk load, and network bandwidth load.

11. The method according to claim 9, characterized in that, The method further includes: Obtain the functional scenario when the application client triggers the target event; Based on the functional scenarios and the requirements for analyzing the event logs under the functional scenarios, the usage requirement scenarios are obtained; Based on the usage scenarios of the event logs and the device operating status, the corresponding encoding method is determined, including: The scenario information representing the usage demand scenario and the state information representing the device operating state are input into the pre-trained policy model to obtain the encoding method.

12. A log data processing device, characterized in that, include: The generation module is used to generate event logs after a target event is triggered by the application client; The determination module is used to determine the corresponding encoding method according to the usage scenario of the event log, and to compress and encode the event log based on the encoding method to generate log data. The resource overhead of the event log compression and encoding is related to the encoding method. The upload module is used to upload the log data to the application server when the target is triggered.

13. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the log data processing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the log data processing method as described in any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the log data processing method as described in any one of claims 1 to 11.

Images