Log processing method, device and storage medium

By splitting and reorganizing the original logs, generating single tag information, and querying them according to preset dimensions, the problem of low processing efficiency of large-scale user log data analysis and processing is solved, and fast and efficient log analysis is achieved.

CN112069048BActive Publication Date: 2025-02-07BEIJING XUEZHITU NETWORK TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202010943762.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-09-09
Publication Date
2025-02-07
Estimated Expiration
2040-09-09

AI Technical Summary

Technical Problem

The prior art is cumbersome when facing the analysis and processing of massive user log data, and multiple data are cross-aliased to store, resulting in low processing efficiency.

Method used

By obtaining the original log, splitting and reorganizing it according to the preset separator, multiple single tag information are generated, including the correspondence between one user identity information and one type tag. Then, multiple single tag information are queried based on the preset dimension information to obtain the query results.

Benefits of technology

By cutting the original log into multiple single tag information, it can quickly query from any dimension, improving the efficiency of log analysis and processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112069048B_ABST
    Figure CN112069048B_ABST
Patent Text Reader

Abstract

Embodiments of the present invention provide a log processing method, device and storage medium, which relate to the field of data processing technology. The log processing method provided by the present embodiment obtains an original log, splits and reorganizes the original log according to a preset delimiter, and generates multiple single label information; queries the multiple single label information according to preset dimension information to obtain a query result, and the query result includes the number of the single label information corresponding to the dimension information. The original log is split and reorganized to form multiple single label information, and multiple one-to-many data types are converted into multiple one-to-one data types, which simplifies the data type. When performing a specific analysis on the user log, it can be directly queried and sorted according to the set dimension, which helps to improve the efficiency of data processing and analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically, to a log processing method, apparatus, and storage medium. Background Art

[0002] With the rapid development of information and communication technologies, various application software has been widely used and promoted, making it necessary to collect user data logs and analyze user behavior.

[0003] For different applications and systems, current solutions only address the collection of basic user information, such as simple usage records, application access volume, access frequency, interests, etc., generating logs and performing simple analysis on mobile phone user logs. However, existing technologies are cumbersome in analyzing and processing massive amounts of user log data, with multiple data sets being stored in a mixed and overlapping manner, resulting in low processing efficiency when the data volume is large. Summary of the Invention

[0004] The purpose of this invention is to provide a log processing method, apparatus, and storage medium to quickly perform statistical analysis on user data from mobile phones and improve processing efficiency.

[0005] The technical solution adopted in this invention is as follows:

[0006] In a first aspect, embodiments of the present invention provide a log processing method, the method comprising:

[0007] Obtain the raw logs, which include multiple user identity information and the correspondence between the user identity information and multiple type tags;

[0008] The original log is split and reassembled according to a preset delimiter to generate multiple single tag information; the single tag information includes a correspondence between a user identity information and a type tag;

[0009] The query results are obtained by querying multiple single tag information based on the preset dimension information, and the query results include the number of single tag information corresponding to the dimension information.

[0010] In an optional implementation, the step of splitting and recombining the original log according to a preset delimiter to generate multiple single-tag information includes:

[0011] The original log is initially segmented based on the user identity information to obtain intermediate logs, and the intermediate logs include a correspondence between the user identity information and multiple type tags;

[0012] The intermediate logs are segmented according to a preset delimiter to obtain user identity information and multiple type tags;

[0013] The user identity information is combined with each of the multiple type tags to generate multiple single tag information.

[0014] In an optional implementation, after the step of querying multiple single-label information based on preset dimension information to obtain query results, the method further includes:

[0015] Delete the intermediate logs and the single tag information.

[0016] In an optional implementation, the step of querying multiple single-label information based on preset dimension information to obtain query results includes:

[0017] Read a first preset number of data blocks, each data block including a second preset number of the single tag information;

[0018] Each data block is queried according to a preset dimension based on the single tag information, and preliminary results are generated. The preliminary results include the number of single tag information corresponding to the preset dimension queried in the data block.

[0019] The preliminary results corresponding to multiple data blocks are classified and statistically analyzed to obtain the query results.

[0020] In an optional implementation, before querying multiple single-label information based on preset dimension information, the log processing method further includes:

[0021] Obtain a query instruction, the query instruction including preset dimension information, the dimension information being at least one of the plurality of tag information.

[0022] In a second aspect, embodiments of the present invention provide a log processing apparatus, the log processing apparatus being used to implement the log processing method as described in any of the foregoing embodiments, the log processing apparatus comprising:

[0023] The acquisition module is used to acquire raw logs, which include multiple user identity information and the correspondence between the user identity information and multiple type tags;

[0024] The splitting module is used to split and reassemble the original log according to a preset delimiter to generate multiple single tag information; the single tag information includes a correspondence between a user identity information and a type tag;

[0025] The processing module is used to query multiple single tag information based on preset dimension information to obtain query results, the query results including the number of single tag information corresponding to the dimension information.

[0026] In an optional implementation, the splitting module is used to perform preliminary segmentation of the original log based on the user identity information to obtain intermediate logs, the intermediate logs including a correspondence between the user identity information and multiple type tags; to segment the intermediate logs according to a preset delimiter to obtain user identity information and multiple type tags; and to combine the user identity information with the multiple type tags one by one to generate multiple single tag information.

[0027] In an optional implementation, the splitting module is further configured to delete the intermediate log and the single tag information after the step of querying multiple single tag information according to preset dimension information to obtain query results.

[0028] In an optional implementation, the processing module is used to query each data block according to a preset dimension based on the single tag information within each data block, and generate preliminary results. The preliminary results include the number of single tag information corresponding to the preset dimension queried within the data block. The multiple preliminary results corresponding to multiple data blocks are classified and statistically analyzed to obtain the query results.

[0029] Thirdly, embodiments of the present invention provide a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the log processing method as described in any of the foregoing embodiments.

[0030] Compared to existing technologies, this application provides a log processing method, apparatus, and storage medium. The log processing method includes: acquiring raw logs, the raw logs including multiple user identity information and the correspondence between the user identity information and multiple type tags; splitting and recombining the raw logs according to a preset delimiter to generate multiple single tag information; each single tag information includes a correspondence between one user identity information and one type tag; querying the multiple single tag information according to preset dimension information to obtain query results, the query results including the number of single tag information corresponding to the dimension information. By segmenting the raw logs to generate multiple single tag information, log analysis and processing results can be quickly generated from any dimension, improving the efficiency of log analysis and processing. Attached Figure Description

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A schematic diagram of the electronic device provided in this embodiment;

[0033] Figure 2 This is a flowchart illustrating a log processing method provided in this embodiment;

[0034] Figure 3 This is a flowchart illustrating another log processing method provided in this embodiment;

[0035] Figure 4 This is a flowchart illustrating another log processing method provided in this embodiment;

[0036] Figure 5 This is a flowchart illustrating another log processing method provided in this embodiment;

[0037] Figure 6 A schematic diagram illustrating the fusion of preliminary results provided in this embodiment;

[0038] Figure 7 This is a flowchart illustrating another log processing method provided in this embodiment;

[0039] Figure 8 This is a schematic diagram of the functional modules of a log processing device provided in this embodiment.

[0040] Explanation of reference numerals in the attached drawings: 210-processor; 211-memory; 212-bus; 213-communication interface; 300-log processing device; 310-acquisition module; 320-splitting module; 330-processing module. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0042] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0043] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0044] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the product of the invention is usually placed when in use. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0045] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0046] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0047] With the rapid development of information and communication technologies, various application software has been widely used and promoted, making the collection of user data logs and the analysis of user behavior necessary. Current solutions for different application software and systems only address the collection of basic user information, such as simple usage records, application access volume, access frequency, and interests, generating logs. Simple analysis of mobile phone user logs is then performed. However, existing technologies are cumbersome in processing massive amounts of user log data, with multiple data sets being stored in a mixed and overlapping manner, resulting in low processing efficiency when the data volume is large.

[0048] To address the aforementioned problems, this application provides an electronic device, please refer to... Figure 1 , Figure 1 A schematic diagram of the electronic device provided in this embodiment is shown. The electronic device includes a processor 210, a memory 211, and a bus 212. The processor 210 and the memory 211 are connected via the bus 212. The processor 210 is used to execute executable modules, such as computer programs, stored in the memory 211.

[0049] The processor 210 can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the log processing method provided in this embodiment can be completed by the integrated logic circuitry in the processor 210 or by software instructions. The processor 210 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0050] The memory 211 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0051] Bus 212 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Figure 1 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus 212 or one type of bus 212.

[0052] The memory 211 is used to store programs, such as programs corresponding to a log processing device. The log processing device includes at least one software functional module that can be stored in the memory 211 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, the processor 210 executes the program to implement the steps of the log processing method.

[0053] The electronic device provided in this application embodiment may also include a communication interface 213. The communication interface 213 is connected to the processor 210 via a bus. The communication interface 213 can be used to connect external devices, such as at least one camera, audio acquisition device, etc.

[0054] It should be understood that Figure 1 The structure shown is only a partial schematic diagram of the electronic device; the electronic device may also include components that are larger than... Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0055] based on Figure 1 The electronic device shown in this embodiment provides a log processing method. Please refer to [link / reference]. Figure 2 , Figure 2 A flowchart of the log processing method provided in this embodiment is shown, which includes steps 110 to 130.

[0056] Step 110: Obtain the raw logs, which include multiple user identity information and the correspondence between user identity information and multiple type tags.

[0057] The raw logs consist of collected user data, including user identity information and the corresponding relationships of multiple type tags. These type tags include information such as user interests, access source, access time, and city of residence. The user data is stored according to a pre-defined format, as shown in Table 1, which provides one possible storage format for the raw logs.

[0058] In one possible implementation, the raw logs are stored on a server, which could be a distributed server.

[0059] Table 1

[0060]

[0061] Step 120: Split and reorganize the original log according to the preset delimiter to generate multiple single tag information; the single tag information includes the correspondence between a user identity information and a type tag.

[0062] The original logs store multiple user identity information entries and their corresponding relationships with various type tags. In actual processing, a single user identity entry can correspond to too many tag types, which reduces the efficiency of analysis and processing when the data volume is large. In this embodiment, the original logs are split and reassembled according to a preset delimiter to generate multiple single tag information entries. A single tag information entry refers to the correspondence between a single user identity entry and a single type tag. Splitting and reassembling the original logs converts multiple one-to-many data types into multiple one-to-one data types, simplifying the data types and facilitating subsequent data processing and statistical analysis.

[0063] Step 130: Query multiple single-label information based on the preset dimension information to obtain query results, including the number of single-label information corresponding to the dimension information.

[0064] After splitting and reorganizing the original logs to generate multiple single-tag information, the multiple single-tag information can be queried according to preset dimension information to obtain query results. In this embodiment, the query results include the number of single-tag information corresponding to the dimension information. Dimension information is a tag of one or more types. For example, querying all single-tag information with "fond of football" as the dimension can determine the number of all users who like football. If querying all single-tag information with "fond of football" and "city Beijing" as the dimension can determine the number of all users in Beijing who like football.

[0065] The log processing method provided in this embodiment obtains the original log, splits and reassembles the original log according to a preset delimiter to generate multiple single tag information; queries the multiple single tag information according to preset dimension information to obtain query results, the query results including the number of single tag information corresponding to the dimension information. By splitting and reassembling the original log to form multiple single tag information, multiple one-to-many data types are converted into multiple one-to-one data types, simplifying the data types. When performing specific analysis on user logs, queries and organization can be performed directly according to the set dimensions, which helps to improve the efficiency of data processing and analysis.

[0066] exist Figure 2 Based on this, refer to Figure 3 Regarding how to segment and reassemble the original logs, this embodiment provides another possible implementation method. Figure 3 This embodiment shows a flowchart of another log processing method, where step 120 includes the following sub-steps:

[0067] Step 120-1: Perform preliminary segmentation of the original log based on the user identity information to obtain intermediate logs. The intermediate logs include the correspondence between a user identity information and multiple type tags.

[0068] As shown in Table 1, the original log information is indexed by user identity information, storing the correspondence between multiple user identity information and multiple type tags. In other words, the original log stores multiple one-to-many data types. In one possible implementation, the original log is split into intermediate logs using user identity information as a delimiter. As shown in Tables 2 and 3, the split intermediate logs can be stored in the formats shown in Tables 2 and 3.

[0069] Table 2

[0070] User ID User Interests City Name Visit Source Access time Project ID Keywords other User08 Football, mountain climbing Beijing Taobao 2020-07-21 10012 Philips …

[0071] Table 3

[0072]

[0073] In one possible implementation, if the original log contains N user identity information entries and a mapping between these N user identity information entries and multiple type tags, then the original log is split into N intermediate logs. Each intermediate log entry includes a mapping between one user identity information entry and multiple type tags.

[0074] Step 120-2: Segment the intermediate logs according to the preset delimiter to obtain user identity information and multiple type tags.

[0075] The intermediate logs are further segmented according to the preset delimiter to obtain user identity information and multiple type tags.

[0076] In one possible implementation, an intermediate log is further segmented using punctuation marks or tabs as delimiters to obtain a user identity information and multiple type tags.

[0077] Taking the intermediate log shown in Table 2 as an example, if we divide it by punctuation marks, it will be divided into user identity information User08, and tags including: likes football; likes mountain climbing; city is Beijing; access source is Taobao; access time is July 21, 2020; project ID is 10012; keyword is Philips, etc.

[0078] Step 120-3: Combine the user identity information with multiple type tags one by one to generate multiple single tag information.

[0079] User identity information is combined with multiple type tags to generate multiple single tag information. For example, in order to obtain user's preferences, user identity information and the type of preference tag are recombined to generate single tag information.

[0080] For example, for the intermediate logs shown in Table 2, two single-tag information entries are generated: User08 with the user's identity information being "football" and User08 with the user's hobby being "mountain climbing". For multiple intermediate logs, multiple single-tag information entries can be generated. In one possible implementation, the generated single-tag information entries can be stored using the user's identity information as an index, as shown in Table 4. Table 4 shows one possible storage format for single-tag information entries.

[0081] Table 4

[0082] User ID User Interests Access time User01 football 2020-07-21 User01 mountain climbing 2020-07-21 User01 game 2020-07-21 User01 travel 2020-07-21 User02 football 2020-07-21 User02 travel 2020-07-21 User02 photography 2020-07-21 User03 photography 2020-07-21 User03 gourmet food 2020-07-21 User03 Dance 2020-07-21 User04 travel 2020-07-20 User04 Dance 2020-07-20 User03 photography 2020-07-26 User03 gourmet food 2020-07-26 User03 Dance 2020-07-26

[0083] Understandably, since the same user may have multiple hobbies, there may be multiple hobby tags corresponding to the same user's identity information.

[0084] Once single-tag information is generated, user logs can be analyzed and statistically processed based on that single-tag information. See [link / reference] for one possible implementation method. Figure 4 Before generating single-label information, this log processing method also includes:

[0085] Step 111: Obtain the query instruction. The query instruction includes pre-defined dimension information, which is at least one of multiple label information.

[0086] Retrieve query instructions, which include the dimension information the user needs to query. The dimension information is at least one of multiple type tags. For example, if you need to count how many users have "football" as their hobby, you need to count the users' hobbies; "hobbies" is the dimension information included in the query instruction.

[0087] After obtaining the query command, the original log is processed based on the dimension information to obtain single tag information corresponding to the hobbies.

[0088] In one possible implementation, after generating single tag information, user logs can be analyzed and statistically processed based on that single tag information. (See [reference needed]). Figure 5 Step 130 includes the following steps:

[0089] Step 130-1: Read a first preset number of data blocks, each data block including a second preset number of single tag information.

[0090] The existing single tag information is divided into multiple data blocks. In one possible implementation, the processor's computing power is combined, and a second preset number of single tag information is used as a data block, and all single tag information is divided into a first preset number of data blocks.

[0091] For example, if there are only 12 single tag information, three single tag information are used as a data block to form a data block. When performing statistical processing on the data, these four data blocks are read first.

[0092] In one possible implementation, due to the large volume of logs, the Hadoop MapReduce computing framework is used for grouped computation, allowing the same processor to read the four data blocks. However, in other possible implementations, the four data blocks can be read separately by distributed processors to improve processing efficiency.

[0093] Step 130-2: Query each data block according to the preset dimensions based on the single tag information, and generate preliminary results. The preliminary results include the number of single tag information corresponding to the preset dimensions queried in the data block.

[0094] Each data block is queried according to a preset dimension for each individual label information, generating preliminary results. The preliminary results include the number of individual label information items corresponding to the preset dimensions retrieved within that data block. For example, the information included in the first data block is shown in Table 5:

[0095] Table 5

[0096] User01 football User01 mountain climbing User01 game

[0097] Therefore, querying and statistically analyzing the first data block using the "hobby" category tag as a dimension yields the following preliminary results, as shown in Table 6:

[0098] Table 6

[0099] football 1 person mountain climbing 1 person game 1 person

[0100] Step 130-3: Classify and statistically analyze the multiple preliminary results corresponding to multiple data blocks to obtain the query results.

[0101] After processing multiple data blocks separately, the preliminary results obtained from processing the multiple data blocks are fused together using Rayga to obtain the query result.

[0102] For example, querying the second data block yields a preliminary result for the second data block; querying the third data block yields a preliminary result for the third data block; and querying the fourth data block yields a preliminary result for the fourth data block.

[0103] The preliminary results from the first, second, third, and fourth data blocks are merged to obtain the final query result. For example... Figure 6 As shown.

[0104] See Figure 7 After step 130, the method further includes:

[0105] Step 140: Delete the intermediate logs and single tag information.

[0106] After processing the data, delete the generated intermediate logs and single-label information to avoid occupying too much space.

[0107] To perform the corresponding steps in the above embodiments and various possible implementations, an implementation of a log processing device is given below. Please refer to [link / reference]. Figure 8 , Figure 8 This invention provides a preferred embodiment of a log processing device 300. It should be noted that the log processing device 300 provided in this embodiment has the same basic principle and technical effects as the air conditioning control method provided in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The log processing device 300 provided in this embodiment includes an acquisition module 310, a splitting module 320, and a processing module 330.

[0108] The acquisition module 310 is used to acquire the raw log, which includes multiple user identity information and the correspondence between the user identity information and multiple type tags.

[0109] Understandably, in one possible implementation, the acquisition module 310 can be used to perform step 110 in the above figures to achieve the corresponding technical effects.

[0110] The splitting module 320 is used to split and reorganize the original logs according to the preset delimiters to generate multiple single tag information; the single tag information includes the correspondence between a user identity information and a type tag.

[0111] Understandably, in one possible implementation, the split module 320 can be used to perform step 120 in the above figures to achieve the corresponding technical effects.

[0112] The processing module 330 is used to query multiple single-label information based on preset dimension information to obtain query results, including the number of single-label information corresponding to the dimension information.

[0113] Understandably, in one possible implementation, the processing module 330 can be used to execute steps 130 in the above figures to achieve the corresponding technical effects.

[0114] In an optional implementation, the splitting module 320 is used to perform preliminary segmentation of the original log based on the user identity information to obtain intermediate logs, the intermediate logs including the correspondence between a user identity information and multiple type tags; to segment the intermediate logs according to a preset delimiter to obtain user identity information and multiple type tags; and to combine the user identity information with the multiple type tags one by one to generate multiple single tag information.

[0115] Understandably, in one possible implementation, the split module 320 can be used to execute steps 120-1 to 120-3 in the above figures to achieve the corresponding technical effects.

[0116] In an optional implementation, the splitting module is also used to delete intermediate logs and single-label information after the step of querying multiple single-label information according to preset dimension information to obtain query results.

[0117] Understandably, in one possible implementation, the split module 320 can be used to perform step 140 in the above figures to achieve the corresponding technical effects.

[0118] In an optional implementation, the processing module is used to query each data block according to a preset dimension based on the single tag information within each data block, and generate preliminary results. The preliminary results include the number of single tag information corresponding to the preset dimension queried within the data block. The multiple preliminary results corresponding to multiple data blocks are classified and statistically analyzed to obtain the query results.

[0119] Understandably, in one possible implementation, the processing module 330 can be used to execute steps 130-1 to 130-3 in the above figures to achieve the corresponding technical effects.

[0120] This invention provides a storage medium storing a computer program, which, when executed by a processor, implements a log processing method as described in any of the foregoing embodiments.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0123] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A log processing method, characterized in that: The method comprises: Acquire an original log, wherein the original log includes multiple user identity information and a corresponding relationship between the user identity information and multiple type tags; The original log is split and reorganized according to a preset separator to generate multiple single label information, including: preliminarily splitting the original log according to the user identity information to obtain an intermediate log, the intermediate log including a corresponding relationship between the user identity information and multiple type labels; splitting the intermediate log according to the preset separator to obtain user identity information and multiple type labels; combining the user identity information with the multiple type labels one by one to generate multiple single label information; the single label information includes a corresponding relationship between the user identity information and a type label; The plurality of single tag information are queried according to the preset dimension information to obtain a query result, wherein the query result includes the quantity of the single tag information corresponding to the dimension information.

2. The log processing method according to claim 1, characterized in that: After the step of querying the plurality of single tag information according to the preset dimension information to obtain the query result, the method further includes: The intermediate log and the single tag information are deleted.

3. The log processing method according to claim 1, characterized in that: The step of querying the plurality of single tag information according to the preset dimension information to obtain the query result includes: Reading a first preset number of data blocks, each data block including a second preset number of the single tag information; Querying according to the preset dimension according to the single label information in each data block respectively, and generating preliminary results, wherein the preliminary results include the number of the single label information corresponding to the preset dimension queried in the data block; The multiple preliminary results corresponding to the multiple data blocks are classified and counted to obtain the query result.

4. The log processing method according to claim 1, characterized in that: Before querying the plurality of single tag information according to the preset dimension information, the log processing method further includes: A query instruction is obtained, where the query instruction includes preset dimension information, and the dimension information is at least one of a plurality of tag information.

5. A log processing device, characterized in that: The log processing device is used to implement the log processing method according to any one of claims 1 to 4, and the log processing device includes: An acquisition module, used to acquire an original log, wherein the original log includes multiple user identity information and a corresponding relationship between the user identity information and multiple type tags; The splitting module is used to split and reorganize the original log according to a preset separator to generate multiple single label information, including: preliminarily splitting the original log according to the user identity information to obtain an intermediate log, the intermediate log including a corresponding relationship between the user identity information and multiple type labels; splitting the intermediate log according to the preset separator to obtain the user identity information and multiple type labels; combining the user identity information with the multiple type labels one by one to generate multiple single label information; the single label information includes a corresponding relationship between the user identity information and a type label; The processing module is used to query the plurality of single tag information according to preset dimension information to obtain a query result, wherein the query result includes the quantity of the single tag information corresponding to the dimension information.

6. The log processing device according to claim 5, characterized in that: The splitting module is also used to delete the intermediate log and the single tag information after the step of querying the plurality of single tag information according to the preset dimension information to obtain the query result.

7. The log processing device according to claim 5, characterized in that: The processing module is used to query according to the preset dimension according to the single label information in each data block, and generate a preliminary result, wherein the preliminary result includes the number of the single label information corresponding to the preset dimension queried in the data block; The multiple preliminary results corresponding to the multiple data blocks are classified and counted to obtain the query result.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the log processing method according to any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Log data hierarchical storage method and device and storage medium

    CN109739821A