Data query method and device, storage medium and electronic device

Abstract

The present invention discloses a data query method and device, a storage medium and an electronic device, wherein the method comprises: obtaining a target log set, and dividing the target log set into a plurality of log subsets corresponding to a plurality of time periods, wherein the data of the time parameters included in the logs in each log subset are located within the corresponding time period; clustering the plurality of log subsets respectively to obtain a plurality of clustering results, wherein each of the plurality of clustering results comprises a clustering cluster obtained by clustering the corresponding log subsets and a clustering label corresponding to the clustering cluster, and each clustering cluster comprises one or more logs; performing an intersection operation on the clustering clusters with the same clustering label in the plurality of clustering results according to a user identifier to obtain a target intersection result; and when the target intersection result comprises one or more target log subsets, storing the one or more target log subsets in the memory of a target server.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to a data query method and device, a storage medium and an electronic device. Background Art

[0002] With the rapid development and popularization of computer information and information data, the scale of industry application systems has expanded rapidly, and the data generated by industry applications has exploded. Industries that easily reach hundreds of TB or even tens to hundreds of PB in scale also require higher-configuration server performance. Therefore, seeking effective big data processing technologies, methods and means has become an urgent need in the real world. The era of big data has brought new challenges to human data processing capabilities, and also provided unprecedented space and potential for people to gain deeper and more comprehensive insights. Implementing efficient big data processing solutions is of great significance to data query and reducing server performance requirements.

[0003] Moreover, as the system is deployed to the server and more and more user log data is generated, the processing of user log data has become a research hotspot today. Data analysis is the basis of data processing. Therefore, developing an effective user log data analysis is of great significance to log data processing on the server.

[0004] However, existing data queries all directly query data from a database stored outside the server, resulting in low data query efficiency.

[0005] Regarding the related technology, when querying log data, the server directly queries from the external storage database, resulting in low query efficiency. Currently, no effective solution has been proposed.

[0006] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0007] The embodiments of the present invention provide a data query method and device, a storage medium and an electronic device, so as to at least solve the problem that when a server queries log data, it directly queries from an external storage database, resulting in low query efficiency.

[0008] According to one aspect of an embodiment of the present invention, a data query method is provided, comprising: obtaining a target log set, wherein each log in the target log set includes data of a set of parameters, wherein the set of parameters includes a user identifier and a time parameter; dividing the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is located within the corresponding time period; clustering the plurality of log subsets respectively to obtain a plurality of clustering results, wherein each of the plurality of clustering results includes a clustering cluster obtained by clustering the corresponding log subset and a clustering label corresponding to the clustering cluster, and each clustering cluster includes one or more logs; performing an intersection operation on the clustering clusters with the same clustering label in the plurality of clustering results according to the user identifier to obtain a target intersection result; and in a case where the target intersection result includes one or more target log subsets, storing the one or more target log subsets in a memory of a target server, wherein the one or more target log subsets stored in the memory of the target server are used to be preferentially queried in a case where the target server obtains a target query instruction.

[0009] According to another aspect of an embodiment of the present invention, a data query device is also provided, including: an acquisition module, used to acquire a target log set, wherein each log in the target log set includes a set of parameter data, and the set of parameters includes a user identifier and a time parameter; a division module, used to divide the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the time parameter data in each log, wherein the time parameter data included in the logs in each log subset is located within the corresponding time period; a clustering module, used to cluster the plurality of log subsets respectively to obtain a plurality of clustering results, wherein each cluster in the plurality of clustering results has a corresponding time period. The result includes clustering clusters obtained by clustering corresponding log subsets and clustering labels corresponding to the clustering clusters, each clustering cluster includes one or more logs; a processing module is used to perform an intersection operation on the clustering clusters with the same clustering labels in the multiple groups of clustering results according to the user identifier to obtain a target intersection result; a storage module is used to store the one or more target log subsets in the memory of the target server when the target intersection result includes one or more target log subsets, wherein the one or more target log subsets stored in the memory of the target server are used to be queried first when the target server obtains a target query instruction.

[0010] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above data query method when running.

[0011] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the data query method through the computer program.

[0012] Through the present invention, a target log set is obtained, and the target log set is divided into multiple log subsets corresponding to multiple time periods, and then the multiple log subsets are clustered respectively to obtain multiple groups of clustering results, and then the cluster clusters with the same clustering label in the multiple groups of clustering results are subjected to intersection operation according to the user identifier to obtain the target intersection result, and when the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried first when the target server obtains the target query instruction. The above technical solution solves the problem that when the server queries log data, it directly queries from the external storage database, resulting in low query efficiency. Through the above technical solution, part of the data that meets the requirements is stored in the memory of the server, and then when the query instruction is obtained, it is searched in the memory first, thereby improving the query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0014] Figure 1 It is a hardware structure block diagram of a computer terminal of a data query method according to an embodiment of the present invention;

[0015] Figure 2 is a flow chart of a data query method according to an embodiment of the present invention;

[0016] Figure 3 is a clustering schematic diagram according to an embodiment of the present invention;

[0017] Figure 4 is a schematic diagram according to an embodiment of the present invention;

[0018] Figure 5 is a schematic diagram of a data filtering algorithm according to an embodiment of the present invention;

[0019] Figure 6 is a data processing effect diagram according to an embodiment of the present invention;

[0020] Figure 7is a structural block diagram of a data query device according to an embodiment of the present invention (I);

[0021] Figure 8 2 is a structural block diagram of a data query device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware structure diagram of a computer terminal of a data query method according to an embodiment of the present invention. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data query method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] In order to solve the above problem, a data query method is provided in this embodiment. Figure 2 is a flow chart of a data query method according to an embodiment of the present invention, the flow chart includes the following steps:

[0028] Step S202, obtaining a target log set, wherein each log in the target log set includes data of a set of parameters, and the set of parameters includes a user identifier and a time parameter;

[0029] It should be noted that the target log set is located in the external memory of the server, that is, the hard disk of the server.

[0030] Step S204, dividing the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is within the corresponding time period;

[0031] It should be noted that the time period can be set by the administrator. For example, the time period can be divided into 8:00-20:00 and 20:00-8:00; and the target log set can be divided into two log subsets. If there are three time periods, there are also three corresponding log subsets.

[0032] Step S206, clustering the multiple log subsets respectively to obtain multiple groups of clustering results, wherein each group of clustering results in the multiple groups of clustering results includes a cluster cluster obtained by clustering the corresponding log subset and a cluster label corresponding to the cluster cluster, and each cluster cluster includes one or more logs;

[0033] That is to say, each log subset in the multiple log subsets needs to be clustered respectively, and then a set of clustering results corresponding to each log subset can be obtained.

[0034] It should be noted that the clustering label may be frequently used social software, frequently used shopping software, etc.

[0035] Step S208, performing an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result;

[0036] Step S210, when the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains a target query instruction.

[0037] It should be noted that, in the process of storing the one or more target log subsets in the memory of the target server, if there are other logs in the memory, the other logs need to be deleted from the memory.

[0038] Through the present invention, a target log set is obtained, and the target log set is divided into multiple log subsets corresponding to multiple time periods, and then the multiple log subsets are clustered respectively to obtain multiple groups of clustering results, and then the cluster clusters with the same clustering label in the multiple groups of clustering results are subjected to intersection operation according to the user identifier to obtain the target intersection result, and when the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried first when the target server obtains the target query instruction. The above technical solution solves the problem that when the server queries log data, it directly queries from the external storage database, resulting in low query efficiency. Through the above technical solution, part of the data that meets the requirements is stored in the memory of the server, and then when the query instruction is obtained, it is searched in the memory first, thereby improving the query efficiency.

[0039] For better understanding, in an exemplary embodiment, clustering the multiple log subsets separately to obtain multiple groups of clustering results can be achieved in the following way: performing the following operations on each log subset of the multiple log subsets separately, wherein when performing the following operations, each log subset is the current log subset: clustering the current log subset according to the data of part or all of the parameters in the set of parameters to obtain a current set of clustering results, wherein, when the current set of clustering results includes at least one current clustering cluster, the feature distance between the log features of each log in the current clustering cluster determined according to the data of part or all of the parameters meets the preset clustering condition.

[0040] It should be noted that the parameters referenced during clustering can be attribute parameters or behavior parameters. Clustering a log subset can obtain one cluster or multiple clusters. In addition, if the feature distance between the log features determined by the parameters in the log subset does not meet the preset clustering conditions, clustering fails and there is no cluster.

[0041] It should be noted that the clustering algorithm used in the embodiments of the present application includes but is not limited to K-means.

[0042] In an exemplary embodiment, when the current set of clustering results includes at least one current clustering cluster, a corresponding current clustering label is set for the current clustering cluster; or when the current set of clustering results includes at least one current clustering cluster, the current clustering label corresponding to the current clustering cluster is determined in the target label set based on the data of the set of parameters included in each log in the current clustering cluster and the preset clustering condition.

[0043] That is to say, in one case, the clustering algorithm can determine the data labels by automatically analyzing the data of the clustering parameters during clustering.

[0044] It should be noted that in another case, if the administrator pre-sets the target tag set and determines the correspondence between the target tag set and the data of the clustering parameters, the server can determine the clustering tag corresponding to the clustering cluster in the target tag set through a set of parameters in the clustering cluster (equivalent to the above-mentioned clustering parameters) and preset clustering conditions.

[0045] In an exemplary embodiment, the performing of the intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain the target intersection result can be implemented in the following manner: when the multiple groups of clustering results are N groups of clustering results, for the j-th cluster in the i-th group of clustering results in the N groups of clustering results, the following operations are performed, wherein N is a positive integer greater than or equal to 2, 1≤i≤N, and the i-th group of clustering results includes M i clusters, M is a positive integer greater than or equal to 1, 1≤j≤M, and the cluster label of the j-th cluster is the current cluster label: searching for a cluster whose cluster label is the current cluster label in N-1 groups of cluster results other than the i-th group of cluster results in the N groups of cluster results; when a cluster whose cluster label is the current cluster label is found, performing an intersection operation on the j-th cluster and the found cluster according to the user identifier to obtain a current intersection result, wherein when the current intersection result includes at least 2 logs, determining the at least 2 logs as a target log subset.

[0046] For a better understanding, the following specific description assumes that there are two groups of clustering results, where the first group of clustering results has two clusters, and the cluster labels are A and B respectively; the second group of clustering results also has two clusters, and the cluster labels are A and C respectively. Each cluster includes one or more logs. Then, in the process of performing the intersection operation on the above two groups of clustering results, the clusters labeled A in the first group of clustering results and the clusters labeled A in the second group of clustering results are specifically intersected, and the clusters labeled A in the first group of clustering results and the logs with the same user identification label in the clusters labeled A in the second group of clustering results are determined as the target log subset. It should be noted that the number of logs in the target log subset is an even number.

[0047] In an exemplary embodiment, when the target intersection result does not include a log subset with an intersection, the target log set is clustered according to data of some or all of the parameters in the set of parameters to obtain one or more cluster clusters and cluster labels corresponding to the one or more cluster clusters.

[0048] That is, if no intersection exists in the process of grouping and clustering the target log set to obtain the intersection, the target log set is directly clustered to obtain one or more clusters and cluster labels corresponding to the one or more clusters.

[0049] In an exemplary embodiment, Figure 3 is a clustering schematic diagram according to an embodiment of the present invention, such as Figure 3 As shown, the target log set is divided into two log subset sets, and the two log subset sets are clustered respectively to obtain two groups of clustering results, and then the two groups of clustering results are subjected to an intersection operation.

[0050] In an exemplary embodiment, after storing the one or more target log subsets in the memory of the target server, it also includes: when the target server obtains a target query instruction and the target query instruction indicates a target query condition, in response to the target query instruction, searching the one or more target log subsets stored in the memory for logs that meet the target query condition; when no logs that meet the target query condition are found in the one or more target log subsets stored in the memory, searching the target log set for logs that meet the target query condition.

[0051] It should be noted that the target query condition includes but is not limited to the user identifier, and then after the server obtains the target query instruction, it first searches from the memory, and if it is not found in the memory, it searches for logs that meet the target query condition from the target log subset stored in the server. The above technical solution can improve the query efficiency.

[0052] In an exemplary embodiment, when the one or more target log subsets are stored in the memory of the target server, the clustering labels corresponding to the one or more target log subsets are stored in the memory, wherein the clustering labels stored in the memory are used when performing a target query operation in response to the target query instruction.

[0053] In an exemplary embodiment, after storing the one or more target log subsets in the memory of the target server, it also includes: when the target server obtains a target query instruction sent by the target account, and the target query instruction indicates a tag to be matched and a target query condition, in response to the target query instruction, querying the one or more target log subsets stored in the memory for a target log subset whose clustering tag matches the tag to be matched, wherein the tag to be matched is a tag of the target account; when the target log subset whose clustering tag matches the tag to be matched is queried, querying the queried target log subset for logs that meet the target query condition.

[0054] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. In order to better understand the above-mentioned data query method, the above-mentioned process is described below in conjunction with the embodiments, but it is not intended to limit the technical solution of the embodiments of the present invention, specifically:

[0055] In an alternative embodiment, Figure 4 This is a schematic diagram according to an embodiment of the present invention, specifically:

[0056] Step 1: Determine each cluster (equivalent to the log subset in the above embodiment): obtain the log source file from the server (equivalent to the target log set in the above embodiment), count the user data into different clusters according to the time period, find the central object in different clusters (equivalent to the clustering result in the above embodiment), and thus count each cluster in different time periods;

[0057] Step 2: Perform intersection operation: Figure 5 is a schematic diagram of a data filtering algorithm according to an embodiment of the present invention, through the central object UBehavior data filtering algorithm principle (such as Figure 5 As shown), each cluster in different time periods is counted for intersection operation, and the intersection result information and data information with high similarity to the intersection information (that is, data that is not within the intersection range but is closest to the intersection center object) are calculated, thereby determining the hotspot data (equivalent to the target intersection result in the above embodiment);

[0058] Step 3: Set hotspot data: put the hotspot data into the cache (equivalent to memory) so that the next time you query the data, you can query the hotspot data first. If the data is not found, query the database information, which improves data query efficiency; clear the log information of non-hotspot data to reduce the server memory size.

[0059] In addition, this embodiment also has the following steps:

[0060] Step 4: Compare query efficiency: Figure 6 is a data processing effect diagram according to an embodiment of the present invention, such as Figure 6 As described above, the data filtering algorithm based on the central object UBehavior improves the data query efficiency and greatly reduces the size of the log, thereby saving memory space. When verifying the scalability of the memory, by increasing the central object data node, the memory space can be increased by an average of 1.5 times. It can be seen that user data processing improves the scalability of the memory.

[0061] Step 5: Analyze and summarize the data processing method: The data processing module proposes a data filtering algorithm based on the central object UBehavior. The algorithm pre-processes the data and forms each cluster according to the chronological order of the time period in which the data appears. The clusters of different time periods are intersected to obtain the intersection results and the information with a high similarity to the intersection results as hot data, which is convenient for querying data. The log information of non-hot data is deleted to reduce the server memory size, save space for other server information, and lay the foundation for subsequent analysis and processing.

[0062] It should be noted that this embodiment proposes a method for processing log hotspot data based on a clustering algorithm, and proposes a data filtering algorithm based on a central object UBehavior in the data processing model. The algorithm introduces the intersection of user attribute data and user behavior data (such as Figure 5 ), the intersection result information and the information with high similarity to the intersection information are stored in the cache to facilitate data query; the information that is not the intersection result is filtered out, and the logs of non-intersection information are removed from the server log information to reduce the memory usage.

[0063] In addition, this embodiment introduces clustering algorithm technology to address load issues and ensure the uniformity and accuracy of data processing. It is based on a parallel distributed user data processing method and has strong scalability. And when bandwidth permits, user data can be processed by simply adding central object nodes to improve data query efficiency. At present, most user log data processing is based on user consumption patterns or habits to conduct data mining, so as to recommend products that users are interested in. This solution provides a solution for processing user data, and introduces a central object UBehavior data filtering algorithm for processing, determines hot data in massive user log data, ensures consistency between user data and hot data, and ensures data accuracy.

[0064] In addition, the above technical solution of the embodiment of the present invention has strong scalability: in terms of service pressure, by introducing the k-means algorithm technology in the clustering algorithm, a relationship with high object similarity in the same cluster is established, the consumption pattern or habit of each type of consumer is guaranteed, and the common pattern or habit data of consumers is found. And based on the distributed series clustering method of parallel data nodes, the data object nodes are vertically expanded, and the all-round intelligent role empowerment has extremely strong scalability. It lays a strong foundation for subsequent big data research and processing.

[0065] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0066] In the present embodiment, a data query device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the equipment described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0067] Figure 7 1 is a structural block diagram of a data query device according to an embodiment of the present invention (I), the device comprising:

[0068] An acquisition module 70, configured to acquire a target log set, wherein each log in the target log set includes data of a set of parameters, wherein the set of parameters includes a user identifier and a time parameter;

[0069] A division module 72, configured to divide the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is within a corresponding time period;

[0070] A clustering module 74, configured to cluster the plurality of log subsets respectively to obtain a plurality of clustering results, wherein each of the plurality of clustering results comprises a cluster obtained by clustering the corresponding log subset and a clustering label corresponding to the cluster, and each cluster comprises one or more logs;

[0071] A processing module 76 is used to perform an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result;

[0072] The storage module 78 is used to store the one or more target log subsets in the memory of the target server when the target intersection result includes one or more target log subsets, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains the target query instruction.

[0073] Through the above device, a target log set is obtained, and the target log set is divided into multiple log subsets corresponding to multiple time periods, and then the multiple log subsets are clustered respectively to obtain multiple groups of clustering results, and then the cluster clusters with the same clustering label in the multiple groups of clustering results are subjected to intersection operation according to the user identifier to obtain the target intersection result, and when the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried first when the target server obtains the target query instruction. The above technical solution solves the problem that when the server queries log data, it directly queries from the database of the hard disk, resulting in low query efficiency. Through the above technical solution, part of the data that meets the requirements is stored in the memory of the server, and then when the query instruction is obtained, it is searched in the memory first, thereby improving the query efficiency.

[0074] In an exemplary embodiment, the clustering module 74 is also used to perform the following operations on each log subset of the multiple log subsets, wherein when performing the following operations, each log subset is a current log subset: clustering the current log subset according to the data of part or all of the parameters in the set of parameters to obtain a current set of clustering results, wherein, when the current set of clustering results includes at least one current clustering cluster, the feature distance between the log features of each log in the current clustering cluster determined according to the data of part or all of the parameters meets the preset clustering conditions.

[0075] In an exemplary embodiment, the clustering module 74 is also used to, when the current set of clustering results includes at least one current clustering cluster, set a corresponding current clustering label for the current clustering cluster; or when the current set of clustering results includes at least one current clustering cluster, determine the current clustering label corresponding to the current clustering cluster in the target label set based on the data of the set of parameters included in each log in the current clustering cluster and the preset clustering condition.

[0076] In an exemplary embodiment, the processing module 76 is also used to, when the multiple groups of clustering results are N groups of clustering results, perform the following operations on the j-th cluster in the i-th group of clustering results in the N groups of clustering results, wherein N is a positive integer greater than or equal to 2, 1≤i≤N, the i-th group of clustering results includes Mi clusters, M is a positive integer greater than or equal to 1, 1≤j≤M, and the clustering label of the j-th cluster is the current clustering label: search for a cluster whose clustering label is the current clustering label in the N-1 groups of clustering results other than the i-th group of clustering results in the N groups of clustering results; when a cluster whose clustering label is the current clustering label is found, perform an intersection operation on the j-th cluster and the found cluster according to the user identifier to obtain a current intersection result; wherein, when the current intersection result includes at least 2 logs, determine the at least 2 logs as a subset of the target logs.

[0077] In an exemplary embodiment, the clustering module 74 is also used to cluster the target log set according to the data of part or all of the parameters in the set of parameters when the target intersection result does not include a log subset with an intersection, to obtain one or more cluster clusters and cluster labels corresponding to the one or more cluster clusters.

[0078] Figure 8 2 is a structural block diagram of a data query device according to an embodiment of the present invention. The device further includes: a query module 80.

[0079] In an exemplary embodiment, the query module 80 is also used to, when the target server obtains a target query instruction and the target query instruction indicates a target query condition, query the one or more target log subsets stored in the memory for logs that meet the target query condition in response to the target query instruction; and when no logs that meet the target query condition are found in the one or more target log subsets stored in the memory, query the target log collection for logs that meet the target query condition.

[0080] In an exemplary embodiment, the storage module 78 is also used to store clustering labels corresponding to the one or more target log subsets in the memory of the target server when the one or more target log subsets are stored in the memory, wherein the clustering labels stored in the memory are used when performing a target query operation in response to the target query instruction.

[0081] In an exemplary embodiment, the query module 80 is also used to, when the target server obtains a target query instruction sent by the target account and the target query instruction indicates a tag to be matched and a target query condition, query the one or more target log subsets stored in the memory for a target log subset whose clustering tag matches the tag to be matched in response to the target query instruction, wherein the tag to be matched is a tag of the target account; and when the target log subset whose clustering tag matches the tag to be matched is queried, query the logs that meet the target query condition in the queried target log subset.

[0082] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0083] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0084] S1, obtaining a target log set, wherein each log in the target log set includes data of a set of parameters, and the set of parameters includes a user identifier and a time parameter;

[0085] S2, dividing the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is within the corresponding time period;

[0086] S3, clustering the multiple log subsets respectively to obtain multiple groups of clustering results, wherein each group of clustering results in the multiple groups of clustering results includes a cluster cluster obtained by clustering the corresponding log subset and a cluster label corresponding to the cluster cluster, and each cluster cluster includes one or more logs;

[0087] S4, performing an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result;

[0088] S5. When the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains a target query instruction.

[0089] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0090] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0091] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0092] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0093] S1, obtaining a target log set, wherein each log in the target log set includes data of a set of parameters, and the set of parameters includes a user identifier and a time parameter;

[0094] S2, dividing the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is within the corresponding time period;

[0095] S3, clustering the multiple log subsets respectively to obtain multiple groups of clustering results, wherein each group of clustering results in the multiple groups of clustering results includes a cluster cluster obtained by clustering the corresponding log subset and a cluster label corresponding to the cluster cluster, and each cluster cluster includes one or more logs;

[0096] S4, performing an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result;

[0097] S5. When the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains a target query instruction.

[0098] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0099] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0100] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data query method, characterized in that: include: Acquire a target log set, wherein each log in the target log set includes data of a set of parameters, and the set of parameters includes a user identifier and a time parameter; According to the data of the time parameter in each log, the target log set is divided into a plurality of log subsets corresponding to a plurality of time periods, wherein the data of the time parameter included in the logs in each log subset is within the corresponding time period; Clustering the multiple log subsets respectively to obtain multiple groups of clustering results, wherein each group of clustering results in the multiple groups of clustering results includes a cluster cluster obtained by clustering the corresponding log subset and a cluster label corresponding to the cluster cluster, and each cluster cluster includes one or more logs; Performing an intersection operation on clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result; In the case where the target intersection result includes one or more target log subsets, the one or more target log subsets are stored in the memory of the target server, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains a target query instruction.

2. The method according to claim 1, characterized in that The clustering of the multiple log subsets is performed respectively to obtain multiple groups of clustering results, including: Perform the following operations on each of the multiple log subsets, respectively, wherein when performing the following operations, each log subset is a current log subset: According to the data of part or all of the parameters in the set of parameters, the current log subset is clustered to obtain a current set of clustering results, wherein, when the current set of clustering results includes at least one current clustering cluster, the feature distance between the log features of each log in the current clustering cluster determined according to the data of part or all of the parameters meets the preset clustering conditions.

3. The method according to claim 2, characterized in that The method further comprises: In a case where the current set of clustering results includes at least one current clustering cluster, setting a corresponding current clustering label for the current clustering cluster; or In the case where the current set of clustering results includes at least one current clustering cluster, the current clustering label corresponding to the current clustering cluster is determined in the target label set according to the set of parameter data included in each log in the current clustering cluster and the preset clustering condition.

4. The method according to claim 1, characterized in that The performing an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result includes: When the plurality of clustering results is N groups of clustering results, for the j-th cluster in the i-th group of clustering results in the N groups of clustering results, the following operations are performed, wherein N is a positive integer greater than or equal to 2, 1≤i≤N, and the i-th group of clustering results includes M i clusters, M is a positive integer greater than or equal to 1, 1≤j≤M, and the cluster label of the jth cluster is the current cluster label: Searching for a cluster whose clustering label is the current clustering label in N-1 groups of clustering results other than the i-th group of clustering results in the N groups of clustering results; In the case of finding a cluster whose cluster label is the current cluster label, performing an intersection operation on the j-th cluster and the found cluster according to the user identifier to obtain a current intersection result; When the current intersection result includes at least 2 logs, the at least 2 logs are determined as a target log subset.

5. The method according to claim 1, characterized in that: The method further comprises: When the target intersection result does not include a log subset with an intersection, the target log set is clustered according to data of part or all of the parameters in the set of parameters to obtain one or more cluster clusters and cluster labels corresponding to the one or more cluster clusters.

6. The method according to claim 1, characterized in that After storing the one or more target log subsets in the memory of the target server, the method further includes: When the target server obtains a target query instruction and the target query instruction indicates a target query condition, in response to the target query instruction, searching for logs that meet the target query condition in the one or more target log subsets stored in the memory; When no log meeting the target query condition is found in the one or more target log subsets stored in the memory, the target log set is searched for a log meeting the target query condition.

7. The method according to claim 1, characterized in that The method further comprises: When storing the one or more target log subsets in the memory of the target server, the clustering labels corresponding to the one or more target log subsets are stored in the memory, wherein the clustering labels stored in the memory are used when performing a target query operation in response to the target query instruction.

8. The method according to claim 7, characterized in that After storing the one or more target log subsets in the memory of the target server, the method further includes: When the target server obtains a target query instruction sent by a target account, and the target query instruction indicates a tag to be matched and a target query condition, in response to the target query instruction, searching the one or more target log subsets stored in the memory for a target log subset whose clustering tag matches the tag to be matched, wherein the tag to be matched is a tag of the target account; In the case where the target log subset whose clustering label matches the to-be-matched label is found, the found target log subset is searched for logs that meet the target query condition.

9. A data query device, characterized in that: include: An acquisition module, used for acquiring a target log set, wherein each log in the target log set includes data of a set of parameters, and the set of parameters includes a user identifier and a time parameter; A partitioning module, configured to divide the target log set into a plurality of log subsets corresponding to a plurality of time periods according to the data of the time parameter in each log, wherein the data of the time parameter included in the logs in each log subset is within a corresponding time period; A clustering module, used to cluster the multiple log subsets respectively to obtain multiple groups of clustering results, wherein each group of clustering results in the multiple groups of clustering results includes a cluster cluster obtained by clustering the corresponding log subset and a cluster label corresponding to the cluster cluster, and each cluster cluster includes one or more logs; A processing module, configured to perform an intersection operation on the clusters with the same clustering label in the multiple groups of clustering results according to the user identifier to obtain a target intersection result; A storage module is used to store the one or more target log subsets in the memory of the target server when the target intersection result includes one or more target log subsets, wherein the one or more target log subsets stored in the memory of the target server are used to be queried preferentially when the target server obtains a target query instruction.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 8 when executed.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 8 through the computer program.

Images