Log classification method and device, equipment and medium

Abstract

The invention discloses a log classification method and device, equipment and a medium. The method comprises the following steps: extracting a first sample set from an error log set, and clustering the first sample set according to K initial centroids of the first sample set to obtain K clusters; eliminating noise points in the first sample set to obtain a second sample set; according to the density and the distance product, mk error logs are screened out from the second sample set; determining K initial centroids of the error log set according to a cosine distance between a center error log of the mk error logs and the error logs; and clustering the error log set according to the K initial centroids to obtain K clusters. According to the embodiment of the invention, screening and calculation can be carried out based on the related information of the error logs in the sample set after the noise points are removed, the K initial centroids which are uniformly distributed are quickly determined to cluster each error log after vectorization processing, the clustering convergence speed is accelerated, and the clustering effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a log classification method, device, equipment and medium. Background Art

[0002] When each user uses the application provided by the financial institution, each user will send an error log used to record the error problem to the business system of the financial institution through the application. The error log can be a text used to describe the error problem that occurred. The business system of the financial institution vectorizes each error log received, and locates the error cause that caused the error problem by analyzing each error log after vectorization. The received error log usually contains a large number of error logs. In order to improve the efficiency of locating the cause of the error, the business system of the financial institution needs to classify each error log after vectorization and divide each error log after vectorization into multiple groups. Similar error logs will be divided into one group. Similar error logs in each group are usually error logs with the same error type. The business system of the financial institution can analyze the error logs in each group, quickly determine the error cause that caused various error problems, and form a solution.

[0003] In the related art, a commonly used log classification scheme is: randomly extract K error logs from each error log after vectorization processing, and then use the randomly extracted K error logs as the initial centroids for clustering by the K-means clustering algorithm, cluster each error log after vectorization processing according to the initial centroids by the K-means clustering algorithm, and divide each error log after vectorization processing into K clusters, thereby dividing each error log after vectorization processing into multiple groups. The log classification scheme in the related art randomly extracts the initial centroids for clustering by the K-means clustering algorithm, which will cause the initial centroid distribution to be too dispersed or concentrated, and not particularly uniform, so that the clustering convergence speed is slow and the clustering effect is not good when clustering each error log after vectorization processing, resulting in low log classification efficiency. Summary of the invention

[0004] The present invention provides a log classification method, device, equipment and medium to solve the problem that the log classification scheme in the related art randomly extracts the initial centroid of the K-means clustering algorithm for clustering, which causes the initial centroid distribution to be too dispersed or concentrated and not particularly uniform, thereby making the clustering convergence speed slow and the clustering effect poor when clustering the error logs after vectorization processing, resulting in low log classification efficiency.

[0005] According to one aspect of the present invention, a log classification method is provided, comprising:

[0006] Extracting a first sample set from the error log set, determining K initial centroids of the first sample set, and clustering the first sample set according to the K initial centroids of the first sample set to obtain K clusters of the first sample set;

[0007] Eliminate noise points in the first sample set to obtain a second sample set;

[0008] Filtering out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs;

[0009] Determine K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs;

[0010] The error log set is clustered according to the K initial centroids of the error log set to obtain K clusters of the error log set.

[0011] According to another aspect of the present invention, there is provided a log classification device, comprising:

[0012] A sample set extraction module, configured to extract a first sample set from an error log set, determine K initial centroids of the first sample set, and cluster the first sample set according to the K initial centroids of the first sample set to obtain K clusters of the first sample set;

[0013] A noise point removal module, used to remove noise points in the first sample set to obtain a second sample set;

[0014] A preliminary extraction module, configured to filter out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs;

[0015] A fine extraction module, used for determining K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs;

[0016] The log clustering module is used to cluster the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor;

[0019] and a memory communicatively coupled to the at least one processor;

[0020] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log classification method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the log classification method described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the log classification method according to any embodiment of the present invention is implemented.

[0023] The technical solution of the embodiment of the present invention extracts a first sample set from an error log set, determines K initial centroids of the first sample set, clusters the first sample set according to the K initial centroids of the first sample set, and obtains K clusters of the first sample set; then removes noise points in the first sample set to obtain a second sample set; selects mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; determines K initial centroids of the error log set according to the cosine distance between the central error log of the mk error logs and the error logs; finally, clusters the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set, thereby solving the problem in the related art that the log classification scheme randomly extracts the initial centroids for clustering using the K-means clustering algorithm, which causes the initial centroids to be too dispersed or concentrated and not particularly uniform, thereby causing the clustering convergence speed to be slow and the clustering to be uneven when clustering the error logs after vectorization. In order to solve the problem of poor effect and low efficiency of log classification, some error logs can be randomly selected from multiple vectorized error logs in the error log set that need to be classified as the sample set, and the noise points in the sampled sample set can be eliminated. The relevant information of the error logs in the sample set after the noise points are eliminated can be screened and calculated to quickly determine K error logs with uniform distribution as the K initial centroids required to cluster the vectorized error logs in the error log set through the K-means clustering algorithm, so as to ensure the quality and speed of the initial centroid selection, and then the vectorized error logs in the error log set can be clustered according to the determined high-quality uniformly distributed K initial centroids, and the vectorized error logs can be quickly and accurately divided into K clusters, so that the vectorized error logs can be quickly and accurately divided into multiple groups, which speeds up the clustering convergence speed, improves the clustering effect, and has high log classification efficiency.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 The present invention provides a flow chart of a log classification method according to the first embodiment of the present invention.

[0027] Figure 2 This is a flow chart of a log classification method provided in Embodiment 2 of the present invention.

[0028] Figure 3 A structural diagram of a log classification device provided in Embodiment 3 of the present invention.

[0029] Figure 4 A schematic diagram of the structure of an electronic device for implementing the log classification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "target", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise", "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data comply with relevant laws, regulations and standards in relevant regions.

[0033] Embodiment 1

[0034] Figure 1 A flow chart of a log classification method provided in the first embodiment of the present invention. This embodiment is applicable to classifying each error log after vectorization processing, and dividing each error log after vectorization processing into multiple groups. The method can be executed by a log classification device, which can be implemented in the form of hardware and / or software, and the log classification device can be configured in the business system of a financial institution. The business system of a financial institution can be a server set up in the financial institution for processing the business of the financial institution. Figure 1 As shown, the method includes:

[0035] Step 101: extract a first sample set from an error log set, determine K initial centroids of the first sample set, cluster the first sample set according to the K initial centroids of the first sample set, and obtain K clusters of the first sample set.

[0036] Optionally, when each user uses the application provided by the financial institution, each user will send an error log for recording the error problems that occurred to the business system of the financial institution through the application. The error log can be a text for describing the error problems that occurred when the user used the application provided by the financial institution. The error problems include but are not limited to errors in the services provided by the application and errors in the data provided by the application. The vectorized error log is an error log that has been converted into a text vector after vectorization.

[0037] Optionally, the error log set is composed of a plurality of vectorized error logs that need to be classified at the current moment. It is necessary to classify all error logs in the error log set, and divide all error logs in the error log set into a plurality of groups.

[0038] Optionally, the first sample set may be a target number of error logs extracted from the error log set. The target number is less than the total number of error logs in the error log set. Exemplarily, the error log set consists of 1,000 error logs that need to be classified at the current moment. The target number is 150. The first sample set is 150 error logs extracted from the error log set. All error logs in the first sample set account for 15% of the error log set.

[0039] Optionally, extracting the first sample set from the error log set includes: randomly extracting a target number of error logs from the error log set as the first sample set. The target number of error logs may be randomly extracted from the error log set using a preset random sampling algorithm, and the extracted target number of error logs may be determined as the first sample set. The preset random sampling algorithm may be a pre-set algorithm for randomly extracting error logs from the error log set.

[0040] Therefore, some error logs can be randomly selected from the error log set as a sample set, so as to facilitate the subsequent rapid determination of the uniformly distributed initial centroid of the error log set based on the sample set obtained by sampling.

[0041] Specifically, the centroid is the cluster center that needs to be used when clustering data using the K-means clustering algorithm, and the K initial centroids are the K initial cluster centers that need to be used when clustering data using the K-means clustering algorithm. The process of clustering multiple vectorized error logs using the K-means clustering algorithm is as follows: select K error logs from multiple error logs as K initial centroids; for each error log except each centroid, calculate the cosine distance between the error log and each centroid, assign the error log to the nearest centroid, and obtain K clusters; wherein the nearest centroid is the centroid with the smallest cosine distance to the error log; calculate the average value of all error logs in each cluster as the new centroid of each cluster; repeat the operations of assigning the error log to the nearest centroid and updating the centroid of each cluster until the centroid of each cluster no longer changes or reaches a predetermined number of iterations. K represents the number of clusters, and K is a positive integer. The error logs in each cluster are similar error logs. Each cluster is a group. Therefore, the multiple error logs are clustered by the K-means clustering algorithm, and the multiple error logs are divided into K clusters, thereby dividing the multiple error logs into multiple groups.

[0042] Optionally, the K initial centroids of the first sample set are the K initial centroids needed to be used when clustering the error logs in the first sample set by the K-means clustering algorithm. The K initial centroids of the first sample set may be K evenly distributed error logs selected by the target user from the first sample set. The target user may be a technician responsible for processing relevant information of the error log.

[0043] Optionally, determining the K initial centroids of the first sample set includes: providing the first sample set to a target user, and obtaining the K initial centroids of the first sample set fed back by the target user. The first sample set may be sent to a terminal device of the target user, so that the target user selects K evenly distributed error logs from the first sample set as the K initial centroids of the first sample set, and the selected K initial centroids of the first sample set are fed back through the terminal device. The terminal device of the target user may be a terminal device used by the target user.

[0044] Therefore, based on the manual selection of the target users, the K uniformly distributed initial centroids of the sample set obtained by sampling can be quickly determined.

[0045] Optionally, the K clusters of the first sample set are K clusters obtained after clustering the error logs in the first sample set according to the K initial centroids of the first sample set using a K-means clustering algorithm. After determining the K initial centroids of the first sample set, the error logs in the first sample set may be clustered according to the K initial centroids of the first sample set using a K-means clustering algorithm to divide the error logs in the first sample set into K clusters, thus obtaining the K clusters of the first sample set.

[0046] Specifically, the process of clustering the first sample set according to the K initial centroids of the first sample set by the K-means clustering algorithm to obtain the K clusters of the first sample set is: for each error log except each centroid in the first sample set, calculate the cosine distance between the error log and each centroid, assign the error log to the centroid with the closest distance, and obtain K clusters; wherein the centroid with the closest distance is the centroid with the smallest cosine distance to the error log; calculate the average value of all error logs in each cluster as the new centroid of each cluster; repeatedly perform the operations of assigning the error log to the centroid with the closest distance and updating the centroid of each cluster until the centroid of each cluster no longer changes or reaches a predetermined number of iterations. Thus, the error logs in the first sample set are clustered by the K-means clustering algorithm, and the error logs in the first sample set are divided into K clusters, and the K clusters of the first sample set are obtained.

[0047] Step 102: Remove noise points in the first sample set to obtain a second sample set.

[0048] Optionally, the average distance of each cluster of the first sample set may refer to the ratio of the total number of samples of the cluster to the cumulative distance of the cluster. The total number of samples of the cluster may refer to the total number of error logs in the cluster. The cumulative distance of the cluster may refer to the sum of the cosine distances between each error log in the cluster except the cluster center of the cluster and the cluster center of the cluster. The noise point in the first sample set may refer to the abnormal error log in the first sample set that deviates from the overall pattern or trend of the first sample set. The second sample set may refer to the first sample set after the noise point removal process.

[0049] Optionally, removing noise points in the first sample set to obtain a second sample set includes: determining an average distance of each cluster of the first sample set; determining a global average distance of the first sample set based on the average distance of each cluster of the first sample set; determining a density of each error log in the first sample set based on the global average distance of the first sample set; and removing noise points in the first sample set based on the density of each error log to obtain the second sample set.

[0050] Therefore, based on the average distance of each cluster of the sample set obtained by sampling, the global average distance of the sample set obtained by sampling can be determined, and then based on the global average distance, the density of each error log in the sample set obtained by sampling can be determined, and the noise points in the sample set obtained by sampling can be eliminated according to the density, thereby improving the accuracy of the subsequent process of quickly determining the uniformly distributed initial centroid of the error log set based on the sample set obtained by sampling.

[0051] Optionally, for each of the K clusters in the first sample set, the total number of error logs in the cluster can be calculated to obtain the total number of samples in the cluster. Then, the cosine distance between each error log in the cluster except the cluster center of the cluster and the cluster center of the cluster is calculated, and the calculated cosine distance between each error log and the cluster center of the cluster is accumulated to obtain the accumulated distance of the cluster. The preset cosine distance calculation formula can be used to calculate the cosine distance between two vectorized error logs. The preset cosine distance calculation formula can be a pre-set formula for calculating the cosine distance between two vectorized error logs. Finally, the ratio of the total number of samples in the cluster to the accumulated distance of the cluster is calculated to obtain the average distance of the cluster. Thus, the average distance of each cluster in the K clusters of the first sample set is determined. For any cluster in the first sample set, the average distance of the cluster can be expressed as: R A =N A / DistA, where R A is the average distance of the clusters, N A is the total number of samples in the cluster, and DistA is the cumulative distance of the cluster.

[0052] Optionally, the global average distance of the first sample set may refer to the ratio of the sum of the average distances of the K clusters of the first sample set to the number of clusters of the first sample set. The number of clusters of the first sample set is the number of clusters obtained after clustering the error logs in the first sample set. The number of clusters of the first sample set is K. The average distances of the K clusters of the first sample set may be first accumulated to obtain the sum of the average distances of the K clusters of the first sample set, and then the ratio of the sum of the average distances of the K clusters of the first sample set to the number of clusters of the first sample set may be calculated to obtain the global average distance of the first sample set. Thus, the global average distance of the first sample set is determined. Exemplarily, the K clusters of the first sample set are cluster 1, cluster 2, cluster 3, cluster 4, ..., cluster K-1, and cluster K. The average distances of the K clusters of the first sample set are R1, R2, R3, R4, ..., R K-1 , R K The global average distance of the first sample set can be expressed as: R avg =(R1+R2+R3+R4+……+R K-1 +R K ) / K, where R avg is the global average distance of the first sample set, (R1+R2+R3+R4+…+R K-1 +R K ) is the sum of the average distances of the K clusters in the first sample set, and K is the number of clusters in the first sample set.

[0053] Optionally, for each error log in the first sample set, the density of the error log may refer to the ratio of the cumulative distance of the close logs of the error log to the parameter of the number of close logs of the error log. The close log of the error log may refer to the error log in the first sample set, except the error log, whose cosine distance with the error log is less than the global average distance of the first sample set. The cumulative distance of the close logs of the error log is the sum of the cosine distances between each close log of the error log and the error log. The parameter of the number of close logs of the error log may be the total number of close logs of the error log plus 1. The total number of close logs of the error log is the total number of close logs of the error log. For each error log in the first sample set, the cosine distance between each error log in the first sample set except the error log and the error log is calculated first, and the error log whose cosine distance with the error log is less than the global average distance of the first sample set is determined as the close log of the error log. Then, the cosine distances between each close log and the error log are accumulated to obtain the cumulative distance of the close logs of the error log. The total number of the close logs of the error log is calculated to obtain the total number of the close logs of the error log. The total number of close logs of the error log is added by 1 to obtain the close log number parameter of the error log. Finally, the ratio of the close log cumulative distance of the error log to the close log number parameter of the error log is calculated to obtain the density of the error log. Thus, the density of each error log in the first sample set is determined. For any error log in the first sample set, the density of the error log can be expressed as: Den = (close log cumulative distance of the error log) / (total number of close logs of the error log + 1), where Den is the density of the error log.

[0054] Optionally, according to the density of each error log, the noise points in the first sample set are eliminated to obtain the second sample set, including: performing the following operations for each error log in the first sample set: determining the error log and the error log in the first sample set except the error log, the error log with the smallest cosine distance with the error log as a pair of neighbor error logs; calculating the difference between the densities of the two error logs in a pair of neighbor error logs; judging whether the difference between the densities of the two error logs in a pair of neighbor error logs is greater than a preset value; if so, determining the error log with a smaller density in a pair of neighbor error logs as a noise point, and deleting the error log with a smaller density in a pair of neighbor error logs from the first sample set; if not, retaining the two error logs in a pair of neighbor error logs. The preset value is a preset value. Generally, if the difference between the densities of the two error logs in a pair of neighbor error logs is greater than a preset value, it can be determined that the error log with a smaller density in each pair of neighbor error logs is a noise point in the first sample set. The first sample set after completing the above operations is the second sample set. Exemplarily, error log A and error log B are a pair of neighbor error logs in the first sample set. The preset value is T. The density of error log A is DenA, and the density of error log B is DenB. If the difference between DenA and DenB is greater than the preset value T, DenA is greater than DenB, then the error log B is deleted from the first sample set. If the difference between DenA and DenB is greater than the preset value T, DenA is less than DenB, then the error log A is deleted from the first sample set.

[0055] Step 103: Filter out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs.

[0056] Optionally, mk is a positive integer greater than or equal to 3, and mk can be adjusted according to the total number of error logs in the second sample set. mk is less than the target number, and mk is greater than K.

[0057] Optionally, based on the density of error logs in the second sample set and the distance product between error logs, mk error logs are screened out from the second sample set, including: determining the error log with the largest density among the error logs in the second sample set as the first error log, and moving the first error log to the first storage location; determining the error log with the largest cosine distance from the first error log among the remaining error logs in the second sample set as the second error log, and moving the second error log to the first storage location; determining the target error log with the largest distance product from the error log in the first storage location among the remaining error logs in the second sample set, and moving the target error log to the first storage location; judging whether the number of error logs in the first storage location is equal to mk; if the number of error logs in the first storage location is not equal to mk, returning to executing the operation of determining the target error log with the largest distance product from the error log in the first storage location among the remaining error logs in the second sample set, and moving the target error log to the first storage location until the number of error logs in the first storage location is equal to mk.

[0058] Therefore, based on the density of error logs in the sample set after noise points are removed and the distance product between error logs, some error logs can be screened out from the sample set after noise points are removed as the initial centroid of the candidate error log set.

[0059] Optionally, the error log with the highest density among the error logs in the second sample set can be determined as the first error log screened out from the second sample set. Then, the first error log is taken out from the second sample set and stored in the first storage location, thereby moving the first error log from the second sample set to the first storage location. The first storage location can be a pre-set file or storage unit for storing error logs screened out from the second sample set. In the initial state, the number of error logs in the first storage location is 0. After the first error log is moved from the second sample set to the first storage location, the number of error logs in the first storage location is updated to 1. Each time an error log is moved from the second sample set to the first storage location, the number of error logs in the first storage location is increased by 1.

[0060] Optionally, the cosine distance between each remaining error log in the second sample set and the first error log can be calculated, and the error log with the largest cosine distance with the first error log among the remaining error logs in the second sample set is determined as the second error log screened out from the second sample set. Then, the second error log is taken out from the second sample set and stored in the first storage location, so that the second error log is moved from the second sample set to the first storage location, and the number of error logs in the first storage location is increased by 1.

[0061] Optionally, for each error log remaining in the second sample set, the distance product between the error log and the error log in the first storage location is the product of the cosine distances between the error log and each error log in the first storage location. The distance product between each error log remaining in the second sample set and the error log in the first storage location can be calculated to determine the target error log with the largest distance product between each error log remaining in the second sample set and the error log in the first storage location. The target error log is an error log newly selected from the second sample set. Then, the target error log is taken out from the second sample set and stored in the first storage location, thereby moving the target error log from the second sample set to the first storage location, and the number of error logs in the first storage location is increased by 1.

[0062] Optionally, after each time the target error log is moved to the first storage location, it can be determined whether the number of error logs in the first storage location is equal to mk. If the number of error logs in the first storage location is equal to mk, it indicates that the total number of error logs screened out from the second sample set is equal to mk, and it can be determined that mk error logs have been screened out from the second sample set. If the number of error logs in the first storage location is not equal to mk, it indicates that the total number of error logs screened out from the second sample set is less than mk, and further screening is required, then the operation of returning to determine the target error log with the largest distance product between the remaining error logs in the second sample set and the error log in the first storage location, and moving the target error log to the first storage location, until the number of error logs in the first storage location is equal to mk.

[0063] Exemplarily, according to the density of the error logs in the second sample set and the distance product between the error logs, the process of selecting mk error logs from the second sample set is as follows: first, the error log with the largest density among the error logs in the second sample set is determined as the first error log Y1 selected from the second sample set, the first error log Y1 is moved from the second sample set to the first storage position, and the number of error logs in the first storage position is increased by 1. Then, the error log with the largest cosine distance from the first error log Y1 among the remaining error logs in the second sample set is determined as the second error log Y2 selected from the second sample set, the second error log Y2 is moved from the second sample set to the first storage position, and the number of error logs in the first storage position is increased by 1. For each error log remaining in the second sample set, the distance product between the error log and the first error log Y1 and the second error log Y2 in the first storage position is calculated, and the distance product = the cosine distance between the error log and the first error log Y1 * the cosine distance between the error log and the second error log Y2. The error log with the largest distance product between the remaining error logs in the second sample set and the first error log Y1 and the second error log Y2 in the first storage position is determined as the third error log Y3 selected from the second sample set, and the third error log Y3 is moved from the second sample set to the first storage position, and the number of error logs in the first storage position is increased by 1. It is determined whether the number of error logs in the first storage position is equal to mk. If the number of error logs in the first storage position is not equal to mk, for each error log remaining in the second sample set, the distance product between the error log and the first error log Y1, the second error log Y2, and the third error log Y3 in the first storage position is calculated, and the distance product = the cosine distance between the error log and the first error log Y1 * the cosine distance between the error log and the second error log Y2 * the cosine distance between the error log and the third error log Y3. The error log with the largest distance product between the remaining error logs in the second sample set and the first error log Y1, the second error log Y2, and the third error log Y3 in the first storage location is determined as the fourth error log Y4 selected from the second sample set, and the fourth error log Y4 is moved from the second sample set to the first storage location, and the number of error logs in the first storage location is increased by 1. This process is repeated until the mkth error log Ymk selected from the second sample set is moved from the second sample set to the first storage location, and the number of error logs in the first storage location is equal to mk, and it is determined that mk error logs have been selected from the second sample set, and the process ends.

[0064] Step 104: Determine K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs.

[0065] Optionally, the K initial centroids of the error log set are K initial centroids that need to be used when clustering each error log in the error log set after vectorization processing using a K-means clustering algorithm.

[0066] Optionally, based on the cosine distance between the central error log of the mk error logs and the error logs, K initial centroids of the error log set are determined, including: calculating the average value of the mk error logs to obtain the central error log of the mk error logs, and storing the central error log to a second storage location; moving the error log with the largest cosine distance from the central error log among the mk error logs to the second storage location; determining the eccentric distance of each remaining error log in the mk error logs based on the cosine distance between each remaining error log in the mk error logs and the error log in the second storage location, and moving the error log with the largest eccentric distance from the error log among the mk error logs to the second storage location; The target error log with the largest error log is moved to the second storage location; whether the number of error logs in the second storage location is equal to K; if the number of error logs in the second storage location is not equal to K, returning to execute the operation of determining the eccentric distance of each error log remaining in the mk error logs according to the cosine distance between each error log remaining in the mk error logs and the error log in the second storage location, and moving the target error log with the largest eccentric distance among each error log remaining in the mk error logs to the second storage location until the number of error logs in the second storage location is equal to K; wherein the K error logs in the second storage location are the K initial centroids of the error log set.

[0067] Therefore, based on the cosine distance between the central error log and the error logs of some error logs screened out from the sample set after removing noise points, K evenly distributed error logs can be determined as the K initial centroids needed to be used when clustering the vectorized error logs in the error log set through the K-means clustering algorithm.

[0068] Optionally, the central error log of mk error logs may refer to the average value of mk error logs. The average value of mk error logs may be calculated to obtain the central error log of mk error logs, and then the central error log of mk error logs is used as the initial centroid of the first determined error log set and stored in the second storage location. Exemplarily, the mk error logs are Y1, Y2, ..., Ymk. The central error log of mk error logs may be expressed as: Ycen = (Y1 + Y2 + ... + Ymk) / mk, where Ycen is the central error log of mk error logs. The second storage location may be a file or storage unit pre-set for storing K initial centroids of the error log set. In the initial state, the number of error logs in the second storage location is 0. The central error log is a new error log obtained by calculating the average value of mk error logs. After the central error log is stored in the second storage location, the number of error logs in the second storage location is updated to 1. Each time an error log is stored in the second storage location and moved from the second sample set to the second storage location, the number of error logs in the second storage location is increased by 1.

[0069] Optionally, for each of the mk error logs, the cosine distance between the error log and the central error log may be calculated, and the error log having the largest cosine distance from the central error log may be used as the initial centroid of the second determined error log set and moved to the second storage location.

[0070] Optionally, after storing the second initial centroid of the determined error log set in the second storage location, for each remaining error log in the mk error logs, the cosine distance between the error log and each error log in the second storage location is calculated, and the calculated cosine distance with the smallest value is determined as the centrifugal distance of the error log. The target error log with the largest centrifugal distance among the remaining error logs in the mk error logs is used as the initial centroid of the newly determined error log set and is moved to the second storage location.

[0071] Optionally, after each time the target error log is moved to the second storage location, it can be determined whether the number of error logs in the second storage location is equal to K. If the number of error logs in the second storage location is equal to K, it indicates that the total number of error logs screened from the second sample set is equal to K, and it can be determined that the K initial centroids of the error log set have been determined. If the number of error logs in the second storage location is not equal to K, it indicates that the total number of initial centroids of the error log set determined is less than K, and further determination is required, then the operation of returning to determine the centrifugal distance of each remaining error log in the mk error logs according to the cosine distance between each remaining error log in the mk error logs and the error log in the second storage location, and moving the target error log with the largest centrifugal distance among each remaining error log in the mk error logs to the second storage location, until the number of error logs in the second storage location is equal to K. The K error logs in the second storage location are the K initial centroids of the error log set.

[0072] Exemplarily, the process of determining the K initial centroids of the error log set according to the cosine distance between the central error log and the error logs of the mk error logs is as follows: the average value of the mk error logs is calculated to obtain the central error log Ycen of the mk error logs, and then the central error log Ycen is used as the initial centroid of the first determined error log set and stored in the second storage location, and the number of error logs in the second storage location is increased by 1. For each error log in the mk error logs, the cosine distance between the error log and the central error log is calculated, and the error log with the largest cosine distance from the central error log is used as the initial centroid Ycen2 of the second determined error log set and moved to the second storage location. For each remaining error log in the mk error logs, the cosine distance between the error log and the central error log Ycen and the initial centroid Ycen2 in the second storage location is calculated, and the calculated cosine distance with the smallest value is determined as the centrifugal distance of the error log. The target error log with the largest centrifugal distance among the remaining error logs of the mk error logs is used as the initial centroid Ycen3 of the third error log set determined, and is moved to the second storage location. For each error log remaining among the mk error logs, the cosine distance between the error log and the central error log Ycen, the initial centroid Ycen2, and the initial centroid Ycen3 in the second storage location is calculated, and the calculated cosine distance with the smallest value is determined as the centrifugal distance of the error log. The target error log with the largest centrifugal distance among the remaining error logs of the mk error logs is used as the initial centroid Ycen4 of the fourth error log set determined, and is moved to the second storage location. And so on, until for each of the remaining error logs in the mk error logs, the cosine distance between the error log and the central error log Ycen, initial centroid Ycen2, initial centroid Ycen3, Ycen4, ..., Ycenk-1 in the second storage location is calculated, and the calculated cosine distance with the smallest value is determined as the eccentric distance of the error log, and the target error log with the largest eccentric distance among the remaining error logs in the mk error logs is used as the initial centroid Ycenk of the Kth determined error log set, and is moved to the second storage location. The number of error logs in the second storage location is equal to K, and it is determined that the K initial centroids of the error log set have been determined, and the process ends. The K error logs in the second storage location are the K initial centroids of the error log set.

[0073] Step 105: Cluster the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set.

[0074] Optionally, the K clusters of the error log set are K clusters obtained after clustering the error logs in the error log set according to the K initial centroids of the error log set using a K-means clustering algorithm. After determining the K initial centroids of the error log set, the error logs in the error log set are clustered according to the K initial centroids of the error log set using a K-means clustering algorithm, and the error logs in the error log set are divided into K clusters, that is, the K clusters of the error log set are obtained.

[0075] Specifically, the error logs in the error log set are clustered according to the K initial centroids of the error log set by the K-means clustering algorithm, and the error logs in the error log set are divided into K clusters, that is, the process of obtaining the K clusters of the error log set is: for each error log in the error log set except each centroid, the cosine distance between the error log and each centroid is calculated, and the error log is assigned to the centroid with the closest distance to obtain K clusters; wherein the centroid with the closest distance is the centroid with the smallest cosine distance with the error log; the average value of all error logs in each cluster is calculated as the new centroid of each cluster; the operation of assigning the error log to the centroid with the closest distance and updating the centroid of each cluster is repeated until the centroid of each cluster no longer changes or reaches a predetermined number of iterations. Each cluster is a group. Thus, the error logs after vectorization processing in the error log set are clustered according to the determined K initial centroids with uniform distribution, and the error logs after vectorization processing are divided into K clusters, thereby dividing the error logs after vectorization processing into multiple groups.

[0076] The technical solution of the embodiment of the present invention extracts a first sample set from an error log set, determines K initial centroids of the first sample set, clusters the first sample set according to the K initial centroids of the first sample set, and obtains K clusters of the first sample set; then removes noise points in the first sample set to obtain a second sample set; selects mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; determines K initial centroids of the error log set according to the cosine distance between the central error log of the mk error logs and the error logs; finally, clusters the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set, thereby solving the problem in the related art that the log classification scheme randomly extracts the initial centroids for clustering using the K-means clustering algorithm, which causes the initial centroids to be too dispersed or concentrated and not particularly uniform, thereby causing the clustering convergence speed to be slow and the clustering to be uneven when clustering the error logs after vectorization. In order to solve the problem of poor effect and low efficiency of log classification, some error logs can be randomly selected from multiple vectorized error logs in the error log set that need to be classified as the sample set, and the noise points in the sampled sample set can be eliminated. The relevant information of the error logs in the sample set after the noise points are eliminated can be screened and calculated to quickly determine K error logs with uniform distribution as the K initial centroids required to cluster the vectorized error logs in the error log set through the K-means clustering algorithm, so as to ensure the quality and speed of the initial centroid selection, and then the vectorized error logs in the error log set can be clustered according to the determined high-quality uniformly distributed K initial centroids, and the vectorized error logs can be quickly and accurately divided into K clusters, so that the vectorized error logs can be quickly and accurately divided into multiple groups, which speeds up the clustering convergence speed, improves the clustering effect, and has high log classification efficiency.

[0077] Embodiment 2

[0078] Figure 2 This is a flow chart of a log classification method provided by Embodiment 2 of the present invention. This embodiment of the present invention can be combined with each optional solution in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0079] Step 201: extract a first sample set from an error log set.

[0080] Step 202: provide the first sample set to a target user, obtain K initial centroids of the first sample set fed back by the target user, cluster the first sample set according to the K initial centroids of the first sample set, and obtain K clusters of the first sample set.

[0081] Step 203: Determine the average distance of each cluster of the first sample set.

[0082] Step 204: Determine the global average distance of the first sample set according to the average distance of each cluster of the first sample set.

[0083] Step 205: Determine the density of each error log in the first sample set according to the global average distance of the first sample set.

[0084] Step 206: According to the density of each error log, remove the noise points in the first sample set to obtain a second sample set.

[0085] Step 207: Filter out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs.

[0086] Step 208: Determine K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs.

[0087] Step 209: Cluster the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set.

[0088] The technical solution of the embodiment of the present invention can quickly determine K uniformly distributed initial centroids of the sample set obtained by sampling based on manual selection of target users, cluster the sample set according to the K uniformly distributed initial centroids to obtain K clusters of the sample set obtained by sampling, and determine the global average distance of the sample set based on the average distance of each cluster of the sample set obtained by sampling, and then determine the density of each error log in the sample set obtained by sampling based on the global average distance, and eliminate noise points in the sample set obtained by sampling based on the density, thereby improving the accuracy of the subsequent process of quickly determining the uniformly distributed initial centroid of the error log set based on the sample set obtained by sampling.

[0089] Exemplarily, the code logic of the process of determining the initial centroid in the log classification method is:

[0090] Semi-supervised distance co-initialization kmeans

[0091] Input: The log sample data set is dataset

[0092] Output: K centroids

[0093] Artificially select K centroids to perform kmeans clustering on the dataset to obtain K clusters

[0094] Calculate the average distance Ri for each cluster i

[0095] The global average distance R is obtained by summing the average distances of all clusters and dividing by the number of clusters. avg

[0096] Calculate the density of each sample

[0097] Eliminate the noise points in the sample set dataset to obtain the sample set dataset1

[0098] In the initial extraction stage, find the sample Y1 with the highest density in the sample set dataset1, and obtain mk samples according to the maximum distance product principle.

[0099] In the fine extraction stage, calculate the center Ycen of mk samples, then find the sample with the farthest cosine distance from Ycen as Ycen2, and then use the minimum and maximum method until K centroids are selected

[0100] Print(K centroids)

[0101] Embodiment 3

[0102] Figure 3 This is a schematic diagram of the structure of a log classification device provided in Embodiment 3 of the present invention. The device can be configured in an electronic device. Figure 3 As shown, the device includes: a sample set extraction module 301, a noise point removal module 302, a preliminary extraction module 303, a fine extraction module 304 and a log clustering module 305.

[0103] Among them, the sample set extraction module 301 is used to extract a first sample set from the error log set, determine K initial centroids of the first sample set, cluster the first sample set according to the K initial centroids of the first sample set, and obtain K clusters of the first sample set; the noise point removal module 302 is used to remove noise points in the first sample set to obtain a second sample set; the preliminary extraction module 303 is used to screen out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; the fine extraction module 304 is used to determine the K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distance between the error logs; the log clustering module 305 is used to cluster the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set.

[0104] The technical solution of the embodiment of the present invention extracts a first sample set from an error log set, determines K initial centroids of the first sample set, clusters the first sample set according to the K initial centroids of the first sample set, and obtains K clusters of the first sample set; then removes noise points in the first sample set to obtain a second sample set; selects mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; determines K initial centroids of the error log set according to the cosine distance between the central error log of the mk error logs and the error logs; finally, clusters the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set, thereby solving the problem in the related art that the log classification scheme randomly extracts the initial centroids for clustering using the K-means clustering algorithm, which causes the initial centroids to be too dispersed or concentrated and not particularly uniform, thereby causing the clustering convergence speed to be slow and the clustering to be uneven when clustering the error logs after vectorization. In order to solve the problem of poor effect and low efficiency of log classification, some error logs can be randomly selected from multiple vectorized error logs in the error log set that need to be classified as the sample set, and the noise points in the sampled sample set can be eliminated. The relevant information of the error logs in the sample set after the noise points are eliminated can be screened and calculated to quickly determine K error logs with uniform distribution as the K initial centroids required to cluster the vectorized error logs in the error log set through the K-means clustering algorithm, so as to ensure the quality and speed of the initial centroid selection, and then the vectorized error logs in the error log set can be clustered according to the determined high-quality uniformly distributed K initial centroids, and the vectorized error logs can be quickly and accurately divided into K clusters, so that the vectorized error logs can be quickly and accurately divided into multiple groups, which speeds up the clustering convergence speed, improves the clustering effect, and has high log classification efficiency.

[0105] In an optional implementation of the embodiment of the present invention, optionally, when executing the operation of extracting the first sample set from the error log set, the sample set extraction module 301 is specifically used to: randomly extract a target number of error logs from the error log set as the first sample set.

[0106] In an optional implementation of an embodiment of the present invention, optionally, when performing the operation of determining the K initial centroids of the first sample set, the sample set extraction module 301 is specifically used to: provide the first sample set to a target user, and obtain the K initial centroids of the first sample set fed back by the target user.

[0107] In an optional implementation of an embodiment of the present invention, optionally, the noise point removal module 302 is specifically used to: determine the average distance of each cluster of the first sample set; determine the global average distance of the first sample set based on the average distance of each cluster of the first sample set; determine the density of each error log in the first sample set based on the global average distance of the first sample set; and remove noise points in the first sample set based on the density of each error log to obtain a second sample set.

[0108] In an optional implementation of the embodiment of the present invention, optionally, the preliminary extraction module 303 is specifically used to: determine the error log with the largest density among the error logs in the second sample set as the first error log, and move the first error log to the first storage position; determine the error log with the largest cosine distance from the first error log among the remaining error logs in the second sample set as the second error log, and move the second error log to the first storage position; determine the target error log with the largest distance product between the error logs remaining in the second sample set and the error log in the first storage position, and move the target error log to the first storage position; determine whether the number of error logs in the first storage position is equal to mk; if the number of error logs in the first storage position is not equal to mk, return to perform the operation of determining the target error log with the largest distance product between the error logs remaining in the second sample set and the error log in the first storage position, and move the target error log to the first storage position until the number of error logs in the first storage position is equal to mk.

[0109] In an optional implementation of an embodiment of the present invention, optionally, the fine extraction module 304 is specifically used to: calculate the average value of the mk error logs to obtain the central error log of the mk error logs, and store the central error log to a second storage location; move the error log with the largest cosine distance from the central error log among the mk error logs to the second storage location; determine the eccentric distance of each remaining error log in the mk error logs based on the cosine distance between each remaining error log in the mk error logs and the error log in the second storage location, and move the target error log with the largest eccentric distance among each remaining error log in the mk error logs to The second storage location; determining whether the number of error logs in the second storage location is equal to K; if the number of error logs in the second storage location is not equal to K, returning to execute the operation of determining the eccentric distance of each error log remaining in the mk error logs according to the cosine distance between each error log remaining in the mk error logs and the error log in the second storage location, and moving the target error log with the largest eccentric distance among each error log remaining in the mk error logs to the second storage location until the number of error logs in the second storage location is equal to K; wherein the K error logs in the second storage location are the K initial centroids of the error log set.

[0110] The log classification device provided in the embodiment of the present invention can execute the log classification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0111] Embodiment 4

[0112] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement the log classification method of an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0113] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0114] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a log classification method.

[0116] In some embodiments, the log classification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on a heterogeneous hardware accelerator via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the log classification method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the log classification method in any other suitable manner (e.g., by means of firmware).

[0117] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or electronic device.

[0119] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein may be implemented on a heterogeneous hardware accelerator having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the heterogeneous hardware accelerator. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0122] The computing system may include a client and an electronic device. The client and the electronic device are generally remote from each other and usually interact through a communication network. The relationship between the client and the electronic device is generated by computer programs running on corresponding computers and having a client-electronic device relationship with each other. The electronic device may be a cloud electronic device, also known as a cloud computing electronic device or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A log classification method, characterized in that: include: Extracting a first sample set from the error log set, determining K initial centroids of the first sample set, and clustering the first sample set according to the K initial centroids of the first sample set to obtain K clusters of the first sample set; Eliminate noise points in the first sample set to obtain a second sample set; Filtering out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; Determine K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs; The error log set is clustered according to the K initial centroids of the error log set to obtain K clusters of the error log set.

2. The log classification method according to claim 1, characterized in that: A first sample set is extracted from the error log set, including: A target number of error logs are randomly selected from the error log set as the first sample set.

3. The log classification method according to claim 1, characterized in that: Determining K initial centroids of the first sample set includes: The first sample set is provided to a target user, and K initial centroids of the first sample set fed back by the target user are obtained.

4. The log classification method according to claim 1, characterized in that: Eliminating noise points in the first sample set to obtain a second sample set includes: determining an average distance for each cluster of the first sample set; Determining a global average distance of the first sample set according to the average distance of each cluster of the first sample set; Determining the density of each error log in the first sample set according to the global average distance of the first sample set; According to the density of each error log, noise points in the first sample set are removed to obtain a second sample set.

5. The log classification method according to claim 4, characterized in that: According to the density of the error logs in the second sample set and the distance product between the error logs, mk error logs are screened out from the second sample set, including: Determine the error log with the highest density among the error logs in the second sample set as the first error log, and move the first error log to the first storage location; Determine the error log having the largest cosine distance with the first error log among the remaining error logs in the second sample set as the second error log, and move the second error log to the first storage location; Determine a target error log whose distance product with the error log in the first storage location is the largest among the remaining error logs in the second sample set, and move the target error log to the first storage location; Determine whether the number of error logs in the first storage location is equal to mk; When the number of error logs in the first storage location is not equal to mk, return to the operation of determining a target error log having the largest distance product with the error log in the first storage location among the remaining error logs in the second sample set, and moving the target error log to the first storage location until the number of error logs in the first storage location is equal to mk.

6. The log classification method according to claim 1, characterized in that: Determining K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs includes: Calculate an average value of the mk error logs to obtain a central error log of the mk error logs, and store the central error log in a second storage location; Move the error log having the largest cosine distance with the central error log among the mk error logs to the second storage location; Determine the eccentric distance of each remaining error log in the mk error logs according to the cosine distance between each remaining error log in the mk error logs and the error log in the second storage position, and move the target error log with the largest eccentric distance among each remaining error log in the mk error logs to the second storage position; Determine whether the number of error logs in the second storage location is equal to K; When the number of error logs in the second storage location is not equal to K, return to execute the operation of determining the eccentric distance of each remaining error log in the mk error logs according to the cosine distance between each remaining error log in the mk error logs and the error log in the second storage location, and moving the target error log with the largest eccentric distance among the remaining error logs in the mk error logs to the second storage location until the number of error logs in the second storage location is equal to K; wherein the K error logs in the second storage location are the K initial centroids of the error log set.

7. A log classification device, characterized in that: include: A sample set extraction module, configured to extract a first sample set from an error log set, determine K initial centroids of the first sample set, and cluster the first sample set according to the K initial centroids of the first sample set to obtain K clusters of the first sample set; A noise point removal module, used to remove noise points in the first sample set to obtain a second sample set; A preliminary extraction module, configured to filter out mk error logs from the second sample set according to the density of the error logs in the second sample set and the distance product between the error logs; A fine extraction module, used for determining K initial centroids of the error log set according to the central error log of the mk error logs and the cosine distances between the error logs; The log clustering module is used to cluster the error log set according to the K initial centroids of the error log set to obtain K clusters of the error log set.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the log classification method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the log classification method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the log classification method according to any one of claims 1 to 6.

Images