Log file-based device detection method and device, and electronic device

Abstract

The application discloses a device detection method and device based on log files and electronic equipment, and relates to the field of financial technology and other related technical fields. The method comprises the following steps: obtaining N log files of a target device, wherein each log file corresponds to a text log information and an index log information; determining A first feature vectors according to the N text log information; determining B second feature vectors according to the N index log information; calculating a first distance between the first feature vectors and a first clustering center and a second distance between the second feature vectors and a second clustering center; and determining that the target device is in an abnormal state when the first distance is greater than a preset distance and / or the second distance is greater than the preset distance. The application solves the technical problem of low detection accuracy in the process of detecting whether the device is abnormal.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology and other related technical fields, in particular, to a device detection method and device based on log files and electronic equipment. BACKGROUND

[0002] The log file of the device contains various log information, for example, the log file of the device contains device operation record information and device index log information. In the prior art, when a user or an operation and maintenance personnel analyzes the log file containing different log information, a single mode analysis method is mainly used, that is, the user or the operation and maintenance personnel separately analyzes the device operation record information and the device index log information, so that the comprehensive judgment of the device log file is lacking, and thus the problem of low accuracy in detecting whether the device is abnormal is caused.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The present application provides a device detection method and device based on log files and electronic equipment to at least solve the technical problem of low accuracy in detecting whether the device is abnormal.

[0005] According to one aspect of the present application, a device detection method based on log files is provided, comprising: obtaining N log files of a target device, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record the operation record information generated when the target device executes an operation command, the index log information is used to record the performance index information of the target device, and N is a positive integer; determining A first feature vectors according to N text log information, wherein A is a positive integer, and the first feature vector is used to represent the repeated text content in at least two text log information of N text log information; determining B second feature vectors according to N index log content, wherein B is a positive integer, and the second feature vector is used to represent the repeated text content in at least two index log information of N index log information; calculating a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center, wherein the first clustering center is the center vector of the cluster where the first feature vector is located, and the second clustering center is the center vector of the cluster where the second feature vector is located; in the case that the first distance is greater than a preset distance and / or the second distance is greater than a preset distance, determining that the target device is in an abnormal state.

[0006] Further, the device detection method based on log files further comprises: performing word segmentation processing on each of the N pieces of text log information to obtain X words corresponding to each piece of text log information, X being a positive integer; performing stop word removal processing on the X words to obtain Y first words, wherein the stop word removal processing is used to filter words in the X words that do not have semantic information, Y being a positive integer less than or equal to X; performing deduplication processing on the Y first words to obtain A target words, wherein the deduplication processing is used to filter, from the Y first words, any first word that is the same as any one of the A target words; and determining A first feature vectors according to the A target words, wherein each target word corresponds to a first feature vector.

[0007] Further, the device detection method based on log files further comprises: generating a clustering result according to each of the A target words to obtain A clustering results, wherein the clustering result corresponding to each target word is used to represent at least one piece of text log information containing the target word; determining target text content from at least one piece of text log information in each clustering result, wherein the target text log content is text content that repeatedly appears in at least one piece of text log information; and generating a first feature vector corresponding to each clustering result according to the target text content corresponding to the clustering result to obtain A first feature vectors.

[0008] Further, the device detection method based on log files further comprises: randomly determining K first feature vectors from the A first feature vectors as K first clustering centers, wherein K is a positive integer less than A; and setting a first label for each first feature vector in the A first feature vectors except the K first clustering centers, wherein the first feature vector corresponding to the first label is a first feature vector that is not allocated to a first cluster, and the first cluster is a cluster corresponding to any one of the first clustering centers.

[0009] Further, the device detection method based on log files further comprises: step one, taking the first feature vector corresponding to the first label as a to-be-processed feature vector; step two, allocating the to-be-processed feature vector to a first cluster corresponding to a target clustering center, wherein the target clustering center is a first clustering center in the K first clustering centers that is closest to the to-be-processed feature vector; step three, updating the first label corresponding to the to-be-processed feature vector to a second label, wherein the second label is used to represent that the to-be-processed feature vector has been allocated to the first cluster corresponding to the target clustering center; step four, recalculating a center vector of the first cluster corresponding to the target clustering center, and updating the target clustering center to the calculated center vector; and step five, repeating the above steps one to four until all first feature vectors corresponding to the first label are allocated to the first cluster.

[0010] Further, the device detection method based on log files further comprises: taking the generation time of each of the A first feature vectors as a first target time, to obtain A first target times; taking the generation time of each of the B second feature vectors as a second target time, to obtain B second target times; performing time sequence processing on the A first feature vectors and the B second feature vectors based on the A first target times and the B second target times, to obtain C third feature vectors, wherein the time sequence processing is used to splice the first feature vectors and the second feature vectors with the same generation time into a third feature vector; taking the distance between the third feature vector and a third clustering center as a third distance, wherein the third clustering center is a center vector of a cluster to which the third feature vector belongs; and determining that the target device is in an abnormal state in a case where the third distance is greater than a preset distance.

[0011] Further, the device detection method based on log files further comprises: determining that the target device is in a normal state in a case where the first distance, the second distance and the third distance are all less than or equal to a preset distance.

[0012] According to another aspect of the present application, a device detection apparatus based on log files is also provided, comprising: an acquisition unit configured to acquire N log files of a target device, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record operation record information generated when the target device executes an operation command, the index log information is used to record performance index information of the target device, and N is a positive integer; a first determination unit configured to determine A first feature vectors according to N text log information, wherein A is a positive integer, and the first feature vector is used to represent text content repeatedly appearing in at least two text log information of the N text log information; a second determination unit configured to determine B second feature vectors according to N index log information, wherein B is a positive integer, and the second feature vector is used to represent text content repeatedly appearing in at least two index log information of the N index log information; a calculation unit configured to calculate a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center, wherein the first clustering center is a center vector of a cluster to which the first feature vector belongs, and the second clustering center is a center vector of a cluster to which the second feature vector belongs; and an abnormality determination unit configured to determine that the target device is in an abnormal state in a case where the first distance is greater than a preset distance and / or the second distance is greater than the preset distance.

[0013] According to another aspect of the present application, a computer readable storage medium having a computer program stored therein is also provided, wherein the computer program controls a device where the computer readable storage medium is located to execute the device detection method based on log files according to any one of the above aspects when the computer program runs.

[0014] According to another aspect of the present application, an electronic device is also provided, which includes one or more processors and a memory storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the above-mentioned device detection method based on log files.

[0015] In the present application, first, N log files of a target device are acquired, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record operation record information generated when the target device executes an operation command, the index log information is used to record performance index information of the target device, N is a positive integer, second, A first feature vectors are determined according to N text log information, wherein A is a positive integer, the first feature vector is used to represent text content repeatedly appearing in at least two text log information of N text log information, then, B second feature vectors are determined according to N index log content, wherein B is a positive integer, the second feature vector is used to represent text content repeatedly appearing in at least two index log information of N index log information, after that, a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center are calculated, wherein the first clustering center is a center vector of a cluster where the first feature vector is located, the second clustering center is a center vector of a cluster where the second feature vector is located, finally, in the case that the first distance is greater than a preset distance and / or the second distance is greater than the preset distance, it is determined that the target device is in an abnormal state.

[0016] From the above content, it can be seen that the present application obtains the first feature vector by analyzing the text log information of the target device, obtains the second feature vector by analyzing the index log information of the target device, and achieves the purpose of analyzing the target device from the dimensions of the text log information and the index log information. Compared with the prior art which only evaluates the device state of the target device from a single dimension, the evaluation dimension of the present application is more diversified, thereby ensuring more comprehensive evaluation of the device state of the device and improving the accuracy of device anomaly detection.

[0017] Therefore, the technical scheme of the present application achieves the purpose of comprehensive analysis of the target device from multiple dimensions, thereby achieving the technical effect of improving the accuracy of device anomaly detection, and further solving the technical problem of low accuracy in detecting whether the device is abnormal in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0019] Figure 1 is a flowchart of an optional log file based device detection method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an optional method for obtaining A first feature vectors according to an embodiment of the present application;

[0021] Figure 3 is a flowchart of an optional first clustering generation method according to an embodiment of the present application;

[0022] Figure 4 is a flowchart of an optional method for determining whether a target device is in an abnormal state according to an embodiment of the present application;

[0023] Figure 5 is a flowchart of another optional log file based device detection method according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an optional log file based device detection apparatus according to an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of another optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the personnel in the art better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It should be noted that the related information (including but not limited to obtaining N log files of the target device) and data (including but not limited to data for display and analyzed data) involved in the present application are information and data authorized by the user or authorized by all parties. For example, an interface is provided between the system and the related user or institution. Before obtaining the related information, the interface needs to send a request to the aforementioned user or institution, and after receiving the consent information feedback from the aforementioned user or institution, the related information is obtained.

[0029] The present application will be further described in conjunction with the embodiments.

[0030] Embodiment 1

[0031] According to the embodiments of the present application, an embodiment of a log file based device detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0032] The present application provides a detection system for executing the log file based device detection method in the embodiments of the present application, Figure 1 is a flowchart of an optional log file based device detection method according to the embodiments of the present application, as Figure 1 shown, the method comprises the following steps:

[0033] Step S101, obtaining N log files of the target device.

[0034] Specifically, each log file corresponds to a text log information and an index log information. The text log information is used to record the operation record information generated when the target device executes the operation command. The index log information is used to record the performance index information of the target device. N is a positive integer.

[0035] For example, the detection system obtains 100 log files of the target device A. Each log file contains 1 text log information, and 100 file log information corresponding to the target device A is obtained based on the 100 log files of the target device A. Each log file contains 1 index log information, and 100 index log information corresponding to the target device A is obtained based on the 100 log files of the target device A.

[0036] Step S102, determining A first feature vectors according to N text log information.

[0037] Specifically, A is a positive integer, and the first feature vector is used to represent the repeated text content in at least two text log information of N text log information.

[0038] Optionally, the detection system first performs word segmentation and stop word removal on each of the N pieces of text log information to obtain Y first words corresponding to the text log information, then removes duplicates from the Y first words to obtain A target words, then generates a clustering result based on each of the A target words to obtain A clustering results, and finally generates a first feature vector corresponding to each of the A clustering results based on the A clustering results to obtain A first feature vectors.

[0039] In step S103, B second feature vectors are determined according to the N pieces of index log content.

[0040] Specifically, B is a positive integer, and the second feature vector is used to represent text content that repeatedly appears in at least two pieces of index log information of the N pieces of index log information.

[0041] Optionally, the detection system first performs word segmentation and stop word removal on each of the N pieces of index log information to obtain Z second words corresponding to the index log information, then removes duplicates from the Z second words to obtain B target index words, then generates a clustering result based on each of the B target index words to obtain B clustering results, and finally generates a second feature vector corresponding to each of the B clustering results based on the B clustering results to obtain B second feature vectors.

[0042] In step S104, a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center are calculated.

[0043] Specifically, the first clustering center is a center vector of a cluster in which the first feature vector is located, and the second clustering center is a center vector of a cluster in which the second feature vector is located.

[0044] In step S105, in a case where the first distance is greater than a preset distance and / or the second distance is greater than the preset distance, it is determined that the target device is in an abnormal state.

[0045] Optionally, after the A first feature vectors and the B second feature vectors are obtained, the detection system performs time sequence processing on the A first feature vectors and the B second feature vectors based on generation times of the first feature vectors and generation times of the second feature vectors to obtain C third feature vectors, then calculates a third distance between the third feature vectors and a third clustering center, and in a case where the third distance is greater than a preset distance, it is determined that the target device is in an abnormal state.

[0046] In addition, the detection system determines that the target device is in a normal state when the first distance, the second distance, and the third distance are all less than or equal to a preset distance.

[0047] From the above, the first feature vector is obtained by analyzing the text log information of the target device, and the second feature vector is obtained by analyzing the index log information of the target device, so as to realize the purpose of analyzing the target device from the dimensions of the text log information and the index log information. Compared with the prior art of evaluating the device state of the target device from a single dimension, the evaluation dimension of the present application is more diversified, so as to ensure more comprehensive evaluation of the device state of the device and improve the accuracy of device anomaly detection.

[0048] Therefore, the technical scheme of the present application realizes the purpose of comprehensive analysis of the target device from multiple dimensions, thereby achieving the technical effect of improving the accuracy of device anomaly detection, and further solving the technical problem of low accuracy in detecting whether the device is abnormal in the prior art.

[0049] In an optional embodiment, Figure 2 is a flowchart of an optional method for obtaining A first feature vectors according to an embodiment of the present application, as shown in Figure 2 The method comprises the following steps:

[0050] Step S201: performing word segmentation processing on each of the N text log information to obtain X words corresponding to each text log information.

[0051] Specifically, X is a positive integer.

[0052] Step S202: performing stop word removal processing on the X words to obtain Y first words.

[0053] Specifically, the stop word removal processing is used to filter out words without semantic information from the X words, and Y is a positive integer less than or equal to X.

[0054] Step S203: performing deduplication processing on the Y first words to obtain A target words.

[0055] Specifically, the deduplication processing is used to filter out first words identical to any one of the A target words from the Y first words.

[0056] Step S204: determining A first feature vectors according to the A target words.

[0057] Specifically, each target word corresponds to a first feature vector.

[0058] In addition, the detection system performs word segmentation processing on each of the N pieces of index log information to obtain T words corresponding to each piece of index log information, where T is a positive integer. Next, the detection system performs stop word removal processing on the T words to obtain Z second words, where Z is a positive integer less than or equal to T. Then, the detection system performs deduplication processing on the Z second words to obtain B target index words, where the deduplication processing is used to filter, from the Z second words, any two second words that are the same as any two of the B target index words. Subsequently, the detection system determines B second feature vectors according to the B target index words, where each target index word corresponds to a second feature vector.

[0059] In an optional embodiment, the detection system generates a clustering result according to each of the A target words to obtain A clustering results, where each target word corresponds to a clustering result that represents at least one text log information containing the target word. Next, the detection system determines target text content from at least one text log information in each clustering result, where the target text log content is text content that repeatedly appears in the at least one text log information. Subsequently, the detection system generates a first feature vector corresponding to each clustering result according to the target text content corresponding to the clustering result to obtain A first feature vectors.

[0060] Optionally, after obtaining the A clustering results, the detection system first performs word frequency analysis on all text log information contained in each clustering result, identifies a word that repeatedly appears at the same position in multiple text log information as a constant, and identifies multiple different words that appear at the same position in multiple text log information as variables, thereby obtaining target text content corresponding to the current clustering result. Then, the detection system performs word segmentation processing and stop word removal operations on the target text content to obtain multiple text words. Finally, the detection system counts the number of times each text word in the multiple text words appears in the target text content, performs vector conversion, and obtains a first feature vector corresponding to the clustering result, where each text word corresponds to an element in the vector, and the number of times the text word appears is the value of the corresponding element. In addition, the detection system generates a clustering result according to each of the B target index words to obtain B clustering results, where each target index word corresponds to a clustering result that represents at least one index log information containing the target index word. Next, the detection system determines target index content from at least one index log information in each clustering result, where the target index log content is index content that repeatedly appears in the at least one index log information. Subsequently, the detection system generates a second feature vector corresponding to each clustering result according to the target index content corresponding to the clustering result to obtain B second feature vectors.

[0061] In an alternative embodiment, the detection system randomly determines K first eigenvectors from the A first eigenvectors as K first cluster centers, wherein K is a positive integer smaller than A, and then sets a first label for each of the first eigenvectors other than the K first cluster centers, wherein the first label corresponds to a first eigenvector that is not assigned to a first cluster, and the first cluster is a cluster corresponding to any one of the first cluster centers.

[0062] In addition, the detection system randomly determines J second eigenvectors from the B second eigenvectors as J second cluster centers, wherein J is a positive integer smaller than B, and then sets a third label for each of the second eigenvectors other than the J second cluster centers, wherein the third label corresponds to a second eigenvector that is not assigned to a second cluster, and the second cluster is a cluster corresponding to any one of the second cluster centers.

[0063] In an alternative embodiment, Figure 3 is a flowchart of an alternative first cluster generation method according to an embodiment of the present application, as shown in Figure 3 The method comprises the following steps:

[0064] Step one, the first label corresponds to a first eigenvector as a to-be-processed eigenvector.

[0065] Step two, the to-be-processed eigenvector is assigned to a first cluster corresponding to a target cluster center.

[0066] Specifically, the target cluster center is a first cluster center with the shortest distance to the to-be-processed eigenvector among the K first cluster centers.

[0067] Step three, the first label corresponding to the to-be-processed eigenvector is updated to a second label.

[0068] Specifically, the second label is used to represent that the to-be-processed eigenvector has been assigned to the first cluster corresponding to the target cluster center.

[0069] Step four, the center vector of the first cluster corresponding to the target cluster center is recalculated, and the target cluster center is updated to the calculated center vector.

[0070] If all the first eigenvectors corresponding to the first label are not assigned to the first cluster, the above steps one to four are repeated until all the first eigenvectors corresponding to the first label are assigned to the first cluster.

[0071] In addition, when the detection system generates the second cluster, the method comprises the following steps:

[0072] Step 1, taking the second feature vector corresponding to the second label as a first to-be-processed feature vector.

[0073] Step 2, assigning the first to-be-processed feature vector to a second cluster corresponding to a second target cluster center.

[0074] Specifically, the second target cluster center is a second cluster center with the shortest distance to the first to-be-processed feature vector among J second cluster centers.

[0075] Step 3, updating the third label corresponding to the first to-be-processed feature vector to a fourth label.

[0076] Specifically, the fourth label is used to represent that the first to-be-processed feature vector has been assigned to the second cluster corresponding to the second target cluster center.

[0077] Step 4, recalculating the center vector of the second cluster corresponding to the second target cluster center, and updating the second target cluster center to the calculated center vector.

[0078] Step 5, repeating the processes of steps 1 to 4 until all second feature vectors corresponding to the third label are assigned to the second cluster.

[0079] In an optional embodiment, Figure 4 is a flowchart of an optional method for judging whether a target device is in an abnormal state according to an embodiment of the present application, as Figure 4 shown, the method comprises the following steps:

[0080] Step S401, taking the generation time of each of A first feature vectors as a first target time, to obtain A first target times.

[0081] Step S402, taking the generation time of each of B second feature vectors as a second target time, to obtain B second target times.

[0082] Step S403, performing time sequence processing on the A first feature vectors and the B second feature vectors based on the A first target times and the B second target times, to obtain C third feature vectors.

[0083] Specifically, the time sequence processing is used to splice the first feature vectors and the second feature vectors with the same generation time into a third feature vector.

[0084] Optionally, the detection system, when concatenating the first feature vector and the second feature vector generated at the same time, concatenates the first feature vector and the second feature vector in a manner including but not limited to horizontal concatenation and vertical concatenation. The detection system preferably concatenates the first feature vector and the second feature vector in a row. For example, the first feature vector a1 = [1, 2, 3] and the second feature vector b1 = [5, 7, 9, 10] are concatenated in a row to obtain a third feature vector c1 = [1, 2, 3, 5, 7, 9, 10], wherein the third feature vector can express the information of the multiple modalities from the time sequence dimension, including the text log information and the index log information of the target device.

[0085] In step S404, the distance between the third feature vector and the third cluster center is taken as a third distance.

[0086] Specifically, the third cluster center is the center vector of the cluster in which the third feature vector is located.

[0087] In step S405, when the third distance is greater than a preset distance, it is determined that the target device is in an abnormal state. In an optional embodiment, when the first distance, the second distance, and the third distance are all less than or equal to the preset distance, the detection system determines that the target device is in a normal state.

[0088] According to another aspect of the embodiments of the present application, another device detection method based on log files is also provided. Figure 5 is a flowchart of another optional device detection method based on log files according to the embodiments of the present application, as shown in Figure 5 The method includes the following steps: template extraction based on the text log information of the target device to obtain a first feature vector corresponding to the target device, template extraction based on the index log information of the target device to obtain a second feature vector corresponding to the target device, feature fusion of the first feature vector and the second feature vector to obtain a third feature vector corresponding to the target device, and abnormality detection of the target device from multiple dimensions based on the first feature vector, the second feature vector, and the third feature vector.

[0089] In the present application, first, N log files of a target device are acquired, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record operation record information generated when the target device executes an operation command, the index log information is used to record performance index information of the target device, N is a positive integer, second, A first feature vectors are determined according to N text log information, wherein A is a positive integer, the first feature vector is used to represent text content that repeatedly appears in at least two text log information of N text log information, then, B second feature vectors are determined according to N index log content, wherein B is a positive integer, the second feature vector is used to represent text content that repeatedly appears in at least two index log information of N index log information, after that, a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center are calculated, wherein the first clustering center is a center vector of a cluster where the first feature vector is located, the second clustering center is a center vector of a cluster where the second feature vector is located, finally, in the case that the first distance is greater than a preset distance and / or the second distance is greater than the preset distance, it is determined that the target device is in an abnormal state.

[0090] From the above, it can be seen that the present application obtains a first feature vector by analyzing text log information of a target device, obtains a second feature vector by analyzing index log information of the target device, and achieves the purpose of analyzing the target device from the dimensions of text log information and index log information. Compared with the prior art which only evaluates the device state of the target device from a single dimension, the evaluation dimension of the present application is more diversified, thereby ensuring more comprehensive evaluation of the device state of the device and improving the accuracy of device anomaly detection.

[0091] Therefore, the technical scheme of the present application achieves the purpose of comprehensive analysis of the target device from multiple dimensions, thereby achieving the technical effect of improving the accuracy of device anomaly detection, and further solving the technical problem of low accuracy in detecting whether the device is abnormal in the prior art.

[0092] Embodiment 2

[0093] According to the embodiments of the present application, an embodiment of a device detection apparatus based on log files is provided. Figure 6 is a schematic diagram of an optional device detection apparatus based on log files according to the embodiments of the present application, as shown in Figure 6 The device detection apparatus based on log files includes an acquisition unit 501, a first determination unit 502, a second determination unit 503, a first calculation unit 504 and an abnormality determination unit 505.

[0094] Specifically, the acquisition unit is configured to acquire N log files of a target device, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record operation record information generated when the target device executes an operation command, the index log information is used to record performance index information of the target device, N is a positive integer, the first determination unit is configured to determine A first feature vectors according to the N text log information, wherein A is a positive integer, and the first feature vector is used to represent text content that repeatedly appears in at least two text log information of the N text log information, the second determination unit is configured to determine B second feature vectors according to the N index log information, wherein B is a positive integer, and the second feature vector is used to represent text content that repeatedly appears in at least two index log information of the N index log information, the first calculation unit is configured to calculate a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center, wherein the first clustering center is a center vector of a cluster to which the first feature vector belongs, and the second clustering center is a center vector of a cluster to which the second feature vector belongs, and the anomaly determination unit is configured to determine that the target device is in an abnormal state when the first distance is greater than a preset distance and / or the second distance is greater than the preset distance.

[0095] Optionally, the first determination unit further includes a first word segmentation subunit, a first processing subunit, a first deduplication subunit, and a first determination subunit.

[0096] Specifically, the first word segmentation subunit is configured to perform word segmentation processing on each text log information in the N text log information to obtain X words corresponding to each text log information, X being a positive integer, the first processing subunit is configured to perform stop word removal processing on the X words to obtain Y first words, wherein the stop word removal processing is used to filter words in the X words that do not have semantic information, Y being a positive integer less than or equal to X, the first deduplication subunit is configured to perform deduplication processing on the Y first words to obtain A target words, wherein the deduplication processing is used to filter, from the Y first words, a first word that is the same as any one of the A target words, and the first determination subunit is configured to determine A first feature vectors according to the A target words, wherein each target word corresponds to a first feature vector.

[0097] Optionally, the first determination subunit further includes a first generation module, a first determination module, and a second generation module.

[0098] Specifically, the first generating module is configured to generate a clustering result according to each target word in the A target words, to obtain A clustering results, wherein the clustering result corresponding to each target word is used to represent at least one text log information containing the target word; the first determining module is configured to determine target text content from at least one text log information in each clustering result, wherein the target text log content is repeatedly appearing text content in the at least one text log information; and the second generating module is configured to generate a first feature vector corresponding to each clustering result according to the target text content corresponding to the clustering result in the A clustering results, to obtain A first feature vectors.

[0099] Optionally, the log file based device detection apparatus further comprises a third determining unit and a first setting unit.

[0100] Specifically, the third determining unit is configured to randomly determine K first feature vectors as K first clustering centers from the A first feature vectors, wherein K is a positive integer less than A; and the first setting unit is configured to set a first label for each first feature vector in the A first feature vectors except the K first clustering centers, wherein the first label corresponds to a first feature vector that is not allocated to a first cluster, and the first cluster is a cluster corresponding to any one of the first clustering centers.

[0101] Optionally, the log file based device detection apparatus further comprises a to-be-processed feature vector determining unit, a first allocating unit, a first updating unit, a second calculating unit and a repeating unit.

[0102] Specifically, the to-be-processed feature vector determining unit is configured to take the first feature vector corresponding to the first label as a to-be-processed feature vector; the first allocating unit is configured to allocate the to-be-processed feature vector to a first cluster corresponding to a target clustering center, wherein the target clustering center is a first clustering center in the K first clustering centers that is closest to the to-be-processed feature vector; the first updating unit is configured to update the first label corresponding to the to-be-processed feature vector to a second label, wherein the second label is used to represent that the to-be-processed feature vector has been allocated to the first cluster corresponding to the target clustering center; the second calculating unit is configured to recalculate a center vector of the first cluster corresponding to the target clustering center, and update the target clustering center to the calculated center vector; and the repeating unit is configured to repeat the processes performed by the to-be-processed feature vector determining unit, the first allocating unit, the first updating unit and the second calculating unit until all the first feature vectors corresponding to the first label are allocated to the first cluster.

[0103] Optionally, the log file based device detection apparatus further comprises a first target time determining unit, a second target time determining unit, a time sequence processing unit, a third distance determining unit and a fourth determining unit.

[0104] Specifically, the first target time determination unit is configured to take the generation time of each of the A first feature vectors as a first target time, thereby obtaining A first target times; the second target time determination unit is configured to take the generation time of each of the B second feature vectors as a second target time, thereby obtaining B second target times; the time sequence processing unit is configured to perform time sequence processing on the A first feature vectors and the B second feature vectors based on the A first target times and the B second target times, thereby obtaining C third feature vectors, wherein the time sequence processing is configured to splice the first feature vectors and the second feature vectors with the same generation time into a third feature vector; the third distance determination unit is configured to take the distance between the third feature vector and a third cluster center as a third distance, wherein the third cluster center is a center vector of a cluster to which the third feature vector belongs; and the fourth determination unit is configured to determine that the target device is in an abnormal state in a case where the third distance is greater than a preset distance.

[0105] Optionally, the device detection apparatus based on log files further comprises a normal determination unit.

[0106] Specifically, the normal determination unit is configured to determine that the target device is in a normal state in a case where the first distance, the second distance, and the third distance are all less than or equal to a preset distance.

[0107] From the above, it can be seen that the present application obtains the first feature vectors by analyzing the text log information of the target device, and obtains the second feature vectors by analyzing the index log information of the target device, thereby achieving the purpose of analyzing the target device from the dimensions of the text log information and the index log information. Compared with the prior art in which the device state of the target device is evaluated from a single dimension, the evaluation dimension of the present application is more diversified, thereby ensuring more comprehensive evaluation of the device state of the device and improving the accuracy of device anomaly detection.

[0108] Therefore, the technical scheme of the present application achieves the purpose of comprehensively analyzing the target device from multiple dimensions, thereby achieving the technical effect of improving the accuracy of device anomaly detection, and further solving the technical problem of low accuracy in detecting whether the device is abnormal in the prior art.

[0109] Embodiment 3

[0110] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which comprises a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the device detection method based on log files in the above-mentioned embodiment 1 when the computer program runs.

[0111] Embodiment 4

[0112] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the log file based device detection method in the above embodiment 1 by executing the executable instructions.

[0113] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application, as Figure 7 shown, the present application provides an electronic device, which comprises a processor, a memory and a program stored in the memory and executable on the processor, and the processor implements the log file based device detection method in the above embodiment 1 when executing the program.

[0114] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0116] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the device embodiment described above is only illustrative, and for example, the division of units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0117] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0118] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0119] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0120] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A log file based device detection method, characterized by, The method comprises the following steps: obtaining N log files of a target device, wherein each log file corresponds to text log information and index log information, the text log information is used for recording operation record information generated when the target device executes an operation command, the index log information is used for recording performance index information of the target device, and N is a positive integer; determining A first feature vectors according to N text log information, wherein A is a positive integer, and the first feature vector is used to represent text content that repeatedly appears in at least two text log information of the N text log information; determining B second feature vectors according to N index log content, wherein B is a positive integer, and the second feature vector is used to represent text content that repeatedly appears in at least two index log information of the N index log information; calculating a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center, wherein the first clustering center is a center vector of a cluster to which the first feature vector belongs, and the second clustering center is a center vector of a cluster to which the second feature vector belongs; in a case where the first distance is greater than a preset distance and / or the second distance is greater than the preset distance, determining that the target device is in an abnormal state; wherein, after determining B second feature vectors according to N index log content, the method further comprises: taking a generation time of each first feature vector in the A first feature vectors as a first target time to obtain A first target times; taking a generation time of each second feature vector in the B second feature vectors as a second target time to obtain B second target times; performing time sequence processing on the A first feature vectors and the B second feature vectors based on the A first target times and the B second target times to obtain C third feature vectors, wherein the time sequence processing is used to splice the first feature vectors and the second feature vectors with the same generation time into a third feature vector; taking a distance between the third feature vector and a third clustering center as a third distance, wherein the third clustering center is a center vector of a cluster to which the third feature vector belongs; and in a case where the third distance is greater than the preset distance, determining that the target device is in an abnormal state.

2. The method of claim 1, wherein, determining A first feature vectors according to N text log information, comprising: performing word segmentation processing on each text log information in the N text log information to obtain X words corresponding to the each text log information, X being a positive integer; performing stop word removal processing on the X words to obtain Y first words, wherein the stop word removal processing is used to filter words without semantic information in the X words, and Y is a positive integer less than or equal to X; performing deduplication processing on the Y first words to obtain A target words, wherein the deduplication processing is used to filter first words identical to any one of the A target words from the Y first words; determining A first feature vectors according to the A target words, wherein each target word corresponds to a first feature vector.

3. The method of claim 2, wherein, determining A first feature vectors according to the A target words, comprising: generating a clustering result according to each target word in the A target words, to obtain A clustering results, wherein the clustering result corresponding to each target word is used to represent at least one text log information containing the target word; determining target text content from at least one text log information in each clustering result, wherein the target text log content is text content that repeatedly appears in the at least one text log information; generating a first feature vector corresponding to each clustering result according to the target text content corresponding to the clustering result in the A clustering results, to obtain A first feature vectors.

4. The method of claim 1, wherein, Before calculating the first distance between the first feature vector and the first clustering center and the second distance between the second feature vector and the second clustering center, the method further comprises: randomly determining K first feature vectors as K first clustering centers from the A first feature vectors, wherein K is a positive integer less than A; setting a first label for each first feature vector in the A first feature vectors except the K first clustering centers, wherein the first label corresponds to a first feature vector that is not allocated to a first cluster, and the first cluster is a cluster corresponding to any first clustering center.

5. The method of claim 4, wherein, After setting a first label for each first feature vector in the A first feature vectors except the K first clustering centers, the method further comprises: Step one, taking the first feature vector corresponding to the first label as a to-be-processed feature vector; Step two, allocating the to-be-processed feature vector to a first cluster corresponding to a target clustering center, wherein the target clustering center is a first clustering center in the K first clustering centers that is closest to the to-be-processed feature vector; Step three, updating the first label corresponding to the to-be-processed feature vector to a second label, wherein the second label is used to represent that the to-be-processed feature vector has been allocated to the first cluster corresponding to the target clustering center; Step four, recalculating the center vector of the first cluster corresponding to the target clustering center, and updating the target clustering center to the calculated center vector; Step five, repeating the processes of steps one to four until all first feature vectors corresponding to the first label are allocated to the first cluster.

6. The method of claim 1, wherein, After taking the distance between the third feature vector and the third clustering center as the third distance, the method further comprises: determining that the target device is in a normal state in a case where the first distance, the second distance, and the third distance are all less than or equal to the preset distance.

7. An apparatus for log file based device detection, for implementing the log file based device detection method according to any one of claims 1 to 6, characterized in that comprising: An acquisition unit is configured to acquire N log files of a target device, wherein each log file corresponds to a text log information and an index log information, the text log information is used to record operation record information generated when the target device executes an operation command, the index log information is used to record performance index information of the target device, and N is a positive integer; A first determination unit is configured to determine A first feature vectors according to N text log information, wherein A is a positive integer, and the first feature vector is used to represent text content repeatedly appearing in at least two text log information of the N text log information; A second determination unit is configured to determine B second feature vectors according to N index log information, wherein B is a positive integer, and the second feature vector is used to represent text content repeatedly appearing in at least two index log information of the N index log information; A calculation unit is configured to calculate a first distance between the first feature vector and a first clustering center and a second distance between the second feature vector and a second clustering center, wherein the first clustering center is a center vector of a cluster where the first feature vector is located, and the second clustering center is a center vector of a cluster where the second feature vector is located; An abnormality determination unit is configured to determine that the target device is in an abnormal state when the first distance is greater than a preset distance and / or the second distance is greater than the preset distance.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program runs, controls a device where the computer readable storage medium is located to execute the device detection method based on log files in any one of claims 1 to 6.

9. An electronic device, comprising: The device includes one or more processors and a memory, and the memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the device detection method based on log files in any one of claims 1 to 6.

Images