Log processor, terminal, method and program

Abstract

To provide a log processor which can determine an important item for determination of an abnormality, etc. in a log.SOLUTION: A criterion for determining whether a log represents occurrence of an abnormality or existence of an abnormality sign is obtained by what item corresponds to what kind of condition among a plurality of items constituting the log by executing software, accepting at least a part of the log of a terminal which acquires the log to be output by the software as the log for learning, accepting a correct label about whether there is the occurrence of the abnormality or the abnormality sign regarding an operation related to the terminal or the operation is normal in the log for learning and performing learning by using the log for learning and the correct label.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a log processing device, a terminal, a method, and a program. [Background technology]

[0002] In Patent Document 1, application logs and OS logs are linked and acquired as a technology for acquiring logs that can be used for reliability analysis from the perspective of software behavior related to the control of in-vehicle devices, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent application No. 2022-143721 Summary of the Invention [Problem to be solved by the invention]

[0004] In the method of Patent Document 1, application logs and OS logs are linked and acquired, allowing a user analyzing the acquired logs to analyze the logs in detail, making it possible to analyze the cause of an abnormality when it occurs.

[0005] On the other hand, Patent Document 1 does not take into consideration the possibility of focusing only on important items in application logs and OS logs, so if all of these logs are sent and received over a network for monitoring, etc., there is a possibility that communication costs will increase and communication lines will become congested.

[0006] In view of the above-mentioned problems of the conventional technology, the first object of the present invention is to provide a log processing device, method, and program that can determine important items in a log for determining abnormalities, etc. The second object is to provide a terminal that determines the important items in the log. [Means for solving the problem]

[0007] To achieve the first object, the present invention provides a log processing device that executes software and acquires logs output by the software, receives at least a portion of the logs from a terminal that acquires the logs as a training log, receives correct labels in the training log indicating whether an abnormality or a sign of an abnormality exists or whether operation related to the terminal is normal, and learns using the training log and the correct labels to obtain a criterion for identifying which of multiple items constituting the log and under what conditions the log indicates or is likely to indicate an abnormality or a sign of an abnormality. The present invention also provides a method and program corresponding to the log processing device. To achieve the second object, the present invention provides a terminal that executes software and acquires logs output by the software, receives a criterion for identifying which of multiple items constituting the log and under what conditions the log indicates or is likely to indicate an abnormality or a sign of an abnormality, and uses the criterion to identify whether the log indicates or is likely to indicate an abnormality or a sign of an abnormality. [Effects of the Invention]

[0008] According to the first feature, by learning using learning data, a determination criterion is obtained that specifies which of the multiple items constituting the log corresponds to what condition, and therefore whether the log indicates or is likely to indicate the occurrence of an abnormality or the presence of a sign of an abnormality, and therefore it is possible to determine items in the log that are important for determining an abnormality, etc. According to the second feature, it is possible to determine the presence or possibility of an abnormality, etc. in the terminal log using the determination criterion. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a configuration diagram of a log use system according to an embodiment. [Figure 2] 1 is a functional block diagram of a log use system according to an embodiment. [Figure 3] 10 is a flowchart of the operation of the log use system according to the embodiment. [Figure 4] 10 is a diagram illustrating an example of the configuration of a log handled by an operation unit and a log acquisition unit. FIG. [Figure 5] FIG. 10 is a diagram illustrating an example of a log. [Figure 6] FIG. 10 is a diagram illustrating a schematic example of learning a decision tree for each attribute by dividing logs by attribute. [Figure 7] FIG. 1 is a diagram illustrating a schematic example of a decision tree to be learned. [Figure 8] FIG. 10 is a diagram illustrating an example of calculating importance of a log according to the present embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example for explaining the use of only highly important criteria in this embodiment. [Figure 10] FIG. 1 is a diagram illustrating an example of a hardware configuration of a general computer. DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 is a configuration diagram of a log usage system 100 according to an embodiment. The log usage system 100 includes a plurality of N terminals 10-1, 10-2, ..., 10-N and a log processing device 20. These components are configured to be able to communicate with each other via a network NW such as the Internet, a LAN (Local Area Network), or a mobile communication network. The terminals 10-1, 10-2, ..., 10-N are configured as devices of the same type for any purpose, such as devices that perform control related to autonomous driving in an autonomous driving vehicle, or control devices that are provided in an autonomous mobile robot for the purpose of delivering goods, etc. Although the terminals 10-1, 10-2, ..., 10-N are devices of the same type, they may include devices with different attributes due to differences in model due to differences in manufacturer, etc., or even if they are the same model, differences in model year, hardware performance, software version, etc.

[0011] 2 is a functional block diagram of a log usage system 100 according to an embodiment. The terminal 10 includes an attribute recording unit 11, a log acquisition unit 12, an operation unit 13, and a first determination unit 14. The log processing device 20 includes a log collection unit 21, a learning unit 22, a determination criterion storage unit 23, and a second determination unit 24. The terminal 10 in FIG. 2 is any one of the multiple terminals 10-1, 10-2, ..., 10-N in FIG. 1; that is, the common functional block configuration of the multiple terminals 10-1, 10-2, ..., 10-N is the terminal 10 in FIG. 2.

[0012] Fig. 3 is a flowchart of the operation of the log use system 100 according to the embodiment. Below, the details of the processing content of each functional block in Fig. 2 will be explained while explaining each step in Fig. 3.

[0013] In step S1, the log processing device 20 collects logs from each of the multiple terminals 10, and at the same time, the log processing device 20 also acquires attribute information of the terminals 10, before proceeding to step S2. Note that the timing of acquiring the attribute information may be before the terminals 10 actually start operating and logs start to be generated, for example, at the time of manufacturing by the manufacturer, so that the attribute information is stored in advance in the log processing device 20. This step S1 can be realized by the attribute recording unit 11, log acquisition unit 12, operation unit 13, and log collection unit 21 performing the following processes.

[0014] The operating unit 13 operates the software processing required for autonomous driving and autonomous traveling on the terminal 10 configured as an autonomous driving vehicle, an autonomous traveling robot, etc., and the log acquisition unit 12 acquires logs related to the software processing in the operating unit 13.

[0015] Fig. 4 is a diagram showing an example of the configuration of logs handled by the operating unit 13 and the log acquisition unit 14. The operating unit 13 is realized by a processor reading and executing a predetermined program (a program that handles processing related to autonomous driving, etc.), and as shown in Fig. 4, the operating unit 13 executes an OS (operating system) as basic software that handles low-level hardware processing, and executes application software controlled on the executed OS.

[0016] The log acquisition unit 14 acquires software logs executed by the operating unit 13 and acquires either or both of the OS log generated by the OS and the application log (application log) generated by the application software. This can be realized by the OS log generation function and the application log generation function, respectively. Acquisition of the OS log by the OS can be achieved by using a log acquisition function available in the OS. For example, if Linux (registered trademark) is running as the OS, OS kernel-level trace log data can be acquired as the OS log by using eBPF (extended Berkeley Packet Filter) technology as all or part of various commonly available OS log acquisition functions. Alternatively, logs related to OS behavior obtained using standard log collection and management functions such as syslog can be acquired as the OS log. Using eBPF enables detailed monitoring of system-level events such as CPU usage, memory allocation, disk IO, and network traffic. An example of monitoring these CPU usage rates and other parameters is shown in Figure 5, which will be described later. Application logs by applications can be acquired by a log output function provided in advance for the application.

[0017] The attribute recording unit 11 pre-records the attributes of the terminal 10, making it possible to associate the attributes with the logs collected by the log collection unit 21. The attributes may be all or part of the following: the version of the application software executed by the operation unit 13, the version of the OS, the model, year of manufacture and number of years of operation of the terminal 10, and the model, year of manufacture and number of years of operation of a mobile object such as a vehicle or robot on which the terminal 10 is mounted and which is controlled by the terminal 10. The attributes may also be all or part of the performance, capacity, model and year of manufacture of the component hardware (CPU, memory, etc.) that constitutes the computer device that constitutes the terminal 10.

[0018] As attributes that can function in the same way as the model year, all or part of the date of operation, the cumulative mileage of the vehicle or robot, etc. may be used.

[0019] The log collection unit 21 collects and stores the logs of the terminal 10 acquired by the log acquisition unit 12 in association with the attributes.

[0020] 5 is a diagram showing a schematic example of logs acquired by the log acquisition unit 14 of each terminal 10 and collected by the log collection unit 21. (FIG. 5 also includes examples of attributes acquired by the attribute recording unit 11, which will be described later, and linked to the logs.) The table on the top side lists the logs acquired at a certain time on eight terminals 10, and the table on the bottom side shows an example of detailed information content that can be included in each individual log.

[0021] In the upper row, column C1 indicates the numbers of eight logs. Column C2 indicates the vehicle identification number of the vehicle in which the on-board device is installed when the terminal 10 is an on-board device. Column C3 indicates whether the vehicle's engine is on or off. Column C4 indicates the vehicle status, which is OK or NG. Column C5 indicates the version of the on-board software. Column C6 indicates the status of the on-board software, which is OK or NG. Columns C7, C8, and C9 indicate the CPU usage rate, memory (primary storage device) usage rate, and disk (storage, secondary storage device) usage rate, respectively, and are given in three levels: low, medium, and high, but may also be given as a percentage or other value. Columns C10, C11, and C12 indicate the status of the LiDAR (Light Detection and Ranging) device, GPS (Global Positioning System) device, and camera device, respectively, which are used to measure the surrounding environment, and are given in OK or NG.

[0022] In the lower row, rows R7, R8, and R9 show examples of log content that provides more detailed information about CPU usage, memory usage, and disk usage, as shown in columns C7, C8, and C9 in the upper row. Column C24 records the maximum, average, and minimum values ​​for CPU usage and other parameters from the most recent period up to the time the log was taken, allowing for log information that reflects not only the instantaneous values ​​at the time the log was taken but also changes over the most recent period. As the names of the items in columns C21 and C22 indicate, in addition to CPU usage and other parameters in rows R7, R8, and R9 and LiDAR status and other parameters in rows R10, R11, and R12, rows R21, R22, R23, R24, and R25 provide hardware status information, including CPU execution time, CPU run queue length, CPU run queue latency, block IO (disk IO) latency, and TCP send / receive throughput, as well as values ​​and recent statuses at the time of log capture. Column C23 indicates whether an alarm (warning) status exists according to the values ​​in columns R7 to R9 and R21 to R25, which indicate the operating status of the computer device, and columns C25 and C26 are threshold settings that distinguish whether the alarm is serious (Major) or normal (Minor). Note that the alarms here are general ones (preliminary alarms) that indicate the possibility of an abnormality when the CPU usage rate is high, for example, and do not directly indicate an abnormality that is determined in detail by this embodiment. (Preliminary) alarms such as those in columns C23, C25, and C26 can be obtained as application logs.

[0023] All of the items in the lower row of Figure 5 can be obtained in the OS log as logs related to hardware behavior. Columns C7 to C12 in the upper row of Figure 5 can also be obtained in the OS log as logs related to hardware behavior. The engine status in column C3 and the vehicle status in column C4 are also hardware-related and can be obtained in the OS log. The onboard software status in column C6 can be obtained as an application log.

[0024] The vehicle identification number in column C2 can be acquired as an attribute, reflecting the vehicle model, etc. The software version in column C5 can also be acquired as an attribute.

[0025] In the example of Figure 5, the log is described as being for the case where terminal 10 is an in-vehicle terminal mounted on a vehicle, but by replacing the vehicle identification number with the robot identification number and the in-vehicle software state and version with the robot software state and version, a similar log can be obtained when terminal 10 is configured as an autonomous robot or other device.

[0026] In the example of Figure 5, the items that make up the log are explained by showing what is acquired at each of multiple terminals 10 at a certain time, but logs can be collected at each terminal 10 over a certain period of time, not just at a single time, and can be collected in the log collection unit 21 in step S1.

[0027] As attributes to be linked to logs collected over a certain period of time, the attribute recording unit 11 may record all or part of the temperature, humidity, air pressure, weather, climate, and time of day (day or night, etc.) at the location of the terminal 10 at the time the log was obtained.

[0028] In step S2, the learning unit 22 learns by referring to the log collected in the log collection unit 21 in step S1, learns the criteria for determining whether an abnormality has occurred in the terminal 10, and stores the learned criteria in the criteria storage unit 23. The process then proceeds to step S3. In one embodiment, the learned criteria can be configured as a decision tree, examples of which will be described later with reference to FIGS. 6 and 7 . That is, the decision tree can determine whether a log is abnormal or normal, and, if abnormal (or normal), the basis for determining the abnormality (or normality) can be obtained as information on the path of the sequential determination process on the decision tree. Therefore, in one embodiment, the entire structure of such a decision tree can be used as the criteria. Alternatively, a conditional expression for the occurrence of an abnormality (e.g., "CPU usage rate > 90%" and "memory usage rate > 80%) can be derived from the decision tree structure and used as the criteria. In this case, in an embodiment described later with reference to FIG. 9, individual criteria or combinations of criteria defined in individual decision nodes of the decision tree can be used as criteria for determining whether an abnormality exists or whether there is a possibility of an abnormality.

[0029] That is, in the former embodiment, an existing decision tree is used to extract individual elements of the judgment criteria, and at the time of judgment, it is possible to trace the entire decision tree from the root node to the leaf nodes to make a judgment, but in the latter embodiment, it is also possible to use individual judgment criteria extracted from the decision tree individually or in combination. The method of extracting the judgment criteria to be used from the decision tree may use the importance described below. Multiple judgment criteria may be combined manually or based on a predetermined rule (for example, by combining all multiple judgment criteria with the logical AND).

[0030] In addition, even when individual judgment criteria are extracted using techniques other than decision trees, such as machine learning or deep learning, the corresponding importance can be calculated to extract important judgment criteria. As an example, the case where individual judgment criteria are obtained using a decision tree will be described below.

[0031] The learning unit 22 refers to the attributes linked together with the logs in the log collection unit 21, and learns the criteria for determining whether an abnormality has occurred for each attribute. In this embodiment, a decision tree is used for this learning, and for each log, correct answer data indicating whether the log corresponds to normal or abnormal is assigned by manual judgment or the like. (In other words, the assignment of correct answer labels of normal / abnormal to the collected logs is performed in advance externally, and in this embodiment, learning is performed using the results.)

[0032] FIG. 6 is a diagram showing a schematic example of learning a decision tree for each attribute by dividing logs by attribute in the learning unit 22 of this embodiment. Here, all logs LG_all recorded in the log collection unit 21 are divided into, for example, three stages according to the attribute "model year" (the "model year" of the terminal 10 or the vehicle or robot controlled by the terminal 10), into logs LG1 corresponding to newer model years, logs LG2 corresponding to intermediate model years, and logs LG3 corresponding to older model years. (That is, the logs LG1, LG2, and LG3 divided by model year are combined to form the total logs LG_all.) In this way, a decision tree T1 is learned using the log LG1 (and the correct answer label) for the newer model years, a decision tree T2 is learned using the log LG2 (and the correct answer label) for the intermediate model years, and a decision tree T3 is learned using the log LG3 (and the correct answer label) for the older model years.

[0033] These decision trees T1, T2, and T3 are dedicated decision trees corresponding to attributes that are used in step S3, described below, depending on whether the "year of manufacture" attribute of the terminal 10 corresponds to new, medium, or old. Here, it is assumed that the way normal / abnormal events occur is similar among terminals 10 with the same attribute, and differs among terminals 10 with different attributes. Therefore, by learning judgment criteria for each attribute as each of the decision trees T1, T2, and T3, appropriate judgment criteria according to the attribute can be used. Furthermore, each of the decision trees T1, T2, and T3 is expected to have a more compact node structure and scale than the decision tree T_all, which is trained using the entire log LG_all without being separated by attribute.

[0034] On the other hand, in step S2, the learning unit 22 of this embodiment may additionally learn a decision tree by using all logs collected in the log collection unit 21 without dividing them by attributes, as shown as decision tree T_all in Figure 6, so that in addition to decision trees according to attributes, it may also learn a general-purpose decision tree that is not based on attributes.

[0035] In this embodiment, the specific attributes by which the logs are divided and the decision tree for each attribute is learned are manually set in advance, as exemplified by the three model year levels in FIG. 6. Attributes may be prepared for each item of interest. For example, it is possible to learn a decision tree for the attribute "model year" with three levels (new, medium, old), and separately learn a decision tree for the attribute "software version" with three levels (new, medium, old).

[0036] 7 is a diagram showing a schematic example of a decision tree learned by the learning unit 22 using logs for each attribute. In this example, the decision tree is composed of seven nodes, n1 to n7. As is known in the art, a decision tree compares data to be determined (logs in this embodiment) with the decision tree, and by moving the node from the root node n1 to one of the leaf nodes n3, n5, n6, and n7 according to the case classification at each node, a final determination result of normal or abnormal (fault present) defined at the leaf node can be obtained. Note that even before making a determination at each of the determination nodes n1, n2, and n4, a provisional determination result such as "normal" can be obtained as a result of learning the decision tree, as a step before reaching one of the leaf nodes n3, n5, n6, and n7.

[0037] In the decision tree of Figure 7, for example, if the log to be judged has runqlat=30000, cputime=50, and biolatency=300, the root node n1's judgment of "runqlat>65636" is No (false), so it moves from n1 to n2, the node n2's judgment of "cputime>100" is No (false), so it moves from n2 to n4, and the node n4's judgment of "biolatency>250" is Yes (true), so it moves from n4 to n7.From the above, the path resulting from tracing the branches based on the condition judgments from the root node of the decision tree for the log in order is "n1→n2→n4→n7", and the final judgment result at leaf node n7 is "abnormal (fault present)".

[0038] Similarly, in the example of Figure 6, when leaf node n3 is reached for a certain log, it is judged to be "abnormal" because the log item runqlat exceeds the threshold value of 65636. In anomaly judgment using a decision tree, the items used in the judgment criteria and the method of judgment for each item are explicitly obtained, i.e., which one or more items in the log meet what conditions (such as a range of values) to determine whether the log is normal or abnormal. In other words, a decision tree contains information on the judgment criteria items, information on how to judge each item, and information on the final judgment result, making it possible to make structural judgments.

[0039] Any existing method in decision tree algorithms may be used as a method for actually learning a decision tree such as that shown in Fig. 6 in the learning unit 22 using logs labeled as normal / abnormal as learning data. That is, the explanatory variables (input) may be the CPU-related, memory-related, disk-related, and network-related resource usage statuses as log items such as those shown in Fig. 5, and the abnormality determination value (0: normal, 1: abnormal) assigned as a correct label may be used as the objective variable (output), and the log items and conditions to be judged at each node may be determined in order starting from the root node.

[0040] The learning unit 22 can further calculate the importance of each node (a decision node other than a leaf node) of the decision tree by calculating the Gini coefficient or the like using the learned decision tree and the log used during learning. FIG. 8 is a diagram showing an example and a schematic example in which a decision tree is learned for the log of this embodiment and the importance of the decision node is calculated, with the magnitude of importance calculated in descending order of importance being represented by a horizontal bar graph at the top. As can be seen from this graph, the magnitude of importance is biased toward the upper side, and by setting a threshold, only those with high importance can be effectively used as a judgment criterion for determining the possibility of an anomaly, as described below.

[0041] When the first judgment unit 14 or the second judgment unit 24 described below judges an abnormality, the abnormality may be judged in an embodiment using the entire decision tree, or only individual judgment criteria may be used to judge the possibility of an abnormality.

[0042] The learning in step S2 has been described above. Returning to the explanation of the flowchart in FIG. 3, in step S3, the judgment criteria for each attribute criterion learned in step S2 are distributed to each terminal 10, and then the process proceeds to step S4. In step S3, judgment criteria according to the attributes of the terminal 10 are distributed from the judgment criterion storage unit 23 to the first judgment unit 14 of the terminal 10. At this time, general-purpose judgment criteria that do not depend on the attributes may also be distributed to the terminal 10. Furthermore, if the importance of the judgment criteria has been calculated, the importance may be distributed to the terminal 10 together with the judgment criteria.

[0043] When attributes are set separately for each item of interest and judgment criteria are learned, judgment criteria for each item may be distributed to the terminal 10. For example, in the above example, when "three judgment criteria for each attribute of three levels of model year" and "three judgment criteria for each attribute of three levels of software version" are learned, if the attributes of a certain terminal 10 are "model year: new" and "software version: medium", two judgment criteria may be distributed to this terminal 10: a judgment criterion based on the decision tree for "model year: new" and a judgment criterion based on the decision tree for "software version: medium". Furthermore, a new judgment criterion may be generated for this terminal 10 by integrating the judgment criterion based on the decision tree for "model year: new" and the judgment criterion based on the decision tree for "software version: medium", and the generated judgment criterion may be distributed.

[0044] Steps S4 and S5 correspond to a step of determining anomalies in any new log (acquired by terminal 10) other than those used for learning by utilizing the results of the log collection and learning in steps S1 and S2, and a step of collecting additional logs and re-learning a decision tree that represents the judgment criteria, respectively. For convenience, step S5 is placed after step S4 in Fig. 3, but steps S4 and S5 may be reversed or may be performed in parallel.

[0045] In step S4, the first judgment unit 14 uses a new log (separate from the one used for learning) acquired by the log acquisition unit 12 in the terminal 10 to make an abnormality judgment. For this abnormality judgment, the decision tree distributed to the terminal 10 is read as a decision tree corresponding to the attributes of the terminal 10, and is compared with the newly acquired log to find a path from the root node to any of the leaf nodes corresponding to the judgment result, as explained in FIG. 7. The normal or abnormal distinction set in the leaf node finally reached while following the conditional branching by nodes becomes the judgment result of the new log. (Note that this judgment is made when the entire decision tree is distributed to the terminal 10 as the judgment criterion.)

[0046] If the first determination unit 14 of the terminal 10 has received multiple determination criteria, it makes a determination based on all of them, and if any one of them determines that there is an abnormality (or a possibility of an abnormality), it issues an alarm to the log processing device 20. In the above example, for example, a certain terminal 10 makes an abnormality determination based on the determination criterion based on the decision tree of "year: new" and also based on the determination criterion based on the decision tree of "software version: medium", making a total of two determinations, and if there is an abnormality (or a possibility of an abnormality) determined in at least one of them, it issues an alarm.

[0047] In another embodiment, the log acquired by the log acquisition unit 12 of the terminal 10 may be transmitted via the network NW to the log processing device 20 serving as a server, and then an abnormality determination may be performed by the second determination unit 24 of the log processing device 20. The second determination unit 24 may read the corresponding determination criterion from the determination criterion storage unit 23 and perform an abnormality determination.

[0048] When the second determination unit 24 of the log processing device 20 performs an abnormality determination, it is necessary to transmit the log of the terminal 10 from the terminal 10 that is the target of the abnormality determination to the log processing device 20. However, for this transmitted log, it is possible to transmit only the log items that appear in the determination node of the decision tree of the determination criteria corresponding to the attributes of the terminal 10 for determining an abnormality for the terminal 10, rather than all the items that constitute the log acquired by the log acquisition unit 12. For example, if the decision tree corresponding to the attributes of a certain terminal 10 is the one illustrated in Fig. 7, the log items to be transmitted to the log processing device 20 are only the three items runqlat, cputime, and biolatency, and other items (for example, memory usage rate, disk usage rate, etc.) do not need to be transmitted.

[0049] As described above, the first determination unit 14 and the second determination unit 24 may determine the presence or absence of an abnormality by using the entire decision tree according to the attribute. On the other hand, when the second determination unit 24 performs an abnormality determination, although it is expected that the number of items constituting the determination criteria will be reduced by using the decision tree according to the attribute, there is a concern that a considerable amount of data will be generated if all of the necessary log items are transmitted over the network NW to perform all of the determinations in the decision tree. When transmitting logs from the terminal 10 to the log processing device 20, an alert indicating that an abnormality may have occurred in the terminal 10 (an abnormality that is suspected) is issued, and logs are transmitted to the log processing device 20 only from the terminal 10 that issued the alert, thereby reducing the amount of data generated in the log use system 100 as a whole.

[0050] In order to reduce the amount of data in this way (or similarly, to select only logs that are suspected of having an abnormality and subject them to detailed analysis), the first judgment unit 14 and the second judgment unit 24 may make judgments based on the judgment criteria of only a part of the decision tree (an excerpt of only the individual judgment nodes that make up the decision tree) rather than the entire decision tree.

[0051] Specifically, the first judgment unit 14 is distributed with only individual judgment criteria with high importance as judgment criteria, rather than the entire decision tree, and when a terminal 10 judges that there is an abnormality using a judgment criterion with high importance, it sends an alert to that effect to the log processing device 20, and the log processing device 20 that receives the alert may make a judgment using the entire decision tree corresponding to the attributes of the terminal 10 as judgment criteria in order to conduct a more detailed investigation.

[0052] The importance value of the criterion at each decision node of the decision tree may be, for example, the Gini coefficient used during learning by the learning unit 22. Of the criterion constituting the decision tree, the criterion that determines an abnormality as the final result (leaf node) by the judgment based on that criterion and that has a high importance in threshold judgment may be used as the individual criterion for issuing an alert that an abnormality may have occurred.

[0053] 9 is a diagram showing an example for explaining the use of only highly important judgment criteria in this embodiment. As shown in FIG. 9, the entire trained decision tree is composed of judgment nodes n10 to n14 as individual judgment criteria and leaf nodes n20 to n25 as final judgment results, with judgment starting from the root node n10. In this decision tree, judgment criteria that can give the judgment result "abnormal" (leaf nodes n22, n23, n25) are n12, n13, and n14. In this embodiment, judgment criteria that give the judgment result "abnormal" are selected to have high importance from among the judgment criteria that give the judgment result "abnormal," and for example, judgment criteria n12 and n13 are selected to have high importance in the importance threshold judgment.

[0054] In this embodiment, to issue an alert indicating the possibility of an abnormality, it is sufficient to use the criteria n12 and n13, which are determined to have high importance as follows, individually. Note that, as described above, multiple such individual criteria may be combined on a rule basis, etc. For example, if an abnormality is determined in at least one of n12 and n13, it may be determined that "an abnormality is possible." "Anomaly was detected only in the highly important criterion n12" → "Possible anomaly" "Anomaly was detected only in the highly important criterion n13" → "Possible anomaly"

[0055] That is, in the normal use of a decision tree, whether or not there is an abnormality based on the judgment criterion n12 is determined by a series of judgments from the root node n10 to the judgment criterion n12, such as "n10 → n11 → n12." However, this method increases the number of items to be judged. Therefore, in this embodiment, judgment is made only for individual judgment criteria, such as n12 and n13, which have high importance, and if an abnormality is judged, it is judged that "there is a possibility of an abnormality." The actual detailed judgment can be made automatically or manually on the server side that receives the alert.

[0056] When the terminal 10 receives the distribution of N determination criteria with high importance as a plurality of determination criteria, the first determination unit 14 of the terminal 10 may use all of these N determination criteria to determine the presence or absence of the possibility of abnormality, or according to the situation where the terminal 10 is placed, it may be possible to distinguish whether to use all or only a part of the N determination criteria from all of the N determination criteria. Here, it is assumed that the more determination criteria are used, the higher the processing load of the terminal 10 will be. Conversely, the fewer the determination criteria, the higher the probability of issuing an alert as having the possibility of abnormality, and each time, it is necessary to perform a detailed determination on the side of the log processing device 20, and it is assumed that the data transmission load on the network NW will increase.

[0057] Therefore, for example, when the processing load of the terminal 10 (the load can be evaluated by, for example, the aforementioned CPU usage rate, memory usage rate, disk usage rate, etc.) is larger, the terminal 10 may use only a smaller part of the N determination criteria from all of the N determination criteria to determine the possibility of abnormality, so that the number of determination criteria used can be adjusted. (For example, when the processing load of the terminal 10 is "small", all N determination criteria are used, when it is "medium", N / 2 determination criteria are used, and when it is "large", N / 4 determination criteria are used, etc. Set a predetermined rule for how many of the total N determination criteria to use according to the specific load situation, and adjust the number of determination criteria used according to this.) Here, when using only a part of N' (N' < N) determination criteria from all N, by selecting N' from the side with higher associated importance and using them, it is possible to ensure the accuracy as much as possible even with fewer determination criteria and determine the possibility of abnormality. Conversely, when the traffic volume of the network NW (the load of the network NW) is large, it may be possible to use more partial determination criteria to determine the possibility of abnormality, so that the number of determination criteria used can be adjusted.

[0058] In step S5, the learning unit 22 performs learning again using the results learned in steps S1 and S2, and also using the learning data newly added to the log collection unit 21, and re-learns the judgment criteria using the decision tree for each attribute. During this re-learning, it is necessary to assign a correct answer label of normal / abnormal to each log that constitutes the added learning data. However, instead of assigning the correct answer label manually, the correct answer label may be automatically assigned to the log by performing the same process as the first judgment unit 14 or the second judgment unit 24 using the decision tree for each attribute that has already been learned.

[0059] As described above, according to the embodiment of the present invention, in a log that is expected to contain a large number of items, the learning results using the decision tree can be used to obtain important items and the conditions under which those items fall to determine whether the log represents an anomaly.

[0060] Various supplementary examples, alternative examples, additional examples, etc. will be described below.

[0061] (1) According to an embodiment of the present invention, the ability to automatically determine judgment criteria can improve the efficiency of log analysis, which is necessary for the development and verification of next-generation technologies such as autonomous robots and autonomous vehicles, and can facilitate remote log monitoring via a network while reducing the amount of communication data. This can contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "build infrastructure, promote inclusive and sustainable industrialization, and promote innovation and foster resilience."

[0062] (2) In the above, the labels of the correct answer data during learning were either normal or abnormal. However, even if the definition of “abnormal” is expanded to “normal” or “abnormality has occurred or a sign of abnormality has been detected,” the labeling of 0 or 1 and the target variable are the same, and therefore the embodiments of the present invention can be applied in the same manner as described above. When the definition is expanded in this way, the determination result of the first determination unit 14 or the second determination unit 24 is also expanded to “normal” or “abnormality has occurred or a sign of abnormality has been detected.” Note that, regarding “abnormality,” a predetermined event may be defined as an abnormality in the terminal 10 itself and / or an abnormality in an object controlled by the terminal 10, and a correct answer label indicating whether or not the predetermined event applies during learning may be assigned. When the terminal 10 is responsible for autonomous driving, for example, predetermined abnormal events may be defined in advance with respect to the behavior of sensors such as LiDAR required for autonomous driving, or the processing of the sensor data to estimate the terminal's position, etc.

[0063] (3) FIG. 10 is a diagram showing an example of the hardware configuration of a general computer device 70. The terminal 10 and the log processing device 20 in the log utilization system 100 can be realized as one or more computer devices 70 having such a configuration. When the terminal 10 and the log processing device 20 are realized by two or more computer devices 70, information required for processing may be transmitted and received via a network. The computer device 70 includes a CPU (Central Processing Unit) 71 that executes predetermined instructions, a GPU (Graphics Processing Unit) 72 as a dedicated processor that executes some or all of the CPU 71's execution instructions in place of or in cooperation with the CPU 71, a RAM 73 as a main memory device that provides a work area for the CPU 71 (and GPU 72), a ROM 74 as an auxiliary memory device, a communication interface 75, a display 76 that displays and outputs images, an input interface 77 that accepts user input via a mouse, keyboard, touch panel, etc., a speaker 78 that outputs audio, and a bus BS for transmitting and receiving data among them.

[0064] Each functional unit in the terminal 10 and the log processing device 20 can be realized by a CPU 71 and / or a GPU 72 that reads from a ROM 74 and executes a predetermined program corresponding to the function of each unit. Both the CPU 71 and the GPU 72 are types of arithmetic units (processors). Here, when display-related processing is performed, a display 76 also operates in conjunction with the processing, when communication-related processing related to data transmission and reception is performed, a communication interface 75 also operates in conjunction with the processing, and when audio output-related processing is performed, a speaker 78 also operates in conjunction with the processing. [Explanation of symbols]

[0065] 100... log utilization system, 10... terminal, 20... log processing device, 11... attribute storage unit, 12... log acquisition unit, 13... operation unit, 14... first judgment unit, 21... log collection unit, 22... learning unit, 23... judgment criterion storage unit, 24... second judgment unit

Claims

1. A log processing device, Execute the software, and accept at least a part of the log output by the software from a terminal that acquires the log as a learning log; receiving a correct answer label indicating whether an abnormality or a sign of an abnormality has occurred or whether the operation related to the terminal is normal in the learning log; By learning using the learning log and the correct answer label, A log processing device characterized by obtaining a judgment criterion for identifying which of the multiple items constituting the log corresponds to what condition and whether the log indicates or is likely to indicate the occurrence of an abnormality or a sign of an abnormality.

2. 2. The log processing device according to claim 1, wherein the log items accepted as the learning log include at least one of items related to processor usage, items related to memory usage, and items related to storage usage.

3. The log processing device according to claim 1 , wherein the judgment criterion is obtained for each attribute of the terminal by learning using the learning log and the correct label for each attribute of the terminal.

4. 2. The log processing device according to claim 1, wherein the attributes of the terminal include at least one of the version of the software, the model of the terminal, the year of manufacture of the terminal, and the number of years in operation of the terminal.

5. 2. The log processing device according to claim 1, wherein the terminal is mounted on a mobile object, and the software performs processing related to the movement of the mobile object.

6. The log processing device according to claim 1 , wherein the judgment criteria are obtained by learning using a decision tree.

7. 2. The log processing device according to claim 1, wherein a plurality of the criteria are obtained and the importance of each of the criteria is calculated.

8. A terminal that executes software and acquires logs output by the software, receiving a determination criterion for identifying which item among the plurality of items constituting the log corresponds to what condition, thereby indicating or possibly indicating that the log indicates the occurrence of an abnormality or the presence of a sign of an abnormality; A terminal characterized in that it uses the judgment criteria to identify whether the log indicates or is likely to indicate the occurrence of an abnormality or the presence of a sign of an abnormality.

9. receiving the criteria as a plurality of criteria corresponding to a plurality of attributes corresponding to the terminal; 9. The terminal according to claim 8, wherein the terminal uses each of the plurality of criteria to determine whether the log indicates the occurrence of an abnormality or a sign of an abnormality.

10. receiving a plurality of said criteria and a calculated importance for each criterion; The terminal according to claim 8, characterized in that when determining whether the log may indicate the occurrence of an abnormality or the presence of a sign of an abnormality, the number of judgment criteria used is adjusted by giving priority to judgment criteria with higher importance.

11. A log processing method, comprising: Execute the software, and accept at least a part of the log output by the software from a terminal that acquires the log as a learning log; receiving a correct answer label indicating whether an abnormality or a sign of an abnormality has occurred or whether the operation related to the terminal is normal in the learning log; By learning using the learning log and the correct answer label, A log processing method characterized by obtaining a judgment criterion for identifying which of the multiple items constituting the log corresponds to what condition, and whether the log indicates or is likely to indicate the occurrence of an abnormality or the presence of a sign of an abnormality.

12. A program that causes a computer to function as a log processing device, The log processing device includes: Execute the software, and accept at least a part of the log output by the software from a terminal that acquires the log as a learning log; receiving a correct answer label indicating whether an abnormality or a sign of an abnormality has occurred or whether the operation related to the terminal is normal in the learning log; By learning using the learning log and the correct answer label, A program characterized by obtaining a judgment criterion for identifying which of the multiple items constituting the log corresponds to what condition and whether the log indicates or is likely to indicate the occurrence of an abnormality or a sign of an abnormality.

Images