Apparatus for providing notification information, and method therefor

Abstract

According to the present disclosure, a method by which an electronic device provides information is disclosed, the method comprising the steps of: identifying one or more indicators related to the provision of a service; acquiring history information related to a first indicator included in the one or more indicators; acquiring, on the basis of the history information, threshold information corresponding to the first indicator; applying the threshold information; and determining, on the basis of the threshold information, whether to provide a notification related to the first indicator.

Description

Device and method for providing notification information

[0001] The present disclosure relates to a device and method for providing notification information. More specifically, the present disclosure relates to an electronic device and method for obtaining threshold information corresponding to a first indicator related to service provision based on history information, applying the threshold information, and determining whether to provide a notification related to the first information based on the threshold information.

[0002] With the advancement of electronic technology, electronic services have become ubiquitous in our daily lives. In relation to the provision of electronic services, various indicators, such as CPU usage and the number of log errors, can be collected. Based on these indicators, service-related anomalies can be manually determined or automatically determined by electronic devices.

[0003] However, when determining whether there is an abnormality related to a service based on these indicators, if the criteria for determination are static, it is difficult to quickly detect and respond to cases where the criteria are not properly set, and thus there is a possibility of undesirable situations occurring, such as cases where the presence of an abnormality related to a service is not properly detected or detected overly sensitively.

[0004] In this regard, reference can be made to prior literature such as KR102183897B1 and KR100898339B1.

[0005] The problem that the present embodiment seeks to solve is to provide an electronic device and method for identifying one or more indicators related to the provision of a service, obtaining history information related to a first indicator included in the one or more indicators, obtaining threshold information corresponding to the first indicator based on the history information, applying the threshold information, and determining whether to provide a notification related to the first indicator based on the threshold information, in order to solve the above-described problem.

[0006] The technical tasks to be achieved by this embodiment are not limited to the technical tasks described above, and other technical tasks can be inferred from the following embodiments.

[0007] A method for providing information in an electronic device according to one embodiment includes: a step of identifying one or more indicators related to provision of a service; a step of obtaining history information related to a first indicator included in the one or more indicators; a step of obtaining threshold information corresponding to the first indicator based on the history information; a step of applying the threshold information; and a step of determining whether to provide a notification related to the first indicator based on the threshold information.

[0008] According to one embodiment, the step of acquiring the critical information may include the step of checking a plurality of past data related to the first indicator included in the history information; and the step of setting the critical condition so that a group of data satisfying the critical condition among the plurality of past data satisfies a set criterion.

[0009] According to one embodiment, the method for providing the information further includes: a step of determining whether a problem related to the first indicator has actually occurred in the provision of the service based on the history information; and a step of setting the criterion based on the result of determining whether a problem related to the first indicator has actually occurred.

[0010] According to one embodiment, the step of obtaining the critical information may include the step of providing one or more inputs including the history information to a machine learning model; and the step of obtaining an output of the machine learning model.

[0011] According to one embodiment, the one or more inputs may include criteria set in relation to threshold conditions.

[0012] According to one embodiment, the threshold information may include upper bound information and lower bound information for determining whether data related to the first indicator is normal.

[0013] According to one embodiment, the critical information may include at least one of a first critical condition associated with a warning notification related to the first indicator and a second critical condition associated with an error occurrence notification related to the first indicator.

[0014] According to one embodiment, the method for providing the information may further include a step of determining a time point for applying the threshold information.

[0015] According to one embodiment, the step of determining a time point for applying the critical information may include a step of determining a chronological relationship between a time point for receiving data related to the first indicator and a time point for applying the critical information, based on one or more attributes related to the first indicator.

[0016] According to one embodiment, the method for providing the information may further include the step of verifying whether the critical information is appropriate; and, if the critical information is not appropriate, the step of providing information notifying the user that the critical information is inappropriate.

[0017] According to one embodiment, the method for providing the information may further include a step of setting a period for obtaining the history information based on one or more attributes related to the first indicator.

[0018] An electronic device providing information according to one embodiment includes a memory storing instructions and a processor, wherein the processor is connected to the memory, verifies one or more indicators related to provision of a service, obtains history information related to a first indicator included in the one or more indicators, obtains threshold information corresponding to the first indicator based on the history information, applies the threshold information, and determines whether to provide a notification related to the first indicator based on the threshold information.

[0019] Specific details of other embodiments are included in the detailed description and drawings.

[0020] According to the present disclosure, an electronic device and method for providing notification information can increase the efficiency of service operation by dynamically acquiring critical information related to notification provision.

[0021] In addition, according to the present disclosure, an electronic device and method for providing notification information can reduce the cost required for setting critical information by increasing the scalability of critical information.

[0022] The effects of the invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

[0023] Figure 1 is a schematic diagram showing a system that provides notification information according to one embodiment.

[0024] Figure 2 is a drawing for exemplifying a conventional notification information provision method.

[0025] FIG. 3 is a flowchart illustrating a method for providing notification information in an electronic device according to an embodiment.

[0026] FIG. 4 is a drawing for exemplarily explaining a screen that displays critical information corresponding to one or more indicators according to one embodiment.

[0027] Figure 5 is a drawing for exemplarily explaining a screen for setting threshold information according to one embodiment.

[0028] FIG. 6 is a drawing for explaining an example of a method for providing notification information according to one embodiment.

[0029] Figure 7 is a drawing for exemplarily explaining notification information according to one embodiment.

[0030] Figure 8 is an exemplary diagram of the configuration of an electronic device according to one embodiment.

[0031] The terms used in the embodiments have been selected from widely used and common terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the present disclosure.

[0032] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part" and "module" used in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0033] The expression “at least one of a, b, and c” described throughout the specification may encompass ‘a alone’, ‘b alone’, ‘c alone’, ‘a and b’, ‘a and c’, ‘b and c’, or ‘all of a, b, and c’.

[0034] The "terminal" mentioned below may be implemented as a computer or portable terminal that can connect to a server or other terminal via a network. Here, the computer includes, for example, a notebook, desktop, laptop, etc. equipped with a web browser, and the portable terminal may include, for example, a wireless communication device that guarantees portability and mobility, and may include all types of handheld-based wireless communication devices such as communication-based terminals such as IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), LTE (Long Term Evolution), smartphones, tablet PCs, etc.

[0035] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0036] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0037] In describing the embodiments, descriptions of technical details that are well known in the technical field to which the present invention pertains and are not directly related to the present invention will be omitted. This is to avoid obscuring the gist of the present invention by omitting unnecessary explanations and to convey the gist more clearly.

[0038] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0039] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0040] At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions. These computer program instructions can be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in the flow diagram block(s). These computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce a manufactured item that includes an instruction means for performing the functions described in the flow diagram block(s). Since the computer program instructions may be installed on a computer or other programmable data processing device, a series of operational steps may be performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for performing the functions described in the flowchart block(s) may also provide steps for performing the functions described in the flowchart block(s).

[0041] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0042] Figure 1 is a schematic diagram showing a system that provides notification information according to one embodiment.

[0043] According to various embodiments, a system for providing notification information includes an electronic device (110). According to one embodiment, the system for providing notification information may further include a user terminal (120). According to one embodiment, the system for providing notification information may further include a network that supports information transmission and reception between the electronic device (110), the user terminal (120), and at least some of the external devices.

[0044] Each of the electronic device (110) and the user's terminal (120) may include a transceiver, memory, and a processor. In addition, each of the electronic device (110) and the user's terminal (120) refers to a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software.

[0045] Meanwhile, throughout the embodiments, the electronic device (110) and the user's terminal (120) are each referred to as separate devices or servers, but this may be a logically divided structure, and at least some of them may be implemented by separate functions in a single device or server. For example, throughout the present disclosure, the electronic device (110) and the user's terminal (120) may be implemented by a single device, and in this case, the process of transmitting and receiving information between the electronic device (110) and the user's terminal (120) may actually be understood as a process in which data is exchanged within a single device.

[0046] According to one embodiment, the electronic device (110) and the user's terminal (120) may include a plurality of computer systems or computer software implemented as network servers. For example, at least some of the electronic device (110) and the user's terminal (120) may refer to a computer system and computer software that are connected to a lower-level device that can communicate with other network servers through a computer network such as an intranet or the Internet, receive a task execution request, perform the task in response, and provide the execution result. In addition, at least some of the electronic device (110) and the user's terminal (120) may be understood as a broad concept that includes a series of application programs that can operate on a network server, and various databases built on other internal or connected nodes. For example, at least some of the electronic device (110) and the user's terminal (120) may be implemented using a network server program that is provided in various ways according to an operating system such as DOS, Windows, Linux, UNIX, or MacOS.

[0047] Meanwhile, for the convenience of explanation, each operating entity is referred to as an electronic device (110) and a user's terminal (120), but these should be understood as comprehensive devices that correspond to various types of devices such as computer devices and mobile communication terminals, include various types of devices, or can be included in various types of devices.

[0048] The electronic device (110) is a device that acquires and provides various information. The electronic device (110) can perform various tasks to provide information. Specifically, the electronic device (110) verifies one or more indicators related to the provision of a service, acquires history information related to a first indicator included in one or more indicators, acquires threshold information corresponding to the first indicator based on the history information, applies the threshold information, and determines whether to provide a notification related to the first indicator based on the threshold information.

[0049] The electronic device (110) may be a device that controls, manages, or supervises the overall operation of the service. Alternatively, the electronic device (110) may be a device used by a person who sets threshold conditions (e.g., a developer or a person in charge of a specific function related to the service). Furthermore, the electronic device (110) may include various types of devices and is not limited to a specific device.

[0050] According to one embodiment, the electronic device (110) may provide a user with a notification related to a first indicator. Users who receive notification information may include the owner or administrator of the service, users of the service (e.g., customers), or even individuals responsible for specific functions related to the service. Depending on the embodiment, various entities may be included in the term "user."

[0051] The user's terminal (120) may be a device operated and managed by the user. The user's terminal (120) may receive input from the user or information from a device such as an electronic device (110) and perform corresponding actions. For example, the user's terminal (120) may receive information from the electronic device (110) indicating that critical information is inappropriate and provide the information to the user (e.g., display it on a screen).

[0052] More detailed information regarding the operation of the electronic device (110) and the user's terminal (120) will be described later in FIGS. 2 to 7, etc.

[0053] The electronic device (110) and the user's terminal (120) may include a computer device, a mobile communication terminal, a server, etc. The electronic device (110) may include or be connected to an input device such as a touchpad, a mouse, or a keyboard for receiving input, and in some embodiments, the user's terminal (120) may also be the same. In addition, the electronic device (110) may include or be connected to an output device such as a screen, a speaker, or an interface device for providing information to a user, and in some embodiments, the user's terminal (120) may also be the same. Furthermore, the input devices and output devices of the electronic device (110) and the user's terminal (120) may be configured as one unit or may be interrelated, and for example, an interface for receiving input may be displayed on at least one of the electronic device (110) and the user's terminal (120).

[0054] The operations related to a series of information providing methods according to various embodiments may be implemented by a single physical device, or may be implemented in a manner in which multiple physical devices are organically combined. For example, some of the components included in a system for providing notification information may be implemented by a single physical device, and the remaining parts may be implemented by another physical device. For example, one physical device may be implemented as part of the electronic device (110), and the other physical device may be implemented as part of the user's terminal (120). In some cases, each component included in a system for providing notification information may be distributed and arranged in different physical devices, and the distributed components may be organically combined to perform the functions and operations of the system for providing notification information. For example, the electronic device (110) of this specification includes at least one sub-device, and some operations described as being performed by the electronic device (110) may be performed by a first sub-device, and other operations may be performed by a second sub-device.

[0055] Figure 2 is a drawing for exemplifying a conventional notification information provision method.

[0056] Referring to FIG. 2, in the conventional notification information provision according to one embodiment, threshold conditions are set by a person, and the monitoring center periodically checks whether the threshold conditions are satisfied and provides a notification to the user when the threshold conditions are exceeded. According to the conventional method, there was a problem in that the threshold conditions may not effectively reflect the presence of an abnormality related to the service because there was no process for automatically determining whether the threshold conditions were set appropriately or a process for automatically setting (or resetting) the threshold conditions. For example, in the case of the conventional method, there was a high possibility that the threshold conditions were set too tightly so that a notification was provided to the user even though no abnormality occurred, or too loosely so that a notification was not provided to the user even though an abnormality occurred. In addition, there was a limitation in that there was no process for correcting such situations when they occurred.

[0057] FIG. 3 is a flowchart illustrating a method for providing notification information in an electronic device according to an embodiment.

[0058] Referring to FIG. 3, an electronic device (110) according to one embodiment checks one or more indicators related to the provision of a service (310). The one or more indicators related to the provision of a service may include indicators that can be referenced to determine whether an anomaly has occurred related to the service. The anomalies related to the service may include anomalies occurring during the service implementation process and anomalies occurring during the service operation process, as well as various other types of anomalies.

[0059] An example of one or more indicators related to the provision of a service can be understood with reference to FIG. 4. Referring further to FIG. 4, an example of a screen (400) displaying threshold information corresponding to one or more indicators is illustrated. In the example of FIG. 4, the one or more indicators may include the number of container restarts (401), CPU usage (402), memory usage (403), the number of times a POD is detected as unhealthy (404), the number of times the current thread is detected as busy (405), Full GC time (406), Minor GC time (407), and the number of log errors (408). In addition, in the example of FIG. 4, threshold conditions set for each indicator can be utilized. For example, in the case of CPU usage (402), it can be understood that the first threshold condition is satisfied when the value exceeds 90 10 times, and the second threshold condition is satisfied when the value exceeds 70 5 times. Meanwhile, in the case of CPU usage (402), the values ​​specified as 90, 70, etc. represent the ratio of CPU capacity being used as a percentage, and it can be understood that the unit has been omitted.

[0060] The screen (400) illustrated in FIG. 4 can be understood as being provided to the administrator or owner of the service. That is, the screen (400) illustrated in FIG. 4 can be understood as a screen for viewing and managing critical conditions at a glance. In addition to displaying the critical conditions of each indicator, the screen (400) illustrated in FIG. 4 may further display a user interface (410) for creating a new rule, a user interface for copying a rule, or other various functions, depending on the embodiment, on the screen (400).

[0061] Meanwhile, in the example of Fig. 4, it can be confirmed that multiple critical conditions exist in relation to each indicator. In the example of Fig. 4, the first critical condition is a critical condition related to an error occurrence notification, and the second critical condition can be understood as a critical condition related to a warning notification. For convenience of explanation, the critical condition related to a warning notification will be referred to as the first critical condition, and the critical condition related to an error occurrence notification will be referred to as the second critical condition.

[0062] Meanwhile, while FIG. 4 illustrates an example in which the first and second critical conditions exist for all indicators, the scope of the present disclosure is not limited thereto. For example, indicators that do not have at least one of the first and second critical conditions, as well as indicators that have a new type of third critical condition, may also fall within the scope of the present disclosure. Each critical condition and further details will be described below.

[0063] Returning to Figure 3, the electronic device (110) acquires historical information related to a first indicator included in one or more indicators (320). The historical information may include multiple past data points related to the first indicator. For example, if the first indicator is "CPU usage," the historical information may include data measuring CPU usage at various points in the past.

[0064] According to one embodiment, the electronic device (110) can acquire history information for a preset period of time. According to another embodiment, the electronic device (110) can set a period of time for acquiring history information and acquire history information based on the set period of time.

[0065] In a more specific example related to the operation of setting a period for acquiring history information, the electronic device (110) may set a period for acquiring history information based on one or more attributes related to the first indicator. The "one or more attributes related to the first indicator" for setting the period for acquiring history information may include, for example, whether the indicator is related to payment. In this regard, in one example, the electronic device (110) may set the period for acquiring history information for an indicator related to payment to 30 days, while setting the period for acquiring history information for other indicators to 7 days, thereby enabling more precise threshold information acquisition for indicators related to payment.

[0066] Meanwhile, regarding the operation of drawing number 320 and the operations below, the explanation based on "the first indicator included in one or more indicators" is for convenience of explanation, and should not be understood as acquiring history information or critical information only for one of the indicators related to service provision. In other words, the operations related to the above or below "first indicators" should be understood to mean that such operations can be performed for at least some of the one or more indicators related to service provision.

[0067] The electronic device (110) acquires threshold information corresponding to the first indicator based on the history information (330). As described above in the description with reference to FIG. 4 , the threshold information corresponding to the first indicator according to an embodiment may include at least one of a first threshold condition associated with a warning notification related to the first indicator and a second threshold condition associated with an error occurrence notification related to the first indicator; however, the scope of the present disclosure is not limited thereto.

[0068] According to one embodiment, the electronic device (110) may check a plurality of past data related to the first indicator included in the history information, and set a threshold condition so that a group of data satisfying the threshold condition among the plurality of past data satisfies the set criterion. As an example, the electronic device (110) may set the threshold condition so that the proportion of data satisfying the threshold condition among the plurality of past data is 80% to 85%. As another example, the electronic device (110) may set the threshold condition so that the number of data satisfying the threshold condition among the plurality of past data is 100 or more and 120 or less.

[0069] As a more direct example to help understanding, if the values ​​of past data are 6, 6, 7, 7, 6, 8, 11, 8, 9, and 6, the electronic device (110) can set the upper threshold included in the critical condition to 8.5 so that the proportion of data satisfying the critical condition among these is 80% (in this case, among 10 data, data with values ​​9 and 11 do not satisfy the critical condition, and the rest satisfy the critical condition).

[0070] According to one embodiment, based on the history information, the electronic device (110) can determine whether a problem related to the first indicator has actually occurred in the provision of the service. The electronic device (110) can set a criterion (e.g., a threshold condition) based on the result of determining whether a problem related to the first indicator has actually occurred. For example, if a problem related to the first indicator has actually occurred within a certain period of time, the electronic device (110) can set the threshold condition so that the proportion of data satisfying the threshold condition among a plurality of past data is 80% to 85%. On the other hand, if a problem related to the first indicator has not actually occurred within a certain period of time, the electronic device (110) can set the threshold condition so that the proportion of data satisfying the threshold condition among a plurality of past data is 90% to 95%. In this way, for indicators where a problem has actually occurred, more data can be determined as data subject to notification provision, thereby enabling more stringent monitoring.

[0071] According to one embodiment, the operation of acquiring the critical condition of drawing number 330 may include the operation of providing one or more inputs including history information to a set model and the operation of acquiring the output of the set model. In this case, the output of the set model may be the critical information itself or may be information that serves as the basis for acquiring the critical information. If the output of the set model is not critical information, the electronic device (110) may further perform the operation of acquiring the critical information based on the output of the set model.

[0072] An example of a configured model may include a "machine learning model." In this case, the "machine learning model" that is provided with one or more inputs containing historical information may be understood as a pre-trained model. The training method of the machine learning model may include a training method similar to (but not limited to) the "method for setting a threshold condition" described above. For example, the machine learning model may be trained to provide an output for setting a threshold condition such that the proportion of data satisfying the condition is 80% to 85%. Alternatively, the machine learning model may be trained to provide an output for setting a threshold condition such that the number of data satisfying the threshold condition among a plurality of historical data is 100 to 120.

[0073] In one embodiment, a model may not be trained targeting a specific threshold condition. In such a case, one or more inputs provided to the model by the electronic device (110) may include "criteria set in relation to the threshold condition." That is, if the model is not trained targeting a specific threshold condition, the model output reflecting the input can be obtained by providing the criteria set in relation to the threshold condition as input to the model.

[0074] According to one embodiment, algorithms such as Prophet, STL, ARIMA, and BiLSTM can be used to train a machine learning model. Furthermore, various types of optimization techniques can be used to train and utilize the machine learning model, including, for example, single-cycle / multi-cycle identification, trend identification, outlier identification, burr identification, and change point identification.

[0075] In some embodiments, the operation of acquiring the threshold condition of drawing number 330 may include an operation of receiving a threshold condition set by a user. In this regard, referring further to FIG. 5, an example of a screen for setting threshold information by a user is illustrated (500). The screen illustrated in FIG. 5 may include, but is not limited to, a page provided based on user input corresponding to the user interface (410) of FIG. 4.

[0076] The page displayed on the screen of FIG. 5 may be a page including a threshold condition setting function. For example, a user may specify a first indicator, such as “CPU usage,” based on the indicator setting function (510) of the page, and may set a threshold condition based on the notification condition setting function (520). The notification condition setting function (520) according to an example may include detailed functions for setting threshold conditions for each detailed notification type, and the detailed functions may include, for example, a detailed function for setting threshold conditions for error notifications (see drawing number 521) and a detailed function for setting threshold conditions for warning notifications (see drawing number 522). Furthermore, depending on the example, detailed functions related to other types of notifications may be further included, such as a detailed function of drawing number 523 for setting threshold conditions for information notifications.

[0077] Meanwhile, with respect to the example of FIG. 5, the critical condition set based on the detailed function of drawing number 521 can be understood as the "second critical condition associated with error occurrence notification" described above, the critical condition set based on the detailed function of drawing number 522 can be understood as the "first critical condition associated with warning notification" described above, and the critical condition set based on the detailed function of drawing number 523 can be understood as the "new type of third critical condition" described above.

[0078] In the example of FIG. 5, if we look at the detailed functions of drawing numbers 521 and 522, we can see that information such as a comparator, a threshold, a duration, and a notification recipient (e.g., “to administrator, writer”) can be input. This is not limited to specific embodiments in which a threshold condition is set by a user, and can be understood as an example of information that the threshold information corresponding to the first indicator can generally have. More generally, the threshold information acquired through the operation of drawing number 330 can include a provision condition of the notification for each of one or more notification types, and the provision condition can include a threshold value (associated with “threshold” in the example of FIG. 5) and whether the threshold value is an upper limit or a lower limit (associated with “comparator” in the example of FIG. 5), and depending on the embodiment, it can further include a duration related to satisfaction or dissatisfaction of the threshold condition (associated with “duration” in the example of FIG. 5).

[0079] Based on the above explanation, to understand the threshold information set according to the information entered in the example of Fig. 5, it can be understood as threshold information including "a threshold condition that provides an error notification to the administrator and author when a state in which CPU usage exceeds 90% continues for 10 minutes or more" and "a threshold condition that provides a warning notification to the author when a state in which CPU usage exceeds 70% continues for 5 minutes or more."

[0080] Returning to FIG. 3, the threshold information according to one embodiment may include both upper bound information and lower bound information for determining whether data related to the first indicator is normal. In one example, some of the multiple threshold conditions included in the threshold information may include upper bound information, and at least some of the remaining conditions may include lower bound information. For example, with respect to the first indicator, threshold information related to a warning notification may include upper bound information, and threshold information related to an error notification may include lower bound information.

[0081] In another example, a specific threshold condition contained in critical information may include both upper and lower bounds. For example, the "critical condition that provides a warning notification to the author when CPU usage is below 10% for more than 10 minutes" can be combined with the aforementioned "critical condition that provides a warning notification to the author when CPU usage exceeds 70% for more than 5 minutes" using an OR operator to form the "critical condition that provides a warning notification to the author when CPU usage is below 10% for more than 10 minutes or CPU usage exceeds 70% for more than 5 minutes." Using CPU usage as an example, if CPU usage is excessively high, a warning should be issued, but if CPU usage is excessively low, there is a possibility that the service operation efficiency is reduced (for example, if CPU capacity is secured too high compared to the resource demand), CPU usage measurement is not being performed correctly, or there are other system problems, so warning conditions can be set for these.

[0082] With regard to the operation of drawing number 330, it has been described that it includes an operation of receiving a threshold condition set by a user. However, even in this case, the threshold condition set by the user does not have to be static information that does not change. For example, the electronic device (110) may acquire the threshold information by partially changing the threshold condition set by the user, and even if it does not change it immediately while performing the operation of drawing number 330, it may be changed in the process of re-acquiring the history information related to the first indicator later.

[0083] The electronic device (110) applies the acquired threshold information (340). The operation of "applying threshold information" of drawing number 340 can be understood as an operation of setting threshold information in the system or an operation of changing system settings based on the threshold information. By applying the threshold information, whether to provide a notification related to the first indicator can be determined based on the applied threshold information. The operation of applying the threshold information can include both an operation of initially setting the threshold information and an operation of updating previous threshold information with newly acquired threshold information.

[0084] According to one embodiment, the electronic device (110) may apply threshold information at a preset time. According to another embodiment, the electronic device (110) may further perform an operation of determining a time for applying threshold information, and apply the threshold information based on the determined time.

[0085] As a more specific example of the operation for determining a time point, the electronic device (110) may determine the chronological relationship between the time point of receiving data related to the first indicator and the time point for applying the threshold information based on one or more attributes related to the first indicator. For example, if data related to the first indicator is regularly received at 7:00 AM every day, the electronic device (110) may determine whether to apply the threshold information before 7:00 AM or after 7:00 AM based on one or more attributes related to the first indicator.

[0086] "One or more attributes related to the first indicator" regarding the timing for applying critical information may include, for example, whether the indicator is traffic-sensitive. In this regard, according to one example, the electronic device (110) may apply critical information in advance for indicators that are traffic-sensitive before the data is received, thereby enabling the determination of data abnormality while reflecting the acquired critical information. On the other hand, for indicators that are not traffic-sensitive, critical information may be applied after the data is received, thereby yielding the timing for reflecting critical information to other indicators that are more traffic-sensitive. By flexibly determining the timing for reflecting critical information in this way, the burden of resources required for reflecting various pieces of information can be effectively distributed.

[0087] Meanwhile, with respect to the operation of drawing number 340, which applies the acquired critical information, the operation of applying the critical information may be omitted for some indicators depending on the embodiment. For example, if the critical information acquired through the operation of drawing number 330 is identical in content to the previously applied critical information, the electronic device (110) may omit the process of applying the critical information, thereby reducing unnecessary resource consumption.

[0088] Based on the applied threshold information, the electronic device (110) determines whether to provide a notification related to the first indicator (350). The operation of drawing number 350 may include an operation of comparing target data corresponding to the first indicator with the threshold information, but the scope of the present disclosure is not limited thereto.

[0089] In one embodiment, the electronic device (110) can verify whether the critical information is appropriate. For example, in connection with the operation of drawing number 330, if the electronic device (110) provides one or more inputs to a set model and obtains an output of the set model, the electronic device (110) can verify whether the output of the model is appropriate.

[0090] If the critical information is inappropriate, the electronic device (110) may provide the user with information notifying that the critical information is inappropriate. The users to whom the information notifying that the critical information is inappropriate may include one or more of the service owner, the service administrator, and a person in charge of the first indicator (e.g., a Slack person in charge). Meanwhile, the users to whom the information notifying that the critical information is inappropriate may include, but are not limited to, a user of the user's terminal (120).

[0091] Cases where critical information is inadequate may include, for example, cases where the first critical condition associated with a warning notification is set more tightly than the second critical condition associated with an error occurrence (wherein "tight" may be interpreted to mean that satisfying the first critical condition is more difficult than satisfying the second critical condition). Cases where critical information is inadequate may also include cases where the critical conditions are set too tightly or too loosely overall.

[0092] In addition, cases where critical information is inappropriate may include various embodiments, and specific embodiments do not limit the scope of the present disclosure. For example, cases where critical information is inappropriate may include various instances where an error is presumed to have occurred during the process of obtaining critical information, and various instances where a critical problem is presumed to exist related to service provision.

[0093] FIG. 6 is a drawing for explaining an example of a method for providing notification information according to one embodiment.

[0094] Referring to FIG. 6, a monitoring center (a type of electronic device (110) described above in FIGS. 1 to 5) according to one embodiment can check indicators related to services 1 to n (an embodiment of the operation of drawing number 310).

[0095] The monitoring center can obtain historical information for at least some of the metrics related to services 1 through n (610) (an example of the operation of drawing number 320). For example, the monitoring center can obtain metric information for the past seven days. However, as described above in FIG. 3, the period for information acquisition can be flexibly set.

[0096] The monitoring center can provide one or more inputs containing historical information to the machine learning model (620) (an example of the operation of drawing number 330). The monitoring center can obtain the output of the machine learning model based on one or more inputs.

[0097] The monitoring center can obtain threshold information for each indicator based on the output of the machine learning model. The process of obtaining threshold information may include obtaining threshold conditions for each indicator. As a more specific example, the monitoring center can calculate upper and lower bounds for determining whether data related to each indicator is normal (630). However, not all associated upper and lower bounds are necessarily required for each indicator.

[0098] The monitoring center can update the critical conditions (an example of the operation of drawing number 340). If previous information does not exist, the monitoring center can "initialize" the critical conditions instead of "updating" them.

[0099] The monitoring center can periodically check whether the threshold information is satisfied, and if the threshold condition is not exceeded, a notification can be omitted, while if the threshold condition is exceeded, a notification can be provided to the user (one embodiment of the operation of drawing number 350).

[0100] Figure 7 is a drawing for exemplarily explaining notification information according to one embodiment.

[0101] Referring to FIG. 7, when an electronic device (110) according to an embodiment decides to provide a notification related to a first indicator (see drawing number 350), an example of notification information provided by the electronic device (110) is as illustrated in drawing number 700. Specifically, the notification information may include time information (“6:53”, “Today at 6:53 AM”) and information (710) specifying the content of the notification. The information (710) specifying the content of the notification may include information indicating the type of the notification (“Notification Type: Warning”) and information of the first indicator associated with the notification (“Metric Alert ‘[Couwatch] Minor GC Time’”).

[0102] Depending on the embodiment, the notification information may include one or more of the information (720) that serves as the basis for the notification and the related link information (730). The information (720) that serves as the basis for the notification may include threshold information ("2s") and data that does not satisfy the threshold condition ("2s440ms").

[0103] Figure 8 is an exemplary diagram of the configuration of an electronic device according to one embodiment.

[0104] Figure 8 is an exemplary diagram of the configuration of an electronic device according to one embodiment.

[0105] Referring to FIG. 8, the electronic device (110) includes a processor (820) and a memory (830), and may further include a transceiver (810) depending on the embodiment. The electronic device (110) is connected to a user's terminal (120) and an external device, etc., through the transceiver (810), and may exchange data.

[0106] The processor (820) may include at least one of the devices described above with reference to FIGS. 1 to 7, or may perform at least one method described above with reference to FIGS. 1 to 7. The memory (830) may store information for performing at least one method described above with reference to FIGS. 1 to 7. The memory (830) may be a volatile memory or a non-volatile memory.

[0107] The processor (820) can execute a program and control an electronic device (110) to provide information. The code of the program executed by the processor (820) can be stored in the memory (830).

[0108] Additionally, the electronic device (110) of the embodiment may further include an interface capable of providing information to a user or the like.

[0109] Meanwhile, the present specification and drawings disclose preferred embodiments of the present invention. Although specific terms have been used, they are used in a general sense only to easily explain the technical contents of the present invention and to assist in understanding the invention, and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modified examples based on the technical concept of the present invention are possible in addition to the embodiments disclosed herein.

[0110] The server or terminal according to the above-described embodiments may include a processor, a memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, a user interface device such as a touch panel, a key, a button, etc. The methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable codes or program commands executable on the processor. Here, the computer-readable recording medium includes a magnetic storage medium (e.g., read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optical reading medium (e.g., CD-ROM, DVD: Digital Versatile Disc)). The computer-readable recording medium may be distributed to computer systems connected to a network, so that the computer-readable code may be stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed by a processor.

[0111] The present embodiment may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the embodiment may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., which may perform various functions under the control of one or more microprocessors or other control devices. Similarly, the present embodiment may be implemented in a programming or scripting language such as C, C++, Java, assembler, Python, etc., including various algorithms implemented as a combination of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms that execute on one or more processors. Furthermore, the present embodiment may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms like "mechanism," "element," "means," and "composition" can be used broadly and are not limited to mechanical or physical components. These terms can also encompass a series of software routines, such as those associated with a processor.

[0112] The above-described embodiments are merely examples, and other embodiments may be implemented within the scope of the claims set forth below.

Claims

1. In a method of providing information in an electronic device, A step of verifying one or more indicators related to the provision of a service; A step of obtaining history information related to a first indicator included in the above one or more indicators; A step of obtaining critical information corresponding to the first indicator based on the above history information; a step of applying the above critical information; and A method for providing information, comprising a step of determining whether to provide a notification related to the first indicator based on the above critical information.

2. In paragraph 1, The step of obtaining the above critical information is A step of confirming a plurality of past data related to the first indicator included in the above history information; and A method for providing information, characterized in that it comprises a step of setting the threshold condition so that a group of data satisfying the threshold condition among the plurality of past data satisfies the set criterion.

3. In paragraph 2, A step of determining whether a problem related to the first indicator actually occurred in the provision of the service based on the above history information; and A method for providing information, characterized in that it further includes a step of setting the criterion based on the result of determining whether a problem related to the first indicator has actually occurred.

4. In paragraph 1, The step of obtaining the above critical information is providing one or more inputs including the above history information to a machine learning model; and A method for providing information, characterized by comprising a step of obtaining an output of the machine learning model.

5. In paragraph 4, One or more of the above inputs A method for providing information, characterized in that it includes criteria set in relation to critical conditions.

6. In paragraph 1, The above critical information is A method for providing information, characterized in that it includes upper bound information and lower bound information for determining whether data related to the first indicator is normal.

7. In paragraph 1, The above critical information is A method for providing information, characterized in that it includes at least one of a first threshold condition associated with a warning notification related to the first indicator and a second threshold condition associated with an error occurrence notification related to the first indicator.

8. In paragraph 1, A method for providing information, characterized in that it further comprises a step of determining a time point for applying the above critical information.

9. In paragraph 8, The step of determining the point in time to apply the above critical information is A method for providing information, characterized in that it comprises a step of determining a chronological relationship between a time of receiving data related to the first indicator and a time of applying the critical information based on one or more attributes related to the first indicator.

10. In paragraph 1, a step of verifying whether the above critical information is appropriate; and A method for providing information, characterized in that it further comprises a step of providing information to a user informing that the critical information is inappropriate when the critical information is inappropriate.

11. In paragraph 1, A method for providing information, characterized in that it further comprises a step of setting a period for obtaining the history information based on one or more attributes related to the first indicator.

12. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of paragraph 1 on a computer.

13. In an electronic device that provides information, Includes a memory and a processor that stores instructions, The above processor is connected to the above memory, Identify one or more indicators related to the provision of the Service; Obtaining history information related to a first indicator included in one or more of the above indicators, Obtaining critical information corresponding to the first indicator based on the above history information, Apply the above critical information, An electronic device that determines whether to provide a notification related to the first indicator based on the above threshold information.

Images