Task early warning methods, devices, computer equipment and storage media

Abstract

This application belongs to the field of infrastructure operation and maintenance, and relates to a task early warning method, device, computer equipment, and storage medium. The method includes: acquiring real-time logs corresponding to each task; for each task, extracting multi-dimensional task value assessment information from the task's real-time logs; calculating the task value assessment value based on the multi-dimensional task value assessment information; configuring an early warning strategy for the task according to the task value assessment value; and issuing anomaly warnings for the task based on the early warning strategy. Furthermore, the multi-dimensional task value assessment information can be stored in a blockchain. This application achieves refined management of early warnings.

Description

Technical Field

[0001] This application relates to the field of infrastructure maintenance technology, and in particular to a task early warning method, device, computer equipment and storage medium. Background Technology

[0002] Production and operational activities are typically broken down into tasks. For example, in a big data platform, there may be tens of thousands of tasks operating simultaneously, with complex dependencies between them. To prevent cascading failures caused by task anomalies, an early warning system is needed to monitor each task and issue alerts when an anomaly is detected.

[0003] However, current early warning systems lack specificity for each task, have insufficient granularity in alarms, and cannot perform real-time analysis or achieve refined management of early warnings. Summary of the Invention

[0004] The purpose of this application is to provide a task early warning method, device, computer equipment, and storage medium to achieve refined management of task early warning.

[0005] To address the aforementioned technical problems, this application provides a task early warning method, employing the following technical solution:

[0006] Obtain the real-time logs corresponding to each task;

[0007] For each task, extract multi-dimensional task value assessment information from the task's real-time logs;

[0008] Calculate the task value assessment value based on the multi-dimensional task value assessment information;

[0009] Configure the early warning strategy for the task based on the task value assessment value;

[0010] Anomalies are detected in the task based on the aforementioned early warning strategy.

[0011] To address the aforementioned technical problems, this application also provides a task early warning device, which employs the following technical solution:

[0012] The log acquisition module is used to acquire real-time logs corresponding to each task;

[0013] The information extraction module is used to extract multi-dimensional task value assessment information from the real-time logs of each task.

[0014] The evaluation calculation module is used to calculate the task value evaluation value of the task based on the multi-dimensional task value evaluation information.

[0015] The strategy configuration module is used to configure the early warning strategy for the task based on the task value assessment value;

[0016] An anomaly warning module is used to provide anomaly warnings for the task based on the warning strategy.

[0017] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0018] Obtain the real-time logs corresponding to each task;

[0019] For each task, extract multi-dimensional task value assessment information from the task's real-time logs;

[0020] Calculate the task value assessment value based on the multi-dimensional task value assessment information;

[0021] Configure the early warning strategy for the task based on the task value assessment value;

[0022] Anomalies are detected in the task based on the aforementioned early warning strategy.

[0023] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0024] Obtain the real-time logs corresponding to each task;

[0025] For each task, extract multi-dimensional task value assessment information from the task's real-time logs;

[0026] Calculate the task value assessment value based on the multi-dimensional task value assessment information;

[0027] Configure the early warning strategy for the task based on the task value assessment value;

[0028] Anomalies are detected in the task based on the aforementioned early warning strategy.

[0029] Compared with the prior art, the embodiments of this application have the following advantages: real-time logs corresponding to each task are obtained, and the real-time logs record the latest information related to the task, which has high real-time performance; for each task, multi-dimensional task value assessment information is extracted from the task's real-time logs to comprehensively evaluate the task value from multiple dimensions and obtain the task value assessment value, which is used to display the task value; the early warning strategy for the task is configured in a targeted manner according to the size of the task value, and the task is given anomaly warning according to the early warning strategy, thereby realizing differentiated management of early warning, and tasks with high task value can be monitored in a key manner, which improves the precision of early warning. Attached Figure Description

[0030] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0032] Figure 2 This is a flowchart of an embodiment of the task warning method according to this application;

[0033] Figure 3 This is a schematic diagram of the structure of one embodiment of the task warning device according to this application;

[0034] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0036] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0037] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0038] like Figure 1As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0039] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0040] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0041] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0042] It should be noted that the task warning method provided in this application embodiment is generally executed by the server, and correspondingly, the task warning device is generally set in the server.

[0043] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0044] Continue to refer to Figure 2 A flowchart of an embodiment of the task warning method according to this application is shown. The task warning method includes the following steps:

[0045] Step S201: Obtain the real-time logs corresponding to each task.

[0046] In this embodiment, the task warning method runs on an electronic device (e.g., Figure 1The server shown can communicate with the terminal via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future wireless connection methods.

[0047] Specifically, the first step is to obtain the real-time logs corresponding to each task. Real-time logs are task logs, which can be generated by the task itself or by log generation tools monitoring the task's execution. Real-time logs record various aspects of the task, such as the operations performed during task execution and related information. Real-time logs are dynamically generated, capturing and recording task-related information in real time, and storing the latest task information.

[0048] Step S202: For each task, extract multi-dimensional task value assessment information from the task's real-time logs.

[0049] Specifically, the real-time logs of a task record information about various aspects of the task, including multiple dimensions that can be used to measure the task's value, which can be understood as the task's importance. For each task, information from multiple dimensions used to measure and evaluate the task's value is extracted from the task's real-time logs, thus obtaining multi-dimensional task value assessment information.

[0050] Step S203: Calculate the task value assessment value based on multi-dimensional task value assessment information.

[0051] Specifically, multi-dimensional task value assessment information can include various factors. These factors can be numerical information with potentially large differences in values, or they can be textual information. Based on a pre-defined transformation strategy, each factor in the multi-dimensional task value assessment information is converted into standardized factors, and then the task value assessment value is calculated based on these standardized factors.

[0052] The task value assessment value can be a numerical value. The value is used to measure the value of the task. The higher the value of the task, the higher the task value assessment value.

[0053] Step S204: Configure the early warning strategy for the task based on the task value assessment value.

[0054] Specifically, the task value assessment value displays the magnitude of the task's value. Different early warning strategies can be adopted for tasks with different values. Early warning strategies are used to configure and control early warnings. By configuring early warning strategies based on the task value assessment value, targeted early warnings can be issued according to the task's importance. Important tasks can have higher granularity of alerts, achieving refined management of early warnings.

[0055] Step S205: Issue anomaly warnings for the task based on the early warning strategy.

[0056] Specifically, after configuring the task's alert strategy, the rule engine can issue alerts for the task based on the strategy. When an anomaly is detected in a task, an appropriate alert is issued according to the alert strategy.

[0057] In this embodiment, real-time logs corresponding to each task are obtained. These logs record the latest information related to the task and have high real-time performance. For each task, multi-dimensional task value assessment information is extracted from the task's real-time logs to comprehensively evaluate the task value from multiple dimensions, resulting in a task value assessment value. This value is used to display the task's value. Based on the task's value, a targeted early warning strategy is configured, and anomaly warnings are issued for the task according to the strategy. This achieves differentiated management of warnings, allowing for focused monitoring of tasks with high value and improving the precision of the warnings.

[0058] Furthermore, step S201 above may include: obtaining the task identifier of each task; and obtaining the real-time log corresponding to each task from Atlas based on the task identifier.

[0059] Specifically, real-time logs can be generated by logging tools. These tools can monitor and record data for each task in real time, generating real-time logs for each task. They can also generate separate real-time logs for each task, distinguishing them based on task identifiers.

[0060] For each task currently running in the system, obtain the task identifier of each task, and extract real-time logs from the logging tool based on the task identifier.

[0061] In one embodiment, the logging tool could be Atlas. Atlas, also known as Apache Atlas, is an open-source project developed by the Hadoop (a distributed system infrastructure) community to address metadata governance issues within the Hadoop ecosystem. It provides core metadata governance functionalities for Hadoop clusters, including data classification, a centralized policy engine, data lineage, security, and lifecycle management. Atlas can generate logs in real time and perform real-time analysis and visualization. Compared to static logs, obtaining real-time logs through Atlas improves data accuracy.

[0062] In this embodiment, real-time logs can be generated by Atlas, and the required real-time logs can be accurately extracted from Atlas based on the task identifier of each task.

[0063] Furthermore, step S202 may include: generating task lineage information for each task based on the real-time logs corresponding to each task; determining each lineage evaluation factor and each operational evaluation factor for each task based on the real-time logs and task lineage information; and for each task, determining each lineage evaluation factor and each operational evaluation factor as multi-dimensional task value evaluation information for the task.

[0064] Specifically, multiple tasks can run on the system, and these tasks have dependencies on each other. For example, task A generates data table B, and task C needs to read data from data table B during runtime. Therefore, task C depends on data table B, and task C also depends on task A. Task A is the upstream task of task C, and task C is the downstream task of task A. This dependency and association relationship can be obtained by parsing real-time logs.

[0065] Based on real-time logs and task lineage information, each task's lineage evaluation factor and operational evaluation factor can be extracted. The lineage evaluation factor can be a factor related to task lineage, such as the number of downstream tasks (determining the number of all or some downstream tasks through task lineage information), the number of data tables read by the task, the amount of data read by the task, information about downstream applications (applications are composed of different tasks, tasks ultimately point to applications, and application-related information includes the application's importance, level, etc.), the frequency of changes to the data tables involved in the task, and the stability of the data tables involved in the task.

[0066] The performance evaluation factors can be factors related to task execution, such as task runtime and CPU resource consumption information.

[0067] Each lineage assessment factor and each operational assessment factor of a task can be identified as multi-dimensional task value assessment information.

[0068] In this embodiment, task lineage information is generated based on real-time logs. Based on the real-time logs and task lineage information, multiple lineage evaluation factors and operational evaluation factors are extracted to form multi-dimensional task value evaluation information, thereby comprehensively measuring the task value from multiple dimensions.

[0069] Furthermore, the steps described above for generating task lineage information for each task based on the real-time logs corresponding to each task may include: parsing the real-time logs corresponding to each task to obtain read and write statements in the real-time logs; generating field lineage information and data table lineage information based on the read and write statements; generating task lineage information for each task based on the field lineage information and data table lineage information; the task lineage information includes field lineage information and data table lineage information.

[0070] Specifically, real-time logs are parsed, for example, using natural language processing in artificial intelligence, to obtain read and write statements. These statements can be SQL statements, recording the read and write operations performed by tasks on fields and data tables during runtime. Fields belong to data tables, and data tables belong to tasks. By parsing the read and write statements, the relationships between fields (field lineage information) and between data tables (table lineage information) can be obtained. Therefore, based on the field and table lineage information of all tasks, task lineage information for each task can be generated, where the field and table lineage information can be included within the task lineage information.

[0071] For example, if we obtain the read / write statement from the real-time log of task A1: `select B.id,C.name from B joinC on B.id=C.id`, we can determine from the keywords `from` and `join` that the task retrieves data from tables B and C. The keyword `select` indicates that table A retrieves the `id` field from table B and the `name` field from table C. Assuming the read / write statement retrieves field D, and table B comes from task B1 and table C comes from task C1, then the field lineage information is: field D is associated with fields `id` and `name`, and field D depends on fields `id` and `name`. The table lineage information is: table A is associated with tables B and C, and table A depends on tables B and C. Therefore, the task lineage information is: task A1 is associated with tasks B1 and C1, and task A1 depends on tasks B1 and C1.

[0072] In this embodiment, read and write statements are obtained by parsing the real-time logs. Field lineage information and data table lineage information are generated through the read and write statements. Based on the membership relationship between fields, data tables and tasks, task lineage information can be further generated.

[0073] Furthermore, the steps for calculating the task value assessment value based on multi-dimensional task value assessment information may include: standardizing each lineage assessment factor and each operational assessment factor in the multi-dimensional task value assessment information; determining the factor weights of each lineage assessment factor and each operational assessment factor after standardization using a preset weighting algorithm; and performing a weighted calculation on each lineage assessment factor and each operational assessment factor according to the obtained factor weights to obtain the task value assessment value.

[0074] Specifically, the multi-dimensional task value assessment information includes multiple lineage assessment factors and multiple operational assessment factors. Their existence forms may vary greatly, and each lineage assessment factor and each operational assessment factor can be standardized according to a preset standardized processing strategy.

[0075] The kinship assessment factor and the operational assessment factor can each have their own factor weights, and the sum of the factor weights of all factors is 1. The factor weights of each kinship assessment factor and each operational assessment factor can be determined by a preset weighting algorithm, such as the CRITIC algorithm, the analytic hierarchy process (AHP), or the Relief algorithm. In one embodiment, the factor weights can be directly preset and added to each factor after querying.

[0076] Based on the obtained factor weights, the weighted calculations of each bloodline assessment factor and each operational assessment factor are performed, for example, by weighted summation, to obtain the task value assessment value.

[0077] In this embodiment, each factor is standardized to facilitate calculation, and factor weights are added to each factor to distinguish and differentiate them, thereby ensuring the rationality of the final calculated task value assessment.

[0078] Furthermore, step S204 may include: determining the value assessment range to which the task belongs based on the task value assessment value; determining the value assessment level of the task according to the value assessment range; and configuring the early warning strategy for the task according to the value assessment level.

[0079] Specifically, multiple standard evaluation values ​​can be pre-defined. These standard evaluation values ​​can be different and can serve as endpoints to form multiple value evaluation intervals. Different value evaluation intervals correspond to different value evaluation levels. Based on the task's value evaluation value, the value evaluation interval to which the task belongs is determined, and the value evaluation level represented by that interval is added to the task to obtain its value evaluation level, thus achieving task classification.

[0080] Different early warning strategies can be configured for tasks with different value assessment levels. For high-value tasks, more emphasis can be placed on early warning, thereby achieving differentiated and refined early warning systems.

[0081] In this embodiment, the value assessment level of a task is determined based on its value assessment value, and an early warning strategy is configured based on the value assessment level to achieve differentiated and refined early warning.

[0082] Furthermore, the steps described above for configuring the early warning strategy for tasks based on the value assessment level may include: generating feature values ​​for each early warning feature based on the value assessment level; the early warning features include early warning frequency, early warning method, notification recipients, and processing strategies; and generating an early warning strategy for the task based on the feature values ​​of each early warning feature.

[0083] Specifically, the early warning strategy includes multiple early warning features, each with a feature value. The value of the feature value is related to the value assessment level of the task. These early warning feature values ​​include early warning frequency (i.e., how often an early warning is issued and how often it is generated; the higher the value assessment level, the higher the frequency), early warning method (i.e., how the early warning information is disseminated, including email, telephone, alarms, etc.; the higher the value assessment level, the more methods can be used to notify relevant personnel more directly), notification recipients (the recipients of the early warning information; the higher the value assessment level, the more recipients can be), and processing strategy (automatic processing can be set for detected anomalies). In one embodiment, the early warning feature may also include monitoring methods; the higher the value assessment level, the more monitoring methods can be used to detect the task, thereby improving the accuracy of the early warning.

[0084] The feature values ​​of each early warning feature can be determined based on the value assessment level, thereby generating an early warning strategy. In one embodiment, after the feature values ​​of each early warning feature are automatically generated, the feature values ​​of each early warning feature can also be manually adjusted.

[0085] In this embodiment, the feature values ​​of each early warning feature are determined according to the value assessment level. The early warning features include early warning frequency, early warning method, notification target and processing strategy, so as to realize the targeted configuration of early warning.

[0086] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned multi-dimensional task value assessment information, this information can also be stored in a blockchain node.

[0087] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0088] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0089] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0091] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0092] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a task early warning device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0093] like Figure 3 As shown, the task early warning device 300 described in this embodiment includes: a log acquisition module 301, an information extraction module 302, an evaluation calculation module 303, a strategy configuration module 304, and an anomaly early warning module 305, wherein:

[0094] The log acquisition module 301 is used to acquire the real-time logs corresponding to each task.

[0095] The information extraction module 302 is used to extract multi-dimensional task value assessment information from the real-time logs of each task.

[0096] The evaluation calculation module 303 is used to calculate the task value evaluation value of a task based on multi-dimensional task value evaluation information.

[0097] The strategy configuration module 304 is used to configure the early warning strategy for the task based on the task value assessment value.

[0098] The anomaly warning module 305 is used to provide anomaly warnings for tasks based on warning strategies.

[0099] In this embodiment, real-time logs corresponding to each task are obtained. These logs record the latest information related to the task and have high real-time performance. For each task, multi-dimensional task value assessment information is extracted from the task's real-time logs to comprehensively evaluate the task value from multiple dimensions, resulting in a task value assessment value. This value is used to display the task's value. Based on the task's value, a targeted early warning strategy is configured, and anomaly warnings are issued for the task according to the strategy. This achieves differentiated management of warnings, allowing for focused monitoring of tasks with high value and improving the precision of the warnings.

[0100] In some optional implementations of this embodiment, the log acquisition module 301 may include: an identifier acquisition submodule and a log acquisition submodule, wherein:

[0101] The identifier acquisition submodule is used to obtain the task identifier for each task.

[0102] The log acquisition submodule is used to retrieve the real-time logs corresponding to each task from Atlas based on the task identifier.

[0103] In this embodiment, real-time logs can be generated by Atlas, and the required real-time logs can be accurately extracted from Atlas based on the task identifier of each task.

[0104] In some optional implementations of this embodiment, the information extraction module 302 may include: a bloodline generation submodule, a factor determination submodule, and an information determination submodule, wherein:

[0105] The lineage generation submodule is used to generate the task lineage relationship information of each task based on the real-time logs corresponding to each task.

[0106] The factor determination submodule is used to determine each lineage evaluation factor and each operational evaluation factor for each task based on real-time logs and task lineage information.

[0107] The information determination submodule is used to determine the various lineage assessment factors and operational assessment factors of each task as multi-dimensional task value assessment information for each task.

[0108] In this embodiment, task lineage information is generated based on real-time logs. Based on the real-time logs and task lineage information, multiple lineage evaluation factors and operational evaluation factors are extracted to form multi-dimensional task value evaluation information, thereby comprehensively measuring the task value from multiple dimensions.

[0109] In some optional implementations of this embodiment, the lineage generation submodule may include: a log parsing unit, an information generation unit, and a lineage generation unit, wherein:

[0110] The log parsing unit is used to parse the real-time logs corresponding to each task to obtain the read and write statements in the real-time logs.

[0111] The information generation unit is used to generate field lineage information and data table lineage information based on read and write statements.

[0112] The lineage generation unit is used to generate task lineage relationship information for each task based on field lineage relationship information and data table lineage relationship information; the task lineage relationship information includes field lineage relationship information and data table lineage relationship information.

[0113] In this embodiment, read and write statements are obtained by parsing the real-time logs. Field lineage information and data table lineage information are generated through the read and write statements. Based on the membership relationship between fields, data tables and tasks, task lineage information can be further generated.

[0114] In some optional implementations of this embodiment, the evaluation calculation module 303 may include: a standard processing submodule, a weight determination submodule, and a factor calculation submodule, wherein:

[0115] The standard processing submodule is used to standardize the various lineage assessment factors and operational assessment factors of the task in the multi-dimensional task value assessment information.

[0116] The weight determination submodule is used to determine the factor weights of each bloodline assessment factor and each operational assessment factor after standardization using a preset weighting algorithm.

[0117] The factor calculation submodule is used to perform weighted calculations on each bloodline assessment factor and each operational assessment factor based on the obtained factor weights, so as to obtain the task value assessment value.

[0118] In this embodiment, each factor is standardized to facilitate calculation, and factor weights are added to each factor to distinguish and differentiate them, thereby ensuring the rationality of the final calculated task value assessment.

[0119] In some optional implementations of this embodiment, the policy configuration module 304 may include: an interval determination submodule, a level determination submodule, and an early warning configuration submodule, wherein:

[0120] The interval determination submodule is used to determine the value assessment interval to which a task belongs based on its task value assessment value.

[0121] The Level Determination submodule is used to determine the value assessment level of a task based on the value assessment range.

[0122] The early warning configuration submodule is used to configure early warning strategies for tasks based on their value assessment levels.

[0123] In this embodiment, the value assessment level of a task is determined based on its value assessment value, and an early warning strategy is configured based on the value assessment level to achieve differentiated and refined early warning.

[0124] In some optional implementations of this embodiment, the early warning configuration submodule may include: a feature value generation unit and a policy generation unit, wherein:

[0125] The feature value generation unit is used to generate feature values ​​for each early warning feature based on the value assessment level; the early warning features include early warning frequency, early warning method, notification target and processing strategy.

[0126] The strategy generation unit is used to generate early warning strategies for tasks based on the feature values ​​of each early warning feature.

[0127] In this embodiment, the feature values ​​of each early warning feature are determined according to the value assessment level. The early warning features include early warning frequency, early warning method, notification target and processing strategy, so as to realize the targeted configuration of early warning.

[0128] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0129] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0130] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0131] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for task alerting methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0132] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the task alerting method.

[0133] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0134] The computer device provided in this embodiment can execute the above-described task warning method. The task warning method here can be any of the task warning methods described in the various embodiments above.

[0135] In this embodiment, real-time logs corresponding to each task are obtained. These logs record the latest information related to the task and have high real-time performance. For each task, multi-dimensional task value assessment information is extracted from the task's real-time logs to comprehensively evaluate the task value from multiple dimensions, resulting in a task value assessment value. This value is used to display the task's value. Based on the task's value, a targeted early warning strategy is configured, and anomaly warnings are issued for the task according to the strategy. This achieves differentiated management of warnings, allowing for focused monitoring of tasks with high value and improving the precision of the warnings.

[0136] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the task warning method described above.

[0137] In this embodiment, real-time logs corresponding to each task are obtained. These logs record the latest information related to the task and have high real-time performance. For each task, multi-dimensional task value assessment information is extracted from the task's real-time logs to comprehensively evaluate the task value from multiple dimensions, resulting in a task value assessment value. This value is used to display the task's value. Based on the task's value, a targeted early warning strategy is configured, and anomaly warnings are issued for the task according to the strategy. This achieves differentiated management of warnings, allowing for focused monitoring of tasks with high value and improving the precision of the warnings.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A task alerting method, characterized by, Includes the following steps: Obtain the real-time logs corresponding to each task, wherein the real-time logs are generated by Atlas in real time; For each task, extract multi-dimensional task value assessment information from the task's real-time logs; Calculate the task value assessment value based on the multi-dimensional task value assessment information; Configure the early warning strategy for the task based on the task value assessment value; Based on the aforementioned early warning strategy, anomaly warnings are issued for the task; The step of extracting multi-dimensional task value assessment information from the real-time logs of each task includes: Based on the real-time logs corresponding to each task, generate task lineage information for each task; Based on the real-time logs and the task lineage information, determine each lineage evaluation factor and each operational evaluation factor for each task; For each task, the lineage assessment factors and operational assessment factors of the task are determined as the multi-dimensional task value assessment information of the task. The step of calculating the task value assessment value based on the multi-dimensional task value assessment information includes: The lineage assessment factors and operational assessment factors of the task in the multi-dimensional task value assessment information are standardized. The factor weights of each bloodline assessment factor and each operational assessment factor after standardization are determined by a preset weighting algorithm. The task value assessment value is obtained by weighting each bloodline assessment factor and each operational assessment factor according to the obtained factor weights.

2. The task early warning method according to claim 1, characterized in that, The steps for obtaining the real-time logs corresponding to each task include: Obtain the task identifier for each task; Based on the task identifier, retrieve the real-time logs corresponding to each task from Atlas.

3. The task early warning method according to claim 1, characterized in that, The step of generating task lineage information for each task based on the real-time logs corresponding to each task includes: The real-time logs corresponding to each task are parsed to obtain the read and write statements in the real-time logs; Based on the read / write statements, generate field lineage information and data table lineage information; Based on the lineage relationship information in the fields and the lineage relationship information in the data table, the task lineage relationship information for each task is generated; the task lineage relationship information includes the lineage relationship information in the fields and the lineage relationship information in the data table.

4. The task early warning method according to claim 1, characterized in that, The step of configuring the early warning strategy for the task based on the task value assessment includes: Based on the task value assessment value, determine the value assessment range to which the task belongs; The value assessment level of the task is determined based on the value assessment range; Configure the early warning strategy for the task based on the value assessment level.

5. The task early warning method according to claim 4, characterized in that, The step of configuring the early warning strategy for the task based on the value assessment level includes: Feature values ​​for each early warning feature are generated based on the value assessment level; the early warning feature includes early warning frequency, early warning method, notification recipient, and processing strategy; Based on the feature values ​​of each warning feature, a warning strategy for the task is generated.

6. A mission early warning device, characterized in that, include: The log acquisition module is used to acquire the real-time logs corresponding to each task, wherein the real-time logs are generated by Atlas in real time; The information extraction module is used to extract multi-dimensional task value assessment information from the real-time logs of each task. The evaluation calculation module is used to calculate the task value evaluation value of the task based on the multi-dimensional task value evaluation information. The strategy configuration module is used to configure the early warning strategy for the task based on the task value assessment value; Anomaly warning module, used to provide anomaly warnings for the task based on the warning strategy; The information extraction module includes: a bloodline generation submodule, a factor determination submodule, and an information determination submodule, wherein: The lineage generation submodule is used to generate the task lineage relationship information of each task based on the real-time logs corresponding to each task. The factor determination submodule is used to determine each lineage evaluation factor and each operation evaluation factor of each task based on the real-time logs and the task lineage information. The information determination submodule is used to determine each lineage assessment factor and each operational assessment factor of the task as multi-dimensional task value assessment information for each task. The evaluation calculation module includes: a standard processing submodule, a weight determination submodule, and a factor calculation submodule, wherein: The standard processing submodule is used to standardize the lineage assessment factors and operational assessment factors of the task in the multi-dimensional task value assessment information. The weight determination submodule is used to determine the factor weights of each bloodline assessment factor and each operational assessment factor after standardization through a preset weight algorithm. The factor calculation submodule is used to perform weighted calculations on each bloodline assessment factor and each operational assessment factor based on the obtained factor weights to obtain the task value assessment value of the task.

7. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the task warning method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the task warning method as described in any one of claims 1 to 5.

Images