A high-precision map computing job execution failure processing method and device

Abstract

This disclosure provides a method and apparatus for handling high-precision map calculation job execution failures, relating to the field of computer technology, and particularly to the field of high-precision maps. The specific implementation scheme is as follows: in response to a calculation job execution failure, the operation log of the calculation job is obtained; based on the operation log, the error type of the calculation job execution failure is determined, the error type including: system stability error and human error; when the error type is a system stability error, the calculation job is re-executed; when the error type is a human error, a prompt message for manual intervention is generated. This saves the time required for handling calculation job execution failures, thereby improving the execution efficiency and stability of the calculation job.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to the field of high-precision mapping technology. Background Technology

[0002] High-precision maps, also known as high-resolution maps, are used by autonomous vehicles. They possess precise vehicle location information and rich road element data, helping cars anticipate complex road conditions such as slope, curvature, and heading, thus better avoiding potential risks. Compared to ordinary maps, high-precision maps have a larger data scale, thus requiring higher computing performance and efficiency. During the execution of numerous computational tasks, these tasks may fail due to various issues. To improve the efficiency of these computational tasks, it is necessary to analyze and handle these failures. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for handling high-precision map calculation job execution failures.

[0004] According to one aspect of this disclosure, a method for handling the failure of high-precision map calculation operations is provided, including:

[0005] In response to a computing job failure, the execution log of the computing job is retrieved;

[0006] Based on the operation log, the error type of the computing job execution failure is determined, and the error type includes: system stability error and human-induced development error;

[0007] If the error type is a system stability error, the computation job will be re-executed.

[0008] When the error type is a human-caused R&D error, a prompt message for manual intervention is generated;

[0009] The runtime logs include driver logs and task logs;

[0010] The step of determining the error type of the computation job execution failure based on the runtime log includes:

[0011] The error type that caused the computation job to fail is determined based on the driver log and the task log.

[0012] When error information is found in the driver log, the error type of the computing job execution failure is determined to be a system stability error.

[0013] If there are error messages in the task log, the error type of the computation job execution failure is determined based on the error messages in the task log.

[0014] According to another aspect of this disclosure, a processing apparatus for high-precision map calculation job failure is provided, comprising:

[0015] The acquisition module is used to acquire the execution log of the computing job in response to the failure of the computing job execution;

[0016] The determination module is used to determine the error type of the computing job execution failure based on the running log, the error type including: system stability error and human-caused development error;

[0017] The execution module is used to re-execute the computation job when the error type is a system stability error;

[0018] The generation module is used to generate a prompt message for manual intervention when the error type is a human-caused R&D error.

[0019] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0020] At least one processor; and

[0021] A memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a method for handling failures in high-precision map calculation operations.

[0023] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to handle a method for causing the computer to fail to perform a high-precision map calculation task.

[0024] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method for handling failures in high-precision map calculation operations.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0027] Figure 1 This is a schematic flowchart of a method for handling high-precision map calculation job execution failure provided in an embodiment of this disclosure;

[0028] Figure 2 This is another flowchart illustrating the method for handling high-precision map calculation job execution failure provided in this embodiment of the disclosure;

[0029] Figure 3 This is a schematic diagram of a method for handling high-precision map calculation job execution failure provided in an embodiment of this disclosure;

[0030] Figure 4 This is a block diagram of an apparatus for implementing the method for handling high-precision map calculation job execution failure provided in the embodiments of this disclosure;

[0031] Figure 5 This is a block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0033] In the field of computer technology, many computational tasks consist of large-scale computational jobs. During the execution of these large-scale computational jobs, they may fail due to various problems. Current solutions for handling computational job failures mainly include the following two approaches:

[0034] Option 1: After a calculation error occurs, manual intervention is required to check the cause of the failure and determine the handling method based on the cause of the failure.

[0035] Option 2: Retry the calculation directly after an error occurs.

[0036] However, in practical applications, both of these solutions have certain drawbacks. The responsible personnel cannot immediately identify the cause of the error and handle the failed computational job, thus solution one reduces the efficiency of the computational job.

[0037] Furthermore, job failures can only be resolved through retries in certain cases. In other cases, retries will not resolve the job failure. Therefore, the direct retries approach in Option 2 not only reduces the efficiency of job execution but also consumes computing resources.

[0038] To address the aforementioned issues, this disclosure provides a method, apparatus, electronic device, and storage medium for handling high-precision map calculation job execution failures.

[0039] Figure 1 This is a flowchart illustrating a method for handling high-precision map calculation job execution failures provided in this embodiment of the disclosure, such as... Figure 1 As shown, the method includes the following steps:

[0040] Step S101: In response to the failure of the computation job execution, obtain the execution log of the computation job.

[0041] In this embodiment of the disclosure, the computing job can be a job running on a computing platform, such as a job running on the Spark platform.

[0042] During the execution of a computational job, various problems may occur, resulting in the inability to obtain the expected results, i.e., the computational job fails to execute.

[0043] When a computation job fails to execute, retrieve the job's execution log.

[0044] Step S102: Determine the error type of the computing job execution failure based on the running log. The error types include: system stability errors and human error caused by development.

[0045] Specifically, human-caused development errors are errors that occur during the development of computational job code. Examples include incorrect parameter configurations or abnormal business logic. In contrast, system stability errors are errors caused by external factors and have a degree of randomness, such as excessive server load. These system stability errors can be resolved by re-executing the computational job.

[0046] Step S103: If the error type is a system stability error, re-execute the calculation job.

[0047] In one embodiment of this disclosure, re-execution here can be understood as retrying the computation job.

[0048] Step S104: When the error type is a human-caused R&D error, generate a prompt message for manual intervention.

[0049] Specifically, in this embodiment, when the error type is a system stability error, the calculation job is re-executed to achieve automated processing. When the error type is a human-caused development error, the calculation job execution failure is not directly processed; instead, a prompt message for manual intervention is generated for relevant personnel to handle.

[0050] In this embodiment of the disclosure, after a computational job fails to execute, the error type is determined through the operation log of the computational job. Based on the error type, it is determined whether the failure can be resolved by re-executing the computational job. If it can be resolved by re-executing the computational job, i.e., the error type is a system stability error, then it is re-executed. If it cannot be resolved by re-executing the computational job, i.e., the error type is a human error, then manual intervention is prompted.

[0051] As can be seen, different handling methods are adopted according to different error types of computational job execution failures. When system stability errors exist, re-execution is performed without checking for failures, which can save the time increase caused by stability issues and improve the execution efficiency of computational jobs. When research and development errors exist, instead of retrying, a prompt message is generated, avoiding the problem of consuming computing resources due to directly setting retries, and also making it easier for relevant personnel to troubleshoot computational job execution failures. When applied to computational jobs that may fail due to different error types, it can improve the execution efficiency and stability of computational jobs.

[0052] As an example, the method for handling computation job failures provided in this disclosure is applied to Spark jobs. To facilitate understanding of the technical solutions in this disclosure, Spark will be introduced below:

[0053] Spark is an open-source, in-memory computing environment for big data clusters. The specific workflow of Spark is as follows:

[0054] After a user submits a Spark job, a corresponding driver process is started for that computation job. Once started, the driver process first requests the resources required to run the computation job from the cluster manager; these resources are the executor processes.

[0055] After acquiring the resources needed to run a computation job, the driver process schedules the user-submitted job. Specifically, data in Spark can be in RDD (Resilient Distributed Dataset) structure. Based on the dependencies between RDDs, the computation job is divided into multiple stages, each executing a portion of the computation job's code. Each stage is further divided into multiple tasks, which are assigned to various executor processes for execution. Once all tasks in a stage have been executed, the driver process schedules the next stage until the computation job is completed.

[0056] Figure 2This is another flowchart illustrating the method for handling high-precision map calculation job execution failure provided in this disclosure embodiment. The following is a combined description of... Figure 2 Another embodiment of this disclosure will be described, specifically including the following steps:

[0057] Step S201: In response to the failure of the computation job execution, obtain the execution log of the computation job.

[0058] Specifically, the runtime log is a file that records various important information during the execution of a computational job. Therefore, the term "information" may be used later to describe the data that can be obtained from the runtime log.

[0059] In one embodiment of this disclosure, the runtime log includes driver logs and task logs.

[0060] In one embodiment of this disclosure, the execution logs of the computation job are collected and output using a pre-written JAR package, which can be implemented in the following way:

[0061] Once the driver for the computation job starts, methods in the JAR file are called to collect driver information and write the collected information to the database, thus enabling the acquisition of driver-related information even if the computation job fails.

[0062] During the execution of the computation job, the methods in the JAR package also collect task information. When a task in the computation job fails, the task failure error information is also written to the database.

[0063] The pre-written JAR package can be applied to different computing jobs. Therefore, collecting and outputting the running logs of computing jobs through the JAR package can not only improve the efficiency of obtaining running logs, but also improve the convenience of applying the computing job execution failure handling method provided in this embodiment to different computing jobs.

[0064] Step S202: Determine the error type of the computation job execution failure based on the driver log and task log.

[0065] Specifically, if error messages are found in the driver log, and the cause of the error is determined to be a system stability error, then step S203 will be executed.

[0066] If error messages are found in the task log, the error type that caused the computation job to fail is determined based on these messages. Depending on the error type, step S203 or step S204 will then be executed.

[0067] The error messages here refer to various types of information indicating that errors have occurred during the execution of the computational job.

[0068] As can be seen, step S202 can also be understood as first determining the failure category of the computation job execution failure based on whether error information exists in the driver log and task log, i.e., driver category and task category. After obtaining the failure category, the error type is further determined. If the failure category is the driver category, i.e., error information exists in the driver log, no further analysis of the error information is performed; instead, the error type is directly presumed to be a system stability error. This approach improves the efficiency of determining the error type.

[0069] The specific details involved in this step are explained below:

[0070] For cases where error messages are found in the driver logs, taking Spark jobs as an example, there are often situations where the load on a single server in the Spark cluster is too high, causing the driver to malfunction.

[0071] Furthermore, the driver is actually responsible for the scheduling and management of computing tasks, such as task distribution and resource allocation. It does not directly run the business code of the computing tasks. The business code of the computing tasks is executed through the task. Therefore, if the driver encounters an exception, it is usually unrelated to the business logic, that is, it is not a human error caused by development.

[0072] Therefore, when error messages are found in the driver logs, the error type can be directly identified as a system stability error. This approach aligns with the driver's operating logic and improves the efficiency of handling errors in the driver.

[0073] In one embodiment of this disclosure, when error messages are present in the task logs, the error type of the computation job execution failure is determined based on the error messages in the task logs as follows:

[0074] When the error message in the task log indicates that a system stability error occurred during the execution of the computation job's task, the error type of the computation job execution failure is determined to be a system stability error;

[0075] When the error message in the task log indicates that any subprocess of the task in the computing job has an exit code error, the error type of the computing job execution failure is determined to be a human-caused development error.

[0076] In one embodiment of this disclosure, system stability errors include one or more of the following:

[0077] External storage anomaly, server overload, network congestion, and timeouts for some interface accesses.

[0078] Specifically, system stability errors refer to temporary anomalies that occur in a system due to instability factors, which can recover on their own. The error types described above are only examples.

[0079] Specifically, external storage anomalies refer to anomalies in the external storage on which the computation job depends. Taking Spark jobs as an example, external storage anomalies may include excessive AFS (the Andrew File System, a distributed file storage system) I / O, excessive concurrent access to tables, etc.

[0080] Regarding how to determine whether the error type of the computation job execution failure belongs to these system stability errors, in one embodiment of this disclosure, the error information in the task log includes information characterizing the specific error cause, and the error type is determined based on the error cause to determine whether it is a system stability error.

[0081] Taking Spark jobs as an example, in one embodiment of this disclosure, the error messages in the task execution logs of Spark jobs, such as Errors, Exceptions, Failed, and Caused by, are examined to determine whether the computation job execution failure is caused by a system stability issue. It can be seen that determining whether the computation job execution failure is caused by a stability issue based on the specific content included in the error message has high accuracy.

[0082] When determining whether a computational job failure is due to a human error (developmental error), the exit code is a constant used to characterize the exit state when the program is terminated. In one embodiment of this disclosure, an exit code of zero indicates a normal task exit, while an exit code within a certain constant range agreed upon in another program indicates an abnormal exit caused by the task itself. This type of abnormal exit indicates a logical error in the business code during execution, such as a serious bug (defect). The root cause of such errors is mistakes made by developers during the development process. In this case, the computational job failure can be considered a human error (developmental error). Therefore, the exit code in the task log can be used to determine whether a human error (developmental error) occurred during the execution of the computational job, improving the efficiency of error type determination.

[0083] It is worth noting that when determining the failure category based on the driver log and task log, this embodiment of the disclosure does not limit the order in which it determines whether there is error information in the driver log and whether there is error information in the task log.

[0084] In one embodiment of this disclosure, considering that the driver cannot run normally when there are error messages in the driver log, that is, it cannot perform task distribution and resource scheduling normally, and thus the task cannot be performed, the subsequent step S203 is directly executed when it is determined that there are error messages in the driver log, so as to improve the efficiency of handling the failure of computing job execution.

[0085] Step S203: If the error type is a system stability error, re-execute the calculation job.

[0086] Based on the descriptions in steps S201 and S202, it can be seen that in one embodiment of this application, if there are error messages in the driver log of the computing job, or if there are error messages in the task log and these error messages indicate that the execution of the computing job has encountered a system stability error, the error type is determined to be a system stability error, and the computing job is re-executed.

[0087] As mentioned above, a system stability error refers to a temporary anomaly that occurs in the system due to instability factors, which can recover on its own.

[0088] For example, if the current load on a single server is too high, causing the driver process to terminate, but after a period of time, the load on the server returns to normal, ensuring the resources used by the driver process to run.

[0089] Therefore, in the event of a system stability error, the computation job can be re-executed based on a preset restart strategy. For example, after a system stability error is detected, the computation job can be re-executed after a preset time.

[0090] Step S204: When the error type is a human-caused R&D error, generate a prompt message for manual intervention.

[0091] In this embodiment of the disclosure, the error type is a human-caused R&D error, which indicates that the R&D personnel made an error during the R&D process. This type of error involves business code and cannot be resolved by restarting. Therefore, a prompt message for manual intervention is generated, and then the relevant responsible personnel resolve the issue.

[0092] In one embodiment of this disclosure, the task log also includes multiple task location information, each task location information being used to identify a task in the computation job.

[0093] When the error type is a human-caused R&D error, a prompt message for manual intervention is generated based on the error message and the corresponding task location information.

[0094] Taking Spark jobs as an example, in one embodiment of this disclosure, the task location information is the taskId (a unique identifier for a task) of each task in the computation job.

[0095] In another embodiment of this disclosure, the task location information may include, in addition to taskId, the attemptId (a unique identifier of an attempt) for each attempt (instance) of the task.

[0096] By generating prompts for manual intervention based on error messages and task location information, relevant personnel can promptly identify which task in the calculation job has encountered an error and the corresponding error message when handling human-caused R&D errors. This allows for more efficient confirmation of the handling method and improves the efficiency of dealing with human-caused R&D errors.

[0097] Figure 3 This is a schematic diagram of a method for handling high-precision map calculation job execution failure provided in this embodiment of the disclosure. For ease of understanding, the following is combined with... Figure 3 The method for handling high-precision map calculation job execution failure provided in this disclosure embodiment is further described, and the method specifically includes the following steps:

[0098] Step S301: Submit the Spark job.

[0099] In one embodiment of this disclosure, the submitted Spark job is a high-precision map drawing job.

[0100] Step S302: Determine whether the driver has started running. If yes, proceed to step S304; otherwise, proceed to step S303.

[0101] Step S303: Write the driver-related information into the database.

[0102] Step S304: Determine whether the calculation job ultimately failed. If yes, proceed to step S305; otherwise, proceed to step S307.

[0103] Specifically, based on the preceding description of the Spark workflow, it can be seen that the driver is responsible for requesting the resources needed to execute computation jobs and scheduling tasks. Therefore, if the driver is not running, the computation job cannot be executed, and there is no need to further determine the error category of the computation job execution failure. In the case that the driver is not running, driver-related information can be written to the database.

[0104] Step S305: Determine whether the calculation job failed due to a stability issue. If yes, proceed to step S306; otherwise, proceed to step S308.

[0105] Step S306: Retry the calculation job.

[0106] Step S307: Processing complete.

[0107] Step S308: Prompt for manual intervention.

[0108] Based on the description of steps S304-S306, in this embodiment of the disclosure, if the computation job ultimately does not fail (i.e., the computation job executes successfully), no processing is required. If the computation job ultimately fails, and the failure is due to a stability issue, the computation job is retried. If the failure is not due to a stability issue, no automated processing is performed; instead, manual intervention is required.

[0109] Furthermore, when the submitted Spark job is a high-precision map drawing job, the failure of the Spark job will prevent the subsequent parts of the high-precision map generation from continuing. Therefore, by applying the processing method for computational job execution failure provided in this embodiment, different processing methods can be adopted according to different error types of computational job execution failure, thereby improving the drawing efficiency of high-precision maps.

[0110] refer to Figure 4 , Figure 4 This is a block diagram of an apparatus for implementing a method for handling high-precision map calculation operation failures according to embodiments of the present disclosure. Figure 4 As shown, the device includes:

[0111] The acquisition module 401 is used to acquire the execution log of the computing job in response to the failure of the computing job execution;

[0112] Module 402 is used to determine the error type of the computing job execution failure based on the operation log. The error types include: system stability errors and human error caused by development.

[0113] Execution module 403 is used to re-execute the computation job when the error type is a system stability error;

[0114] The generation module 404 is used to generate a prompt message for manual intervention when the error type is a human-caused R&D error.

[0115] The runtime logs include driver logs and task logs, identifying module 402, specifically used for:

[0116] Determine the error type that caused the computation job to fail based on driver logs and task logs;

[0117] When error messages are found in the driver log, the error type that caused the computation job to fail is determined to be a system stability error.

[0118] If error information is found in the task log, the error type of the computation job execution failure is determined based on the error information in the task log.

[0119] In one embodiment of this disclosure, the step of determining the error type of a computation job execution failure based on error information in the task log includes:

[0120] When the error message in the task log indicates that a system stability error occurred during the execution of the computation job's task, the error type of the computation job execution failure is determined to be a system stability error;

[0121] When the error message in the task log indicates that any subprocess of the task in the computing job has an exit code error, the error type of the computing job execution failure is determined to be a human-caused development error.

[0122] In this embodiment of the disclosure, after a computational job fails to execute, the error type is determined through the operation log of the computational job. Based on the error type, it is determined whether the failure can be resolved by re-executing the computational job. If it can be resolved by re-executing the computational job, i.e., the error type is a system stability error, then it is re-executed. If it cannot be resolved by re-executing the computational job, i.e., the error type is a human error, then manual intervention is prompted.

[0123] As can be seen, different handling methods are adopted according to different error types of computational job execution failures. When system stability errors exist, re-execution is performed without checking for failures, which can save the time increase caused by stability issues and improve the execution efficiency of computational jobs. When research and development errors exist, instead of retrying, a prompt message is generated, avoiding the problem of consuming computing resources due to directly setting retries, and also making it easier for relevant personnel to troubleshoot computational job execution failures. When applied to computational jobs that may fail due to different error types, it can improve the execution efficiency and stability of computational jobs.

[0124] In one embodiment of this disclosure, system stability errors include one or more of the following:

[0125] External storage anomaly, server overload, network congestion, and timeouts for some interface accesses.

[0126] In one embodiment of this disclosure, the task log further includes multiple task location information, each task location information being used to identify a task in a computation job. The generation module 404 is specifically used for:

[0127] When the error type is a human-caused R&D error, a prompt message for manual intervention is generated based on the error message and the corresponding task location information.

[0128] In one embodiment of this disclosure, the computational job is a high-precision map rendering computational job running on Spark.

[0129] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0130] This disclosure provides an electronic device, including:

[0131] At least one processor; and

[0132] A memory that is communicatively connected to at least one processor; wherein,

[0133] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform a method for handling the failure of a computational job.

[0134] This disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used for a method of handling the failure of a computer to perform a computational job.

[0135] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements a method for handling computational job execution failures.

[0136] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0137] like Figure 5As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0138] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for handling computing job execution failure. For example, in some embodiments, the method for handling computing job execution failure may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method for handling computing job execution failure described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured, by any other suitable means (e.g., by means of firmware), to perform a method for handling computing job execution failures.

[0140] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0145] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0146] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for handling high-precision map calculation job failures, comprising: In response to a computing job failure, the execution log of the computing job is retrieved; Based on the operation log, the error type of the computing job execution failure is determined, and the error type includes: system stability error and human-induced development error; If the error type is a system stability error, the computation job will be re-executed. When the error type is a human-caused R&D error, a prompt message for manual intervention is generated; The runtime logs include driver logs and task logs; The step of determining the error type of the computation job execution failure based on the runtime log includes: The error type that caused the computation job to fail is determined based on the driver log and the task log. If error information is found in the driver log, the error type of the computing job execution failure is determined to be a system stability error. If there are error messages in the task log, the error type of the computation job execution failure is determined based on the error messages in the task log.

2. The method according to claim 1, wherein, The step of determining the error type of the computation job execution failure based on the error information in the task log includes: When the error information in the task log indicates that a system stability error occurred during the execution of the computing job's task, the error type of the computing job's execution failure is determined to be a system stability error. When the error information in the task log indicates that any subprocess of the task in the computing job has an exit code error, the error type of the computing job execution failure is determined to be a human-caused development error.

3. The method according to claim 1, wherein, The system stability error includes one or more of the following: External storage anomaly, server overload, network congestion, and timeouts for some interface accesses.

4. The method according to claim 2, wherein the task log further includes multiple task location information, each of the task location information being used to identify a task in the computing job; The step of generating a manual intervention prompt when the error type is a human-caused development error includes: When the error type is a human-caused R&D error, a prompt message for manual intervention is generated based on the error information and the task location information corresponding to the error information.

5. The method according to claim 1, wherein, The computational job is a high-precision map rendering job running on Spark.

6. A processing device for high-precision map calculation job failure, comprising: The acquisition module is used to acquire the execution log of the computing job in response to the failure of the computing job execution; The determination module is used to determine the error type of the computing job execution failure based on the running log, the error type including: system stability error and human-caused development error; The execution module is used to re-execute the computation job when the error type is a system stability error; The generation module is used to generate a prompt message for manual intervention when the error type is a human-caused R&D error. The runtime log includes driver logs and task logs. The determining module is specifically used for: The error type that caused the computation job to fail is determined based on the driver log and the task log. If error information is found in the driver log, the error type of the computing job execution failure is determined to be a system stability error. If there are error messages in the task log, the error type of the computation job execution failure is determined based on the error messages in the task log.

7. The apparatus according to claim 6, wherein, The step of determining the error type of the computation job execution failure based on the error information in the task log includes: When the error information in the task log indicates that a system stability error occurred during the execution of the computing job's task, the error type of the computing job's execution failure is determined to be a system stability error. When the error information in the task log indicates that any subprocess of the task in the computing job has an exit code error, the error type of the computing job execution failure is determined to be a human-caused development error.

8. The apparatus according to claim 6, wherein, The system stability error includes one or more of the following: External storage anomaly, server overload, network congestion, and timeouts for some interface accesses.

9. The apparatus according to claim 7, wherein the task log further includes multiple task location information, each of the task location information being used to identify a task in the computation job, and the generation module is specifically used for: When the error type is a human-caused R&D error, a prompt message for manual intervention is generated based on the error information and the task location information corresponding to the error information.

10. The apparatus according to claim 6, wherein, The computational job is a high-precision map rendering computational job running on Spark.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images