System and method for artificial intelligence based analysis of over-the-air client logs

Abstract

A method and system for analyzing over-the-air (OTA) client logs from vehicles includes: receiving an OTA client log file containing error logs generated during OTA software updates; preprocessing the log file to generate a cleaned log file by removing duplicate log lines and standardizing the format; segmenting the cleaned log file into log blocks; and analyzing the blocks using rule-based and artificial intelligence (AI) methods. The AI ​​analysis includes generating vector embeddings for error-related log lines; comparing the embeddings with a nuanced error database to filter out minor errors; retrieving error context information from the vector database; and generating error summaries using a large language model. The system displays the analysis results via a Fault Analysis Engineer (FAE) review interface and updates the database based on FAE feedback. This hybrid approach achieves efficient processing of OTA client logs while maintaining accuracy through rule-based filtering and contextual AI analysis.

Description

Technical Field

[0001] This disclosure generally relates to the field of software error analysis, and more specifically to systems and methods for analyzing over-the-air (OTA) client logs from vehicles using artificial intelligence and rule-based approaches. Background Technology

[0002] As vehicles incorporate increasingly sophisticated electronic systems and software, the automotive industry is making greater use of over-the-air (OTA) software updates. When an OTA update fails, the original equipment manufacturer (OEM) collects error logs from the affected vehicle and sends them to the OTA service provider for analysis. These logs contain information about the nature and cause of the update failure, but analyzing these logs presents significant technical challenges.

[0003] Traditional OTA client log analysis methods rely heavily on manual review by Fault Analysis Engineers (FAEs). These engineers must examine numerous log files to identify error codes, understand error patterns, and determine the root cause of update failures. This manual process is both time-consuming and resource-intensive, with FAEs spending approximately 20% of their time analyzing client logs. The complexity of these logs is further amplified by their length and the interconnected nature of vehicle subsystems, making it difficult to distinguish root causes from related subsystem errors. The varying implementations across different OEMs exacerbate the challenges of log analysis. Each manufacturer can customize its OTA system with specific plugins and modifications, resulting in thousands of potential error codes and behaviors. Additionally, OEMs may use test vehicles for road testing, generating logs containing test-specific errors that must be differentiated from production vehicle issues. These customization and test scenarios add to the complexity of identifying and diagnosing errors in client logs from OTA software updates.

[0004] Current log analysis methods are also limited by their inability to effectively handle duplicate information. When an OTA update fails, the system typically attempts to complete the update multiple times, generating duplicate error messages in the logs. This redundancy increases the amount of data that must be processed and may mask the root cause of the failure.

[0005] The limitations of manual log analysis highlight the need for more automated and intelligent methods to process OTA client logs. Systems that can efficiently preprocess logs, differentiate between critical and minor errors, and provide contextual analysis will significantly reduce the time and effort required for log analysis, while improving the accuracy and consistency of results. Such systems will be particularly valuable given the increasing volume of OTA updates and the growing complexity of vehicle software systems. Summary of the Invention

[0006] This disclosure at least partially addresses the aforementioned problems. In one embodiment, a method for analyzing over-the-air (OTA) client logs from a vehicle includes: receiving an OTA client log file, which includes error logs generated during an OTA software update of the vehicle; preprocessing the OTA client log file to generate a cleaned log file by removing duplicate log lines and standardizing the log file format; segmenting the cleaned log file into log blocks based on a block size parameter; analyzing the log blocks using a rule engine to extract predefined error codes; and analyzing the log blocks using an artificial intelligence (AI) engine. The AI ​​engine analysis includes: generating vector embeddings for error-related log lines; comparing the vector embeddings with a subtle error database to identify and filter subtle errors; retrieving error context information from the vector database based on remaining errors; and generating an error summary and root cause analysis using a large language model based on the remaining errors, the error context information, and the OTA client information. The method further includes displaying the analysis results, including rule-based error codes and AI-based root cause analysis, via a Fault Analysis Engineer (FAE) review interface. This implementation scheme provides efficient preprocessing and hybrid analysis of OTA client logs through rule-based and artificial intelligence methods, which enables more accurate identification of the root cause of errors while filtering out known minor errors, thereby reducing the time and workload required for log analysis.

[0007] In another embodiment, a system for analyzing over-the-air (OTA) client logs from a vehicle includes: at least one processor; and a non-transitory memory storing instructions that, when executed by the at least one processor, cause the processor to perform operations including: receiving an OTA client log file, the OTA client log file including error logs generated during OTA software updates of the vehicle; preprocessing the OTA client log file to generate a cleaned log file by removing duplicate log lines and standardizing the log file format; segmenting the cleaned log file into log blocks based on a block size parameter; analyzing the log blocks using a rule engine to extract predefined error codes; and analyzing the log blocks using an artificial intelligence (AI) engine. The AI ​​engine analysis includes: generating vector embeddings for error-related log lines; comparing the vector embeddings with a subtle error database to identify and filter subtle errors; retrieving error context information from the vector database based on remaining errors; and generating an error summary and root cause analysis using a large language model based on the remaining errors, the error context information, and the OTA client information. The system displays the analysis results, including rule-based error codes and AI-based root cause analysis, via a Fault Analysis Engineer (FAE) review interface. This system implementation provides an integrated approach to analyzing OTA client logs by combining rule-based error code extraction with AI-driven contextual analysis. The rule engine identifies error patterns through regular expression matching and predefined rules, while the AI ​​engine performs analysis by generating vector embeddings of error-related log lines and filtering based on similarity comparisons with a nuanced error database. By maintaining separate databases for nuanced errors and error context information, the system distinguishes between errors requiring investigation and those that can be filtered, such as those caused by OEM-specific plugins or test vehicle scenarios. The combination of rule-based filtering and contextual AI analysis reduces the time required for FAEs to analyze client logs while maintaining accuracy, a practice particularly advantageous given the varying implementations of OTA systems by different OEMs and the potential for thousands of potential error codes from frequent introductions of new plugins and customizations.

[0008] In another embodiment, a method for analyzing over-the-air (OTA) client logs from a vehicle includes: receiving an OTA client log file, the OTA client log file including error logs generated during an OTA software update of the vehicle and associated OTA client information; preprocessing the OTA client log file to generate a cleaned log file; accessing predetermined block size parameters and overlap parameters from a configuration database based on the log file type of the cleaned log file; and segmenting the cleaned log file into log blocks based on the accessed block size parameters and overlap parameters, wherein each log block is assigned a unique identifier and indexed with reference to its original log file position. The method includes: analyzing each log block using a rule engine to extract predefined error codes and analyzing log-related log lines using an artificial intelligence (AI) engine by generating vector embeddings for the extracted error-based log lines; comparing the vector embeddings with a subtle error database to identify and filter subtle errors; after filtering out the subtle errors, retrieving error context information from the vector database based on the remaining errors; and filtering the retrieved error context information based on the associated OTA client information. The method further includes: aggregating filtered error context information from the analyzed log blocks; generating error summaries and root cause analyses using a large language model based on the aggregated filtered error context information, the associated OTA client information, and error patterns identified in multiple log blocks; displaying the analysis results via a Fault Analysis Engineer (FAE) review interface; and updating one or more of the nuanced error database and the vector database based on FAE feedback received via the FAE review interface. This implementation provides a technical solution for continuously improving OTA client log analysis through a feedback loop between Fault Analysis Engineers and an AI-based analysis system. By capturing FAE feedback on the analyzed errors and integrating this expertise into the nuanced error database and vector database, the system increasingly accurately distinguishes between errors requiring investigation and known nuanced errors, such as those caused by OEM-specific plugins or test vehicle scenarios. The feedback mechanism enables the system to adapt to dynamic changes in OTA implementations across different OEMs, ensuring proper classification of custom-specific errors and test-related issues without requiring manual updates to system specifications. This adaptability is particularly advantageous given the varying implementations of OTA systems by manufacturers and the frequent introduction of new plugins and customizations that can generate thousands of potential error codes.

[0009] The advantages and other advantages and features of this specification will become apparent from the following detailed description, alone or in conjunction with the accompanying drawings. It should be understood that the above summary is provided to introduce, in a simplified form, the selection of concepts further described in the detailed description. This does not imply confirmation of the key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims appended to the detailed description. Furthermore, the claimed subject matter is not limited to implementations that address any of the shortcomings pointed out above or in any part of this disclosure. Attached Figure Description

[0010] A better understanding of various aspects of this disclosure can be achieved by reading the following description of non-limiting embodiments with reference to the accompanying drawings, wherein:

[0011] Figure 1 This is a schematic diagram illustrating a process for analyzing over-the-air (OTA) client logs according to an embodiment of the present disclosure;

[0012] Figure 2 This is a block diagram illustrating an OTA log analysis system according to an embodiment of the present disclosure;

[0013] Figure 3 This is a flowchart illustrating a method for analyzing OTA client logs using hybrid analytics according to an embodiment of this disclosure;

[0014] Figure 4 This is a flowchart illustrating a method for preprocessing OTA client logs according to an embodiment of the present disclosure;

[0015] Figure 5 This is a flowchart illustrating a method for generating log blocks from a cleaned log file according to an embodiment of the present disclosure;

[0016] Figure 6 This is a flowchart illustrating a method for analyzing log blocks using a rules engine according to an embodiment of this disclosure;

[0017] Figure 7 This is a flowchart illustrating a method for analyzing log blocks using an artificial intelligence engine according to an embodiment of the present disclosure;

[0018] Figure 8 This is a flowchart illustrating a method for identifying subtle errors using vector embedding according to an embodiment of the present disclosure;

[0019] Figure 9 This is a flowchart illustrating a method for retrieving error context information from a vector database according to an embodiment of the present disclosure; and

[0020] Figure 10This is a flowchart illustrating a method for updating a database based on feedback from a fault analysis engineer, according to an embodiment of this disclosure. Detailed Implementation

[0021] This disclosure provides a system and method for analyzing over-the-air (OTA) client logs using a hybrid approach that combines rule-based error code extraction with AI-driven contextual analysis. Traditional OTA client log analysis methods rely heavily on manual review by fault analysis engineers (FAEs), requiring significant time to examine log files to identify error codes, understand error patterns, and determine the root causes of update failures. The complexity of these logs is compounded by their length, the interconnected nature of vehicle subsystems, and the varying implementations by different original equipment manufacturers (OEMs). Each manufacturer can customize its OTA system with specific plugins and modifications, resulting in thousands of potential error codes and behaviors that may need to be distinguished from errors in test vehicles.

[0022] The systems and methods disclosed in this paper address these challenges through an integrated approach illustrated in several views. Figure 1 A schematic diagram of the process for analyzing OTA client logs is shown, depicting the flow from the raw log file to the preprocessing, chunking, and hybrid analysis stages to generate actionable results. Figure 2 A block diagram of an OTA log analysis system is shown, illustrating the components that implement a hybrid analysis approach. Figure 3 This paper details the overall methodology for analyzing OTA client logs, outlining the steps from log file reception to analysis and feedback incorporation. Figure 4 This section details log file preprocessing used to remove duplicate information and standardize the format. Figure 5 A method for generating log blocks is shown, which enables the processing of large log files while maintaining contextual relationships. Figure 6 This section details the rule-based analytics components, demonstrating how to extract predefined error codes using regular expression matching and other rule-based methods. Figure 7 The AI-based analysis process is described, including vector embedding generation and comparison with a database of subtle errors. Figure 8 The details illustrate how to identify subtle errors through vector embedding comparisons, and Figure 9 The process of retrieving and filtering error context information from a vector database is illustrated. Figure 10 A feedback mechanism is shown that enables system improvement through FAE input, allowing the database to adapt to new error patterns and OEM-specific implementations.

[0023] These components collectively enable the analysis of OTA client logs through a combination of rule-based filtering and contextual AI analytics, while maintaining accuracy. This can be particularly advantageous given the varying implementations of OTA systems by different OEMs and the potential for introducing new plugins and customizations to generate thousands of potential error codes. The system's ability to be adjusted via FAE feedback allows it to handle changes in OTA implementations without requiring manual updates to system specifications.

[0024] In one implementation scheme, such as Figure 1 The OTA client log analysis process described in the text can be performed by, for example... Figure 2 The OTA log analysis system 200 shown is used to execute this process. The OTA client log analysis process may include... Figure 3 The method 300 includes one or more operations. Method 300 can be performed by... Figure 4 One or more operations in method 400 shown are used to preprocess log files, by Figure 5 The method 500 shown in the diagram generates log blocks through operations performed separately in... Figure 6 and Figure 7 The rule-based and AI-based methods detailed in methods 600 and 700 illustrate log analysis. AI-based analysis may include... Figure 8 The vector embedding comparison operation of method 800 shown is used to identify subtle errors, and by... Figure 9 The method 900 shown retrieves error context information. The system can further retrieve error context information based on FAE feedback. Figure 10 The method 1000 shown is used to update the database, enabling continuous improvement of analytical capabilities.

[0025] refer to Figure 1 This paper illustrates a hybrid process 100 for analyzing over-the-air (OTA) client logs, demonstrating an implementation scheme for processing and analyzing client logs using rule-based and AI-based methods. Process 100 is particularly advantageous in applications such as automotive OTA software updates, as efficient analysis of error logs can quickly identify and resolve update issues. Process 100 combines rule-based analysis with AI technology to provide more robust analysis of OTA client logs, enabling rapid identification of known error patterns and better root cause analysis.

[0026] OTA client log file 102 represents the raw log file generated during an OTA software update for a vehicle. In one embodiment, OTA client log 102 may include an error log containing information about the nature and cause of the update problem, including error codes, timestamps, and system messages. In another embodiment, OTA client log 102 may include additional client information such as client version, device identifier, model name, manufacturer name, and configuration parameters. These logs are collected from the affected vehicle when an OTA update encounters a problem and sent to the OTA service provider for analysis. OTA client log 102 may contain useful diagnostic information that, when properly analyzed, can provide insights into the root cause of the update problem.

[0027] Log preprocessing unit 104 processes OTA client logs 102 to generate cleaned log files by removing duplicate information and standardizing the format. In one embodiment, log preprocessing unit 104 identifies and removes duplicate log lines that occur when the OTA system attempts to complete an update multiple times. These duplicate log lines generate duplicate error messages based on pre-configured retry parameters. For example, if an OTA update activity is configured to retries a maximum of three times, the same error message may appear up to three times repeatedly in the log file. In another embodiment, preprocessing unit 104 standardizes the format of timestamps and file paths, trimming timestamps to a standardized format and retaining only filenames instead of full directory paths. For example, excessively long timestamps can be trimmed to a uniform format, and file paths can be shortened to include only the relevant filenames instead of the entire directory structure. This preprocessing reduces the amount of data that needs to be processed while maintaining the integrity of error information. Log preprocessing unit 104 may also perform additional formatting and standardization operations to ensure consistent analysis of logs from different vehicle models or manufacturers, such as standardizing line endings, character encodings, and error message formats across different OEM implementations.

[0028] The cleaned log 106 represents a preprocessed log file after deduplication and format standardization. In one implementation, the cleaned log 106 contains only unique error messages and events, with timestamps and file paths formatted normally. In another implementation, the cleaned log 106 can be stored in memory for subsequent chunking and analysis, while maintaining a reference to the original log file location. The cleaned log 106 provides a more efficient and standardized input for subsequent analysis steps while preserving the continuity of diagnostic information contained in the original log.

[0029] The chunking unit 108 divides the cleaned log 106 into chunks based on a predetermined chunk size parameter. In one embodiment, the chunking unit 108 sets the chunk size and overlap parameters based on the log file type, identifying log file boundaries and structure to ensure correct segmentation. In another embodiment, the chunking unit 108 assigns a unique identifier to each chunk and indexes it with reference to its original log file location, thereby achieving traceability of the analysis results back to the source log. The chunking unit 108 can dynamically adjust the chunk size based on the content and structure of different log types to improve subsequent analysis steps.

[0030] Log block 110 represents a segment of a cleaned log file. In one embodiment, each log block 110 contains a portion of the log file sized according to a predetermined block size parameter, with overlap between blocks to maintain context across block boundaries. In another embodiment, log block 110 is stored along with its unique identifier and index information for processing by the hybrid analysis engine 112. Log block 110 enables parallel processing of large log files while maintaining contextual relationships between different parts of the log.

[0031] The hybrid analysis engine 112 comprises both a rules engine 114 and an AI engine 116, enabling comprehensive analysis through rule-based methods and AI technology. The rules engine 114 uses predefined rules and regular expression patterns to analyze log blocks to extract error codes. In one implementation, the rules engine 114 scans log blocks to find specific error code patterns, identify timeout events, and verify binary file size constraints. In another implementation, the rules engine 114 generates a rule-based error report that includes the identified error codes, error definitions, associated log lines, timeout events, and binary size violations. The hybrid analysis engine 112 leverages the complementary strengths of the two analysis methods to provide comprehensive and efficient log analysis.

[0032] AI Engine 116 uses vector embeddings and a large language model for analysis. In one implementation, AI Engine 116 generates vector embeddings for error-related log lines, compares them to a nuanced error database to filter out known nuanced errors, and retrieves error context information from the vector database. For example, when analyzing a log block containing multiple error messages, AI Engine 116 can identify error patterns that match known plugin-specific errors or test vehicle scenarios stored in the nuanced error database. In another implementation, AI Engine 116 uses a large language model to generate error summaries and root cause analyses based on residual errors, error context information, and OTA client information. AI Engine 116 contextualizes errors by considering OTA client version information, device specifications, manufacturer customization, and test status to provide more accurate and relevant analysis. For example, when analyzing errors from custom plugins with a specific OEM implementation, AI Engine 116 references relevant plugin specifications and error codes to correctly interpret the error context. AI Engine 116 provides analytical capabilities that can identify patterns and relationships that cannot be captured by rule-based analysis alone, especially where error messages may not directly correspond to specific error codes but describe system behavior indicating underlying problems.

[0033] Vector database 118 stores error context information and associated vector embeddings. In one implementation, vector database 118 includes error definitions, known root causes, historical solutions, OEM-specific information, and software version dependencies. In another implementation, vector database 118 is updated based on feedback from fault analysis engineers to incorporate new error patterns and solutions. Vector database 118 serves as a knowledge base, becoming increasingly comprehensive over time through continuous updates and feedback.

[0034] The minor error database 120 stores vector embeddings of known minor errors. In one implementation, these errors include those caused by OEM-specific plugins or test vehicle scenarios that do not require investigation. In another implementation, the minor error database 120 is continuously updated based on the frequency of error occurrence and FAE classification. The minor error database 120 helps improve analysis efficiency by filtering out known benign errors that do not require detailed investigation.

[0035] The rule-based result 124 and the AI-based result 126 are combined to generate a comprehensive analysis result 122. In one implementation, the analysis result 122 includes both rule-based error codes and AI-generated root cause analysis, providing multiple perspectives on the identified errors. In another implementation, the analysis result 122 may include suggested solutions based on historical solution data. Combining the analysis results provides a comprehensive understanding of the log content while leveraging the advantages of both analysis methods.

[0036] The FAE review interface 128 displays analysis results and captures feedback from failure analysis engineers. In one implementation, the interface allows FAEs to review rule-based and AI-based analysis results and provide feedback on error classification. In another implementation, the interface enables FAEs to log new error patterns related to OEM customization or test scenarios. The FAE review interface 128 facilitates continuous improvement of the analysis system through professional feedback and validation.

[0037] Feedback database 130 stores FAE feedback to continuously improve the analysis system. In one embodiment, feedback database 130 captures plug-in errors, test vehicle errors, and custom errors identified by the FAE, including specific information about OEM customization, test scenarios, and associated error patterns. For example, when the FAE identifies certain error codes associated with newly added OEM plug-ins, this information is stored in feedback database 130 to improve future analysis of similar errors. In another embodiment, this feedback is used to update the nuanced error database 120 and the vector database 118, improving the accuracy of future analyses by incorporating expertise on error classification and contextual relationships. Feedback database 130 enables the system to be tweaked and improved based on expertise and practical experience, especially when OEMs frequently modify their implementations or use dedicated test vehicles for testing. The database may store metadata about feedback entries, such as timestamps, FAE identifiers, and classification types (plug-in errors, test errors, or custom errors), making it possible to track feedback patterns and system improvements over time.

[0038] The OTA DevOp team 132 receives the verified analysis results and implements corresponding system modifications and enhancements based on them. In one implementation, the OTA DevOp team 132 uses the analysis results to fix identified errors within the OTA system architecture and associated binaries. In another implementation, the OTA DevOp team 132 may modify system specifications in response to new error patterns identified through the analysis process. Therefore, the OTA DevOp team 132 constitutes the final stage in the analysis workflow, where analytical insights are translated into actionable system improvements and enhancements. Such improvements and enhancements may include, but are not limited to, modifying the OTA client implementation, updating error handling protocols, and optimizing the binary generation process.

[0039] refer to Figure 2 This paper illustrates an OTA log analysis system 200 for analyzing over-the-air (OTA) download logs, demonstrating one implementation for processing and analyzing client logs using rule-based and AI-based methods. The OTA log analysis system 200 is configured to analyze error logs generated during OTA software updates in a vehicle and provide the analysis results to a fault analysis engineer.

[0040] The OTA log analysis device 202 includes a processor 204 and a non-transitory storage 206 for storing instructions and modules for analyzing OTA client logs. In one embodiment, the OTA log analysis device 202 can be implemented as a server system accessible via a network connection. In another embodiment, the OTA log analysis device 202 can be implemented as a distributed system, with components deployed across multiple computing nodes.

[0041] Processor 204 is configured to execute instructions stored in non-transitory storage 206 to perform log analysis operations. In one embodiment, processor 204 may include multiple processing cores for parallel processing of log blocks. In another embodiment, processor 204 may include a dedicated hardware accelerator for comparing embedded vector operations.

[0042] Non-transitory storage 206 stores various software modules and databases used during log analysis. In one embodiment, non-transitory storage 206 may include a solid-state storage device. In another embodiment, non-transitory storage 206 may include a combination of fast-access storage for active processing and slower archive storage for historical logs and feedback data.

[0043] Rule engine 208 uses predefined rules and regular expression patterns to analyze log blocks to extract error codes. In one implementation, rule engine 208 scans log blocks to find specific error code patterns, identify timeout events, and verify binary file size constraints. In another implementation, rule engine 208 generates a rule-based error report that includes the identified error code, error definition, associated log line, timeout event, and binary size violation.

[0044] Vector database 210 stores error context information and associated vector embeddings. In one implementation, vector database 210 includes error definitions, known root causes, historical solutions, OEM-specific information, and software version dependencies. In another implementation, vector database 210 is updated based on feedback from fault analysis engineers to incorporate new error patterns and solutions.

[0045] The minor error database 212 stores vector embeddings of known minor errors. In one implementation, these errors include those caused by OEM-specific plug-ins or test vehicle scenarios, which do not require investigation. In another implementation, the minor error database 212 is continuously updated based on the frequency of error occurrence and FAE classification.

[0046] Large language model 214 generates error summaries and root cause analyses based on contextualized errors and OTA client information. In one implementation, large language model 214 processes filtered error contextual information to generate a natural language description of the error causes. In another implementation, large language model 214 proposes potential solutions based on historical solution data stored in vector database 210.

[0047] Log preprocessing module 216 processes raw OTA client logs to generate cleaned log files. In one implementation, log preprocessing module 216 removes duplicate log lines resulting from multiple update attempts and standardizes timestamps and file path formats. In another implementation, log preprocessing module 216 performs additional formatting operations to ensure consistent analysis of logs from different vehicle models and manufacturers.

[0048] The chunking module 218 divides the cleaned log file into chunks based on predetermined parameters. In one embodiment, the chunking module 218 sets chunk size and overlap parameters based on the log file type, and identifies log file boundaries and structure to ensure correct segmentation. In another embodiment, the chunking module 218 assigns a unique identifier to each chunk and indexes it with reference to its original log file location.

[0049] Feedback database 220 stores FAE feedback to continuously improve the analysis system. In one embodiment, feedback database 220 captures plug-in errors, test vehicle errors, and customization errors identified by the FAE, including specific information about OEM customizations, test scenarios, and associated error patterns. In another embodiment, feedback database 220 stores metadata about feedback entries to enable tracking of feedback patterns and system improvements over time.

[0050] User device 250 enables the FAE to interact with OTA log analysis system 200. In one embodiment, user device 250 may include a desktop or laptop computer with secure network access to OTA log analysis device 202. In another embodiment, user device 250 may include a mobile device with a dedicated application for reviewing analysis results and providing feedback.

[0051] Display device 230 presents analysis results and captures FAE feedback through a graphical interface. In one embodiment, display device 230 displays both rule-based error codes and AI-generated root cause analysis, allowing the FAE to review and verify the results. In another embodiment, display device 230 provides interactive visualization tools for exploring error patterns and relationships across multiple log files.

[0052] refer to Figure 3 The diagram illustrates a flowchart of method 300 for analyzing over-the-air (OTA) client log files, showcasing one implementation scheme for processing and analyzing client logs using rule-based and AI-based methods. Method 300 effectively analyzes error logs from vehicle OTA software updates through a hybrid approach combining traditional rule-based error code extraction with AI-driven contextual analysis.

[0053] At operation 302, the OTA log analysis system receives OTA client log files, including error logs generated during OTA software updates for the vehicle. In one implementation, the OTA log analysis system processes the OTA client log files, including error logs containing information about the nature and cause of update problems, including error codes, timestamps, and system messages. The OTA client log files may further include additional client information such as client version, device identifier, model name, manufacturer name, and configuration parameters. In another implementation, when an OTA update encounters problems, OTA client log files are collected from the affected vehicle and sent to the OTA service provider for analysis via a secure network connection.

[0054] At Operation 304, the OTA log analysis system preprocesses the OTA client log file to generate a cleaned log file by removing duplicate log lines and standardizing the log file format. In one implementation, preprocessing includes identifying and removing duplicate log lines that occur when the OTA system attempts to complete an update multiple times based on pre-configured retry parameters. For example, if an OTA update activity is configured to retries a maximum of three times, the same error message might appear up to three times repeatedly in the log file. In another implementation, preprocessing includes standardizing the format of timestamps and file paths by cropping timestamps to a consistent format and retaining only filenames instead of full directory paths to optimize token usage for subsequent AI analysis.

[0055] At operation 306, the OTA log analysis system generates log blocks from the cleaned log files based on a predetermined block size parameter. In one implementation, the block size and overlap parameters are set based on the log file type, where log file boundaries and structure are identified to ensure correct segmentation. Each block can be assigned a unique identifier and indexed with reference to its original log file location. In another implementation, the blocks, along with their unique identifiers and index information, are stored for subsequent analysis, enabling parallel processing of large log files while maintaining contextual relationships between different parts of the log.

[0056] At operation 308, the OTA log analysis system uses a rule engine to analyze log blocks to extract predetermined error codes. In one implementation, the rule engine uses regular expression matching to scan log blocks for specific error code patterns, identify timeout events, and verify binary file size constraints. For example, the rule engine may identify error code 001 when the binary file size exceeds the available vehicle storage capacity; or it may identify a timeout event when the update duration exceeds a predetermined threshold. In another implementation, the rule engine generates a rule-based error report that includes the identified error code, error definition, associated log line, timeout event, and binary size violation.

[0057] At operation 310, the OTA log analysis system uses an AI engine based on a subtle error database and an error context vector database to analyze log blocks. In one implementation, the AI ​​engine generates vector embeddings for error-related log lines and compares them with the subtle error database to identify and filter out known subtle errors, such as those caused by OEM-specific plugins or test vehicle environments. This comparison uses similarity thresholds and error code consistency checks to ensure accurate filtering. In another implementation, after filtering, the AI ​​engine retrieves error context information from the vector database based on the remaining errors, where the context information includes error definitions, known root causes, historical solutions, OEM-specific information, and software version dependencies.

[0058] At operation 312, the OTA log analysis system generates analysis results, including rule-based error codes and descriptions, as well as AI-based error summaries and root cause analyses. In one implementation, the analysis results include both rule-based error codes and AI-generated root cause analyses, providing multiple perspectives on the identified errors. AI-based analysis may include natural language summaries describing the causes of the errors, as well as suggested solutions based on historical solution data. In another implementation, the OTA log analysis system contextualizes errors by considering OTA client version information, device specifications, manufacturer customization, and testing status to provide more accurate and relevant analysis.

[0059] At operation 314, the OTA log analysis system displays the analysis results via a Fault Analysis Engineer (FAE) review interface. In one implementation, this interface displays both rule-based error codes and AI-based root cause analysis, allowing the FAE to review and verify the results. The interface can highlight specific error lines in the logs and provide summaries, including error location, number of errors, potential root causes, and possible solutions. In another implementation, the interface provides interactive visualization tools for exploring error patterns and relationships across multiple log files.

[0060] At point 316, the OTA log analysis system receives FAE feedback on the analysis results via a FAE review interface. In one implementation, particularly for plug-in errors, test vehicle errors, or customization errors, this interface allows the FAE to review rule-based and AI-based analysis results and provide feedback on error classification. In another implementation, the interface enables the FAE to log new error patterns related to OEM customizations or testing scenarios, including specific information about plug-in specifications, test batch information, and associated error patterns.

[0061] In Operation 318, the OTA log analysis system updates one or more of the subtle error database and the error context vector database based on FAE feedback. In one implementation, the OTA log analysis system determines whether a newly identified error should be added to the subtle error database based on its frequency of occurrence and FAE classification. For example, if the FAE identifies certain error codes as associated with newly added OEM plugins, this information can be stored to improve future analysis of similar errors. In another implementation, when an error is classified as a critical error based on its impact on OTA functionality or its impact on multiple OEM implementations, the OTA log analysis system updates the vector database using the new error context information and associated vector embeddings.

[0062] Method 300 provides a technical solution for analyzing OTA client logs by combining traditional rule-based analytics with AI-driven contextual understanding. This hybrid approach enables the rapid identification of known error patterns through rule-based analytics and the sophisticated interpretation of complex error scenarios through AI analysis. The system significantly reduces noise in the analysis results by comparing vector embeddings with a continuously updated database of known subtle errors to filter out minor errors. Contextualizing residual errors using an error context vector database allows for more accurate root cause analysis by considering OEM-specific implementations, plugin specifications, and historical solutions. The continuous feedback loop between the FAE and the analytics system allows for continuous improvement in error classification and contextual understanding, which is particularly valuable given the varying implementations of OTA systems by different OEMs and the potential for thousands of potential error codes from frequent introductions of new plugins and customizations. This adaptability ensures the system remains effective even as OEM implementations evolve and new error patterns emerge.

[0063] refer to Figure 4 The diagram illustrates a flowchart of method 400 for preprocessing OTA client log files, showing one implementation for cleaning and standardizing log files before analysis. Method 400 enables efficient preprocessing of error logs from vehicle OTA software updates by removing duplicate information and standardizing the format, thereby optimizing subsequent analysis steps.

[0064] At operation 402, the OTA log analysis system receives OTA client log files, including error logs generated during OTA software updates for the vehicle. In one implementation, the OTA client log file includes error logs containing information about the nature and cause of the update problem, including error codes, timestamps, system messages, and client information such as client version, device identifier, model name, manufacturer name, and configuration parameters. In another implementation, when an OTA update encounters a problem, the OTA client log file is collected from the affected vehicle and sent to the OTA service provider for analysis via a secure network connection.

[0065] At operation 404, the OTA log analysis system identifies duplicate log lines in the OTA client log file. In one implementation, the system scans the log file for duplicate error messages that occur when the OTA system attempts to complete the update multiple times based on pre-configured retry parameters. For example, if an OTA update activity is configured to retries a maximum of three times, the same error message may appear up to three times repeatedly in the log file. In another implementation, the system identifies duplicate log lines by comparing message content, error codes, and associated timestamps across log files.

[0066] At operation 406, the OTA log analysis system removes duplicate log lines from the OTA client log files. In one implementation, the system only retains the first occurrence of each duplicate log line while preserving the temporal order of the events. In another implementation, the system retains metadata about the number of duplicates removed to provide context about retry attempts during subsequent analysis.

[0067] At operation 408, the OTA log analysis system identifies timestamps in the OTA client log files. In one implementation, the system uses pattern matching to identify common timestamp formats to locate the timestamp information associated with each log entry. In another implementation, the system extracts absolute and relative time information to maintain the temporal relationships between events.

[0068] At operation 410, the OTA log analysis system trims timestamps into a standardized format. In one implementation, the system converts timestamps to a consistent format for efficient token use in subsequent AI analysis. In another implementation, the system standardizes the timezone information and time resolution in all log entries to ensure consistency in time-based analysis.

[0069] At operation 412, the OTA log analysis system identifies file paths in the OTA client log files. In one implementation, the system uses pattern matching to locate full directory paths and file references within log entries. In another implementation, the system identifies both absolute and relative file paths to ensure comprehensive preprocessing.

[0070] At operation 414, the OTA log analysis system trims file paths to preserve filenames. In one implementation, the system extracts only the relevant filenames from the full directory path to reduce token usage while maintaining referencing information. In another implementation, the system preserves the mapping between the shortened filenames and their original paths to maintain traceability.

[0071] At operation 416, the OTA log analysis system stores the preprocessed log file as a cleaned log file. In one implementation, the system saves the cleaned log file in a standardized format and removes duplicates while maintaining references to its original source file. In another implementation, the system includes metadata about the preprocessing operations performed, including the number of duplicates removed and the format standardization applied.

[0072] Method 400 provides efficient preprocessing of OTA client logs by removing redundant information and standardizing the format. This preprocessing is particularly valuable for improving the computational efficiency of subsequent analysis steps, including rule-based error code extraction and AI-driven contextual analysis. Removing duplicate log lines reduces noise in the analysis while maintaining information about system behavior. Standardization of timestamps and file paths enables consistent processing across different OEM implementations and log formats, while optimizing token usage in AI analysis steps.

[0073] refer to Figure 5 The diagram illustrates a flowchart of method 500 for segmenting log files for analysis, showing one implementation for processing cleaned log files and segmenting them into chunks for subsequent analysis. Method 500 enables efficient processing of large log files while maintaining contextual relationships between different parts of the log.

[0074] At operation 502, the OTA log analysis system receives a cleaned log file that has been preprocessed to remove duplicate log lines and standardize its format. In one implementation, the cleaned log file includes an error log containing information about the nature and cause of the update problem, where timestamps and file paths are standardized for efficient use of tokens in subsequent AI analysis. In another implementation, the cleaned log file maintains a reference to its original source file and includes metadata about the preprocessing operations performed, including the number of duplicates removed and the format standardization applied.

[0075] At Operation 504, the OTA log analysis system sets block size and overlap parameters based on the log file type. In one implementation, the system accesses predetermined block size and overlap parameters from a configuration database based on the log file type, where these parameters are determined through experimental validation on different types of log files. For example, structured log files that primarily contain error codes and system messages may use smaller block sizes than log files containing lengthy descriptive text. In another implementation, the block size and overlap parameters are dynamically adjusted based on the content density and distribution of error messages within the log file.

[0076] At operation 506, the OTA log analysis system identifies log file boundaries and structure. In one implementation, the system analyzes cleaned log files to identify natural segmentation points, such as timestamp-based partitions, error code sequences, or logical groupings of related events. For example, the system may identify boundaries between different update attempts or between different stages of the OTA update process. In another implementation, the system determines structural elements such as header information, error code blocks, and system message sequences to ensure proper segmentation, thereby maintaining the logical flow of information.

[0077] At operation 508, the OTA log analysis system segments the cleaned log file into blocks based on block size and overlap parameters. In one implementation, the system creates blocks of uniform size with a predetermined overlap between adjacent blocks to maintain context across block boundaries, particularly for error patterns that may span multiple blocks. For example, if an error sequence spans the end of one block and the beginning of another, the overlap ensures that the complete error sequence is captured in at least one block. In another implementation, the system adjusts the block boundaries to align with natural breakpoints in the log file structure, such as error sequences or the completion of update phases, while maintaining the specified overlap parameters.

[0078] At operation 510, the OTA log analysis system assigns a unique identifier to each block. In one implementation, the system generates an identifier that contains information about the block's location within the original log file, the log file type, and a timestamp, enabling efficient tracking and retrieval of the block during subsequent analysis. In another implementation, the system includes metadata about the block's content in the identifier, such as the number of error codes or the presence of specific types of system messages, thereby aiding in targeted analysis of blocks containing specific types of information.

[0079] At operation 512, the OTA log analysis system indexes blocks by referencing their original log file locations. In one implementation, the system maintains a mapping between each block and its source location in the original log file, thereby achieving traceability and context preservation during analysis. For example, when an error pattern is identified in a block, the system can reference the original log file location as necessary to examine the surrounding context. In another implementation, the system creates an index structure that includes relationships between blocks, such as overlapping content or sequential relationships, thereby aiding in the analysis of error patterns spanning multiple blocks.

[0080] At operation 514, the OTA log analysis system stores blocks in memory for subsequent analysis. In one implementation, the system organizes the blocks into a data structure optimized for parallel processing, where the content, metadata, and relationships of each block to other blocks are easily accessible for rule-based and AI-driven analysis. In another implementation, the system implements a caching strategy that prioritizes frequently accessed blocks or blocks containing critical error patterns, thereby improving processing efficiency during analysis.

[0081] Method 500 provides a log file segmentation approach that strikes a balance between efficient processing and maintaining contextual relationships within log data. The combination of configurable block parameters and intelligent segmentation based on the log file structure enables effective analysis of error patterns in logs from different types of OTA clients. The method's block identification and indexing approach ensures traceability to the source logs while facilitating parallel processing and efficient access in subsequent analysis steps. This segmentation strategy is particularly useful for processing large log files from different OEM implementations, as maintaining the relevant context of error patterns enables accurate analysis.

[0082] refer to Figure 6 The diagram illustrates a flowchart of a method 600 for generating rule-based error reports from log data, and shows one implementation for analyzing log blocks using a rule-based approach to identify and log error patterns. Method 600 achieves efficient extraction of error codes and associated information through pattern matching and constraint validation.

[0083] At operation 602, the OTA log analysis system receives preprocessed log blocks segmented from a cleaned log file. In one embodiment, the log blocks comprise portions of a log file sized according to a predetermined block size parameter, with overlap between blocks to maintain context across block boundaries. In another embodiment, the log blocks are received along with their unique identifiers and index information to reference their original log file location, thereby enabling traceability during analysis.

[0084] At operation 604, the OTA log analysis system initializes a regular expression pattern for error code matching. In one implementation, the system loads a predefined regular expression pattern designed to identify specific error code formats and patterns within a log block. For example, the pattern can be initialized to match error codes in a format such as "ERR_001" or numeric codes such as "0x640a". In another implementation, the system can initialize a context-aware pattern that takes into account surrounding text and formatting to improve the accuracy of error code identification.

[0085] At operation 606, the OTA log analysis system uses regular expression matching to scan log blocks for error code patterns. In one implementation, the system applies an initialized regular expression pattern to each log block to identify the occurrence of an error code, maintaining a reference to the original log line where the error was found. In another implementation, the system can employ parallel processing to scan multiple blocks simultaneously, thereby improving the processing efficiency of large log files.

[0086] At operation 608, the OTA log analysis system identifies timeout events in the log block. In one implementation, the system searches for specific patterns indicating that the update duration exceeds a predetermined threshold, such as connection timeout or update process timeout. In another implementation, the system analyzes time information within the log block to identify sequences of events indicating timeout conditions, such as repeated connection attempts or incomplete update processes.

[0087] At operation 610, the OTA log analysis system verifies binary file size constraints. In one implementation, the system compares the binary file size mentioned in the log block with the available vehicle storage capacity to identify potential size-related faults. For example, if the binary file size exceeds the available vehicle storage capacity, this can trigger an identification of a size constraint violation. In another implementation, the system may analyze the compression ratio and incremental file size to ensure they meet the specific requirements of OTA updates.

[0088] At operation 612, the OTA log analysis system extracts the log code and associated log line. In one implementation, the system creates a mapping between the identified error code and its surrounding log context, including timestamps and related system messages. In another implementation, the system may extract additional metadata associated with each error occurrence, such as client version, device specifications, and configuration parameters.

[0089] At operation 614, the OTA log analysis system identifies error code definitions in the error code database. In one implementation, the system queries the database, which contains a comprehensive error code definition, known causes, and potential solutions for each identified error code. In another implementation, the system can retrieve OEM-specific error definitions and handling guidelines based on manufacturer information associated with the log block.

[0090] At operation 616, the OTA log analysis system generates a rule-based error report, which includes the identified error code, error definition, associated log line, timeout event, and binary size violation. In one implementation, the report includes detailed information about each error occurrence, including chronological order, contextual information, and potential impact on the OTA update process. In another implementation, the system can organize the report hierarchically, grouping related errors and highlighting issues that may require immediate attention.

[0091] At operation 618, the OTA log analysis system stores rule-based error reports for hybrid analysis. In one implementation, the system saves the reports in a format compatible with subsequent AI-based analysis, enabling the integration of rule-based findings with AI-driven contextual understanding. In another implementation, the system may include metadata about the rule-based analysis process, such as the patterns used and the confidence levels of the matches, to guide the hybrid analysis process.

[0092] Method 600 effectively identifies and logs errors through rule-based analysis of log blocks. This approach is particularly valuable for rapidly identifying known error patterns and verifying system constraints. A combination of regular expression pattern matching, timeout detection, and binary file size verification enables comprehensive error detection, while integration with an error code database provides context and definition for the identified issues. The resulting rule-based error reports serve as the basis for hybrid analysis, complementing AI-based methods with structured identification of known error patterns. This method is especially effective for handling common error scenarios and system constraint violations during OTA updates across different OEM implementations.

[0093] refer to Figure 7 The diagram illustrates a flowchart of method 700 for analyzing log errors using an AI engine, and shows one implementation for processing and analyzing log blocks using an artificial intelligence-based approach. Method 700 enables efficient analysis of error logs during vehicle OTA software updates through AI-driven contextual analysis and error filtering.

[0094] At operation 702, the OTA log analysis system receives preprocessed log blocks segmented from a cleaned log file. In one embodiment, the log blocks comprise portions of a log file sized according to predetermined block size parameters, with overlap between blocks to maintain context across block boundaries. In another embodiment, the log blocks are received along with their unique identifiers and index information to reference their original log file location, thereby enabling traceability during analysis.

[0095] At operation 704, the OTA log analysis system counts the number of errors in each log block. In one implementation, the system performs an initial scan of each block to identify and enumerate error patterns, including error codes, timeout events, and binary size violations. In another implementation, the system maintains cumulative counts of different error types within each block, providing statistics on error distribution and frequency.

[0096] At operation 706, the OTA log analysis system extracts error-related log lines from each log block. In one implementation, the system identifies and extracts log lines containing error codes, system messages, and contextual information related to the occurrence of the error. In another implementation, the system may extract additional metadata associated with each error occurrence, such as timestamps, device specifications, and configuration parameters.

[0097] At operation 708, the OTA log analysis system generates vector embeddings for the extracted error-related log lines. In one implementation, the system uses a large language model to generate vector embeddings that capture the semantic meaning of the error message and its surrounding context. In another implementation, the system may employ a specialized embedding model specifically trained for OTA update logs to better capture domain-specific error patterns and relationships.

[0098] In step 710, the OTA log analysis system compares the vector embeddings with a subtle error database to identify subtle errors, such as... Figure 8 The details are as follows. In one implementation, the system calculates a similarity score between the generated vector embedding and known subtle error embeddings stored in a database. In another implementation, the system can apply multiple similarity thresholds for different types of errors, allowing for more granular filtering based on error category or OEM-specific requirements.

[0099] At operation 712, the OTA log analysis system filters out identified minor errors. In one implementation, the system removes error entries that match known minor errors above a predetermined similarity threshold, retaining only significant errors for further analysis. In another implementation, the system may retain metadata about the filtered errors for statistical analysis and system improvement.

[0100] At operation 714, the OTA log analysis system retrieves error context information from the vector database based on the remaining errors, such as... Figure 9 The details are as follows. In one implementation, the system uses the embeddings of critical errors to query a vector database to retrieve relevant error definitions, known root causes, historical solutions, and OEM-specific information. In another implementation, the system can retrieve software version dependencies and configuration requirements associated with each error type.

[0101] At operation 716, the OTA log analysis system contextualizes the remaining errors using the retrieved error context information. In one implementation, the system combines error messages with their associated contextual information, including the OTA client version, device specifications, and manufacturer customizations. In another implementation, the system can analyze the temporal relationship between errors and system events to provide additional context regarding error occurrence patterns.

[0102] At operation 718, the OTA log analysis system uses a large language model to generate error summaries and root cause analyses based on contextualized information and OTA client information. In one implementation, the system provides the large language model with contextualized errors, OTA client information (including client version and device specifications), error definitions, and known root causes to generate a natural language summary describing the identified error type, potential root causes, and suggested solutions. In another implementation, the system can customize the prompts provided to the large language model based on OEM-specific requirements or error patterns.

[0103] At operation 720, the OTA log analysis system stores error summaries and root cause analyses. In one implementation, the system saves the analysis results in a format compatible with subsequent reviews by fault analysis engineers, including references to the original log locations and filtered errors. In another implementation, the system may include metadata about the analysis process, such as similarity scores used for filtering and confidence levels for root cause determination.

[0104] Method 700 provides efficient analysis of OTA client logs through AI-driven processing and filtering. This approach is particularly valuable for identifying and analyzing critical errors while filtering out known minor errors, such as those caused by OEM-specific plugins or test scenarios. A combination of vector embeddings, similarity-based filtering, and large-scale language model analysis enables error analysis that considers the semantic content of error messages and their broader context within the OTA update process. This approach's contextualization of errors and generation of natural language summaries allows for more efficient review by fault analysis engineers, which can be particularly valuable given the varying implementations of OTA systems by different OEMs and the potential for thousands of potential error codes generated by frequent introductions of new plugins and customizations.

[0105] refer to Figure 8 The diagram illustrates a flowchart of method 800 for identifying subtle errors using vector embedding, and shows one implementation for filtering known subtle errors from OTA client logs via vector similarity comparison. Method 800 achieves efficient identification and removal of errors that do not require investigation, such as errors caused by OEM-specific plugins or test vehicle scenarios.

[0106] At operation 802, the OTA log analysis system receives vector embeddings of error-related log lines generated from the extracted error information. In one implementation, the vector embeddings are generated using a large language model to capture the semantic meaning of the error message and its surrounding context. In another implementation, the system may employ a specialized embedding model specifically trained for OTA update logs to better capture domain-specific error patterns and relationships.

[0107] At operation 804, the OTA log analysis system accesses a null error database that includes vector embeddings of known null errors. In one implementation, this database contains vector embeddings of errors previously classified as null errors by fault analysis engineers, including OEM-plugin-specific errors, test vehicle scenarios, and known benign system behaviors. In another implementation, the database includes metadata about each null error, such as error code, error description, and associated OEM-specific information.

[0108] At operation 806, the OTA log analysis system initializes a similarity threshold for subtle error matching. In one implementation, the system sets a predetermined threshold based on experimental validation of error classification accuracy. In another implementation, the system can employ multiple thresholds for different error categories, allowing for more granular filtering based on error type or OEM-specific requirements.

[0109] At operation 808, the OTA log analysis system determines a similarity score between each received vector embedding and vector embeddings stored in a null error database. In one implementation, the system calculates a cosine similarity score between vector embeddings to measure their semantic similarity. In another implementation, the system may employ multiple similarity measures and combine their results to achieve a more robust match.

[0110] At operation 810, the OTA log analysis system identifies candidate minor errors based on a similarity score exceeding an initial threshold. In one implementation, the system selects the error message with the highest similarity score exceeding the threshold that matches any minor error in the database. In another implementation, the system may classify an error as a candidate minor error in response to finding multiple highly similar matches within the database.

[0111] At operation 812, the OTA log analysis system processes each candidate subtle error by extracting the associated error code for each subtle error, retrieving the associated error context from the vector database, and verifying whether the error type matches the database entry. In one implementation, the system compares the error code and description to ensure consistency between the candidate error and the matched subtle error. In another implementation, the system analyzes the surrounding log context to verify whether the error occurred under conditions similar to known subtle errors.

[0112] At operation 814, the OTA log analysis system filters out false positive matches based on error code consistency, context similarity, and OEM-specific rules. In one implementation, the system applies a set of validation rules that consider specific OEM implementations, plugin specifications, and test scenarios to ensure accurate classification of subtle errors. In another implementation, the system may employ a machine learning model trained on historical error classifications to identify and remove false positive matches.

[0113] At operation 816, the OTA log analysis system generates a list of confirmed minor errors. In one implementation, the system generates a report containing the identified minor errors, their matching criteria, and verification results. In another implementation, the system may include metadata about the filtered errors for statistical analysis and system improvement.

[0114] Method 800 provides efficient identification and filtering of subtle errors through vector embedding comparison and multi-stage verification. This approach is particularly useful for reducing noise in error analysis by removing known benign errors while maintaining high accuracy through context-aware verification. The combination of semantic similarity matching and rule-based verification ensures reliable identification of subtle errors across different OEM implementations and testing scenarios. The method's ability to consider OEM-specific rules and contextual information is especially effective in handling different implementations of OTA systems from different manufacturers.

[0115] refer to Figure 9 A flowchart of a method 900 for processing and contextualizing error information is shown, illustrating one implementation for retrieving and filtering error context information from a vector database. By retrieving relevant error context information while filtering out irrelevant information based on client-specific parameters, method 900 can efficiently contextualize errors identified in OTA client logs.

[0116] At operation 902, the OTA log analysis system receives an error message, which includes an error code and associated log lines that have been passed through... Figure 8 The detailed process described herein filters for removing minor errors. In one implementation, the error information includes significant error codes identified through rule-based analysis and their surrounding log context, including timestamps and relevant system messages, where minor errors have been filtered out based on a similarity comparison of vector embeddings with known minor errors. In another implementation, the error information may include additional metadata associated with each significant error occurrence, such as client version, device specifications, and configuration parameters, where the significance of the error has been verified through error code consistency checks and OEM-specific validation rules.

[0117] At operation 904, the OTA log analysis system generates vector embeddings for the error message. In one implementation, the system uses a large language model to generate vector embeddings that capture the semantic meaning of the error message and its surrounding context. In another implementation, the system may employ a specialized embedding model specifically trained for OTA update logs to better capture domain-specific error patterns and relationships.

[0118] At Operation 906, the OTA log analysis system accesses a vector database that includes error context information and associated vector embeddings. In one implementation, the vector database contains error definitions, known root causes, historical solutions, and OEM-specific information, with each entry having an associated vector embedding. In another implementation, the database includes software version dependencies and configuration requirements associated with each error type.

[0119] At operation 908, the OTA log analysis system determines a similarity score between the error vector embedding and the error context vector embedding. In one implementation, the system calculates a cosine similarity score between the vector embeddings to measure their semantic similarity. In another implementation, the system may employ multiple similarity measures and combine their results to achieve a more robust match of the error context.

[0120] At operation 910, the OTA log analysis system identifies error context information whose similarity score exceeds a threshold. In one implementation, the system selects error context entries whose similarity score to the error information exceeds a predetermined threshold. In another implementation, the system can apply multiple thresholds for different categories of errors, allowing for more granular selection based on error type or OEM-specific requirements.

[0121] At operation 912, the OTA log analysis system retrieves the context information of the identified errors, including error definitions, known root causes, historical solutions, OEM-specific information, and software version dependencies. In one implementation, the system retrieves context information for each error, including a detailed description of the error cause and potential solutions based on historical data. In another implementation, the system may retrieve OEM-specific handling guidelines and configuration requirements associated with each error type.

[0122] At operation 914, the OTA log analysis system filters out irrelevant error context information based on the OTA client version, device specifications, OEM customization, and test status. In one implementation, the system applies filtering rules that consider specific OEM implementations, client version compatibility, and device specifications to ensure that the retrieved context information is applicable to the current error scenario. For example, if the OTA client version is 2.1.0 and the error context is related to a feature introduced in version 3.0.0, the system filters out the context because it is irrelevant. Similarly, if the error context information references a 2 GB storage requirement, while the device specifications indicate only 512 MB of available storage, the system deletes the context because it is inapplicable. In another implementation, the system may employ a machine learning model trained on historical error solutions to identify and remove irrelevant context information. For example, the system may analyze patterns of successfully resolved errors across different OEM implementations to identify which error contexts are relevant to combinations of specific client versions and device configurations. The system may also consider the vehicle's test status, filtering out error contexts applicable only to production vehicles when analyzing logs from test vehicles, and vice versa. In addition, the system can evaluate OEM-specific customizations, such as proprietary plugins or modified update protocols, to ensure that error context information is consistent with the details of the current OEM's specific implementation.

[0123] At operation 916, the OTA log analysis system stores relevant error context information for subsequent analysis. In one implementation, the system saves the filtered context information in a format compatible with subsequent processing steps, including references to the original error location and the filtered context entry. In another implementation, the system may include metadata about the contextualization process, such as similarity scores used for matching and confidence levels of context relevance.

[0124] Method 900 achieves efficient retrieval and filtering of error context information through vector embedding comparison and multi-stage filtering. This approach is particularly valuable for ensuring that relevant historical solutions and OEM-specific information guide error analysis while filtering out irrelevant context information that may not be applicable to the current error scenario. The combination of semantic similarity matching and client-specific filtering ensures reliable contextualization of errors across different OEM implementations and testing scenarios. The method's ability to consider OEM-specific requirements and client configurations makes it particularly effective for handling various implementations of OTA systems from different manufacturers and for addressing the thousands of potential error codes that may arise from frequent introductions of new plugins and customizations.

[0125] refer to Figure 10 A flowchart of method 1000 for processing FAE feedback and updating the error database is shown, illustrating one implementation scheme for incorporating feedback from fault analysis engineers to improve error classification and contextual understanding. Method 1000 enables continuous improvement of error analysis through professional feedback integration and database updates.

[0126] At operation 1002, the OTA log analysis system receives FAE feedback on the analysis results via a FAE review interface. In one implementation, particularly for plug-in errors, test vehicle errors, or customization errors, this interface allows the FAE to review rule-based and AI-based analysis results and provide feedback on error classification. In another implementation, the interface enables the FAE to log new error patterns related to OEM customizations or testing scenarios, including specific information about plug-in specifications, test batch information, and associated error patterns.

[0127] At operation 1004, the OTA log analysis system identifies the feedback type as a plugin error, test vehicle error, or custom error. In one implementation, the system categorizes the feedback based on predetermined criteria and metadata associated with the error, such as a plugin-specific identifier or test batch number. In another implementation, the system may employ natural language processing to analyze the FAE's feedback description and automatically categorize the error type.

[0128] At operation 1006, for plugin errors, the OTA log analysis system extracts the plugin specification and error code from the FAE feedback. In one implementation, the system parses the feedback to identify the specific plugin identifier, version information, and the associated error code that occurred during plugin operation. In another implementation, the system can extract additional metadata about the plugin implementation and its interaction with the OTA system.

[0129] At operation 1008, to test vehicle errors, the OTA log analysis system extracts test batch information and associated error patterns. In one implementation, the system identifies the test batch number, test case identifier, and specific error patterns that occurred during the vehicle testing scenario. In another implementation, the system can extract the temporal relationship between errors and testing phases to better understand the context of test-related issues.

[0130] At operation 1010, for customized errors, the OTA log analysis system extracts OEM-specific customization details and error behaviors. In one implementation, the system identifies manufacturer-specific implementations, custom configurations, and associated error patterns arising from OEM customizations. In another implementation, the system can extract information about the interaction between customizations and standard OTA functionality.

[0131] At operation 1012, the OTA log analysis system generates vector embeddings for the extracted error information. In one implementation, the system uses a large language model to generate vector embeddings that capture the semantic meaning of the error description and its surrounding context. In another implementation, the system may employ a specialized embedding model specifically trained for OTA update logs to better capture domain-specific error patterns and relationships.

[0132] At operation 1014, the OTA log analysis system determines whether an error should be added to the minor error database based on its frequency of occurrence and FAE classification. In one implementation, the system analyzes the frequency of similar errors across multiple OTA client logs and compares it to a predetermined threshold. In another implementation, the system may consider the FAE's explicit classification of the error (significant or minor) along with supporting reasons.

[0133] At operation 1016, if an error is classified as minor, the OTA log analysis system updates the minor error database with the error vector embedding and associated metadata. In one implementation, the system adds the generated vector embedding along with contextual information, such as plugin specifications, test batch information, or OEM customization details. In another implementation, the system can update existing entries in the database with additional context or modify the similarity threshold based on the new information.

[0134] At operation 1018, if the error is classified as significant, the OTA log analysis system updates the vector database with new error context information and associated vector embeddings. In one implementation, the system adds error context information, including the error definition, known root causes, and potential solutions, along with the generated vector embeddings. In another implementation, the system can update existing entries with the additional context or create new relationships between related error patterns.

[0135] At operation 1020, the OTA log analysis system stores the updated database for subsequent analysis. In one implementation, the system maintains version control over the database to track changes and perform rollbacks when necessary. In another implementation, the system can generate analysis on database updates to monitor the evolution of error classifications and contextual information over time.

[0136] Method 1000 provides a systematic approach to incorporate expert feedback into an error analysis system, enabling continuous improvement in error classification and context understanding. This feedback loop is particularly valuable for handling the thousands of potential error codes that can arise from the diverse implementations of OTA systems by different OEMs and the frequent introduction of new plugins and customizations. This method distinguishes between plugin errors, test vehicle errors, and customization errors, while maintaining separate databases for subtle errors and error context information. This ensures the system effectively filters out known benign errors while maintaining comprehensive context for critical issues. This adaptability ensures the system remains effective even as OEM implementations evolve and new error patterns emerge.

[0137] When describing the elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to indicate the presence of one or more elements. The terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one element from another. The terms “comprising” and “having” are intended to be inclusive and indicate that there may be additional elements besides those listed. When the terms “connected to,” “coupled to,” etc., are used herein, an object (e.g., a material, element, structure, component, etc.) may be connected to or coupled to another object, regardless of whether the first object is directly connected to or coupled to the second object, or whether there are one or more intermediate objects between the first and second objects. Furthermore, references to “an embodiment” or “an embodiment” in this disclosure are not intended to exclude the existence of additional embodiments that also include the listed features.

[0138] In addition to any modifications previously indicated, those skilled in the art can devise many other variations and alternative arrangements without departing from the spirit and scope of this specification, and the appended claims are intended to cover such modifications and arrangements. Therefore, while the information has been described in particular and detail above in conjunction with what is now considered the most practical and preferred aspects, it will be apparent to those skilled in the art that various modifications can be made without departing from the principles and concepts set forth herein, including but not limited to changes in form, function, operation, and manner of use. Furthermore, as used herein, the examples and embodiments are intended in all respects to be illustrative only and should not be construed as limiting in any way.

Claims

1. A method for analyzing over-the-air (OTA) client logs from a vehicle, comprising: Receive OTA client log files, which include error logs generated during OTA software updates for the vehicle; The OTA client log files are preprocessed to generate cleaned log files by removing duplicate log lines and standardizing the log file format. The cleaned log file is divided into log blocks based on the block size parameter; The log blocks are analyzed using a rules engine to extract predefined error codes; Analyze the log block using an artificial intelligence (AI) engine using the following steps: Generate vector embeddings for error-related log lines; The vector embeddings are compared with a database of minor errors to identify and filter out minor errors; Retrieve error context information from the vector database based on the remaining errors; as well as Based on the remaining errors, the error context information, and the OTA client information, a large language model is used to generate an error summary and root cause analysis. as well as The analysis results are displayed via a Fault Analysis Engineer (FAE) review interface, and these results include rule-based error codes and AI-based root cause analysis.

2. The method of claim 1, wherein preprocessing the OTA client log file comprises: Identify the timestamp in the OTA client log file; Trim the timestamps into a standardized format; Identify the file path in the OTA client log file; as well as Trim the file path to preserve the file name.

3. The method of claim 1, wherein dividing the cleaned log file into log blocks comprises: Set block size and overlap parameters based on log file type; Identify log file boundaries and structure; Assign a unique identifier to each block; as well as Index the blocks based on the original log file location.

4. The method of claim 1, wherein analyzing the log block using the rule engine comprises: Initialize the regular expression pattern used for error code matching; Use regular expressions to scan the log blocks to see error code patterns; Identify timeout events in the log block; Verify binary file size constraints; as well as Generate rule-based error reports, which include the identified error code, error definition, associated log line, timeout event, and binary size violation.

5. The method of claim 1, wherein comparing the vector embedding with the subtle error database comprises: Access the subtle error database, which includes vector embeddings of known subtle errors; Initialize the similarity threshold to allow for subtle mismatches; Determine the similarity score between the received vector embedding and the vector embeddings in the subtle error database; Minor errors in candidates are identified based on similarity scores exceeding the aforementioned similarity threshold; as well as False alarm matches are filtered out based on error code consistency, context similarity, and original equipment manufacturer (OEM) specific rules.

6. The method of claim 1, wherein retrieving error context information from the vector database comprises: Determine the similarity score between the error vector embedding and the error context vector embedding; Error context information indicating similarity scores exceeding a threshold; Retrieve contextual information for identified errors, including error definition, known root causes, historical solutions, original equipment manufacturer (OEM) specific information, and software version dependencies; as well as Filter out irrelevant error context information based on OTA client version, device specifications, original equipment manufacturer (OEM) customization, and test status.

7. The method of claim 1, further comprising: Receive FAE feedback on the analysis results via the FAE review interface; Identify the feedback type as plugin error, test vehicle error, or custom error; Generate vector embeddings for the error information extracted from the FAE feedback; Whether the error should be added to the minor error database is determined based on its frequency of occurrence and FAE classification; as well as Based on the FAE feedback, update one or more of the minor error database and the vector database.

8. The method of claim 1, wherein the OTA client log file includes information, the information including: Client version; Device identifier; Model name; Manufacturer name; as well as Configuration parameters.

9. A system for analyzing over-the-air (OTA) client logs from a vehicle, comprising: At least one processor; as well as A non-transitory memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations, the operations including: Receive OTA client log files, which include error logs generated during OTA software updates for the vehicle; The OTA client log files are preprocessed to generate cleaned log files by removing duplicate log lines and standardizing the log file format. The cleaned log file is divided into log blocks based on the block size parameter; The log blocks are analyzed using a rules engine to extract predefined error codes; Analyze the log block using an artificial intelligence (AI) engine using the following steps: Generate vector embeddings for error-related log lines; The vector embeddings are compared with a database of minor errors to identify and filter out minor errors; Retrieve error context information from the vector database based on the remaining error; and Based on the remaining errors, the error context information, and the OTA client information, a large language model is used to generate an error summary and root cause analysis; and The analysis results are displayed via a Fault Analysis Engineer (FAE) review interface, and these results include rule-based error codes and AI-based root cause analysis.

10. The system of claim 9, wherein the non-transitory memory further stores: The subtle error database includes vector embeddings of known subtle errors and associated metadata; The vector database includes error context information, which includes error definitions, known root causes, historical solutions, and software version dependencies. as well as A web-based interface, configured as follows: Receive OTA client log file uploads; The log analysis results are displayed, including rule-based error codes and AI-based root cause analysis. as well as Feedback is captured from fault analysis engineers to update one or more of the nuanced error database and the vector database.

11. The system of claim 10, wherein comparing the vector embedding with the subtle error database to identify and filter subtle errors comprises: Access the subtle error database, which includes vector embeddings of known subtle errors; Initialize the similarity threshold to allow for subtle mismatches; Determine the similarity score between the received vector embedding and the vector embeddings in the subtle error database; Minor errors in candidates are identified based on similarity scores exceeding the aforementioned similarity threshold; Extract the associated error code and error context for each candidate subtle error; as well as False alarm matches are filtered out based on error code consistency, context similarity, and original equipment manufacturer (OEM) specific rules.

12. The system of claim 10, wherein retrieving error context information from the vector database based on residual errors comprises: Determine the similarity score between the error vector embeddings and the error context vector embeddings stored in the vector database; Identify erroneous contextual information where the similarity score exceeds the contextual similarity threshold; Retrieve contextual information for identified errors, including error definition, known root causes, historical solutions, original equipment manufacturer (OEM) specific information, and software version dependencies; as well as Filter out irrelevant error context information based on OTA client version, device specifications, original equipment manufacturer (OEM) customization, and test status.

13. The system of claim 9, wherein generating error summaries and root cause analysis using the large language model comprises: Provided for the large language model: Contextualized errors from the vector database; OTA client information, which includes client version, device specifications, and original equipment manufacturer (OEM) customization; Error definition and known root causes; as well as Generate a natural language summary describing the following: The type of error identified; The underlying root cause based on the error context; as well as Suggested solutions based on historical solution data.

Images