Method for locating source code location of software issue by means of log, and electronic device

Abstract

Disclosed in the present invention are a method for locating a source code location of a software issue by means of a log, and an electronic device. The method comprises: acquiring log output points of target software, and creating index records for the log output points; detecting source code location information of each log output point, analyzing the log content of each log output point on the basis of the source code location information, and extracting and generating a feature value corresponding to the log content; constructing a log index database; and receiving an actual running log, extracting a feature value of the log, performing matching in the log index database to determine source code locations corresponding to the log output points, and locating the specific location of a source code on the basis of a matching result. The present invention can improve the code performance and readability, does not depend on code location information outputted in a log, reduces requirements for a log format, enhances code readability, improves the integrity of the log by scanning the code and providing suggestions for perfecting the log.

Description

A method for locating software problem source code location through log, electronic device TECHNICAL FIELD

[0001] The present application relates to a method for locating software problem source code location, electronic device, and particularly relates to a method for locating software problem source code location through log, electronic device. BACKGROUND

[0002] The main problems existing in the prior art are:

[0003] I. The car machine operating system is a very complex software system:

[0004] The car machine operating system is a complex software system with a large amount of code and complex logical branches. Its complexity is not only reflected in the initial development of the software, but also in the subsequent maintenance and functional iteration of the software. The system code amount is in the order of hundreds of millions, how to ensure that these codes run in the expected way, how to identify problems or exceptions, how to accurately and quickly locate the cause of the problem and fix the problem, how to verify the problem fixing and not affect other normal functions, how to let new people quickly learn and participate in the development and operation of the system, and so on, are all complex problems.

[0005] It is a very challenging task to develop a car machine system and keep the system continuously updated to meet the rapidly changing user needs and the constantly updated hardware environment.

[0006] II. The importance of software log in system development:

[0007] In the development process, log files play an indispensable important role and can provide great help in the development and operation process, including the following aspects:

[0008] 1. Error detection and repair: When software errors or exceptions occur, logs will record detailed error information, including error type, error occurrence code location, stack trace, etc. These information helps developers quickly locate problems and fix them.

[0009] 2. Performance tuning: Log records the use of memory, CPU, etc. and the execution time of functions or code segments, helping developers identify performance bottlenecks and optimize them, or find and solve resource leaks or improper resource usage problems.

[0010] 3. Code behavior verification: Log records the value and state change of variables, helping developers understand the state change of the program during running. Through log output, developers can verify whether the code executes according to the expected logic, confirm the execution order and result of each step, and confirm whether the code executes according to the expected logic.

[0011] 4. Test support: In independent testing, logs provide a history of system operation, allowing developers to trace past events, understand the system's state at a specific point in time, and identify when and why problems occurred. During automated testing, logs record test steps and results, helping developers understand problems during testing. By comparing logs from different versions, developers can identify issues introduced in new versions and perform regression testing.

[0012] 5. Debugging assistance: During debugging, real-time log output can help developers understand the current state of the program and quickly locate problems. For remotely deployed applications, logs are an important means for developers to understand the operation of remote systems, especially when direct debugging is not possible.

[0013] 6. Knowledge accumulation: For a complex dynamic system, logs can help developers understand the actual operation of the system, record key statistics, and understand the actual operation in the real environment. This not only helps to optimize the system, but also helps to pass on experience and lessons to new members, improving the overall development and debugging efficiency of the team.

[0014] By effectively using software logs, developers can improve development efficiency, reduce debugging time, and efficiently improve software stability and performance.

[0015] Three, the association between logs and source code:

[0016] Source code is the expected design of the software system by the developer, and logs are one of the actual behaviors of the software system. By comparing the two, we can confirm whether the system behavior is executed as expected.

[0017] 1. Logs are pre-written by developers in source code, and when the program executes the corresponding logical branch code, it will output log information, which dynamically reflects the running state and real-time data of the software.

[0018] 2. Software system problems are caused by errors or imperfections in source code. To fix software problems, you need to track and locate the corresponding source code location based on the problem phenomenon, then modify the code and fix the problem. Error information in logs is one of the specific manifestations of system problems, and log information needs to be combined with source code to locate the problem cause or verify the correctness of the program.

[0019] Therefore, the log and the source code need to be combined to efficiently locate and solve problems. However, in reality, the log and the source code are difficult to effectively combine, and possible reasons include that the software user can obtain the log but usually does not have the source code; the system cannot obtain the log in real time and needs to obtain the log at a specific operation and maintenance time; and even if the source code and the log are available, professional technical personnel are still needed to analyze and locate the problem. Ultimately, the problem locating and solving are challenging and inefficient.

[0020] Four, challenges of locating software problems through logs:

[0021] In actual operation, a complex system like a car machine can be triggered by hundreds of different signals to run various logic branches and generate a large amount of logs. The problem information and corresponding source code location contained in the software system log are not obvious, and usually a technician needs multiple steps to locate the problem.

[0022] 1. Preliminary screening of logs. A complex system is developed and maintained by a large team, so the log needs to be preliminarily confirmed as being responsible for which small team or developer before further analysis and processing.

[0023] 2. Complexity of source code. The log content does not correspond to the source code one by one, and it is necessary to be very familiar with the source code or analyze and locate it in combination with the log context, compare multiple source code files, or supplement the log to reproduce the problem to achieve the purpose of locating.

[0024] 3. Logs and source code are dynamically changing. Software is iteratively developed, and the source code and log content can change. After obtaining the log, it is necessary to determine the corresponding source code version, otherwise it will mislead the analyst and cannot solve the problem or introduce new problems, further increasing the difficulty of log analysis.

[0025] 4. Multiple log types increase the difficulty of analysis. There are multiple types of logs in the system, and each type of log content format is different and needs to be viewed using different tool software, increasing the difficulty of analyzing the problem.

[0026] 5. Personnel factors. The person who writes the source code and analyzes the log may not be the same person or team, and different technical personnel may draw different conclusions due to differences in technical ability, development experience, etc., affecting problem identification.

[0027] In summary, the above factors make the analysis of logs and the location of problems challenging, difficult to fully play the important role of logs, and inefficient for development, which cannot meet people's requirements and needs to be improved. SUMMARY

[0028] The present application aims to provide a method for locating source code positions of software problems through logs, and an electronic device, to solve the problem that logs cannot be combined with source code in the prior art.

[0029] The present application provides the following solutions:

[0030] A method for locating source code positions of software problems through logs, comprising:

[0031] By scanning the source code of the target software, the log output points of the target software are obtained, and index records are created for the log output points, the index records corresponding to unique identifiers;

[0032] The source code position information of each log output point is detected, the source code position information including file names, line numbers, column numbers, function or method names, the log content of each log output point is analyzed based on the source code position information, and feature values corresponding to the log content are extracted and generated;

[0033] According to the length of the log content, the log content is directly used as a feature value, or: a key part of the log content is extracted, and a feature value corresponding to the key part is calculated;

[0034] The output frequency and log level of each log output point are determined, a log index library is constructed, and the index records are stored;

[0035] Actual running logs are received, the feature values of the logs are extracted, and matching is performed in the log index library, to determine the source code positions corresponding to the log output points, and according to the matching result, the specific positions of the source code are located;

[0036] If the matching fails, the logs are recorded and the user is prompted to perform manual analysis or add them to the log index library, and the actual running logs are used to verify and optimize the log index library.

[0037] Further, the method for locating source code positions of software problems through logs further comprises:

[0038] The pre-processing function of a compiler or an interpreter is used to perform syntax analysis on the source code of the target software, and all log output functions or method calls in the source code are identified;

[0039] The position information of each identified log output point in the source code is extracted, the position information specifically being: file names, line numbers, or optional column numbers;

[0040] generating at least one index record for each log output point, the index record comprising: a unique identifier associated with the log output point, the unique identifier corresponding to location information in the source code;

[0041] establishing an index database, and storing all generated index records into the index database, and establishing a mapping relationship between the index records and data retrieval in the index database.

[0042] Further, in the process of identifying all log output functions or method calls in the source code, further comprising:

[0043] performing context analysis on the code near the log output point, extracting key variable names, function names or class names of the code, and adding the key variable names, function names or class names as context keywords to the index record;

[0044] analyzing the log formatting string at the log output point based on a regular expression matching method to obtain parameter placeholders of the log formatting string;

[0045] obtaining the data type of the log content according to the type of the parameter placeholder in the log formatting string, and adding information of the data type of the log content to the index record.

[0046] Further, the source code location information of each log output point includes a file name, a line number, a column number and a function or method name, the log content of each log output point is analyzed based on the source code location information, a feature value corresponding to the log content is extracted and generated, and further comprising:

[0047] traversing the source code of the target software based on a source code parser, identifying all log output statements and their location information in the source code, and extracting the file name, line number, column number and function or method name in the location information;

[0048] constructing a feature vector of the log content according to the log output statement, in combination with the location information of the log output statement and the context analysis result, the feature vector comprising: key words, data types, formatting patterns or context keywords;

[0049] associating the generated feature value with the source code location information of the corresponding log output point, and storing in the index database for subsequent log matching and source code positioning.

[0050] Further, according to the length of the log content, directly using the log content as the feature value, or: extracting a key part of the log content and calculating a feature value corresponding to the key part, further comprising:

[0051] According to the length of the log content, the length of the log content generated by each log output point is obtained, and for log content with a length less than or equal to a predetermined threshold, the entire log content is directly used as a feature value, or:

[0052] For log content with a length greater than the predetermined threshold, a key part extraction operation is performed to generate an overall feature value; a feature value calculation algorithm is applied to the extracted key part to generate a corresponding compact feature value;

[0053] The overall feature value or the compact feature value is mapped and associated with the source code location information of the log output point, and is stored in an index database.

[0054] Further, the actual running log is received, the feature value of the log is extracted, and matching is performed in the log index library to determine the source code location corresponding to the log output point, according to the matching result, the specific location of the source code is located, and further comprising:

[0055] In the actual running environment of the software system, real-time log capturing operation is performed, and generated log data is received in real time, and preprocessing operation is performed on the captured log data, and the preprocessing operation is specifically: removing noise, formatting operation or standardization operation;

[0056] Feature extraction operation is performed using a natural language processing algorithm to extract key feature values from the preprocessed log data, and the key feature values are specifically: word frequency, key phrase or language pattern;

[0057] A log matching module is constructed to perform similarity evaluation and analysis on the extracted key feature values and the index records in the log index library;

[0058] According to the similarity evaluation and analysis result, the index record with the highest matching degree is selected to determine the source code location with the highest matching degree.

[0059] A system for locating the source code location of a software problem through logs, for implementing the method for locating the source code location of a software problem through logs, comprising:

[0060] A target software log output point module: by scanning the source code of the target software, the log output points of the target software are obtained, and index records are created for the log output points, and the index records correspond to unique identifiers;

[0061] A log content corresponding feature value providing module detects the source code location information of each log output point, the source code location information includes file name, line number, column number and function or method name, analyzes the log content of each log output point based on the source code location information, and extracts and generates feature values corresponding to the log content;

[0062] The key part corresponding feature value calculation module directly uses the log content as the feature value according to the length of the log content, or extracts the key part of the log content and calculates the feature value corresponding to the key part;

[0063] The log index library construction module determines the output frequency and log level of each log output point, constructs a log index library, and stores the index record;

[0064] The source code specific position positioning module receives the actual running log, extracts the feature value of the log, and matches in the log index library to determine the source code position corresponding to the log output point, and according to the matching result, the specific position of the source code is located.

[0065] The log index library verification and optimization module records the log and prompts the user to manually analyze or add to the log index library if the matching fails, and verifies and optimizes the log index library by using the actual running log.

[0066] An intelligent cockpit, wherein the intelligent cockpit is provided with the system for locating the source code position of software problems through logs.

[0067] An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method.

[0068] A computer readable storage medium stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method.

[0069] Compared with the prior art, the present application has the following advantages:

[0070] The present application can improve the accuracy and efficiency of software problem source code positioning, reduce the technical requirements of problem positioning, and can perform efficient automatic log analysis, import the obtained logs, automatically analyze and extract the features of each line of log, and compare with the index, and quickly locate the source code position. A large amount of manual analysis time is saved, and errors can be reduced.

[0071] The application can identify multiple similar matching source code positions, accurately identify problems and less missed. Since the log content is spliced according to the actual running of the software, there is no completely matching string in the source code. The software can automatically match and provide unique matching or multiple matching recommendations for selection, which can significantly improve accuracy and positioning efficiency. Real-time log output analysis can be performed to identify abnormal problems and locate the source code position during software running.

[0072] The application reduces the technical requirements for problem analysis personnel. Even if the developer is not very familiar with the system, the application can quickly and accurately locate the source code position corresponding to the problem, reduce the research scope, help to quickly and effectively solve the problem, and is independent of the source code. The log index information is fully utilized, and the corresponding file and position of the source code can be located without the source code. To some extent, the safety of the development process is improved, the security measures of the source code are enhanced, and the application range is wide. It is independent of the content format of the log, and can be compatible with various commonly used logs, such as Android logcat, Linux slog, DLT-log, etc.

[0073] In summary, the application is beneficial to improve the code performance and readability, is independent of the code position information output in the log, reduces the requirement for the log format, and at the same time, due to the simplification of the log output content, is beneficial to improve the efficiency of program running, enhance the code readability, and through scanning the code and providing suggestions for improving the log, including removing redundant and repeated logs, supplementing necessary logs in the key logic branch, and improving the log content, the integrity of the log is improved. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the specific embodiments of the application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0075] Fig. 1 is a flowchart of a method for locating software problem source code position through a log.

[0076] Fig. 1A is a method flowchart of an optimized technical solution of step S1.

[0077] Fig. 1B is a method flowchart of an optimized technical solution of step S11.

[0078] Fig. 1C is a method flowchart of an optimized technical solution of step S2.

[0079] Figure 1D is a method flow chart of the optimization technical solution of step S3.

[0080] Figure 1E is a method flow chart of the optimization technical solution of step S5.

[0081] Figure 2 is an architecture diagram of a system for locating a source code location of a software problem through logs.

[0082] Figure 3 is a flow chart of real-time log analysis.

[0083] Figure 4 is a flow chart of locating a source code location through a log index database.

[0084] Figure 5 is a structural schematic diagram of an electronic device. DETAILED DESCRIPTION

[0085] The technical solutions of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0086] The method for locating a source code location of a software problem through logs shown in Figure 1 comprises:

[0087] Step S1: obtaining log output points of a target software by scanning source code of the target software, creating index records for the log output points, and the index records corresponding to unique identifiers;

[0088] Step S2: detecting source code location information of each log output point, the source code location information including a file name, a line number, a column number, a function or method name, analyzing log content of each log output point based on the source code location information, extracting and generating feature values corresponding to the log content;

[0089] Step S3: directly using log content as a feature value according to a length of the log content, or: extracting a key part of the log content and calculating a feature value corresponding to the key part;

[0090] In step S3, it is worth noting that the length of the log content is relatively long or relatively short, and the key part of the log content can be a relatively long part or a relatively short part. In the present application and examples, the key part of the log content refers to the part of the log content that is most representative and distinguishable, and the key part of the log content can effectively identify and distinguish different log entries, for example:

[0091] Error codes or status information: Error codes or status information typically indicate the nature of the problem and are a key part of log analysis.

[0092] Timestamps: Timestamps of logs provide the exact time of event occurrence, which helps trace the timing of problem occurrence.

[0093] Event level: Levels such as INFO, DEBUG, WARNING, ERROR, etc., indicate the importance and urgency of the log message.

[0094] Key variables or parameter values: In log messages, the values of certain variables or parameters may be crucial to understanding the problem.

[0095] Operation or transaction ID: In cases involving multiple steps or distributed systems, operation or transaction IDs help correlate a series of related log entries.

[0096] Exception stack trace: In exception logs, stack traces provide detailed information about where the problem occurred.

[0097] Service or component name: Indicates which service or component in the system the log message originated from.

[0098] Key operation description: Describes the operation being performed, such as "user login failed," "database connection failed," etc.

[0099] Step S4, determine the output frequency and log level of each log output point, build a log index library, store the index records, and use them to predict the repeatability of logs and distinguish different importance of log information;

[0100] Step S5, receive actual running logs, extract feature values of logs, and match in the log index library to determine the source code location corresponding to the log output point, and locate the specific location of the source code according to the matching result;

[0101] Step S6, if the matching fails, record the log and prompt the user for manual analysis or add to the log index library, use the actual running logs to verify and optimize the log index library, to improve the accuracy of log matching and the efficiency of locating the source code location.

[0102] The technical solutions provided by steps S1 to S6 improve the management and problem positioning efficiency of log output points in software systems through automated log analysis and index construction methods. Based on the scanning results of the target software source code, the detailed information of the log output point is identified and recorded, including its source code location information, log content features, output frequency, and log level, and an efficient log index library is constructed.

[0103] The technical solutions provided by steps S1 to S6 can construct log output point identification and index records, obtain log output points by scanning source code, and create unique index records for each output point, ensuring the accuracy of subsequent analysis. By detecting source code location information, the source code location information of each log output point is obtained, including file name, line number, column number, and function name, providing a basis for subsequent log content analysis. Then, according to the length of the log content, a feature value is generated directly or by extracting its key part, ensuring that the extracted information is most representative and discriminative, determining the output frequency and importance of each log output point, and constructing a log index library for predicting log repetitiveness and distinguishing different importance of log information.

[0104] The technical solutions provided by steps S1 to S6 match real-time running logs in actual operation, receive actual running log data, extract feature values and match them with the log index library, quickly locate the specific location of the source code, record the log and prompt the user for manual analysis if the matching fails, and at the same time, use the actual running log to verify and optimize the index library, improving the accuracy of future matching.

[0105] In summary, the technical solutions provided by steps S1 to S6 improve the efficiency of log management, reduce manual intervention, and improve the efficiency of log management. By constructing a log index library, actual running logs can be quickly matched with source code locations, significantly shortening the problem positioning time. By extracting the key part of the log content and generating a feature value, log analysis becomes more accurate, effectively distinguishing different log entries. Through verification and optimization of actual running logs, the accuracy of log matching is continuously improved, ensuring that the system can still run efficiently in a constantly changing environment. At the same time, when matching fails, the technical solutions provided by steps S1 to S6 provide a prompt for manual analysis, enhancing the usability and user experience of the system, significantly improving the efficiency and accuracy of the software development and maintenance process.

[0106] As shown in FIG. 1A, in step S1, the log output points of the target software are obtained by scanning the source code of the target software, and index records are created for the log output points. The index records correspond to unique identifiers and further include:

[0107] Step S11, using the pre-processing function of the compiler or interpreter, performing syntax analysis on the source code of the target software to identify all log output functions or method calls in the source code;

[0108] Step S12, for each identified log output point, extracting its location information in the source code, the location information specifically including: file name, line number, or optional column number;

[0109] Step S13, at least one index record is generated for each log output point, which is specifically: a unique identifier associated with the log output point, which corresponds to the location information in the source code;

[0110] Step S14, an index database is established, and all generated index records are stored in the index database, and a mapping relationship between the index records and data retrieval is established in the index database.

[0111] The optimization technical solution provided by steps S11 to S14 improves the systematicness of log management, ensures comprehensive identification and management of log output points through automatic scanning and analysis of source code, avoids possible omissions or errors of manual identification, and creates index records for each log output point, which includes detailed source code location information and unique identifiers, so that the specific location of the source code can be quickly and accurately located when a problem occurs. Through the establishment of an index database and a mapping relationship, the retrieval efficiency of log data is optimized, so that it is easier to quickly find related logs in a large amount of log data.

[0112] The optimization technical solution provided by steps S11 to S14 improves the development and maintenance efficiency, reduces the workload of development and maintenance personnel in problem diagnosis and repair through automatic log output point identification and index record generation, improves the overall development and maintenance efficiency, provides good scalability and flexibility through the establishment of an index database, facilitates the addition of new log output points or modification of existing records in the future, and ensures the accuracy and integrity of the log output points through an automatic method, thereby improving the reliability of system operation.

[0113] As shown in FIG. 1B, in the process of step S11 involving identification of all log output functions or method calls in the source code, the process further includes:

[0114] Step S111, context analysis is performed on the code near the log output point, key variable names, function names or class names of the code are extracted, and the key variable names, function names or class names are added to the index record as context keywords;

[0115] Step S112, the log formatting string at the log output point is analyzed based on a regular expression matching method, parameter placeholders of the log formatting string are obtained, and possible formats of log content are inferred;

[0116] Step S113, the data type of the log content is obtained according to the type of the parameter placeholder in the log formatting string, and information of the data type of the log content is added to the index record.

[0117] The optimization technical solution provided by steps S111 to S113 enriches the content of the index record and improves the accuracy and efficiency of log analysis by deeply analyzing the code context and log formatting string of the log output point, extracting key information, and integrating it into the index record.

[0118] By extracting the key variable name, function name, or class name as the context keyword, the semantic information of the log content is increased, making the log message more easily understood and analyzed, and the understandability of the log content is enhanced; by analyzing the log formatting string and parameter placeholder, the format and possible values of the log content can be more accurately predicted, which helps to quickly identify and handle problems in actual log analysis, and improves the accuracy of log analysis; adding the code context keyword and data type information of the log content to the index record makes each index record contain more comprehensive information, providing more clues for subsequent log matching and problem positioning, enriching the information of the index record, and optimizing the problem positioning process. The rich index record information when the problem occurs can help to quickly locate the problem source code, reduce the problem diagnosis time, and improve the development and maintenance efficiency; the detailed information contained in the index record can be used as a retrieval keyword to improve the accuracy and efficiency of log data retrieval, improve the retrieval efficiency of log data, and make the vehicle-mounted system better adapt to complex log scenarios and variable log formats, improving the adaptability and flexibility of the system.

[0119] As shown in FIG. 1C, in step S2, the source code location information of each log output point is detected, including the file name, line number, column number, and function or method name, the log content of each log output point is analyzed based on the source code location information, and the feature value corresponding to the log content is extracted and generated, further comprising:

[0120] Step S21: based on the source code parser, the source code of the target software is traversed, all log output statements and their location information in the source code are identified, and the file name, line number, column number, and function or method name in the location information are extracted;

[0121] Step S22: according to the log output statement, and in combination with the location information and context analysis result of the log output statement, a feature vector of the log content is constructed, and the feature vector is specifically: keyword, data type, formatting mode, or context keyword;

[0122] Step S23: the generated feature value is associated with the source code location information of the corresponding log output point, and is stored in the index database for subsequent log matching and source code positioning.

[0123] The optimization technical solution provided by steps S21 to S23 utilizes automatic source code analysis technology to accurately identify and extract the location information of the log output point, analyzes the log content based on the location information of the log output point, generates a representative feature vector that can capture the key features of the log output and associate these features with the specific location of the source code, and stores the feature vector in an index database to facilitate subsequent log matching and source code positioning, thereby achieving accurate source code positioning, efficient log content analysis, enhanced log matching capability, optimized log processing flow, improving problem solving efficiency, and improving system reliability and maintainability. By automatically analyzing the source code, extracting the location information of the log output point, analyzing the log content, and generating a feature vector, efficient log management and rapid problem positioning are achieved, thereby significantly improving the efficiency and quality of software development, testing, and maintenance.

[0124] The optimization technical solution provided by steps S21 to S23 can be further expanded, for example, using some algorithms of natural language processing technology (bag-of-words model or TF-IDF algorithm) to perform vectorization processing on the log message text to extract text features.

[0125] The extracted text features are subjected to dimension reduction and normalization processing to reduce the dimension of the feature space and improve the comparability of the features.

[0126] A hash algorithm such as MD5 or SHA series is used to perform hash processing on the processed log content feature vector to generate a unique feature value.

[0127] The expanded technical solution based on steps S21 to S23 uses natural language processing (NLP) technology and data processing algorithms to perform in-depth analysis and processing on the log message text, which can extract more accurate and efficient text features and generate unique feature values for subsequent processing and matching.

[0128] The extended technical solution based on steps S21 to S23 further enhances the content of text feature extraction by using NLP techniques such as bag-of-words model or TF-IDF algorithm, which can more effectively convert log message text into vector form and extract key features of the text such as word frequency, keywords, etc. Through dimension reduction and normalization processing, the dimension of the feature vector is reduced, the comparability and processing efficiency of the features are improved, the feature space complexity is reduced, and the dimension disaster is avoided. The feature vector is processed using a hash algorithm to generate a unique feature value, ensuring that each log entry has a stable and unique identifier, improving the accuracy and efficiency of log matching. The generated unique feature value can be directly used for log matching and querying, improving the speed and accuracy of log matching. Finally, by extracting more accurate text features, the log content can be more accurately analyzed and understood, improving the quality of log analysis.

[0129] In summary, the extended technical solution based on steps S21 to S23 uses natural language processing techniques and data processing algorithms for vectorization, dimension reduction, normalization, and hashing of log message text, which not only improves the uniqueness and stability of log features, but also optimizes the performance of log matching and improves the scalability of the entire system and the quality of log analysis.

[0130] As shown in FIG. 1D, in step S3, according to the length of the log content, the log content is directly used as a feature value, or: the key part of the log content is extracted, and the feature value corresponding to the key part is calculated, further comprising:

[0131] Step S31: evaluating the log content according to its length to obtain the length of the log content generated by each log output point. For log content with a length less than or equal to a predetermined threshold, the entire log content is directly used as a feature value, or:

[0132] Step S32: for log content with a length exceeding the predetermined threshold, performing a key part extraction operation to generate an overall feature value; applying a feature value calculation algorithm to the extracted key part to generate a corresponding compact feature value; for example, the feature value calculation algorithm can be a hash function or information entropy calculation;

[0133] Step S33: mapping and associating the overall feature value or compact feature value with the source code location information of the log output point pair, and storing it in the index database for subsequent log matching and source code positioning.

[0134] As shown in FIG. 1E, in step S5, the actual running log is received, the feature value of the log is extracted, and matching is performed in the log index library to determine the source code location corresponding to the log output point. According to the matching result, the specific location of the source code is located, further comprising:

[0135] Step S51, real-time log capture operation is performed in the actual running environment of the software system, real-time log data is received, and preprocessing operation is performed on the captured log data. The preprocessing operation specifically includes removing noise, formatting operation or standardization operation. The function of step S1 is to ensure the consistency and analyzability of the log data.

[0136] Step S52, feature extraction operation is performed using a natural language processing algorithm to extract key feature values from the preprocessed log data. The key feature values specifically include word frequency, key phrase or language pattern.

[0137] Step S53, a log matching module is constructed to perform similarity evaluation analysis on the extracted key feature values and the index records in the log index library.

[0138] Step S54, according to the similarity evaluation analysis result, the index record with the highest matching degree is selected to determine the source code position with the highest matching degree.

[0139] The optimization technical solution provided by steps S51 to S54 realizes real-time log capture in the actual running environment of the software system, and performs preprocessing and feature extraction on the log. Then, through the constructed log matching module, similarity evaluation analysis is performed on the extracted key feature values and the index records in the log index library, and finally the index record with the highest matching degree is selected to determine the specific position of the source code.

[0140] The optimization technical solution provided by steps S51 to S54 realizes real-time log capture in the actual running environment of the software system, and performs preprocessing and feature extraction on the log. Then, through the constructed log matching module, similarity evaluation analysis is performed on the extracted key feature values and the index records in the log index library, and finally the index record with the highest matching degree is selected to determine the specific position of the source code.

[0141] The system for locating the source code position of software problems through logs as shown in FIG. 2 is used to implement the method for locating the source code position of software problems through logs, and includes:

[0142] Target software log output point module: by scanning the source code of the target software, obtaining the log output point of the target software, creating an index record for the log output point, the index record corresponding to a unique identifier;

[0143] Log content corresponding feature value providing module, detecting the source code location information of each log output point, the source code location information including file name, line number, column number and function or method name, analyzing the log content of each log output point based on the source code location information, extracting and generating the feature value corresponding to the log content;

[0144] Key part corresponding feature value calculation module, according to the length of the log content, directly using the log content as the feature value, or: extracting the key part of the log content and calculating the feature value corresponding to the key part;

[0145] Log index library construction module, determining the output frequency and log level of each log output point, constructing a log index library, and storing the index record;

[0146] Source code specific location positioning module, receiving actual running logs, extracting the feature value of the log, and matching in the log index library to determine the source code location corresponding to the log output point, and positioning to the specific location of the source code according to the matching result;

[0147] Log index library verification and optimization module, if the matching fails, record the log and prompt the user to manually analyze or add to the log index library, and use the actual running log to verify and optimize the log index library.

[0148] The embodiments of the system described above are only illustrative, for example: wherein each functional module, unit or subsystem in the system can or can not be physically separated, or can or can not be a physical unit, i.e. can be located in the same place or distributed to multiple different systems and their subsystems or modules. Those skilled in the art can select part or all of the functional modules, units or subsystems to achieve the purpose of the embodiments of the present application according to actual needs, and those skilled in the art can understand and implement without creative labor.

[0149] The embodiments of the present application in the specific application scenario are shown in FIGS. 3-4, specifically:

[0150] 1. Create a log index library:

[0151] Scan source code, create index database for all log output. Program log is output through a specific function or method, so all log output code can be identified. In the log index database, each record corresponds to a log output, including the following information:

[0152] 1) Create a unique ID for each log.

[0153] 2) Log corresponding source code location, including file name, line number, column number, etc.

[0154] 3) Log itself index feature value.

[0155] a) For shorter (fixed part length less than or equal to 64 bytes) log content, directly use the fixed information in the log content as the key.

[0156] b) For longer log content, save the first 64 bytes key and save the complete feature information by extracting feature value information (such as extracting md5 value), thereby shortening the amount of data storage.

[0157] 4) Log output frequency, combined with source code, can infer whether the log will be repeated output, for example, in similar for loop code structure, the log will be repeated output.

[0158] 5) Log level, when analyzing the log, high priority or serious log can be processed first to improve processing efficiency. The way to identify log level:

[0159] a) There are usually specific log level keywords in log output methods, such as error, warning, info, debug, etc. These keywords can be used to identify and label log level.

[0160] b) Key system call execution related logs, such as memory allocation, file creation, etc. Judge operation result from record log, this kind of log should be set as high priority.

[0161] c) Key operation related logs on main flow, such as database connection, closing, data query operation related result output log, should be set as high priority.

[0162] 2. Based on log index to quickly locate source code location:

[0163] Import the actual generated log, analyze the log line by line through the above method and extract the feature value. Use the log feature to quickly retrieve the corresponding index item in the index database, and locate and jump to the corresponding source code location through the source code location information in the index.

[0164] If the corresponding index is not found, log the information of such retrieval failure, prompt the user to manually identify and add to the index library; or for unimportant logs, choose to ignore.

[0165] 3. Verify and optimize the index database using actual generated logs:

[0166] 1) Optimize the index using test process logs: Verify the index database using actual logs, which are closer to the actual situation in the generated environment. Before the system is completed and delivered, actual logs cannot be obtained, so logs obtained in debugging, trial running, testing, etc. environment can be used for verification. These logs are close to the actual generated environment logs, and the detection can accurately locate the source code position of the output log through a certain log.

[0167] 2) Re-learn and optimize based on logs in the actual generated environment: Collect relevant logs when the system runs for a period of time or after problems are found, and locate business problems while checking whether the index database has missed or misjudged log data. Based on this, the log index is perfected and optimized, which can further improve the accuracy of problem positioning.

[0168] 4. Optimize the log index through manual calibration

[0169] On the basis of automatic scanning and optimization, manual adjustment is made according to actual needs, such as adjusting the priority level of the log, the log feature keyword, the context environment range, etc., which can optimize the efficiency of problem identification and positioning.

[0170] 5. Log context information:

[0171] Log context information includes the features of adjacent logs, etc., which are used to identify and locate repeated or close log content. In order to improve the accuracy of identification, multiple logs adjacent to the context information can be included. Since this method will affect the efficiency, different strategies need to be adopted according to actual needs, such as only for a small number of key problem logs, in order to improve accuracy, set a larger context association context.

[0172] 6. Interface operation design:

[0173] 1) The left side of the interface is the log problem list window, which displays the problem list in the log after the software is imported. The log can be classified according to the severity of the problem.

[0174] 2) The middle is the log window, which displays the imported log content. By clicking the problem list, the middle log window automatically jumps to the log corresponding to the problem and highlights it.

[0175] 3) Right side is the code window, which displays the code corresponding to the log content. When browsing the log content in the log window, the mouse-selected log is searched and positioned according to the index. At the same time, the source code can also be browsed, and when the mouse selects a line of log-related code, the middle log window will also be automatically positioned to the related log. If this line of code runs multiple times and there are multiple corresponding logs, a list dialog box is popped up to display the corresponding log list, and after selecting a log item, the corresponding log position is jumped to.

[0176] 7. Compatible with multiple formats of logs:

[0177] Read and analyze multiple common types of log content, compatible with Android logcat, Linux Slog, DLT-log and other log formats.

[0178] 8. Real-time dynamic analysis of logs and identification of problems:

[0179] By dynamically monitoring and reading files or redirecting the log output of the software system, the log analysis software can dynamically monitor the log output in the system running environment, and can analyze the log in real time online. If a problem is identified, the corresponding source code position of the problem can be output in time, significantly shortening the problem positioning and repair time.

[0180] 9. Suggesting log optimization: The log analysis system analyzes the source code log and suggests improving the log, including:

[0181] 2) If necessary information such as version is not detected, prompt to add version and other related necessities;

[0182] 3) Prompt possible redundant logs or repeated logs to improve log analysis efficiency and program running efficiency;

[0183] Supplement and improve key link logs, such as detecting that the main logic link is missing logs or the existing logs are missing important variable outputs, prompting for improvement suggestions.

[0184] As shown in FIG. 5, the present application provides an intelligent cockpit, an electronic device and a storage medium corresponding to the method of locating the source code position of software problems through logs:

[0185] An intelligent cockpit, wherein the intelligent cockpit is provided with the system for locating the source code position of software problems through logs.

[0186] An electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory stores a computer program, when the computer program is executed by the processor, the processor executes the steps of the method.

[0187] A computer readable storage medium stores a computer program executable by an electronic device, when the computer program runs on the electronic device, the electronic device executes the steps of the method.

[0188] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of the method. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM has various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0189] The technical features of the above embodiments can be combined in any way. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered within the scope of the present disclosure.

[0190] In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means within the scope of the present application and forms different embodiments. For example: any one of the embodiments claimed in the claims can be used in any combination of the embodiments of the present application.

[0191] In addition, the technical solutions among various embodiments of the present application can be combined with each other, but must be based on the fact that a person of ordinary skill in the art can realize the combination, and when the combination of the technical solutions is contradictory or unachievable, it should be considered that the combination of the technical solutions does not exist and is not within the protection scope of the present application.

[0192] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for locating a source code location of a software problem by a log, characterized by, The method comprises the following steps: By scanning the source code of the target software, the log output points of the target software are obtained, and index records corresponding to unique identifiers are created for the log output points; The source code location information of each log output point is detected, including the file name, line number, column number, function or method name, the log content of each log output point is analyzed based on the source code location information, the feature values corresponding to the log content are extracted and generated; According to the length of the log content, the log content is directly used as the feature value, or: the key part of the log content is extracted, and the feature value corresponding to the key part is calculated; The output frequency and log level of each log output point are determined, a log index library is constructed, and the index records are stored; The actual running log is received, the feature value of the log is extracted, and matching is performed in the log index library to determine the source code location corresponding to the log output point, and the specific location of the source code is located according to the matching result; If the matching fails, the log is recorded and the user is prompted to manually analyze or add to the log index library, and the actual running log is used to verify and optimize the log index library.

2. The method of claim 1, wherein, The method further comprises the following steps: By using the pre-processing function of the compiler or interpreter, the source code of the target software is syntax analyzed, and all log output functions or method calls in the source code are identified; For each identified log output point, its location information in the source code is extracted, including the file name, line number or optional column number; At least one index record is generated for each log output point, including a unique identifier associated with the log output point, which corresponds to the location information in the source code; An index database is established, and all generated index records are stored in the index database, and the mapping relationship between the index records and data retrieval is established in the index database.

3. The method of claim 2, wherein, In the process of identifying all log output functions or method calls in the source code, the method further comprises the following steps: Context analysis is performed on the code near the log output point, the key variable name, function name or class name of the code is extracted, and the key variable name, function name or class name is added to the index record as a context keyword; The log formatting string at the log output point is analyzed based on the regular expression matching method to obtain the parameter placeholder of the log formatting string; The data type of the log content is obtained according to the type of the parameter placeholder in the log formatting string, and the information of the data type of the log content is added to the index record.

4. The method of claim 1, wherein, The method further comprises the following steps: The source code parser traverses source code of the target software to identify all log output statements and their position information in the source code, and extract file names, line numbers, column numbers and function or method names in the position information; According to the log output statement and in combination with the position information and context analysis result of the log output statement, a feature vector of the log content is constructed, and the feature vector is specifically a keyword, a data type, a formatting mode or a context keyword; The generated feature value is associated with the source code position information of the corresponding log output point and stored in an index database for subsequent log matching and source code positioning.

5. The method of claim 1, wherein, According to the length of the log content, the log content is directly used as a feature value, or: the key part of the log content is extracted, and the feature value corresponding to the key part is calculated, further comprising: According to the length of the log content, the length of the log content generated by each log output point is obtained, and for log content with a length less than or equal to a predetermined threshold, the entire log content is directly used as a feature value, or: For log content with a length greater than the predetermined threshold, a key part extraction operation is performed to generate an overall feature value; and a feature value calculation algorithm is applied to the extracted key part to generate a corresponding compact feature value; The overall feature value or the compact feature value is mapped and associated with the source code position information of the corresponding log output point and stored in the index database.

6. The method of claim 1, wherein, The actual running log is received, the feature value of the log is extracted, and matching is performed in the log index library to determine the source code position corresponding to the log output point, and according to the matching result, the specific position of the source code is located, further comprising: Real-time log capturing is performed in the actual running environment of the software system, and generated log data is received in real time, and preprocessing is performed on the captured log data, and the preprocessing is specifically noise removal, formatting operation or standardization operation; A feature extraction operation is performed using a natural language processing algorithm to extract key feature values from the preprocessed log data, and the key feature values are specifically word frequency, key phrase or language pattern; A log matching module is constructed to evaluate and analyze the similarity of the extracted key feature values and the index records in the log index library; According to the similarity evaluation and analysis result, the index record with the highest matching degree is selected to determine the source code position with the highest matching degree.

7. A system for locating a source code location of a software problem by logs, for implementing the method for locating a source code location of a software problem by logs according to any one of claims 1 to 6, characterized in that, It includes: The target software log output point module: by scanning the source code of the target software, the log output point of the target software is obtained, and an index record is created for the log output point, and the index record corresponds to a unique identifier; The log content corresponding feature value providing module detects the source code position information of each log output point, the source code position information includes file name, line number, column number and function or method name, analyzes the log content of each log output point based on the source code position information, and extracts and generates the feature value corresponding to the log content; The key part corresponding feature value calculation module directly uses the log content as the feature value according to the length of the log content, or extracts the key part of the log content and calculates the feature value corresponding to the key part; The log index library construction module determines the output frequency and log level of each log output point, constructs a log index library, and stores the index record; The source code specific position positioning module receives the actual running log, extracts the feature value of the log, and matches in the log index library to determine the source code position corresponding to the log output point, and according to the matching result, the specific position of the source code is located; The log index library verification and optimization module records the log and prompts the user to manually analyze or add to the log index library if the matching fails, and verifies and optimizes the log index library by using the actual running log.

8. An intelligent cabin, characterized in that, The intelligent cockpit is provided with the system for locating the source code position of software problems through logs according to claim 7.

9. An electronic device, comprising: It comprises: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, It stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method in any one of claims 1 to 6.

Images