Log processing method and device, equipment and storage medium

Abstract

The invention discloses a log processing method and device, equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: obtaining a to-be-compiled source code compiled for embedded equipment on a host machine, and traversing the to-be-compiled source code to recognize a Log statement template to obtain an original Log statement template; generating a unique identifier for the original Log statement template; identifying a formatted character string in the original Log statement template to obtain a target formatted character string, and deleting a non-conversion description character sequence in the target formatted character string to obtain a compressed formatted character string; splicing the unique identifier and the compressed formatted character string to obtain a compressed Log statement template, and replacing an original Log statement template in the to-be-compiled source code with the compressed Log statement template to obtain a new code; and compiling the new code to obtain a mirror image file, and running the mirror image file on the target machine to generate compressed Log information. According to the method, the log compression process is simplified, the higher compression ratio is achieved, and then occupation of memory space resources is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a log processing method, apparatus, device, and storage medium. Background Art

[0002] Log (log) is usually used for troubleshooting, system monitoring, and debugging. Compressing Log information can significantly reduce the demand for storage space resources, so that more storage space can be allocated for other more critical services in the system. At the same time, when the compressed Log is transmitted over the network, the amount of data transmitted can be reduced, saving network resources. For example, during the operation of embedded software with limited storage space resources, the resources can be effectively managed by compressing Log information.

[0003] Currently, the mainstream compression methods include: statistic-based compression methods, prediction-based compression methods, dictionary-based compression methods, embedded database compression methods, and custom compression algorithm-based compression methods. Among them, the statistic-based compression method uses the statistical characteristics of data to compress Log information. For example, techniques such as Huffman Coding or Arithmetic Encoding are used to encode information according to the frequency of occurrence in the data, thereby reducing the required storage space. The prediction-based compression method uses the predictability of data to compress information. For example, the Differential Logging technique is used to record the information that has changed compared to the previous log, thereby reducing the required storage space. The dictionary-based compression method uses a "dictionary" to map the repeated patterns in the data, such as dictionary-based compression algorithms like LZ77 and LZ78. The embedded database compression method uses the built-in compression function provided by the embedded database for information compression, such as information compression through compression options such as Quicklz, Squicklz, and the nostalgic "LZ Only" provided by SQLite. The custom compression algorithm-based compression method is mainly applied to the compression of specific application scenarios or specific data structures. For example, a compression algorithm for specific hardware can better utilize the hardware characteristics, thereby improving the compression efficiency.

[0004] However, when compressing Log information by the above compression methods, complex compression algorithms are involved, which are relatively complex to implement. Moreover, when the non-conversion description part in the Log is more detailed, the compression ratio will be lower. In addition, since the above compression methods all compress the Log information generated after operation, they will occupy the computing power resources of the target machine, such as CPU (Central Processing Unit) resources.

[0005] In summary, how to solve the above problems is still an issue to be further addressed in this field. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a log processing method, device, equipment and storage medium, which simplifies the log compression process, has a higher compression ratio, and thus reduces the demand for storage space resources. More storage space can be allocated for other more critical services in the system. At the same time, when the compressed log is transmitted through the network, the amount of data transmitted can be reduced, thereby saving network resources. The specific solutions are as follows:

[0007] In the first aspect, this application discloses a log processing method, which is applied to a host computer and includes:

[0008] Obtain the to-be-compiled source code written for an embedded device on the host computer, and traverse the to-be-compiled source code to identify the Log statement template in the to-be-compiled source code to obtain the original Log statement template;

[0009] Generate a unique identifier for the original Log statement template, and establish a mapping relationship between the unique identifier and the original Log statement template;

[0010] Identify the formatted string in the original Log statement template to obtain the target formatted string, and delete the non-conversion description character sequence in the target formatted string to obtain a compressed formatted string containing conversion descriptions;

[0011] Concatenate the unique identifier and the compressed formatted string to obtain a compressed Log statement template, and replace the original Log statement template in the to-be-compiled source code with the compressed Log statement template to obtain new code;

[0012] Compile the new code to obtain an image file, and run the image file on the target machine to generate compressed Log information; the compressed Log information is the information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file.

[0013] Optionally, the step of concatenating the unique identifier and the compressed formatted string to obtain a compressed Log statement template includes:

[0014] Concatenate all the conversion descriptions in the compressed formatted string according to a preset character to obtain a first concatenated string;

[0015] Concatenate the unique identifier and the first concatenated string to obtain a second concatenated string;

[0016] Add the preset character between the unique identifier in the second concatenated string and the first concatenated string to obtain a compressed Log statement template.

[0017] Optionally, after running the image file on the target machine to generate compressed Log information, the method further includes:

[0018] Obtain the compressed Log information exported by the target machine, and determine the original Log statement template according to the unique identifier included in the compressed Log information;

[0019] Decompress the compressed Log information according to the original Log statement template to restore the target Log information including the non-converted description character sequence.

[0020] Optionally, the decompressing the compressed Log information according to the original Log statement template to restore the target Log information including the non-converted description character sequence includes:

[0021] Parse the parameters in the compressed Log information to obtain respective target parameters corresponding to the respective conversion descriptions in the compressed Log statement template;

[0022] Successively replace each of the target parameters into the conversion descriptions of the target format string in the original Log statement template to obtain the target Log information including the non-converted description character sequence.

[0023] Optionally, the establishing the mapping relationship between the unique identifier and the original Log statement template includes:

[0024] Establish the mapping relationship between the unique identifier and the original Log statement template, and record the mapped unique identifier and the original Log statement template in a preset mapping file; wherein, the preset mapping file is located on the host machine;

[0025] Correspondingly, the determining the original Log statement template according to the unique identifier included in the compressed Log information includes:

[0026] Query the Log statement template corresponding to the unique identifier included in the compressed Log information from the preset mapping file located on the host machine to obtain the original Log statement template.

[0027] Optionally, the generating a unique identifier for the original Log statement template includes:

[0028] Sort all the original Log statement templates to obtain the sorted Log statement templates;

[0029] Use a counter with a preset number of bytes to generate corresponding unique identifiers for each Log statement template in the sorted Log statement templates.

[0030] Optionally, the traversing the to-be-compiled source code to identify Log statement templates in the to-be-compiled source code to obtain original Log statement templates includes:

[0031] Traverse the to-be-compiled source code line by line, and during the traversal, determine whether there is a Log statement template in the current statement;

[0032] If there is a Log statement template in the current statement, extract the Log statement template existing in the current statement to obtain the original Log statement template.

[0033] In a second aspect, the present application discloses a log processing device, including:

[0034] A source code acquisition module, configured to acquire the to-be-compiled source code written for the embedded device on the host;

[0035] A traversing module, configured to traverse the to-be-compiled source code to identify Log statement templates in the to-be-compiled source code to obtain original Log statement templates;

[0036] An identifier generation module, configured to generate unique identifiers for the original Log statement templates;

[0037] A mapping relationship establishment module, configured to establish a mapping relationship between the unique identifiers and the original Log statement templates;

[0038] An identification module, configured to identify formatted strings in the original Log statement templates to obtain target formatted strings;

[0039] A deletion module, configured to delete non-conversion description character sequences in the target formatted strings to obtain compressed formatted strings containing conversion descriptions;

[0040] A splicing module, configured to splice the unique identifiers and the compressed formatted strings to obtain compressed Log statement templates;

[0041] A replacement module, configured to replace the original Log statement templates in the to-be-compiled source code with the compressed Log statement templates to obtain new code;

[0042] A compilation and execution module, configured to compile the new code to obtain an image file, and execute the image file on a target machine to generate compressed Log information; the compressed Log information is the information generated after executing the compressed Log statement template in the image file and does not include the non-conversion description character sequence.

[0043] In a third aspect, the present application discloses an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the foregoing log processing method is implemented.

[0044] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing log processing method is implemented.

[0045] It can be seen that this application is applied to a host computer. First, the source code to be compiled that has been written for an embedded device on the host computer is obtained, and the source code to be compiled is traversed to identify the Log statement template in the source code to be compiled to obtain the original Log statement template. Then, a unique identifier is generated for the original Log statement template, and a mapping relationship is established between the unique identifier and the original Log statement template. Next, the formatted string in the original Log statement template is identified to obtain the target formatted string, and the non-conversion description character sequence in the target formatted string is deleted to obtain the compressed formatted string containing conversion descriptions. Then, the unique identifier and the compressed formatted string are concatenated to obtain the compressed Log statement template, and the compressed Log statement template is used to replace the original Log statement template in the source code to be compiled to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file is run on the target machine to generate compressed Log information; wherein, the compressed Log information is the information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file. This application establishes a unique identifier for the Log statement template in the source code to be compiled, and then concatenates the unique identifier and the formatted string from which the non-conversion description character sequence has been deleted to obtain the compressed Log statement template. The compressed Log statement template is used to replace the original Log statement template in the source code to be compiled to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file obtained by compiling the new code is run on the target machine, thereby generating compressed Log information that does not contain the non-conversion description character sequence. In this way, the larger the data volume of the non-conversion description character sequence in the original Log statement template, the higher the compression ratio; since no complex compression algorithm is involved, the log compression process is simplified, thereby reducing the demand for storage space resources. More storage space can be allocated for other more critical services in the system. At the same time, when the compressed log is transmitted over the network, the amount of data transmitted is reduced, thereby saving network resources; in addition, since the compression operation is performed on the host computer, the computing power resources of the target machine are saved. Description of the Drawings

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0047] Figure 1 It is a flowchart of a log processing method disclosed in the present application;

[0048] Figure 2 A flowchart of a specific log processing method disclosed in this application;

[0049] Figure 3 A schematic diagram of a specific log processing method disclosed in this application;

[0050] Figure 4 A schematic diagram of the structure of a log processing device disclosed in this application;

[0051] Figure 5 A structural diagram of an electronic device disclosed in this application. Specific embodiments

[0052] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0053] The embodiments of this application disclose a log processing method, which is applied to a host computer. Refer to Figure 1 As shown, the method includes:

[0054] Step S11: Obtain the source code to be compiled that has been written for the embedded device on the host computer, and traverse the source code to be compiled to identify the Log statement template in the source code to be compiled to obtain the original Log statement template.

[0055] It should be noted that the log processing solution proposed in this application is different from the traditional log compression method when compressing logs. The compression operation in this application is performed on the host computer, and the compression operation of Log is advanced to before the compilation of the embedded software, and the Log compression is directly completed at the source code level. Specifically, first obtain the source code to be compiled that has been written for the embedded device on the host computer, and then traverse the obtained source code to be compiled, and identify the Log statement template in the source code to be compiled during the traversal, so as to obtain the original Log statement template; wherein, the assembly language corresponding to the source code to be compiled includes but is not limited to C language, C++ language, Java language, etc., and the Log statement template refers to the Log statement that has not assigned values to the parameters.

[0056] Specifically, traversing the source code to be compiled to identify the Log statement templates in the source code to be compiled to obtain the original Log statement templates may include: traversing the source code to be compiled line by line, and determining whether there is a Log statement template in the current statement during the traversal; if there is a Log statement template in the current statement, extracting the Log statement template existing in the current statement to obtain the original Log statement template. In this embodiment, the source code to be compiled written for the embedded device on the host is traversed line by line, that is, the source code to be compiled is identified in the order of code execution. During the traversal, it is determined whether there is a Log statement template in the current statement, that is, whether there is an unassigned Log statement. If there is a Log statement template in the current statement, the Log statement template existing in the current statement is extracted to obtain the corresponding original Log statement template. For example, after traversing the source code to be compiled, the following Log statement templates are extracted:

[0057] LOG_D("My name is%s,I am%d years old,and my height is%.2f meters.\n",name,age,height);

[0058] LOG_I(“[%s,%d]Start to create cli thread”,__FUNCTION__,__LINE__);

[0059] LOG_E(“Failed to create cli thread,Tid=%x”,g_cli_tid);

[0060] LOG_E(“App initfalied.,ret=%x”,ret);

[0061] It can be understood that the format of the Log statement template is specific and can be easily recognized in form. Corresponding algorithms can be used to identify the Log statement template, which is not specifically limited here.

[0062] Step S12: Generate a unique identifier for the original Log statement template and establish a mapping relationship between the unique identifier and the original Log statement template.

[0063] In this embodiment, after identifying the Log statement template in the source code to be compiled and obtaining the original Log statement template, further, a corresponding identifier is generated for the original Log statement template. This identifier can uniquely represent the original Log statement template, such as the unique identifier Log ID (Identity Document, that is, the unique code). Then, a mapping relationship is established between the unique identifier and the original Log statement template, that is, mapping the two. For example, for each identified Log statement template, a corresponding Log ID is generated for the Log statement template, and a mapping relationship is established between the Log statement template and the Log ID. For example, 1, 2, 3, and 4 are used as the unique identifiers for the above four Log statement templates respectively, and the following mapping relationship is obtained:

[0064] 1LOG_D("My name is%s,I am%d years old,and my height is%.2fmeters.\n",name,age,height);

[0065] 2LOG_I(“[%s,%d]Start to create cli thread”,__FUNCTION__,__LINE__);

[0066] 3LOG_E(“Failed to create cli thread,Tid=%x”,g_cli_tid);

[0067] 4LOG_E(“App initfalied.,ret=%x”,ret);

[0068] It can be seen that the first column is the ID number corresponding to each Log statement template, and the second column is the specific Log statement template.

[0069] In another specific implementation manner, a custom coding method can be used to generate the unique identifier of the Log statement template. For example, UUID1, UUID2, UUID 3, and UUID 4 are used as the unique identifiers for the above four Log statement templates respectively, that is, encoding is performed in the way of preset string + number, and the following mapping relationship is obtained:

[0070] UUID1 LOG_D("My name is%s,I am%d years old,and my height is%.2fmeters.\n",name,age,height);

[0071] UUID2 LOG_I("[%s,%d]Start to create cli thread",__FUNCTION__,__LINE__);

[0072] UUID3 LOG_E("Failed to create cli thread,Tid=%x",g_cli_tid);

[0073] UUID4 LOG_E("App initfalied.,ret=%x",ret);

[0074] Of course, the unique identifier of the Log statement template can also be generated according to other custom encoding methods, as long as the uniqueness of the Log statement template can be ensured. By generating a unique identifier and establishing a mapping relationship between the unique identifier and the original Log statement template, it helps to quickly restore the compressed Log information accurately later, improving the speed of Log restoration.

[0075] Step S13: Identify the format string in the original Log statement template to obtain a target format string, and delete the non-conversion description character sequence in the target format string to obtain a compressed format string containing conversion descriptions.

[0076] In this embodiment, after establishing the mapping relationship between the unique identifier and the original Log statement template, the format string (Format String) in the original Log statement template can be further identified to obtain a corresponding target format string, and then the non-conversion description character sequence in the target format string is deleted, so as to obtain a compressed format string containing conversion descriptions. It can be understood that the non-conversion description character sequence is the characters other than the format specifier. These characters will be directly output to the screen or written to a file without being formatted according to the subsequent parameters. For example, in the printf function, "Hello%s!" in the string is a non-conversion description character sequence, which will be directly output to the screen or written to a file without being formatted according to the subsequent parameters. Therefore, in order to compress the log, the non-conversion description character sequence that does not change before and after compilation can be deleted from the format string in the original Log statement template, so as to obtain a format string containing conversion descriptions.

[0077] For example, when the formatting string in the original Log statement template is "LOG_D(\"My name is%s, I am%d years old, and my height is%.2f meters.\n\", name, age, height);", remove the non-conversion description character sequence "My name is, I am years old, and my height is meters.\n" from the above formatting string to obtain a compressed formatting string containing the conversion descriptions "%s%d%.2f".

[0078] Step S14: Concatenate the unique identifier and the compressed formatting string to obtain a compressed Log statement template, and replace the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain new code.

[0079] In this embodiment, after obtaining the compressed formatting string containing the conversion descriptions, concatenate the unique identifier and the compressed formatting string to obtain a compressed Log statement template that does not contain the non-conversion description character sequence, and then replace the original Log statement template in the source code to be compiled with the compressed Log statement template, that is, compress the original Log statement template in the source code to be compiled, and then generate new code.

[0080] In a specific embodiment, the concatenation of the unique identifier and the compressed formatted string to obtain a compressed Log statement template may specifically include: concatenating all the conversion specifications in the compressed formatted string according to a preset character to obtain a concatenated formatted string; concatenating the unique identifier and the first concatenated string to obtain a second concatenated string; adding the preset character between the unique identifier and the first concatenated string in the second concatenated string to obtain a compressed Log statement template. For example, concatenating all the conversion specifications in the above compressed formatted string according to the preset character # to obtain the concatenated formatted string "LOG_D(\"%s#%d#%.2f\",name,age,height);", and then concatenating the corresponding unique identifier LogID to the above concatenated formatted string, such as concatenating the unique identifier Log ID as a prefix to the concatenated formatted string to obtain the compressed Log statement template "LOG_D(\"1%s#%d#%.2f\",name,age,height);". Further, replacing the original Log statement template "LOG_D(\"Myname is %s,I am %d years old,and my height is %.2f meters.\n\",name,age,height);" in the source code to be compiled with the above compressed Log statement template to obtain the new code "LOG_D(\"1%s#%d#%.2f\",name,age,height);".

[0081] Step S15: Compile the new code to obtain an image file, and run the image file on the target machine to generate compressed Log information; the compressed Log information is the information generated after executing the compressed Log statement template in the image file and does not contain the non-conversion specification character sequence.

[0082] In this embodiment, after replacing the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain the new code, compile the above new code to obtain the corresponding image file, and then run the above image file on the target machine so that after executing the compressed Log statement template in the above image file, compressed Log information that does not contain the non-conversion specification character sequence is generated. For example, the following compressed Log information is generated after running the image file:

[0083] 1#Jack#13#1.75

[0084] 2#create_cli_task#135

[0085] 3#0

[0086] 4#15

[0087] It can be understood that since the new code does not contain non-conversion description character sequences, that is, the source code to be compiled is compressed, the resulting image file after compilation also does not contain non-conversion description character sequences, which is equivalent to compressing the generated image file, and the data volume is correspondingly reduced. The reduction of the data volume of the image file can save the occupation of storage resources and can save network resources when transmitted through the network.

[0088] Further, after running the image file on the target machine to generate compressed Log information, it may further include: obtaining the compressed Log information exported by the target machine, and determining the original Log statement template according to the unique identifier included in the compressed Log information; decompressing the compressed Log information according to the original Log statement template to restore the target Log information including the non-conversion description character sequence. In this embodiment, in order to facilitate technicians to view the complete Log information, the compressed Log information can be decompressed to restore the complete Log information, that is, to restore the non-conversion description character sequence. Specifically, after the host machine obtains the compressed Log information exported by the target machine, it can determine the corresponding original Log statement template according to the unique identifier included in the above compressed Log information, and then decompress the above compressed Log information according to the original Log statement template, so as to restore the target Log information including the non-conversion description character sequence.

[0089] Specifically, the decompressing the compressed Log information according to the original Log statement template to restore the target Log information including the non-conversion description character sequence may include: parsing the parameters in the compressed Log information to obtain respective target parameters corresponding to the respective conversion descriptions in the compressed Log statement template; sequentially replacing the respective target parameters into the conversion descriptions of the target format string in the original Log statement template to obtain the target Log information including the non-conversion description character sequence. For example, after parsing the first compressed Log information "1#Jack#13#1.75", parameters including Jack, 13, and 1.75 are obtained, and then the above Jack, 13, and 1.75 are sequentially replaced into the conversion descriptions of the target format string in the original Log statement template, and further the target Log information "My name is Jack, I am 13 years old, and my height is 1.75 meters." including the non-conversion description character sequence is obtained.

[0090] Similarly, decompress each of the remaining Log messages in the compressed Log messages to obtain the following decompressed target Log messages:

[0091] [create_cli_task,135]Start to create cli thread

[0092] Failed to create cli thread,Tid=0

[0093] App initfalied.,ret=15

[0094] From the number of bytes of the Log messages before and after recovery, it can be seen that the Log compression method of the present application can effectively reduce the storage space occupied by the Log, and the more information in the non-converted description character sequence in the Log, the higher the compression ratio.

[0095] It can be seen that the embodiment of the present application is applied to a host computer. First, the to-be-compiled source code written for an embedded device on the host computer is obtained, and the to-be-compiled source code is traversed to identify the Log statement template in the to-be-compiled source code to obtain the original Log statement template. Then, a unique identifier is generated for the original Log statement template, and a mapping relationship is established between the unique identifier and the original Log statement template. Next, the formatted string in the original Log statement template is identified to obtain the target formatted string, and the non-conversion description character sequence in the target formatted string is deleted to obtain the compressed formatted string containing conversion descriptions. Then, the unique identifier and the compressed formatted string are concatenated to obtain the compressed Log statement template, and the original Log statement template in the to-be-compiled source code is replaced with the compressed Log statement template to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file is run on the target machine to generate compressed Log information; wherein, the compressed Log information is the information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file. In the embodiment of the present application, a unique identifier is established for the Log statement template in the to-be-compiled source code, and then the unique identifier and the formatted string from which the non-conversion description character sequence is deleted are concatenated to obtain the compressed Log statement template. The original Log statement template in the to-be-compiled source code is replaced with the compressed Log statement template to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file obtained by compiling the new code is run on the target machine, thereby generating compressed Log information that does not contain the non-conversion description character sequence. In this way, the larger the data volume of the non-conversion description character sequence in the original Log statement template, the higher the compression ratio, that is, the compression ratio is improved; since no complex compression algorithm is involved, the log compression process is simplified, and further the requirement for storage space resources is reduced. More storage space can be allocated for other more critical services in the system. At the same time, when the compressed log is transmitted over the network, the amount of data transmitted is reduced, thereby saving network resources; in addition, since the compression operation is performed on the host computer, the computing power resources of the target machine are saved.

[0096] An embodiment of the present application discloses a specific log processing method, which is applied to a host computer. Refer to Figure 2 As shown, the method includes:

[0097] Step S201: Obtain the to-be-compiled source code written for the embedded device on the host computer, and traverse the to-be-compiled source code to identify the Log statement template in the to-be-compiled source code to obtain the original Log statement template.

[0098] Step S202: Sort all the original Log statement templates to obtain the sorted Log statement templates.

[0099] In this embodiment, after identifying the Log statement templates in the source code to be compiled and obtaining the original Log statement templates, all the above original Log statement templates can be sorted according to the order of recognition to obtain the corresponding sorted Log statement templates.

[0100] Step S203: Use a counter with a preset number of bytes to generate a corresponding unique identifier for each Log statement template in the sorted Log statement templates, establish a mapping relationship between the unique identifier and the original Log statement template, and record the mapped unique identifier and the original Log statement template into a preset mapping file; wherein, the preset mapping file is located on the host machine.

[0101] In this embodiment, when generating corresponding identifiers for each Log statement template in the above sorted Log statement templates, a counter with a preset number of bytes can be used to generate a unique identifier Log ID for it. Specifically, Log ID can be generated by a global counter. For example, it can be stored with 2 bytes (which can map 64K Log statement templates, and usually the ID resources are completely sufficient). Then, establish a mapping relationship between the above unique identifier Log ID and the above original Log statement template, and then record the mapped unique identifier and the original Log statement template into a preset mapping file located on the above host machine. In a specific embodiment, the content of the mapping file is as follows:

[0102] 1LOG_D("My name is%s,I am%d years old,and my height is%.2fmeters.\n",name,age,height);

[0103] 2LOG_I(“[%s,%d]Start to create cli thread”,__FUNCTION__,__LINE__);

[0104] 3LOG_E(“Failed to create cli thread,Tid=%x”,g_cli_tid);

[0105] 4LOG_E(“App initfalied.,ret=%x”,ret);

[0106] ……

[0107] Step S204: Identify the formatting strings in the original Log statement template to obtain target formatting strings, and delete the non-conversion description character sequences in the target formatting strings to obtain compressed formatting strings containing conversion descriptions.

[0108] Step S205: Concatenate all the conversion descriptions in the compressed formatting strings according to a preset character to obtain a first concatenated string.

[0109] In this embodiment, all the conversion descriptions in the above compressed formatting strings can be concatenated according to a preset character to obtain a first concatenated string. For example, all the conversion descriptions in the compressed formatting strings are concatenated according to the preset character & to obtain the first concatenated string "LOG_D(\"%s&%d&%.2f\",name,age,height);".

[0110] Step S206: Concatenate the unique identifier and the first concatenated string to obtain a second concatenated string.

[0111] Next, the corresponding unique identifier Log ID is concatenated into the above first concatenated string. For example, the unique identifier Log ID is used as a prefix and concatenated into the above first concatenated string to obtain the second concatenated string "LOG_D(\"UUID1%s&%d&%.2f\",name,age,height);".

[0112] Step S207: Add the preset character between the unique identifier and the first concatenated string in the second concatenated string to obtain a compressed Log statement template, and replace the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain new code.

[0113] Further, add the above preset character & between the unique identifier and the first concatenated string in the above second concatenated string "LOG_D(\"UUID1%s&%d&%.2f\",name,age,height);" to obtain the compressed Log statement template "LOG_D(\"UUID1&%s&%d&%.2f\",name,age,height);".

[0114] Step S208: Compile the new code to obtain an image file, and run the image file on the target machine to generate compressed Log information; the compressed Log information is the information generated after executing the compressed Log statement template in the image file and does not contain the non-conversion description character sequences.

[0115] Step S209: Obtain the compressed Log information exported by the target machine, and query the Log statement template corresponding to the unique identifier included in the compressed Log information from the preset mapping file located on the host machine, so as to obtain the original Log statement template.

[0116] In this embodiment, after the host machine obtains the compressed Log information exported by the above target machine, it can query the preset mapping file located on the above host machine to identify the Log statement template that has a mapping relationship with the unique identifier included in the above compressed Log information, so as to obtain the above original Log statement template.

[0117] Step S210: Analyze the parameters in the compressed Log information to obtain respective target parameters corresponding to the respective conversion descriptions in the compressed Log statement template, and sequentially replace the target parameters into the conversion descriptions of the target formatting strings in the original Log statement template to obtain the target Log information including the non-conversion description character sequence.

[0118] Among them, for more specific processing procedures of the above steps S201, S202, S204, S208, and S210, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be elaborated here.

[0119] It can be seen that the embodiment of the present application advances the operation of Log compression before the compilation of the embedded software, directly completes Log compression at the source code level, and deletes the non-conversion description character sequence in the Log statement template of the source code to be compiled, thereby realizing the Log compression of the embedded software. Since it does not involve complex compression algorithms, the Log compression process is simplified, the Log compression efficiency is improved, and when the data volume of the non-conversion description character sequence in the Log statement template is larger, the corresponding compression ratio will be higher, thereby reducing the demand for storage space resources and enabling more storage space to be allocated for other more critical services in the system. Since the compression operation is performed on the host machine, the computing power resources of the target machine, such as the CPU, can be effectively saved. In addition, the compressed Log information can be decompressed through the pre-created unique identifier, which is convenient for users to view the complete Log information.

[0120] Specifically, refer to Figure 3As shown, first obtain the source code to be compiled that has been written for the embedded device on the host machine, and then input it into the source code Log statement processing module to traverse the source code to be compiled, so as to identify the original Log statement template in the source code to be compiled, and generate a unique identifier for the identified original Log statement template. Then, establish a mapping relationship between the unique identifier and the original Log statement template, and record the mapped unique identifier and the original Log statement template in the mapping file. Further, the source code Log statement processing module deletes the non-conversion description character sequence contained in the formatted string of the original Log statement template, and splices the unique identifier and the compressed formatted string to obtain the compressed Log statement template. Then, replace the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain new code, and then compile and run the new code to generate the corresponding image file. When running the image file on the target machine, the generated compressed Log information can be sent to the host machine, and then the Log information recovery module on the host machine decompresses the compressed Log information according to the mapping relationship recorded in the mapping file to obtain the complete target Log information containing the non-conversion description character sequence.

[0121] Correspondingly, the embodiment of the present application also discloses a log processing device, which is applied to the host machine. Refer to Figure 4 As shown, the device includes:

[0122] A source code acquisition module 11, configured to acquire the source code to be compiled that has been written for the embedded device on the host machine;

[0123] A traversal module 12, configured to traverse the source code to be compiled to identify the Log statement template in the source code to be compiled to obtain the original Log statement template;

[0124] An identifier generation module 13, configured to generate a unique identifier for the original Log statement template;

[0125] A mapping relationship establishment module 14, configured to establish a mapping relationship between the unique identifier and the original Log statement template;

[0126] An identification module 15, configured to identify the formatted string in the original Log statement template to obtain the target formatted string;

[0127] A deletion module 16, configured to delete the non-conversion description character sequence in the target formatted string to obtain a compressed formatted string containing conversion descriptions;

[0128] A splicing module 17, configured to splice the unique identifier and the compressed formatted string to obtain a compressed Log statement template;

[0129] A replacement module 18, configured to replace the original Log statement template in the source code to be compiled with the compressed Log statement template, to obtain new code;

[0130] A compilation and running module 19, configured to compile the new code to obtain an image file, and run the image file on a target machine to generate compressed Log information; the compressed Log information is information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file.

[0131] Wherein, for the specific working processes of the above-mentioned respective modules, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details are not described herein again.

[0132] It can be seen that the embodiment of the present application is applied to a host computer. First, the to-be-compiled source code written for an embedded device on the host computer is obtained, and the to-be-compiled source code is traversed to identify the Log statement template in the to-be-compiled source code to obtain the original Log statement template. Then, a unique identifier is generated for the original Log statement template, and a mapping relationship between the unique identifier and the original Log statement template is established. Next, the formatted string in the original Log statement template is identified to obtain the target formatted string, and the non-conversion description character sequence in the target formatted string is deleted to obtain a compressed formatted string containing conversion descriptions. Then, the unique identifier and the compressed formatted string are concatenated to obtain a compressed Log statement template, and the original Log statement template in the to-be-compiled source code is replaced with the compressed Log statement template to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file is run on the target machine to generate compressed Log information; wherein, the compressed Log information is the information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file. In the embodiment of the present application, a unique identifier is established for the Log statement template in the to-be-compiled source code, and then the unique identifier and the formatted string from which the non-conversion description character sequence is deleted are concatenated to obtain a compressed Log statement template. The original Log statement template in the to-be-compiled source code is replaced with the compressed Log statement template to obtain new code. Finally, the new code is compiled to obtain an image file, and the image file obtained by compiling the new code is run on the target machine, thereby generating compressed Log information that does not contain the non-conversion description character sequence. In this way, the larger the data volume of the non-conversion description character sequence in the original Log statement template, the higher the compression ratio; since no complex compression algorithm is involved, the log compression process is simplified, thereby reducing the demand for storage space resources, and more storage space can be allocated for other more critical services in the system. At the same time, when the compressed log is transmitted over the network, the amount of data transmitted is reduced, thereby saving network resources; in addition, since the compression operation is performed on the host computer, the computing power resources of the target machine are saved.

[0133] In some specific embodiments, the concatenation module 17 may specifically include:

[0134] A first concatenation unit, configured to concatenate all the conversion descriptions in the compressed formatted string according to a preset character to obtain a first concatenated string;

[0135] A second concatenation unit, configured to concatenate the unique identifier and the first concatenated string to obtain a second concatenated string;

[0136] A character adding unit, configured to add the preset character between the unique identifier in the second concatenated string and the first concatenated string to obtain a compressed Log statement template.

[0137] In some specific embodiments, after the compilation and running module 19, the following may further be included:

[0138] A first information acquisition unit, configured to acquire the compressed Log information exported by the target machine;

[0139] A template determination unit, configured to determine the original Log statement template according to the unique identifier included in the compressed Log information;

[0140] A decompression unit, configured to decompress the compressed Log information according to the original Log statement template to restore the target Log information including the non-conversion description character sequence.

[0141] In some specific embodiments, the decompression unit may specifically include:

[0142] A second acquisition unit, configured to parse the parameters in the compressed Log information to obtain respective target parameters corresponding to the respective conversion descriptions in the compressed Log statement template;

[0143] A replacement unit, configured to sequentially replace the respective target parameters into the conversion descriptions of the target formatting string in the original Log statement template to obtain the target Log information including the non-conversion description character sequence.

[0144] In some specific embodiments, the mapping relationship establishment module 14 may specifically include:

[0145] A mapping relationship establishment unit, configured to establish a mapping relationship between the unique identifier and the original Log statement template, and record the mapped unique identifier and the original Log statement template into a preset mapping file; wherein, the preset mapping file is located on the host machine;

[0146] Correspondingly, the traversal module 12 may specifically include:

[0147] A template query unit, configured to query the Log statement template corresponding to the unique identifier included in the compressed Log information from the preset mapping file located on the host machine to obtain the original Log statement template.

[0148] In some specific embodiments, the identifier generation module 13 may specifically include:

[0149] A template sorting unit for sorting all the original Log statement templates to obtain sorted Log statement templates;

[0150] An identifier generation unit for sequentially generating corresponding unique identifiers for each Log statement template in the sorted Log statement templates by using a counter with a preset number of bytes.

[0151] In some specific embodiments, the traversal module 12 may specifically include:

[0152] A traversal unit for traversing the source code to be compiled line by line;

[0153] A judgment unit for judging whether there is a Log statement template in the current statement during the traversal process;

[0154] A template extraction unit for extracting the Log statement templates existing in the current statement to obtain original Log statement templates if there are Log statement templates in the current statement.

[0155] Furthermore, an embodiment of the present application also discloses an electronic device, Figure 5 which is a structural diagram of an electronic device 20 shown according to an exemplary embodiment. The content in the figure should not be considered as any limitation on the scope of use of the present application.

[0156] Figure 5 This is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the log processing method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0157] In this embodiment, the power supply 23 is used to provide working voltages for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application requirements, and no specific limitation is made here.

[0158] In addition, as a carrier for storing resources, the memory 22 can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc. The storage method can be temporary storage or permanent storage.

[0159] Among them, the operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device 20, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program that can be used to complete the log processing method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks.

[0160] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the foregoing disclosed log processing method is implemented. For the specific steps of this method, reference can be made to the corresponding content disclosed in the foregoing embodiments, and details will not be repeated here.

[0161] In this specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0162] Those skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in this document can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0163] The steps of the method or algorithm described in combination with the embodiments disclosed in this document can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0164] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0165] The above has introduced in detail a log processing method, apparatus, device and storage medium provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A log processing method, characterized in that, Applied to a host computer, including: Obtain the source code to be compiled that has been written for an embedded device on the host computer, and traverse the source code to be compiled to identify the Log statement template in the source code to be compiled to obtain the original Log statement template; Generate a unique identifier for the original Log statement template, and establish a mapping relationship between the unique identifier and the original Log statement template; Identify the formatted string in the original Log statement template to obtain the target formatted string, and delete the non-conversion description character sequence in the target formatted string to obtain a compressed formatted string containing conversion descriptions; Concatenate the unique identifier and the compressed formatted string to obtain a compressed Log statement template, and replace the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain new code; Compile the new code to obtain an image file, and run the image file on the target machine to generate compressed Log information; the compressed Log information is the information that does not contain the non-conversion description character sequence generated after executing the compressed Log statement template in the image file.

2. The log processing method according to claim 1, wherein The concatenating the unique identifier and the compressed formatted string to obtain a compressed Log statement template includes: Concatenate all the conversion descriptions in the compressed formatted string according to a preset character to obtain a first concatenated string; Concatenate the unique identifier and the first concatenated string to obtain a second concatenated string; Add the preset character between the unique identifier and the first concatenated string in the second concatenated string to obtain a compressed Log statement template.

3. The log processing method according to claim 2, wherein After running the image file on the target machine to generate compressed Log information, it further includes: Obtain the compressed Log information exported by the target machine, and determine the original Log statement template according to the unique identifier included in the compressed Log information; Decompress the compressed Log information according to the original Log statement template to restore the target Log information containing the non-conversion description character sequence.

4. The log processing method according to claim 3, wherein The decompressing the compressed Log information according to the original Log statement template to restore the target Log information containing the non-conversion description character sequence includes: Parse the parameters in the compressed Log information to obtain respective target parameters corresponding to the respective conversion descriptions in the compressed Log statement template; Successively replace each target parameter into the conversion description of the target formatted string in the original Log statement template to obtain the target Log information containing the non-conversion description character sequence.

5. The log processing method according to claim 3, wherein The establishing the mapping relationship between the unique identifier and the original Log statement template includes: Establish a mapping relationship between the unique identifier and the original Log statement template, and record the mapped unique identifier and the original Log statement template in a preset mapping file; wherein, the preset mapping file is located on the host machine; Correspondingly, the determining the original Log statement template according to the unique identifier included in the compressed Log information includes: Query the Log statement template corresponding to the unique identifier included in the compressed Log information from the preset mapping file located on the host machine to obtain the original Log statement template.

6. The log processing method according to claim 4, wherein The generating a unique identifier for the original Log statement template includes: Sort all the original Log statement templates to obtain the sorted Log statement templates; Use a counter of a preset number of bytes to generate corresponding unique identifiers for each Log statement template in the sorted Log statement templates in turn.

7. The log processing method according to any one of claims 1 to 6, characterized in that, The traversing the source code to be compiled to identify the Log statement template in the source code to be compiled to obtain the original Log statement template includes: Traverse the source code to be compiled line by line, and during the traversal, determine whether there is a Log statement template in the current statement; If there is a Log statement template in the current statement, extract the Log statement template existing in the current statement to obtain the original Log statement template.

8. A log processing device, characterized in that, Applied to the host machine, it includes: A source code acquisition module, configured to acquire the source code to be compiled written for the embedded device on the host machine; A traversing module, configured to traverse the source code to be compiled to identify the Log statement template in the source code to be compiled to obtain the original Log statement template; An identifier generation module, configured to generate a unique identifier for the original Log statement template; A mapping relationship establishment module, configured to establish a mapping relationship between the unique identifier and the original Log statement template; An identification module, configured to identify the formatted string in the original Log statement template to obtain the target formatted string; A deletion module, configured to delete the non-conversion description character sequence in the target formatted string to obtain a compressed formatted string including conversion descriptions; A splicing module, configured to splice the unique identifier and the compressed formatted string to obtain a compressed Log statement template; A replacement module, configured to replace the original Log statement template in the source code to be compiled with the compressed Log statement template to obtain new code; A compilation and running module, configured to compile the new code to obtain an image file, and run the image file on the target machine to generate compressed Log information; the compressed Log information is the information that does not include the non-conversion description character sequence generated after executing the compressed Log statement template in the image file.

9. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes the computer program saved in the memory, it implements the log processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by a processor, it implements the log processing method according to any one of claims 1 to 7.

Images