Information processing method, apparatus, device, vehicle, medium and program product
By compressing part of the information in the vehicle log file and generating file association information, the problem of low log file search efficiency is solved, and efficient storage and fast search are achieved.
Patent Information
- Application Number
- PCT/CN2025/084174
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-03-27
- Filing Date
- 2025-03-21
- Publication Date
- 2025-10-02
AI Technical Summary
In the prior art, after the vehicle log files are compressed, a large amount of processing resources and time are consumed in the process of searching the log files, resulting in low efficiency.
By obtaining some information from the log file, such as time code, state change code, process ID code and thread ID code, compression processing is performed using different dimensions to generate file association information, which is stored in a dictionary to represent the relationship between the compressed information and the log file, thereby achieving fast search.
This reduces the amount of stored data, saves storage costs, improves the search efficiency and flexibility of log files, and reduces the inconvenience of finding log files.
Smart Images

Figure CN2025084174_02102025_PF_FP_ABST
Abstract
Description
Information processing method, device, equipment, vehicle, medium and program product
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This disclosure claims priority to Chinese patent application No. 2024103614780 filed on March 27, 2024, filed by Beijing Rockwell Technology Co., Ltd., and entitled “A method, device, equipment and vehicle for information processing,” the full text of which is incorporated by reference into this disclosure. Technical Field
[0003] The present disclosure relates to the field of information processing technology, and relates to, but is not limited to, an information processing method, apparatus, device, vehicle, medium, and program product. Background Art
[0004] As vehicles become increasingly intelligent, they generate an increasing number of log files. This excessive number of log files not only takes up a significant amount of storage space, but also wastes significant data transmission resources when uploading them to the cloud. Therefore, existing methods often compress log files to reduce their size. However, this leads to the problem that when log files are needed, the sheer number of log files consumes significant processing resources and time when searching for them. Therefore, finding the right log files while compressing them is crucial. Summary of the Invention
[0005] The present disclosure provides an information processing method, apparatus, device, vehicle, medium, and program product, which improve the efficiency of searching log files.
[0006] The present disclosure provides an information processing method, the method comprising:
[0007] Obtaining at least one piece of file information, where each piece of file information is partial information in a log file corresponding to the file information, and each piece of file information includes at least two sub-file information of time code information, state change code information, process ID code information, and thread ID code information, and dimensions of the at least two sub-file information are different from each other;
[0008] Compressing sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information;
[0009] At least two first compressed information are stored in a dictionary, and file association information is generated based on the at least two first compressed information and the compressed log file. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes a compressed file after the log file is compressed.
[0010] In an optional implementation, obtaining at least one file information includes:
[0011] Get at least one log file;
[0012] For each log file in the at least one log file, extracting information to be processed from the log file, where the information to be processed includes at least two of time information, state change information, a process identifier, and a thread identifier;
[0013] The information to be processed is encoded to obtain file information corresponding to the log file.
[0014] In an optional embodiment, the information to be processed includes state change information;
[0015] For each log file in the at least one log file, extract information to be processed from the log file, including:
[0016] For each log file in the at least one log file, state change information is extracted from the log file using a text matching algorithm.
[0017] In an optional embodiment, before compressing sub-file information having the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information, the method further includes:
[0018] Using a first compression tool to compress at least one file information to obtain second compressed information corresponding to the at least one file information, each second compressed information including dimension information corresponding to at least two sub-file information;
[0019] Compressing sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information includes:
[0020] For each dimension information in the at least two dimension information, extract the dimension information from each second compressed information in the at least one second compressed information to obtain a dimension information group corresponding to the dimension information, where the dimension information group includes at least one dimension information;
[0021] The at least two dimensional information groups are respectively compressed using a second compression tool to obtain first compressed information corresponding to the at least two dimensional information groups. The first compression tool and the second compression tool are different compression tools.
[0022] In an optional embodiment, before compressing sub-file information having the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information, the method further includes:
[0023] For each piece of file information in the at least one piece of file information, if the log file corresponding to the file information meets the target condition, compress the file information based on the streaming compression method to obtain at least one third compressed information, each of the third compressed information including at least two sub-compressed information of different dimensions;
[0024] The target conditions include that the log file is a log file generated by the Android system;
[0025] Compressing sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information includes:
[0026] The sub-compressed information with the same dimension in the at least one third compressed information is compressed according to different dimensions to obtain at least two first compressed information.
[0027] The present disclosure also provides an information processing method, the method comprising:
[0028] Acquire information to be searched, where the information to be searched includes at least one of time search information, state change search information, process search identifier, and thread search identifier;
[0029] Searching the dictionary for target compressed information corresponding to the information to be searched, and searching for the compressed log file corresponding to the target compressed information based on file association information stored in the dictionary, the file association information being used to characterize a relationship between at least two first compressed information and the compressed log file;
[0030] Decompress the compressed log file to obtain the log file corresponding to the information to be searched;
[0031] The dictionary stores at least two pieces of first compression information, and the at least two pieces of first compression information include at least two of time compression information, state change compression information, process ID compression information, and thread ID compression information.
[0032] The present disclosure also provides an information processing device, the device comprising:
[0033] an acquisition module configured to acquire at least one piece of file information, each piece of file information being partial information in a log file corresponding to the file information, each piece of file information including at least two sub-file information of time code information, state change code information, process identifier code information, and thread identifier code information, wherein dimensions of the at least two sub-file information are different from each other;
[0034] a compression module configured to compress sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information;
[0035] The storage module is configured to store at least two first compressed information in a dictionary, and generate file association information based on the at least two first compressed information and the compressed log file. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes a compressed file after the log file is compressed.
[0036] The present disclosure also provides an information processing device, the device comprising:
[0037] an acquisition module configured to acquire information to be searched, the information to be searched comprising at least one of time search information, state change search information, process search identifier, and thread search identifier;
[0038] a search module configured to search for target compressed information corresponding to the information to be searched from a dictionary, and search for a compressed log file corresponding to the target compressed information based on file association information stored in the dictionary, the file association information being used to characterize a relationship between at least two first compressed information and the compressed log file;
[0039] A decompression module is configured to decompress the compressed log file to obtain a log file corresponding to the information to be searched;
[0040] The dictionary stores at least two pieces of first compression information, and the at least two pieces of first compression information include at least two of time compression information, state change compression information, process ID compression information, and thread ID compression information.
[0041] The present disclosure also provides an electronic device, including a memory configured to store computer program instructions; and a processor configured to read and execute the computer program instructions stored in the memory to perform any one of the information processing methods described above.
[0042] The present disclosure also provides a computer storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, any one of the information processing methods described above is implemented.
[0043] The present disclosure also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements any one of the information processing methods described above.
[0044] The present disclosure also provides a vehicle, comprising at least one of the above-mentioned information processing device, electronic device, computer storage medium, and computer program product.
[0045] In an embodiment of the present disclosure, at least one file information can be obtained, each file information being a portion of information in a log file corresponding to the file information. Each file information includes at least two sub-file information of time code information, state change code information, process identification code information, and thread identification code information. The dimensions of the at least two sub-file information are different from each other. Based on this, the sub-file information of the at least one file information having the same dimension can be compressed according to different dimensions to obtain at least two first compressed information. The at least two first compressed information can then be stored in a dictionary. Based on the at least two first compressed information and the compressed log file, file association information is generated. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes the compressed log file after the log file is compressed. In this way, not only can the compressed storage of partial information in the log file significantly reduce the amount of data required for storage and save storage costs, but the generation of file association information can also reduce the inconvenience of subsequent log file search and use, thereby improving the efficiency of log file search. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0047] FIG1 is a flow chart of an information processing method according to an embodiment of the present disclosure;
[0048] FIG2 is a second flow chart of an information processing method provided by an embodiment of the present disclosure;
[0049] FIG3 is a third flow chart of an information processing method provided by an embodiment of the present disclosure;
[0050] FIG4 is a schematic structural diagram of an information processing device provided by an embodiment of the present disclosure;
[0051] FIG5 is a schematic structural diagram of another information processing device provided by an embodiment of the present disclosure;
[0052] FIG6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0053] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.
[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0055] To address the problems existing in the prior art, embodiments of the present disclosure provide an information processing method, apparatus, device, vehicle, medium, and program product capable of obtaining at least one file information item, each file information item being a portion of information from a corresponding log file. Each file information item includes at least two sub-file information items selected from time code information, state change code information, process identification code information, and thread identification code information. The dimensions of the at least two sub-file information items are different. Based on this, sub-file information items with the same dimensions within the at least one file information item can be compressed according to different dimensions to obtain at least two first compressed information items. The at least two first compressed information items can then be stored in a dictionary. File association information is generated based on the at least two first compressed information items and a compressed log file. The file association information is used to characterize the relationship between the at least two first compressed information items in the dictionary and the compressed log file. The compressed log file includes a compressed log file obtained by compressing the log file. In this manner, not only can the compressed storage of partial information from the log file significantly reduce the amount of data required for storage and save storage costs, but the generation of the file association information also reduces the inconvenience of subsequently searching and using the log file, thereby improving the efficiency of log file searches.
[0056] It should be noted that the information processing method provided in the embodiments of the present disclosure may be executed by an information processing device or a control module within the information processing device configured to execute the information processing method. The present embodiments use an information processing device executing the information processing method as an example to describe the information processing method provided in the embodiments of the present disclosure in detail.
[0057] It should also be noted that the information processing device can be applied to the Compressed Log Processor (CLP) tool, which is a tool that can losslessly compress unstructured text logs and can perform fast searches directly on compressed data.
[0058] The information processing method provided by the embodiment of the present disclosure is described in detail below with reference to specific embodiments in conjunction with the accompanying drawings.
[0059] FIG1 is a flow chart of an information processing method provided by an embodiment of the present disclosure.
[0060] As shown in Figure 1, the execution subject of the information processing method may be an information processing device. Based on this, the method may include the following steps:
[0061] S110: Obtain at least one file information.
[0062] In some embodiments, each of the above-mentioned file information may include partial information from the log file corresponding to the file information. For example, each of the above-mentioned file information may include at least two sub-file information items from the log file corresponding to the file information, including time code information, state change code information, process ID code information, and thread ID code information. Furthermore, the dimensions of the at least two sub-file information items are different from each other.
[0063] It should be noted that the above-mentioned time coding information may be information obtained by encoding the time information of the corresponding log file, wherein the above-mentioned time information may be information such as the timestamp of the log file. Correspondingly, the state change coding information may be information obtained by encoding the state change information of the corresponding log file, wherein the above-mentioned state change information may include state information in the log file that represents the system operation process. For example, if the system is a vehicle system, the state change information may include information such as vehicle speed and acceleration, which is not limited here. The above-mentioned process identification coding information may be information obtained by encoding the process identification of the corresponding log file, wherein the above-mentioned process identification may be the process number in the log file. The above-mentioned thread identification coding information may be information obtained by encoding the thread identification of the corresponding log file, wherein the above-mentioned thread identification may be the thread number in the log file.
[0064] S120: Compress sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information.
[0065] Exemplarily, after the information processing device obtains at least one file information, since each file information in the at least one file information may include at least two sub-file information, and the dimensions of the at least two sub-file information are different from each other, the information processing device can compress the sub-file information with the same dimension in the obtained at least one file information according to different dimensions to obtain at least two first compressed information.
[0066] S130: Store at least two pieces of first compression information in a dictionary, and generate file association information according to the at least two pieces of first compression information and the compressed log file.
[0067] In some embodiments, the file association information is used to characterize the relationship between at least two first compressed information in the dictionary and the compressed log file, wherein the compressed log file includes a compressed file obtained by compressing the log file.
[0068] Exemplarily, after obtaining at least two pieces of first compressed information, the information processing apparatus can store the at least two pieces of first compressed information in a dictionary, and can generate file association information according to the at least two pieces of first compressed information and the compressed log file.
[0069] In an embodiment of the present disclosure, at least one file information can be obtained, each file information being a portion of information in a log file corresponding to the file information. Each file information includes at least two sub-file information of time code information, state change code information, process identification code information, and thread identification code information. The dimensions of the at least two sub-file information are different from each other. Based on this, the sub-file information of the at least one file information having the same dimension can be compressed according to different dimensions to obtain at least two first compressed information. The at least two first compressed information can then be stored in a dictionary. Based on the at least two first compressed information and the compressed log file, file association information is generated. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes the compressed log file after the log file is compressed. In this way, not only can the compressed storage of partial information in the log file significantly reduce the amount of data required for storage and save storage costs, but the generation of file association information can also reduce the inconvenience of subsequent log file search and use, thereby improving the efficiency of log file search.
[0070] In one embodiment, the above-mentioned step S110 may specifically include the following steps:
[0071] Get at least one log file;
[0072] For each log file in the at least one log file, extract information to be processed from the log file;
[0073] The information to be processed is encoded to obtain file information corresponding to the log file.
[0074] The information to be processed in each log file may include at least one of the time information, state change information, process ID, and thread ID in the log file, which is not limited here.
[0075] Exemplarily, the information processing device can obtain at least one log file, and for each log file in the at least one log file, extract information to be processed from the log file, and then encode the information to be processed to obtain file information corresponding to the log file.
[0076] In this embodiment, part of the information in the log file can be extracted from the obtained log file as information to be processed, and the information to be processed can be encoded so as to obtain valid file information corresponding to the log file, which also facilitates the subsequent compression and storage of part of the information in the log file, which not only reduces the amount of data that needs to be stored, but also saves storage costs.
[0077] In one embodiment, if the information to be processed includes state change information, the step of extracting the information to be processed from each log file in at least one log file may specifically include the following steps:
[0078] For each log file in the at least one log file, state change information is extracted from the log file using a text matching algorithm.
[0079] The text matching algorithm may be a related algorithm selected based on actual needs and is not limited here.
[0080] Exemplarily, when the information to be processed includes state change information, the information processing apparatus can extract corresponding state change information from each log file in at least one log file using a text matching algorithm.
[0081] In this embodiment, when the information to be processed includes state change information, the corresponding state change information can be extracted from each log file by using a text matching algorithm, thereby effectively improving the efficiency of information acquisition.
[0082] In some embodiments, as shown in FIG2 , before the above-mentioned S120 , the information processing method provided by the embodiment of the present disclosure may further include the following steps:
[0083] S210, obtaining at least one file information, which is the same as the above-mentioned S110 and will not be described in detail here.
[0084] S220: Use a first compression tool to compress at least one piece of file information to obtain second compressed information corresponding to the at least one piece of file information.
[0085] Each of the second compressed information may include dimension information corresponding to at least two sub-file information in the corresponding file information. That is, each of the second compressed information includes dimension information obtained by compressing at least two sub-file information among the time code information, state change code information, process ID code information, and thread ID code information in the corresponding file information.
[0086] In addition, the first compression tool may be a lightweight compression tool, which is not limited here. For example, it may be the clpir tool in the CLP tool, which is used to perform preliminary compression on file information.
[0087] Exemplarily, after acquiring at least one piece of file information, the information processing apparatus can use a first compression tool to compress the at least one piece of file information to obtain second compressed information corresponding to the at least one piece of file information.
[0088] Based on this, the above S120 may specifically include the following steps:
[0089] S230 : For each dimension information in the at least two dimension information, extract the dimension information from each second compressed information in the at least one second compressed information to obtain a dimension information group corresponding to the dimension information.
[0090] The dimension information group includes at least one dimension information, that is, at least one dimension information corresponding to the at least one file information.
[0091] Exemplarily, after obtaining at least one second compressed information, since each second compressed information may include at least two dimensional information, the information processing apparatus may extract the dimensional information from each of the at least two dimensional information to obtain a dimensional information group corresponding to the dimensional information. In this manner, at least two dimensional information groups may be obtained.
[0092] S240: Use a second compression tool to compress the at least two dimensional information groups respectively to obtain first compressed information corresponding to the at least two dimensional information groups respectively.
[0093] In some embodiments, the first compression tool and the second compression tool are different compression tools. For example, the second compression tool may integrate a compression algorithm that is an improved and optimized version of Deflate and LZ77 (Lempel-Ziv-Markov chain-Algorithm, Lzma).
[0094] Exemplarily, after obtaining at least two dimensional information groups, the information processing device can use a second compression tool to compress the at least two dimensional information groups respectively to obtain first compressed information corresponding to the at least two dimensional information groups, that is, it can obtain at least two first compressed information.
[0095] S250: Store at least two first compression information in a dictionary, and generate file association information based on the at least two first compression information and the compressed log file. This is the same as the content of S130 above and will not be described in detail here.
[0096] In this embodiment, the file information can be preliminarily compressed first, and then the compressed information obtained after the preliminarily compression can be compressed again according to different information categories, so that in the subsequent search process, the corresponding compressed information can be searched faster based on actual search needs, and the corresponding log file can be found faster, thereby improving the search efficiency.
[0097] In one embodiment, before S120, the information processing method may further include the following steps:
[0098] For each file information in the at least one file information, if the log file corresponding to the file information meets the target condition, the file information is compressed based on the streaming compression method to obtain at least one third compressed information.
[0099] In some embodiments, the target condition may include that the log file is a log file generated by an Android system. In addition, each of the third compressed information may include sub-compressed information obtained by compressing at least two sub-file information of the time code information, the state change code information, the process identifier code information, and the thread identifier code information in the corresponding file information using a streaming compression method.
[0100] Exemplarily, after obtaining at least one file information, the information processing device can, for each file information in the at least one file information, compress the file information based on a streaming compression method when the log file corresponding to the file information is a log file generated by the Android system, to obtain third compressed information corresponding to the at least one file information, that is, to obtain at least one file information.
[0101] Based on this, the above S120 may specifically include the following steps:
[0102] The sub-compressed information with the same dimension in the at least one third compressed information is compressed according to different dimensions to obtain at least two first compressed information.
[0103] Exemplarily, after obtaining at least one third compressed information, the information processing apparatus can compress sub-compressed information with the same dimension in the at least one third compressed information according to different dimensions to obtain at least two first compressed information.
[0104] It should be noted that the method of compressing sub-compressed information with the same dimension in at least one third compressed information according to different dimensions to obtain at least two first compressed information is similar to the above-mentioned method of compressing sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information, and will not be elaborated here.
[0105] In this embodiment, when the log file meets the target conditions, the file information corresponding to the log file can be compressed using a streaming compression method. In this way, compared with traditional compression methods, the file information can be compressed in real time while being read, and compression does not need to be performed after the log writing is completed, which facilitates reducing the amount of data subsequently written to the disk.
[0106] Based on this, the embodiment of the present disclosure further provides a flowchart of an information processing method, as specifically shown in FIG3 . The information processing method may be executed by an information processing device, and the method may specifically include the following steps:
[0107] S310: Obtain information to be searched.
[0108] In some embodiments, the aforementioned information to be searched may include at least one of time search information, state change search information, process search identifier, and thread search identifier.
[0109] It should be noted that the aforementioned time search information may include information such as the timestamp of a log file; the aforementioned state change search information may include state information in the log file that represents the system's operation. For example, if the system is a vehicle system, the state change search information may include information such as vehicle speed and acceleration, without further limitation. The process search identifier may be the process number in the log file, and correspondingly, the thread search identifier may be the thread number in the log file, without further elaboration.
[0110] S320: Search the dictionary to obtain target compressed information corresponding to the information to be searched, and search for a compressed log file corresponding to the target compressed information based on the file association information stored in the dictionary.
[0111] In some embodiments, the file association information may be used to characterize the relationship between at least two first compressed information and a compressed log file, wherein the compressed log file includes a compressed file obtained by compressing the log file.
[0112] In addition, the above-mentioned dictionary can be understood as a storage space. Based on this, in some embodiments, the above-mentioned dictionary can store at least two first compressed information, and the at least two first compressed information can include at least two of time compressed information, state change compressed information, process ID compressed information, and thread ID compressed information.
[0113] It should be noted that the compressed time information may include at least one compressed time information, and each compressed time information corresponds to a log file. Accordingly, the compressed state change information may include at least one compressed state change information, and each compressed state change information corresponds to a log file. The compressed process ID information may include at least one compressed process ID, and each compressed process ID corresponds to a log file. The compressed thread ID information may include at least one compressed thread ID. Each compressed thread ID may correspond to a log file.
[0114] It should also be noted that the above-mentioned target compressed information may include part of the information in the log file corresponding to the information to be searched, that is, the target compressed information may include at least one of the compressed information of the time information, state change information, process ID and thread ID in the log file corresponding to the information to be searched.
[0115] Exemplarily, since at least two first compression information are stored in the dictionary, and the at least two first compression information can include at least two of time compression information, state change compression information, process ID compression information and thread ID compression information, the information processing device can, after obtaining the information to be searched, search for the target compression information corresponding to the information to be searched from at least one first compression information stored in the dictionary, and can search for the compressed log file corresponding to the target compression information based on the file association information stored in the dictionary.
[0116] Since a log file may include information such as time information, state change information, process ID, and thread ID, as well as information such as static text content, in some embodiments, the file association information mentioned above may also include sub-association information, which is used to characterize the relationship between at least two first compressed information and the compressed text. Based on this, the above-mentioned S320 may include the following steps:
[0117] The target compressed information corresponding to the information to be searched is searched from the dictionary, and the compressed text corresponding to the target compressed information is searched based on the file association information stored in the dictionary; the compressed log file is determined based on the target compressed information and the compressed text.
[0118] Exemplarily, after obtaining the information to be searched, the information processing device can search for the target compression information corresponding to the information to be searched from the dictionary, and based on the file association information stored in the dictionary, search for the compressed text corresponding to the target compression information, and then determine the compressed log file based on the target compression information and the compressed text.
[0119] S330: decompress the compressed log file to obtain a log file corresponding to the information to be searched.
[0120] Exemplarily, after obtaining the compressed log file through searching, the information processing apparatus can decompress the compressed log file to obtain a log file corresponding to the information to be searched.
[0121] In the embodiment of the present disclosure, it is possible to obtain information to be searched, which may include at least one of time search information, state change search information, process search identifier, and thread identifier search information. Based on this, it is possible to search for target compressed information corresponding to the information to be searched from a dictionary, and based on the file association information stored in the dictionary, it is possible to search for a compressed log file corresponding to the target compressed information, and then it is possible to decompress the compressed log file to obtain a log file corresponding to the information to be searched. In this way, it is possible to directly search for compressed information stored in the dictionary, and decompress the compressed log file obtained by the search to obtain the corresponding log file. Compared with the prior art method of decompressing all log files and then searching for the corresponding log file during the log file search process, this method not only reduces the inconvenience in the use of log files, but also reduces the search resources for log files, and improves the search efficiency and flexibility of log files.
[0122] Based on the same inventive concept, an embodiment of the present disclosure further provides an information processing device, and the information processing device provided by the embodiment of the present disclosure can be described in detail with reference to FIG. 4 .
[0123] FIG4 is a schematic structural diagram of an information processing device provided by an embodiment of the present disclosure.
[0124] As shown in FIG. 4 , the information processing device 400 may include: an acquisition module 410 , a compression module 420 and a storage module 430 .
[0125] An acquisition module 410 is configured to acquire at least one piece of file information, where each piece of file information is a portion of information in a log file corresponding to the file information, and each piece of file information includes at least two sub-file information of time code information, state change code information, process ID code information, and thread ID code information, where the dimensions of the at least two sub-file information are different from each other;
[0126] The compression module 420 is configured to compress sub-file information with the same dimension in at least one file information according to different dimensions to obtain at least two first compressed information;
[0127] The storage module 430 is configured to store at least two first compressed information in a dictionary, and generate file association information based on the at least two first compressed information and the compressed log file. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes a compressed file after the log file is compressed.
[0128] In one embodiment, the above-mentioned information processing device may further include an extraction module and an encoding module.
[0129] an acquisition module, specifically configured to acquire at least one log file;
[0130] an extraction module configured to extract, for each log file in the at least one log file, information to be processed from the log file, the information to be processed including at least two of time information, state change information, a process identifier, and a thread identifier;
[0131] The encoding module is configured to encode the information to be processed to obtain file information corresponding to the log file.
[0132] In one embodiment, the information to be processed includes state change information. Based on this, the extraction module is specifically configured as follows:
[0133] For each log file in the at least one log file, state change information is extracted from the log file using a text matching algorithm.
[0134] In one embodiment, the compression module is further configured to compress the at least one file information using the first compression tool to obtain second compressed information corresponding to the at least one file information, each second compressed information including dimension information corresponding to at least two sub-file information;
[0135] Based on this, the information processing device provided by the embodiment of the present disclosure may further include an extraction module.
[0136] an extraction module configured to extract dimension information from each second compressed information in the at least one second compressed information for each dimension information in the at least two dimension information, and obtain a dimension information group corresponding to the dimension information, the dimension information group including at least one dimension information;
[0137] The compression module is specifically configured to use a second compression tool to compress at least two dimensional information groups respectively to obtain first compressed information corresponding to the at least two dimensional information groups respectively. The first compression tool and the second compression tool are different compression tools.
[0138] In one embodiment, the compression module is further configured to compress each file information in the at least one file information based on a streaming compression method to obtain at least one third compressed information, where each third compressed information includes at least two sub-compressed information of different dimensions, if the log file corresponding to the file information meets a target condition; the target condition includes that the log file is a log file generated by the Android system;
[0139] Based on this, the compression module is specifically configured to compress sub-compressed information with the same dimension in at least one third compressed information according to different dimensions to obtain at least two first compressed information.
[0140] In an embodiment of the present disclosure, at least one file information can be obtained, each file information being a portion of information in a log file corresponding to the file information. Each file information includes at least two sub-file information of time code information, state change code information, process identification code information, and thread identification code information. The dimensions of the at least two sub-file information are different from each other. Based on this, the sub-file information of the at least one file information having the same dimension can be compressed according to different dimensions to obtain at least two first compressed information. The at least two first compressed information can then be stored in a dictionary. Based on the at least two first compressed information and the compressed log file, file association information is generated. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes the compressed log file after the log file is compressed. In this way, not only can the compressed storage of partial information in the log file significantly reduce the amount of data required for storage and save storage costs, but the generation of file association information can also reduce the inconvenience of subsequent log file search and use, thereby improving the efficiency of log file search.
[0141] The various modules in the information processing device provided in the embodiments of the present disclosure can be used to implement the method steps of the embodiments shown in Figure 1 or Figure 2, and can achieve the corresponding technical effects. For the sake of brevity, they will not be repeated here.
[0142] Based on the same inventive concept, an embodiment of the present disclosure further provides an information processing device, and the information processing device provided by the embodiment of the present disclosure can be described in detail with reference to FIG. 5 .
[0143] FIG5 is a schematic structural diagram of an information processing device provided by an embodiment of the present disclosure.
[0144] As shown in FIG. 5 , the information processing device 500 may include: an acquisition module 510 , a search module 520 and a decompression module 530 .
[0145] An acquisition module 510 is configured to acquire information to be searched, where the information to be searched includes at least one of time search information, state change search information, process search identifier, and thread search identifier;
[0146] A search module 520 is configured to search the dictionary for target compressed information corresponding to the information to be searched, and search for the compressed log file corresponding to the target compressed information based on file association information stored in the dictionary, the file association information being used to characterize a relationship between at least two first compressed information and the compressed log file;
[0147] The decompression module 530 is configured to decompress the compressed log file to obtain the log file corresponding to the information to be searched;
[0148] The dictionary stores at least two pieces of first compression information, and the at least two pieces of first compression information include at least two of time compression information, state change compression information, process ID compression information, and thread ID compression information.
[0149] In the embodiment of the present disclosure, it is possible to obtain information to be searched, which may include at least one of time search information, state change search information, process search identifier, and thread identifier search information. Based on this, it is possible to search for target compressed information corresponding to the information to be searched from a dictionary, and based on the file association information stored in the dictionary, it is possible to search for a compressed log file corresponding to the target compressed information, and then it is possible to decompress the compressed log file to obtain a log file corresponding to the information to be searched. In this way, it is possible to directly search for compressed information stored in the dictionary, and decompress the compressed log file obtained by the search to obtain the corresponding log file. Compared with the prior art method of decompressing all log files and then searching for the corresponding log file during the log file search process, this method not only reduces the inconvenience in the use of log files, but also reduces the search resources for log files, and improves the search efficiency and flexibility of log files.
[0150] The various modules in the information processing device provided in the embodiment of the present disclosure can be used to implement the method steps of the embodiment shown in Figure 3 and can achieve the corresponding technical effects. For the sake of brevity, they will not be repeated here.
[0151] FIG6 shows a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure.
[0152] The electronic device may include a processor 601 and a memory 602 storing computer program instructions.
[0153] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.
[0154] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 602 may include removable or non-removable (or fixed) media. Where appropriate, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.
[0155] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0156] The processor 601 implements any one of the information processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .
[0157] In one example, the electronic device may further include a communication interface 603 and a bus 610. As shown in FIG6, the processor 601, the memory 602, and the communication interface 603 are connected via the bus 610 and communicate with each other.
[0158] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present disclosure.
[0159] Bus 610 includes hardware, software or both, and the components of online data flow metering equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. Where appropriate, bus 610 may include one or more buses. Although the present disclosure describes and shows a specific bus, the present disclosure considers any suitable bus or interconnection.
[0160] In addition, in conjunction with the information processing method in the above embodiment, the present disclosure can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the information processing method provided in the present disclosure can be implemented.
[0161] The embodiments of the present disclosure further provide a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the information processing method provided by the embodiments of the present disclosure.
[0162] It should be understood that the present disclosure is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present disclosure is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present disclosure.
[0163] The functional blocks shown in the above structured block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in unit, a function card or the like. When implemented in software, the elements of the present disclosure are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0164] It should also be noted that the exemplary embodiments described in this disclosure describe methods or systems based on a series of steps or devices. However, this disclosure is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.
[0165] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable information processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable information processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0166] The above is only a specific embodiment of the present disclosure. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present disclosure is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the scope of protection of the present disclosure.
[0167] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element. Industrial Applicability
[0168] The present disclosure provides an information processing method, apparatus, device, vehicle, medium, and program product, comprising: obtaining at least one file information; compressing sub-file information of the at least one file information having the same dimension according to different dimensions to obtain at least two first compressed information; storing the at least two first compressed information in a dictionary, and generating file association information based on the at least two first compressed information and a compressed log file, wherein the file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file, wherein the compressed log file includes a compressed file obtained by compressing the log file. Embodiments of the present disclosure reduce the amount of data required for storage while also alleviating the inconvenience of subsequently searching and using the log file.
Claims
1. An information processing method, comprising: Acquire at least one file information, where each file information is partial information in a log file corresponding to the file information, and each file information includes at least two sub-file information of time code information, state change code information, process ID code information, and thread ID code information, and dimensions of the at least two sub-file information are different from each other; compressing sub-file information with the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information; The at least two first compressed information are stored in a dictionary, and file association information is generated based on the at least two first compressed information and the compressed log file. The file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file. The compressed log file includes a compressed file after the log file is compressed.
2. The method according to claim 1, wherein The obtaining of at least one piece of file information includes: Get at least one log file; For each log file in the at least one log file, extracting information to be processed from the log file, the information to be processed including at least two of time information, state change information, a process identifier, and a thread identifier; The information to be processed is encoded to obtain file information corresponding to the log file.
3. The method according to claim 2, wherein: The information to be processed includes state change information; The extracting information to be processed from each log file in the at least one log file includes: For each log file in the at least one log file, the state change information is extracted from the log file using a text matching algorithm.
4. The method according to any one of claims 1 to 3, wherein: Before compressing the sub-file information of the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information, the method further includes: Using a first compression tool to compress the at least one file information respectively to obtain second compressed information corresponding to the at least one file information respectively, each second compressed information including dimension information corresponding to the at least two sub-file information respectively; The compressing sub-file information with the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information includes: For each dimension information in the at least two dimension information, extract the dimension information from each second compressed information in the at least one second compressed information to obtain a dimension information group corresponding to the dimension information, where the dimension information group includes the at least one dimension information; A second compression tool is used to compress at least two dimensional information groups respectively to obtain first compressed information corresponding to the at least two dimensional information groups respectively, and the first compression tool and the second compression tool are different compression tools.
5. The method according to any one of claims 1 to 3, wherein: Before compressing sub-file information having the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information, the method further includes: For each piece of file information in the at least one piece of file information, if the log file corresponding to the piece of file information meets the target condition, compress the piece of file information based on a streaming compression method to obtain at least one piece of third compressed information, each piece of the third compressed information including at least two sub-compressed information of different dimensions; The target condition includes that the log file is a log file generated by an Android system; The compressing sub-file information with the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information includes: The sub-compressed information with the same dimension in the at least one third compressed information is compressed according to different dimensions to obtain at least two first compressed information.
6. An information processing method, the method comprising: Acquire information to be searched, where the information to be searched includes at least one of time search information, state change search information, process search identifier, and thread search identifier; Searching a dictionary to obtain target compressed information corresponding to the information to be searched, and searching for a compressed log file corresponding to the target compressed information based on file association information stored in the dictionary, the file association information being used to characterize a relationship between the at least two first compressed information and the compressed log file; Decompressing the compressed log file to obtain a log file corresponding to the information to be searched; The dictionary stores at least two pieces of first compression information, and the at least two pieces of first compression information include at least two of time compression information, state change compression information, process ID compression information, and thread ID compression information.
7. An information processing device, comprising: an acquisition module configured to acquire at least one piece of file information, each piece of file information being partial information in a log file corresponding to the file information, each piece of file information including at least two sub-file information of time code information, state change code information, process identifier code information, and thread identifier code information, wherein dimensions of the at least two sub-file information are different from each other; a compression module configured to compress sub-file information with the same dimension in the at least one file information according to different dimensions to obtain at least two first compressed information; The storage module is configured to store at least two first compressed information in the dictionary, and generate file association information based on the at least two first compressed information and the compressed log file, wherein the file association information is used to characterize the relationship between the at least two first compressed information in the dictionary and the compressed log file, and the compressed log file includes a compressed file after the log file is compressed.
8. An information processing device, comprising: an acquisition module configured to acquire information to be searched, wherein the information to be searched includes at least one of time search information, state change search information, process search identifier, and thread search identifier; a search module configured to search the dictionary for target compressed information corresponding to the information to be searched, and search for a compressed log file corresponding to the target compressed information based on file association information stored in the dictionary, the file association information being used to characterize a relationship between the at least two first compressed information and the compressed log file; a decompression module configured to decompress the compressed log file to obtain a log file corresponding to the information to be searched; The dictionary stores at least two pieces of first compression information, and the at least two pieces of first compression information include at least two of time compression information, state change compression information, process ID compression information, and thread ID compression information.
9. An electronic device, the device comprising a processor and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the information processing method according to any one of claims 1 to 6.
10. A vehicle comprising at least one of the following: The information processing device according to claim 7; The information processing device according to claim 8; The electronic device according to claim 9.
11. A computer storage medium comprising a memory configured to store computer program instructions; and a processor configured to read and execute the computer program instructions stored in the memory to perform the information processing method according to any one of claims 1 to 6.
12. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the information processing method according to any one of claims 1 to 6.
Citation Information
Patent Citations
OLAP storage query system capable of efficiently fusing CDN real-time logs and storage query method of OLAP storage query system
CN112667590A
Log file compression method and system, equipment and medium
CN113407508A
Log processing method and related product
CN117056299A
Compression and distribution of meteorological data using machine learning
US20240014827A1