This invention relates to the field of
data processing technology, providing a log
processing method and
system. The method includes: recording the location information of sub-variables that need to be compressed in a log block as key positions and corresponding
metadata of the encoded
stream into a preset
data structure based on a preset template; compressing the log block into different storage units based on the preset
data structure; locating the storage unit corresponding to the search keyword based on the distribution of the location vectors corresponding to each log block on the location matrix, and completing the retrieval of log entries based on the matching result of the search keyword and the storage unit, where the location vector is the matching result of the search keyword and the decompressed preset template. This invention efficiently extracts dynamic patterns from logs through preset templates and stores
log data as fine-grained units, thereby improving log compression bandwidth; it uses vectorization technology to achieve low-latency retrieval of fine-grained units, thus providing effective support for log analysis and
system modeling.