Data processing method and apparatus

Abstract

The invention discloses a data processing method and apparatus. The method comprises the steps of obtaining to-be-combined-and-stored attributes and corresponding attribute values; obtaining preset digital numbers corresponding to the attribute values; performing byte string conversion on the digital numbers according to a first preset rule to obtain corresponding codes; and performing combination and storage on the codes. According to a bit compression-based storage method, the attribute values are combined and stored by utilizing corresponding byte string storage formats through number conversion; and compared with an existing mode of performing simple attribute value splicing storage by using a splicing connector and performing storage by utilizing a hash function, the method has the advantages that the storage space can be greatly saved, so that the server resource waste is reduced and the utilization rate is increased.

Description

technical field [0001] The invention belongs to the technical field of communication, and in particular relates to a data processing method and device. Background technique [0002] In data storage and analysis, multiple attribute values ​​are usually combined and stored. Generally, this storage method is called "multi-value combination". Currently, multi-valued composite storage mostly uses byte[] byte strings. For the sake of illustration, it is assumed that there are the following three ordered attributes and related specific attribute values ​​that need to be stored in combination. [0003] For example, three ordered attributes include operating system (OS, Operating System), Internet Protocol address (IP, Internet Protocol) and uniform resource locator (URL, Uniform ResourceLocator); wherein, the attribute value of OS includes Android, Mac OS X, windows mobile, Symbian, etc., the attribute values ​​of IP include 172.10.1.1, 172.10.1.2...172.10.225.225, and the attribu...

AI Technical Summary

Problems solved by technology

[0003] For example, three ordered attributes include operating system (OS, Operating System), Internet Protocol address (IP, Internet Protocol) and uniform resource locator (URL, Uniform ResourceLocator); wherein, the attribute value of OS includes Android, Mac OS X, windows mobile, Symbian, etc., the attribute values ​​of IP include 172.10.1.1, 172.10.1.2...172.10.225.225, and the attribute values ​​of URL include http: / / www.baidu.com / , http: / / www.google. com.hk / , http: / / www.qq.com / , etc.; in multi-valued combined storage, the more common method is to use "_" as the splicing character to perform simple splicing of attribute values, such as "v=Android_172.10.1 .2_http: / / www.baidu.com / ", but this storage method takes up 40 bytes and requires a large storage space; another method is to use the hash (hash) value of the above simple splicing results as a new The combined value of the method, the storage space of this method depends on the range of the return value of the hash function. If you use the hash function that comes with java, you will get a 32-bit integer number, which can be represented by only 4 bytes, but additional The overhead of maintaining the correspondence between all hash values ​​and original values ​​does not actually save storage space. Therefore, the multi-value combination storage method in the prior art has a large storage space and is likely to cause waste of server resources.

Images