Data storage method and device, electronic equipment and storage medium

Abstract

The application provides a data storage method and device, electronic equipment and storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring signaling data of a user; constructing a bit slice index (BSI) of a bit map based on the signaling data; and constructing a compact matrix corresponding to the bit slice index. The columns in the compact matrix do not contain all-zero columns. The all-zero column can be understood as no signaling data at this moment, so the all-zero column in the bit slice index is removed and converted into the compact matrix, which can save storage resources. In addition, the bitmap formed by the compact matrix is compressed and stored according to a Roaring Bitmap data structure. Since the Roaring Bitmap data structure can efficiently compress data, the storage resources can be further saved, and the occupation of the signaling data storage resources can be effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data storage method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of mobile Internet, the penetration rate of mobile terminals (such as mobile phones) has reached a very high level. When the mobile phone is turned on, it will continue to interact with the base station. When the mobile phone is attached to the corresponding sector of the base station, it can be considered that the current position of the mobile phone is within the coverage range of the base station, so that the communication between the mobile phone and the base station can be realized by using the cellular data network.

[0003] When the mobile phone is in the state of being attached to the base station, the behaviors such as call, short message and network access will continuously generate signaling data. With the passage of time, these signaling data will continue to accumulate, and thus a large amount of storage resources will be consumed. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a data storage method, device, electronic equipment and storage medium, so as to improve the problem of large storage resource consumption in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a data storage method, which comprises:

[0006] Obtaining signaling data of a user, wherein the signaling data comprises signaling data of the user interacting with a plurality of base stations at a plurality of time points;

[0007] Based on the signaling data, constructing a bit slice index BSI of a bit map, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows are related to the number of time points, and each column in the j columns represents the binary encoding of the base station identifier corresponding to the signaling data at the corresponding time point;

[0008] Constructing a compact matrix corresponding to the bit slice index, wherein the columns in the compact matrix do not contain all-zero columns;

[0009] Extracting the binary encoding corresponding to each row in the compact matrix to form a bitmap, and obtaining a plurality of bitmaps;

[0010] Compressing and storing the plurality of bitmaps according to the Roaring Bitmap data structure.

[0011] In the implementation process, signaling data of a user is acquired, a bit slice index BSI of a bitmap is then constructed based on the signaling data, and a compact matrix corresponding to the bit slice index is then constructed. The columns in the compact matrix do not contain all-zero columns. The all-zero columns can be understood as no signaling data at the moment, so the all-zero columns in the bit slice index are removed and converted into the compact matrix, so as to save storage resources. In the present scheme, the bitmap formed by the compact matrix is compressed and stored according to a Roaring Bitmap data structure. Since the Roaring Bitmap data structure can efficiently compress data, the storage resources can be further saved, and the storage resources of the signaling data can be effectively reduced.

[0012] Optionally, the binary encoding of the base station identifier is obtained by:

[0013] According to the spatial positions of the plurality of base stations, corresponding identifiers are allocated to the base stations, and the identifiers are binary encoded to obtain the binary encoding of the identifier of each base station.

[0014] In the implementation process, the identifiers of the base stations are allocated according to the spatial positions of the base stations, so that the identifiers allocated to adjacent base stations are as close as possible, which facilitates subsequent processing of the compact matrix.

[0015] Optionally, after the compact matrix corresponding to the bit slice index is constructed, the extracting the binary encoding corresponding to each row of the compact matrix is performed before the following steps are performed:

[0016] The values corresponding to the binary encoding of the columns in the compact matrix are sorted in size order to obtain a sorted compact matrix.

[0017] In the implementation process, since the identifiers of the base stations are allocated according to the spatial positions of the base stations, after the values corresponding to the binary encoding of the columns in the compact matrix are sorted in size order, it can be ensured that more bitmaps formed from the compact matrix contain continuous 0 or 1. Therefore, when data compression is performed, the RunContainer in the Roaring Bitmap data structure can be selected for storage. The storage structure can further increase the compression rate of the data, thereby reducing the storage space.

[0018] Optionally, after the sorted compact matrix is obtained, the following steps are further included:

[0019] A first change relationship bitmap between the compact matrix before sorting and the compact matrix after sorting is recorded. The first change relationship bitmap is used to restore the compact matrix after sorting to the compact matrix before sorting when data decompression is performed. In this way, data restoration can be performed when data decompression is performed.

[0020] Optionally, the constructing the bit slice index of the bit map based on the signaling data comprises:

[0021] dividing the signaling data into signaling data of a set time period;

[0022] constructing a bit slice index of a corresponding bit map for the signaling data of the set time period.

[0023] In the above implementation process, one BSI is constructed for the signaling data of each day, so that the number of columns in each BSI is less, and when the bitmap of the compact matrix is extracted, more bitmaps can contain continuous 0 or 1, thereby further improving the compression rate and further saving storage resources.

[0024] Optionally, after the constructing the bit slice index of the bit map based on the signaling data comprises:

[0025] recording a second change relationship bitmap between the bit slice index and the compact matrix, the second change relationship bitmap being used for restoring the compact matrix to the bit slice index when data is decompressed. In this way, data restoration can be performed when data is decompressed.

[0026] In a second aspect, an embodiment of the present application provides a data storage device, the device comprising:

[0027] a data acquisition module configured to acquire signaling data of a user, the signaling data comprising signaling data of interaction between the user and a plurality of base stations at a plurality of time points;

[0028] a BSI construction module configured to construct a bit slice index BSI of a bit map based on the signaling data, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows are related to the number of time points, and each column in the j columns represents binary encoding of a base station identifier corresponding to the signaling data at a corresponding time point;

[0029] a compact matrix construction module configured to construct a compact matrix corresponding to the bit slice index, wherein a column in the compact matrix does not contain all-zero columns;

[0030] a bit map generation module configured to extract binary encoding corresponding to each row in the compact matrix to form a bitmap, and obtain a plurality of bitmaps;

[0031] a storage module configured to compress and store the plurality of bitmaps according to a Roaring Bitmap data structure.

[0032] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores computer readable instructions. When the computer readable instructions are executed by the processor, the steps in the method according to the first aspect are performed.

[0033] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps in the method according to the first aspect are performed.

[0034] In a fifth aspect, a computer program product is provided, which includes computer program instructions. When the computer program instructions are read and executed by a processor, the steps in the method according to the first aspect are performed.

[0035] Other features and advantages of the present application will be further described in the following description with reference to the drawings, and part of the features and advantages will become apparent from the description, or will be learned by practice of the application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0037] Figure 1 A flow chart of a data storage method provided by the embodiments of the present application;

[0038] Figure 2 A schematic diagram of a bit slice index (BSI) provided by the embodiments of the present application;

[0039] Figure 3 A schematic diagram of a compact matrix before sorting provided by the embodiments of the present application;

[0040] Figure 4 A schematic diagram of a compact matrix after sorting provided by the embodiments of the present application;

[0041] Figure 5 A process schematic diagram of restoring the compact matrix after sorting to the compact matrix before sorting provided by the embodiments of the present application;

[0042] Figure 6 A structural block diagram of a data storage device provided by the embodiments of the present application;

[0043] Figure 7 A structural schematic diagram of an electronic device for performing a data storage method is provided in the embodiments of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the accompanying drawings in the embodiments of the present application.

[0045] It should be noted that the terms "system" and "network" in the embodiments of the present application can be used interchangeably. "Multiple" means two or more, and therefore, "multiple" can also be understood as "at least two" in the embodiments of the present application. "And / or" describes the association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / ", if not specially stated, generally represents an "or" relationship between the associated objects before and after it.

[0046] To facilitate the understanding of the embodiments of the present application, first, the Bit Sliced Indexes (BSI) will be introduced as follows.

[0047] Suppose there is a table T, which has a column representing the object instance ID (Identity document) and multiple columns representing the different field attribute values of the object instance. For a certain numerical attribute column G, a BSI object can be used to represent it, that is, BSI is defined as a set composed of multiple Bitmaps, which is used to store the relationship and value data of the ID column and the G column of the table T. Each independent bitmap in the BSI is called a bit slice of the G column of the table T, and the data stored in the bitmap is determined by the encoding strategy of the BSI.

[0048] For example, in the application scenario of the present application, the present application stores signaling data, which includes user ID, base station ID and signaling occurrence time. The signaling data can be stored by using a table T, as shown in Table 1 below. The table T includes the signaling data interaction between each user and the base station at different time.

[0049] Table 1

[0050]

[0051] In actual situations, if there are a large number of users, if the data is directly stored according to such a table structure, a large storage resource will be occupied, which can cause the situation that the storage resource is not enough.

[0052] In order to save resources, the data shown in Table 1 can be converted into BSI for storage in the scheme of the present application, as shown in Table 2 below. Each row can be referred to as a BSI. Since the signaling data can represent the trajectory data of a user, the BSI can store the trajectory information of a user for a lifetime. The time in the table is the time when the signaling occurs.

[0053] Table 2

[0054]

[0055] To improve the above problems, the embodiment of the present application provides a data storage method. The method obtains signaling data of a user, then constructs a bit slice index BSI of a bitmap based on the signaling data, and then constructs a compact matrix corresponding to the bit slice index. The columns in the compact matrix do not contain all-zero columns. An all-zero column can be understood as no signaling data at this time. Therefore, the all-zero columns in the bit slice index are removed and converted into a compact matrix, which can save storage resources. In addition, the bitmap formed by the compact matrix is compressed and stored according to a Roaring Bitmap data structure. Since the Roaring Bitmap data structure can efficiently compress data, the storage resources can be further saved, and the occupation of the signaling data storage resources can be effectively reduced.

[0056] Please refer to Figure 1 , Figure 1 A flowchart of a data storage method provided by the embodiment of the present application is shown in FIG. 1. The method includes the following steps:

[0057] Step S110: Obtain signaling data of a user.

[0058] The signaling data includes signaling data of the user interacting with a plurality of base stations at a plurality of times.

[0059] The user herein can refer to one user or a plurality of users. In the subsequent processing, the signaling data of each user can be distinguished first, and then the subsequent processing is performed on the signaling data of each user.

[0060] The signaling data interaction of the plurality of users with the base stations at different times can be shown in the content of Table 2 above. In some application scenarios, a large amount of signaling data needs to be obtained for analysis, such as user behavior analysis and user trajectory analysis. In this case, the signaling data can be stored according to the method provided by the present application in each time period, such as storing the signaling data after processing according to the method of the present application every month. In this way, the storage resources can be saved.

[0061] The obtaining of the signaling data of the user can refer to reading the signaling data of the user from the storage device, and then storing the signaling data of the user in the storage device after processing the signaling data of the user according to the method of the present scheme. In this way, the storage resources of the storage device can be saved, and the storage device can store more signaling data.

[0062] Step S120: constructing a bit slice index of a bit map based on the signaling data.

[0063] The bit slice index is the BSI described above. If the signaling data is the signaling data of one user, the bit slice index corresponding to the signaling data of the user can be constructed. The bit slice index BSI is a matrix of i rows and j columns. The i rows are related to the number of time instants. Each column in the j columns is the binary encoding of the base station identifier corresponding to the signaling data at the corresponding time instant. i and j are both integers greater than or equal to 1.

[0064] It can be understood that, for the convenience of storage, the identifier of the base station can be converted into binary encoding to obtain the binary encoding corresponding to the identifier of the base station. For example, if the identifier of a base station is 1, the corresponding binary encoding is 000000000001. If the identifier of another base station is 2, the corresponding binary encoding is 000000000010. It can be understood that the number of bits of the binary encoding of the base station identifier can be determined according to the maximum value of the base station identifier in the actual situation. For example, the number of bits can be set to N in the actual situation.

[0065] The plurality of time instants can be defined according to the actual situation. For example, if the time instants are accurate to seconds, each second can be regarded as a time instant. Of course, the time instants can be determined according to the signaling occurrence time in the signaling data. For example, if the signaling occurrence time is a timestamp, the time instants can be accurate to seconds. If the signaling occurrence time is accurate to minutes, the time instants can be accurate to minutes, and so on.

[0066] In some embodiments, if the obtained signaling data is the signaling data of one user in one month, the number of columns of the constructed BSI can be large if the time instants are accurate to seconds, which can not be convenient for subsequent processing. In order to improve the processing efficiency, the signaling data can also be divided into signaling data in a smaller time period. For example, the signaling data can be divided into signaling data in a set time period, and then the bit slice index of the corresponding bit map can be constructed for the signaling data in the set time period. For example, if the set time period is one day, 30 or 31 corresponding BSIs can be constructed for the signaling data of each user in one month, and then each BSI can be stored according to the method provided in the present application.

[0067] Of course, the signaling data can also be divided into smaller time periods. For example, the signaling data can be divided into signaling data every 8 hours, and the specific division period can be flexibly set according to actual needs.

[0068] Taking the signaling data divided by day as an example, if each second is taken as a time point, there are 86400 seconds in a day, and if each second is taken as a time point, there are 86400 time points. When constructing the BSI, the signaling generation time in the signaling data can be converted into a timestamp and placed on the X axis of the BSI, so that there are 86400 time points on the X axis, and then the binary encoding of the base station ID is placed on the value of the BSI, that is, the Y axis, as shown in Figure 2 , where each table represents the binary encoding of a base station ID (the number of bits of the binary encoding of the base station ID shown in Figure 2 is 12), so that the base station ID in the signaling data at each time point is arranged in a hierarchical manner in binary.

[0069] Step S130: constructing a compact matrix corresponding to the bit slice index.

[0070] In the compact matrix, the column does not contain all-zero columns. Understandably, the compact matrix is a simplification of the bit slice index BSI, that is, the all-zero columns in the bit slice index are deleted in the compact matrix.

[0071] Understandably, since the timestamp is used and the unit is second, there is not necessarily signaling data every second, so the base station binary identification at some time points in the BSI can be all 0 (for example, the columns corresponding to time point 1 and time point 4 in Figure 2 are all 0), indicating that there is no signaling data at the time point. The obtained compact matrix can be as shown in Figure 3 .

[0072] Step S140: extracting the binary encoding corresponding to each row in the compact matrix to form a bitmap, and obtaining a plurality of bitmaps.

[0073] The compact matrix is essentially also a BSI, which is a simplified version of the BSI, Figure 2 , and the BSI therein can be understood as a sparse matrix. The binary encoding of each row in the compact matrix can form a bitmap, such as the bitmap formed by the first layer (BSI1) of the compact matrix in Figure 3 is 1010..., the bitmap formed by the second layer (BSI2) is 0111..., and if the compact matrix has N layers, that is, N rows, N bitmaps can be obtained.

[0074] Step S150: compressing and storing the plurality of bitmaps according to the Roaring Bitmap data structure.

[0075] Among them, the main idea of Roaring Bitmap is: 32-bit unsigned integer is divided into buckets according to the high 16 bits, that is, there are at most 2 16 equal to 65536 containers, called containers. When storing data, find the container according to the high 16 bits of the data, and if not found, create a new one, and then put the low 16 bits into the container. That is, a RoaringBitmap is a collection of many containers.

[0076] Here, the container can be divided into ArrayContainer, BitmapContainer and RunContainer.

[0077] When the cardinality of the data in the bucket is not greater than 4096, ArrayContainer is used to store it, which is essentially an ordered array of unsigned short type. The initial length of the array is 4, and it will automatically expand as the data increases, but the maximum length is 4096. In addition, a counter is maintained to record the cardinality in real time.

[0078] When the cardinality of the data in the bucket is greater than 4096, BitmapContainer is used to store it, which is represented by an unsigned long array with a fixed length of 1024, that is, the size of the bitmap is fixed at 2 16 bits (8KB), which also has a counter.

[0079] Run in RunContainer refers to the run length compression algorithm (Run Length Encoding), which has good compression effect on continuous data. Its principle is that for consecutive numbers, only the initial number and the subsequent number are recorded, such as: for a 65536 segment, all 1, record as 1:65536. The performance of this compression algorithm is closely related to the continuity of the data. For 100 consecutive short, it can be compressed from 200 bytes to 4 bytes, so it can achieve good compression of data.

[0080] If the compact matrix has N bitmaps, when each bitmap is compressed and stored, different types of containers can be used to store each bitmap according to its different conditions, such as RunContainer if there are many consecutive 1s or 0s in a bitmap. This way, N bitmaps can be compressed and stored to reduce the occupation of storage resources. RunContainer can achieve efficient compression of data, thereby further saving data storage space.

[0081] In the implementation process, signaling data of a user is acquired, a bit slice index BSI of a bitmap is then constructed based on the signaling data, and a compact matrix corresponding to the bit slice index is then constructed. The columns in the compact matrix do not contain all-zero columns. The all-zero columns can be understood as no signaling data at the moment, so the all-zero columns in the bit slice index are removed and converted into the compact matrix, which can save storage resources. In addition, the bitmap formed by the compact matrix is compressed and stored according to a Roaring Bitmap data structure. Since the Roaring Bitmap data structure can efficiently compress data, the storage resources can be further saved, and the storage resources of the signaling data can be effectively reduced.

[0082] On the basis of the above embodiment, in the manner of obtaining the binary encoding of the base station identifier, the identifiers corresponding to the base stations can be allocated according to the spatial positions of the base stations, and the identifiers are binary encoded to obtain the binary encoding of the identifiers of the base stations.

[0083] It can be understood that when the base stations are initially allocated with the identifiers, the spatial positions of the base stations can be considered, so that the base stations in each city can be arranged together as much as possible, and the base stations close to each other can be arranged together, so that the base stations close to each other can be arranged together as much as possible after being converted into binary encoding. For example, identifiers are allocated to all base stations in the country, a starting city is first selected, and for K base stations in a city, a starting base station is first selected, and an identifier is allocated to the base station, for example, the first base station is allocated with an identifier 0, a second base station close to the first base station is selected, and an identifier 1 is allocated to the second base station, and the base stations are sequentially selected and allocated with identifiers. The principle is to minimize the difference between the identifiers allocated to the base stations close to each other.

[0084] For example, if the movement trajectory of the user does not change much, the values of the binary encodings of the base stations corresponding to each moment in the bit slice index BSI are close, that is, the high layers in the BSI are basically the same, and only the low layers change, so that a large number of values in the bitmap in the compact matrix are continuous 0 or 1 when stored subsequently, for example, Figure 3 The bitmap formed by BSI4-BSI12 in the compact matrix contains continuous 0, so a large number of bitmap can be compressed and stored by using a RunContainer, and the storage resources can be greatly saved.

[0085] In the implementation process, the identifiers corresponding to the base stations are allocated according to the spatial positions of the base stations, so that the identifiers allocated to the base stations close to each other are close as much as possible, which facilitates the subsequent processing of the compact matrix.

[0086] On the basis of the above embodiment, since the full zero column in the bit slice index is removed from the compact matrix, the time points in the compact matrix may not be arranged continuously, at this time, the binary encoding corresponding to the value of the base station is out of order, if directly stored, there may be fewer continuous 0 or 1 bitmap. In order to solve this problem, in the embodiment of the application, the binary encoding corresponding to the value of the column in the compact matrix can be sorted in size order to obtain a sorted compact matrix.

[0087] As Figure 3 In the compact matrix before sorting, the binary encoding corresponding to the value of the column at each time point is 1, 6, 7, and 6 in turn, if sorted in size order, such as from small to large, the sorted value is 1, 6, 6, and 7, of course, it can also be sorted from large to small, and the specific sorting rule can be flexibly set.

[0088] The sorted compact matrix can be as shown in Figure 4 The 0 or 1 of the high layer to the bottom layer is more compact, such as the binary encoding of the base station identifier of the same city, the 0 or 1 of the middle layer is continuous.

[0089] It can be understood that the compression ratio of the three containers is different, and the compression ratio of the RunContainer will be higher, so if the RunContainer is used for storage, the data will be compressed at a higher ratio, thereby saving more storage space.

[0090] Therefore, in order to achieve higher compression ratio of data, the compact matrix is sorted in the embodiment of the application to obtain more continuous 0 or 1 bitmap, so that more bitmap can be selected to store RunContainer when the sorted compact matrix is compressed and stored, so as to save more storage resources.

[0091] In the above implementation process, since the identifier of the base station is allocated according to its spatial position, after sorting the binary encoding corresponding to the value of the column in the compact matrix in size order, more bitmap in the bitmap formed by the compact matrix contains continuous 0 or 1, so when data compression is performed, RunContainer in the Roaring Bitmap data structure can be selected for storage, and the storage structure can further increase the compression ratio of data, thereby reducing the occupation of storage space.

[0092] Based on the above embodiments, after obtaining the sorted compact matrix, a first transformation relationship bitmap between the compact matrix before sorting and the compact matrix after sorting can also be recorded. The first transformation relationship bitmap is used to restore the sorted compact matrix to the compact matrix before sorting during data decompression.

[0093] Continue to refer to Figure 3 and Figure 4 The compact matrix before and after sorting shown are swapped at time 5 and time 6 according to the numerical order of the binary encoding. At this time, the first change relationship bitmap can be represented by a bitmap, which can be represented as (1,2,4,3,...), representing the change of bits. According to the first change relationship bitmap, it can be seen that the order of the 4th column (bit) and the 3rd column (bit) of the compact matrix has changed. The first change relationship bitmap can be recorded as sort_all.

[0094] When decompressing and restoring data, the compressed data can first be restored to a BSI structure, resulting in a sorted compact matrix. Then, based on the first change relation bitmap, the sorted compact matrix can be restored to its original, unsorted compact matrix. For example, if the first and second bits of the compact matrix remain unchanged according to the first change relation bitmap, a new, initially empty matrix can be created. The binary codes of the first and second bits of the sorted compact matrix can then be directly copied over. Next, the position of the third bit's binary code within the sorted compact matrix can be determined. Using the first change relation bitmap, it can be found that the third bit has moved to the fourth bit. Therefore, the binary code of the fourth bit of the sorted compact matrix can be copied to the third bit of the original, unsorted compact matrix. This process continues for subsequent bits. The specific restoration process is as follows: Figure 5 As shown. This facilitates data restoration during data decompression.

[0095] Based on the above embodiments, after constructing the compact matrix corresponding to the bit slice index, a second transformation relationship bitmap between the bit slice index and the compact matrix can also be recorded. The second transformation relationship bitmap is used to restore the compact matrix to the bit slice index during data decompression. The second transformation relationship bitmap can be recorded as bitmap_all.

[0096] like Figure 2 The bit slice index BSI shown changes to Figure 3After the compact matrix is shown, the all-zero column in the bit slice index is removed, and the bitmap corresponding to the bit slice index can be expressed as 011011, wherein 0 indicates that the column corresponding to the bit is all-zero, i.e., no signaling data, and 1 indicates that there is signaling data. After being converted into the second changed relationship bitmap, it can be expressed as (2, 3, 5, 6,...), which indicates that the columns corresponding to the 2nd bit, the 3rd bit, the 5th bit and the 6th bit are non-all-zero columns, and the columns corresponding to the 1st bit and the 4th bit are all-zero columns. Alternatively, the second changed relationship bitmap can also be directly expressed by the bitmap corresponding to the bit slice index, i.e., bitmap_all can be recorded as 011011. When restoring, the second changed relationship bitmap can be used to restore the compact matrix to the bit slice index.

[0097] For example, according to the second changed relationship bitmap, the column corresponding to the 1st bit is an all-zero column. When restoring, an initial matrix with all empty is first constructed, then an all-zero column is added at the 1st bit, the 2nd bit takes the binary code of the 1st bit in the compact matrix, the 3rd bit takes the binary code of the 2nd bit in the compact matrix, an all-zero column is added at the 4th bit, the 5th bit takes the binary code of the 3rd bit in the compact matrix, the 6th bit takes the binary code of the 4th bit in the compact matrix, and the subsequent bits are restored in a similar manner. In this way, data restoration can be performed when data is decompressed.

[0098] When decompressing the compressed data, the compressed BSI data can be first read, and then the first changed relationship bitmap is used to restore it to the compact matrix before sorting (i.e., from Figure 4 to Figure 3 ), and then the second changed relationship bitmap is used to restore it to the bit slice index (i.e., from Figure 3 to Figure 2 ), and then the bit slice index is converted into the original signaling data (i.e., from Figure 2 to Table 1 above).

[0099] Please refer to Figure 6 , Figure 6 for a structure block diagram of a data storage device 200 provided by the embodiment of the present application. The device 200 can be a module, a program segment or code on an electronic device. It should be understood that the device 200 corresponds to the above-mentioned Figure 1 method embodiment, and can perform each step involved in the Figure 1 method embodiment. The specific functions of the device 200 can be referred to the description in the above text. To avoid repetition, the detailed description is appropriately omitted here.

[0100] Optionally, the device 200 includes:

[0101] The data acquisition module 210 is configured to acquire signaling data of a user, wherein the signaling data comprises signaling data of the user interacting with a plurality of base stations at a plurality of time points.

[0102] The BSI construction module 220 is configured to construct a bit slice index BSI of a bitmap based on the signaling data, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows are related to the number of time points, and each column in the j columns represents a binary encoding of a base station identifier corresponding to the signaling data at a corresponding time point.

[0103] The compact matrix construction module 230 is configured to construct a compact matrix corresponding to the bit slice index, wherein the compact matrix does not contain an all-zero column.

[0104] The bitmap generation module 240 is configured to extract a binary encoding corresponding to each row in the compact matrix to form a bitmap, and obtain a plurality of bitmaps.

[0105] The storage module 250 is configured to compress and store the plurality of bitmaps in a Roaring Bitmap data structure.

[0106] Optionally, the binary encoding of the base station identifier is obtained by the following method:

[0107] allocating a corresponding identifier to each base station according to the spatial positions of the plurality of base stations, and performing binary encoding on the identifier to obtain the binary encoding of the identifier of each base station.

[0108] Optionally, the apparatus 200 further comprises:

[0109] The sorting module is configured to sort the numerical values corresponding to the binary encodings of the columns in the compact matrix in size order to obtain a sorted compact matrix.

[0110] Optionally, the apparatus 200 further comprises:

[0111] The recording module is configured to record a first change relationship bitmap between the compact matrix before sorting and the compact matrix after sorting, and the first change relationship bitmap is used to restore the compact matrix after sorting to the compact matrix before sorting during data decompression.

[0112] Optionally, the data acquisition module 210 is configured to divide the signaling data into signaling data of a set time period, and construct a bit slice index of a bitmap corresponding to the signaling data of the set time period.

[0113] Optionally, the apparatus 200 further comprises:

[0114] A recording module is configured to record a second change relationship bit map between the bit slice index and the compact matrix, and the second change relationship bit map is used to restore the compact matrix to the bit slice index during data decompression.

[0115] It should be noted that, for the convenience and brevity of description, the specific working process of the apparatus described above can refer to the corresponding process in the foregoing method embodiments, which will not be described herein.

[0116] Please refer to Figure 7 , Figure 7 A structural schematic diagram of an electronic device for performing a data storage method is provided in the embodiment of the present application, and the electronic device can include at least one processor 310, such as a CPU, at least one communication interface 320, at least one memory 330, and at least one communication bus 340. The communication bus 340 is used to realize the connection and communication between the components. The communication interface 320 of the device in the embodiment of the present application is used to communicate with other node devices. The memory 330 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 330 can also be at least one storage device located away from the aforementioned processor. The memory 330 stores computer readable instructions, and when the computer readable instructions are executed by the processor 310, the electronic device performs the method process shown in the foregoing Figure 1 .

[0117] It can be understood that Figure 7 The structure shown is only schematic, and the electronic device can include more or fewer components than those shown in the foregoing Figure 7 , or have a different configuration from the foregoing Figure 7 . Figure 7 The components shown in the foregoing may be realized by hardware, software, or a combination thereof.

[0118] Figure 1 The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor executes the method process performed by the electronic device in the method embodiment shown in the foregoing .

[0119] The embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method provided by the foregoing method embodiments, for example, including:

[0120] acquire signaling data of a user, the signaling data comprising signaling data of the user interacting with a plurality of base stations at a plurality of time instants;

[0121] construct a bit slice index BSI of a bitmap based on the signaling data, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows are related to a number of the plurality of time instants, and each of the j columns represents a binary encoding of a base station identifier corresponding to the signaling data at a corresponding time instant;

[0122] construct a compact matrix corresponding to the bit slice index, wherein no column in the compact matrix contains all zeros;

[0123] extract a binary encoding corresponding to each row in the compact matrix to form a bitmap, and obtain a plurality of bitmaps;

[0124] store the plurality of bitmaps in a compressed manner according to a Roaring Bitmap data structure.

[0125] In summary, the embodiments of the present application provide a data storage method, device, electronic equipment and storage medium. The method acquires signaling data of a user, and then constructs a bit slice index BSI of a bitmap based on the signaling data. Then, a compact matrix corresponding to the bit slice index is constructed. No column in the compact matrix contains all zeros. All zeros can be understood as no signaling data at the time instant. Therefore, all zero columns in the bit slice index are removed and converted into the compact matrix, so as to save storage resources. In addition, the bitmap formed by the compact matrix is stored in a compressed manner according to a Roaring Bitmap data structure. Since the Roaring Bitmap data structure can efficiently compress data, the storage resources can be further saved, and the storage resources occupied by the signaling data can be effectively reduced.

[0126] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The above-described device embodiments are only schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0127] In addition, the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0128] Furthermore, the functional modules in various embodiments of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0129] In this paper, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0130] The above is only an embodiment of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data storage method, characterized by, The method comprises: acquiring signaling data of a user, the signaling data comprising signaling data of the user interacting with a plurality of base stations at a plurality of time instants; based on the signaling data, constructing a bit slice index BSI of a bitmap, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows being related to the number of time instants, and each of the j columns representing binary encoding of a base station identifier corresponding to signaling data at a corresponding time instant; constructing a compact matrix corresponding to the bit slice index, wherein the columns of the compact matrix do not contain all-zero columns; extracting binary encoding corresponding to each row of the compact matrix to form a bitmap, and obtaining a plurality of bitmaps; storing the plurality of bitmaps in a compressed manner according to a Roaring Bitmap data structure.

2. The method of claim 1, wherein, The binary encoding of the base station identifier is obtained by: allocating identifiers corresponding to the base stations to the base stations according to the spatial positions of the base stations, and binary encoding the identifiers to obtain binary encoding of the identifiers of the base stations.

3. The method of claim 2, wherein, After the compact matrix corresponding to the bit slice index is constructed, before the binary encoding corresponding to each row of the compact matrix is extracted, the method further comprises: ordering the values corresponding to the binary encoding of the columns of the compact matrix in size order to obtain an ordered compact matrix.

4. The method of claim 3, wherein, After the ordered compact matrix is obtained, the method further comprises: recording a first change relationship bitmap between the compact matrix before ordering and the compact matrix after ordering, the first change relationship bitmap being used to restore the compact matrix after ordering to the compact matrix before ordering during data decompression.

5. The method of claim 1, wherein, The method comprises: dividing the signaling data into signaling data of a set time period; for the signaling data of the set time period, constructing a bit slice index of a corresponding bitmap.

6. The method of claim 1, wherein, After the compact matrix corresponding to the bit slice index is constructed, the method further comprises: recording a second change relationship bitmap between the bit slice index and the compact matrix, the second change relationship bitmap being used to restore the compact matrix to the bit slice index during data decompression.

7. A data storage device, characterized by The apparatus comprises: a data acquisition module configured to acquire signaling data of a user, the signaling data comprising signaling data of the user interacting with a plurality of base stations at a plurality of time instants; a BSI construction module configured to construct a bit slice index BSI of a bitmap based on the signaling data, wherein the bit slice index BSI is a matrix of i rows and j columns, the i rows being related to the number of time instants, and each of the j columns representing binary encoding of a base station identifier corresponding to signaling data at a corresponding time instant; a compact matrix construction module configured to construct a compact matrix corresponding to the bit slice index, wherein the columns of the compact matrix do not contain all-zero columns; a bitmap generation module configured to extract binary encoding corresponding to each row of the compact matrix to form a bitmap, and obtain a plurality of bitmaps; a storage module configured to store the plurality of bitmaps in a compressed manner according to a Roaring Bitmap data structure.

8. An electronic device, comprising: A computer program product comprising a computer readable medium having stored thereon computer readable program instructions that, when executed by a processor, implement the method of any of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the method of any of claims 1-6.

10. A computer program product, characterised in that, A computer program product comprising computer program instructions that, when read by a processor and executed, implement the method of any of claims 1-6.

Images