Label data storage method, query method, electronic equipment and storage medium

Abstract

The invention discloses a tag data storage method, a query method, electronic equipment and a storage medium. The tag data storage method comprises the following steps: acquiring current tag data; storing the current label data in a data warehouse, and storing the current label data in a cache; wherein the data warehouse uses a column database, and the cache uses a non-relational database; and when a query request of the label data is received, querying from the cached non-relational database. According to the scheme, the tag data query efficiency can be improved by changing the tag data storage strategy.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a tag data storage method, a query method, an electronic device and a storage medium. Background Art

[0002] In many business scenarios, operational activities are carried out from time to time for target subject groups, such as content recommendation, message push, coupon distribution, advertising, etc. Label data provides data support for operational activities. Specifically, label data represents the subject corresponding to the label, which is obtained by pre-labeling the subject through the subject's basic attributes, behavioral events and other source data. Before the operational activities, the target subject group can be determined by querying the label data.

[0003] However, the existing tag data query solutions have low query efficiency and limited concurrency capabilities, which easily cause time lags in operational activities. Summary of the invention

[0004] The present application at least provides a tag data storage method, an electronic device, and a storage medium, which can solve the problem of low efficiency in existing tag data query.

[0005] The present application provides a label data storage method, including: obtaining current label data; storing the current label data in a data warehouse, and storing the current label data in a cache; wherein the data warehouse uses a columnar database and the cache uses a non-relational database; upon receiving a query request for label data, querying from the cached non-relational database.

[0006] The present application provides a tag data query method, including: receiving a tag data query request; querying in a cache according to the tag data query request, and feeding back corresponding query results; wherein the tag data is stored in the cache according to the above tag data storage method.

[0007] The present application provides a label data storage device, including: an acquisition module, a first storage module, and a second storage module. The acquisition module is used to acquire current label data; the first storage module is used to store the current label data in a data warehouse; the second storage module is used to store the current label data in a cache; the data warehouse uses a columnar database, and the cache uses a non-relational database; when a query request for label data is received, a query is performed from the cached non-relational database.

[0008] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned tag data storage method and / or tag data query method.

[0009] The present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned tag data storage method and / or tag data query method are implemented.

[0010] The above solution changes the storage strategy of label data. In addition to storing label data in the data warehouse, label data is also stored in the cache. The cached label data supports label data queries. This decouples the data warehouse and label data query services. On the one hand, the label data query of the data warehouse is transferred to the cache, and the label data query is provided to the outside through the cache. Since the storage structure of the label data in the cached non-relational database is different from that of the columnar database of the data warehouse, it can reduce query latency, improve query efficiency, and support high-concurrency query requirements. On the other hand, the query pressure of the data warehouse is significantly reduced, thereby preventing the data warehouse from crashing. On the other hand, storing label data in the data warehouse can improve the reusability of label data.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0013] Figure 1 This is a flow chart of an embodiment of a tag data storage method provided by the present application;

[0014] Figure 2 This is a flowchart of another embodiment of the tag data storage method provided by the present application;

[0015] Figure 3 yes Figure 2 The specific process diagram of S121;

[0016] Figure 4 This is a flowchart of another embodiment of the tag data storage method provided by the present application;

[0017] Figure 5 It is a flowchart of an embodiment of a tag data query method provided by the present application;

[0018] Figure 6 This is a schematic diagram of a specific example of the tag data storage method provided by the present application;

[0019] Figure 7 This is a schematic diagram of an application scenario of the tag data storage method provided by the present application;

[0020] Figure 8 It is a structural schematic diagram of an embodiment of a tag data storage device provided by the present application;

[0021] Fig. 9 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0022] Fig.10 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0024] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0025] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] Before introducing the embodiments of the present application, the relevant technical terms, application scenarios and technical issues are explained:

[0027] Subject: refers to any object that has characteristics and whose characteristics can be described, such as customer terminals, orders, and equipment.

[0028] Subject data: used to represent the labels corresponding to the subject. That is, which labels the subject corresponds to, or which labels the subject has.

[0029] Label: used to describe the characteristics of a subject. The dimensions of the description can be attributes, behaviors, etc.

[0030] Label data: used to characterize the subject corresponding to the label. In other words, which subjects the label corresponds to, or which subjects have the label. Label data can be obtained by analyzing the original data generated by the subject.

[0031] Data warehouse: A storage system used to integrate, analyze, and correlate data from various sources to facilitate data-driven high-level decision making and analysis.

[0032] Column-based database: Data is stored in a continuous manner on disk by column. Technologies such as data compression can also be introduced to reduce data storage space, improve data analysis performance, and save storage costs. For example, Clickhouse.

[0033] Bitmap data: A data structure used to store a list of entities. It is composed of binary data and converts the entity ID into a string of 0 or 1 bits to facilitate efficient storage and intersection set operations.

[0034] Subject group: refers to the collection of subjects with the same label category.

[0035] Snapshot technology: A method used in the computer and storage fields to copy or archive data on a system or storage device at a certain moment.

[0036] Operational activities can be applied to many business scenarios such as content recommendation and advertising. Tag data query services are often used in operational activities. Tag data provides data support for operational activities. Through the tag data query service, the target subject group of the operational activities can be determined.

[0037] However, the query efficiency of existing tag data query services is low, resulting in a delay in message push.

[0038] The inventor of this application has found through long-term research that many factors such as the storage location of tag data, selection efficiency (configured selection rule complexity, number of entities), etc. may affect query efficiency. In related technologies, most tag data is calculated and stored in data warehouses such as column-based databases, which cannot provide low latency for query services like non-relational databases, and cannot meet high-concurrency query requirements.

[0039] In view of this, the present application provides a tag data storage method to optimize the tag storage strategy and improve query efficiency.

[0040] Figure 1 1 is a flow chart of an embodiment of a tag data storage method provided by the present application. Figure 1 As shown, in an embodiment, the tag data storage method may include:

[0041] S110: Acquire current tag data.

[0042] The execution end of the tag data storage method embodiment of the present application can be an electronic device with computing capabilities, including but not limited to a mobile phone, a microcomputer, a server, a laptop computer, and a tablet computer.

[0043] A subject is any object that has characteristics that can be described, such as a client terminal, a device, or an order. Tags are used to describe the characteristics of a subject. For example, the attribute tag of a subject is used to describe the attributes of a subject, and the behavior tag of a subject is used to describe the behavior events of a subject. A tag includes at least one category.

[0044] The tag data may characterize the subject corresponding to the tag. In some embodiments, the tag data may characterize the subject corresponding to a single tag, that is, which subjects have a single tag (hereinafter referred to as atomic tag data). For example, subjects A, B, and C have tag 1. In some embodiments, the tag data may characterize the subjects corresponding to multiple tags, that is, which subjects have multiple tags at the same time (hereinafter referred to as combined tag data). For example, subjects A and B have both tags 1 and 2.

[0045] It is understandable that in order to support the label data query business of products or platforms and realize the operational activities of products or platforms, it may be necessary to store label data and update label data during the product development stage or the product iteration and update stage. For example, during the product development stage, big data analysis is performed on the source data of the collected subject to obtain label data, and the label data is stored. During the product iteration and update stage, big data analysis is performed on the source data of the re-collected subject to obtain label data again, and the label data is stored. That is, during the product development stage and even the product iteration and update stage, in order to obtain label data and update label data, it may be necessary to obtain source data multiple times for data analysis to obtain label data.

[0046] The current tag data refers to the tag data obtained this time. This time can be the first time or not.

[0047] S120: Storing the current tag data in the data warehouse, and storing the current tag data in the cache.

[0048] The data warehouse uses a columnar database, and the cache uses a non-relational database; when a query request is received, a query is performed from the cached non-relational database.

[0049] The data warehouse is the underlying storage component of big data, and the cache can be REDIS, etc. Column-based databases and non-relational databases use different storage structures when storing data. Among them, column-based databases use column storage structures to store current tag data, while non-relational databases use key-value storage structures to store current tag data.

[0050] The data structure of the label data stored in the data warehouse can be a bitmap or other data structures, such as a data structure obtained by directly analyzing the source data.

[0051] When storing the current label data in the data warehouse and cache respectively, the current label data can be converted into a column storage structure and a row storage structure respectively and stored in the data warehouse and cache with different storage structures. Alternatively, the current label data can be first stored in the data warehouse, and then the label data can be converted from a column storage structure to a row storage structure and then stored in the cache.

[0052] The cached non-relational database supports the query of tagged data. The query of tagged data refers to the query subject by tag. Compared with columnar databases, non-relational databases can meet the query requirements of low latency and high concurrency.

[0053] It is understandable that in the related art, label data is only stored in the columnar database of the data warehouse, the query efficiency of label data in the data warehouse is low, and the data warehouse cannot withstand high-concurrency query requirements.

[0054] Through the implementation of this embodiment, the storage strategy of the label data is changed. In addition to storing the label data in the data warehouse, the label data is also stored in the cache. The cached label data supports the query of the label data. Thereby, the data warehouse and the label data query business are decoupled. On the one hand, the label data query of the data warehouse is transferred to the cache, and the query of the label data is provided to the outside through the cache. Since the storage structure of the label data in the cached non-relational database is different from that of the columnar database of the data warehouse, it can reduce the query latency, improve the query efficiency, and support high-concurrency query requirements. On the other hand, the query pressure of the data warehouse is significantly reduced, thereby preventing the data warehouse from crashing. On the other hand, storing the label data in the data warehouse can improve the reusability of the label data.

[0055] Figure 2 FIG. 1 is a flow chart of another embodiment of the tag data storage method provided by the present application. Figure 2 As shown, compared with the above-mentioned embodiment, in S120 of this embodiment, storing the current tag data in the data warehouse may further include the following steps:

[0056] S121: Compare the current label data with the historical label data in the data warehouse to determine the difference between the current label data and the historical label data.

[0057] The data warehouse maintains the historical records of the label data. The data warehouse can maintain the historical records in a snapshot table, but is not limited to it. The historical records include the latest n times of label data acquired from the first time to the previous time, 1≤n<m, m refers to the total number of times the label data has been acquired (including the current time).

[0058] The historical tag data is the latest tag data obtained in the historical records, that is, the tag data obtained last time.

[0059] The difference part represents the change of the current tag data relative to the historical tag data, and the difference part includes the newly added subjects and / or deleted subjects corresponding to the tags.

[0060] Combined with reference Figure 3 In some embodiments, S121 may further include S1221-S1222.

[0061] S1221: Obtain bitmap data corresponding to current tag data and snapshot bitmaps corresponding to historical tag data.

[0062] The snapshot bitmap represents the historical tag data in the cache.

[0063] The data structures of the current tag data and the historical tag data are consistent. In this embodiment, the data structures are all bitmaps. It can be understood that the bitmap has the advantage of native logical operations, which can be used to calculate the subsequent difference parts and improve the operation speed. In the case where the current tag data is the tag data obtained for the first time, the snapshot bitmap corresponding to the historical tag data is empty.

[0064] S1222: Perform bitwise operation on the bitmap data and the snapshot bitmap to obtain a difference portion.

[0065] The bitmap data and the snapshot bitmap may be subjected to a forward difference operation to obtain a newly added subject; and the bitmap data and the snapshot bitmap may be subjected to a reverse difference operation to obtain a deleted subject.

[0066] Positive difference operation refers to the difference operation between the bitmap data and the snapshot bitmap. Negative difference operation refers to the difference operation between the snapshot bitmap and the bitmap data.

[0067] S122: Based on the difference, update the historical tag data in the cache.

[0068] It is understandable that the cache stores the latest tag data to support accurate query. When the current tag data arrives, the tag data stored in the cache becomes the historical tag data and needs to be updated to ensure that the tag data stored in the cache is the latest.

[0069] In some embodiments, the difference portion may be converted into a data structure (eg, a tag list) consistent with the historical tag data in the cache, and the historical tag data in the cache may be updated using the converted difference portion.

[0070] Through the implementation of this embodiment, when the cache is updated based on the difference part, except for the first update, the subsequent updates are all for the difference part. That is, except for the first full update, the subsequent updates are incremental updates, so it can improve the update efficiency, avoid the long-term existence of cache data vacuum period, reduce the resources consumed by the update, and improve the timeliness of the difference part for query.

[0071] In addition, the data warehouse maintains the historical records of the label data, which facilitates the tracing of the historical versions of the label data in the cache. If snapshots are used to maintain historical records, the historical snapshot bitmap in the snapshot table can quickly complete the operation and maintenance operations such as restoring and rolling back the label data in the cache, improving the fault tolerance of incremental updates of label data and greatly ensuring the consistency of the main data of the data warehouse and the main data of the cache.

[0072] In some embodiments, after S121 , the following step may further include: using the current label data to update the historical label data in the data warehouse.

[0073] The historical tag data needs to be compared with the current tag data to determine the difference between the current tag data and to achieve incremental update. Therefore, after obtaining the difference in S121, the historical tag data in the data warehouse can be updated with the current tag data for the next comparison.

[0074] As mentioned above, the data warehouse maintains historical records of label data, and the historical records include at least the latest n times of label data acquired. To update the historical label data in the data warehouse, the historical label data in the historical records can be directly replaced with the current label data, or the current label data can be added to the historical records.

[0075] In some embodiments, after S121, the following may be included: based on the difference part, updating the historical subject data in the cache. The subject data represents the label corresponding to the subject. For example, subject A corresponds to label 1 and label 2.

[0076] It is understandable that in some business scenarios, there is a need for subject data query, that is, through subject query tags. In order to enable the cache to also support business data query, business data can be stored in the cache to meet efficient business data query. In some embodiments, business data can be stored in a SET set.

[0077] In some embodiments, during the update process, the cache may, but is not limited to, use an HSET structure to maintain tag data and a HASH structure to maintain subject data, that is, maintain the subject corresponding to the tag and the tag corresponding to the subject.

[0078] Since the difference part can also represent the newly added tags and deleted tags corresponding to the subject, the difference part can be converted into a data structure consistent with the historical subject data in the cache (such as a subject list), and the historical subject data in the cache can be updated using the converted difference part, thereby realizing the update of the tags corresponding to the subject.

[0079] Figure 4 FIG. 1 is a flow chart of another embodiment of the tag data storage method provided by the present application. Figure 4 As shown, in an embodiment, the current tag data includes the current combined tag data, and S110 further includes:

[0080] S111: Acquire the existing atomic label data and the existing combined label data in the data warehouse.

[0081] Atomic label data represents the subject corresponding to a single label, and combined label data represents the subject corresponding to multiple labels. Existing atomic label data includes atomic label data obtained by analyzing data from the first time to the current time. Alternatively, it may include atomic label data obtained by analyzing data from part of the first time to the current time (including the current time). Combined label data includes the combination results of atomic label data and / or combined label data from historical times.

[0082] The way to obtain atomic label data through data analysis is:

[0083] In some embodiments, the source data of the collected subject can be analyzed to obtain a subject list, which includes several subject records, each of which includes a subject and its corresponding label; the subject list is converted into a label list, which includes several label records, each of which includes a subject corresponding to a label; and the label list is stored in a data warehouse. In this case, each label record in the label list is an atomic label data.

[0084] In some embodiments, after obtaining several tag lists, bitmap data of the tag lists can be further obtained and stored in a data warehouse to reduce the storage cost of the data warehouse. In this case, the bitmap data corresponding to each tag record in the tag list is an atomic tag data.

[0085] Each row of the existing atomic label data in the data warehouse consists of LabelId (label ID), Value (label value, i.e. label category), and corresponding subject (such as subject bitmap Bitmap) columns. Each row of the existing combined label data in the data warehouse is stored according to label ID and corresponding subject (such as subject bitmap Bitmap).

[0086] The acquisition method of the existing combined tag data is similar to that of the current combined tag data. Please refer to the relevant explanation of S112, which will not be repeated here.

[0087] It is understandable that after each acquisition of atomic label data and combined label data, they are stored in the data warehouse and can be reused in subsequent combinations to form new combined label data, thereby making the atomic label data and combined label data reusable.

[0088] S112: Based on a preset circle selection rule, existing atomic label data and / or existing combined label data are combined to obtain current combined label data.

[0089] The preset selection rules can be determined based on the requirements of the product development stage and the product iteration and update stage. For example, if the product requirement is to implement operation activities on entities with labels a, label b, and label c, the selection rules are determined to combine the entities corresponding to labels a, label b, and label c.

[0090] The combination may be at least two combinations of existing atomic label data, at least two combinations of existing combined label data, or at least two combinations of existing combined label data.

[0091] In one embodiment, the priority of the existing combined label data participating in the combination is higher than the priority of the existing atomic label data participating in the combination. In other words, after determining the selection rule, determine whether the selection rule corresponds to the existing combined label data, and the corresponding part uses the existing combined label data to participate in the combination, and the remaining part uses the existing atomic label data to participate in the combination. For example, the existing combined label data AB is a combination of the existing atomic label data A belonging to label a and the existing atomic label data B belonging to label b, and the existing atomic label data C belonging to label c, then the existing combined label data AB and the existing atomic label data C are combined. For another example, if there is no existing combined label data between the existing atomic label data A, B and C, then the existing atomic label data A, B and C are combined.

[0092] In some embodiments, the data structure of the atomic label data and the combined label data is a bitmap, which has the advantage of native logical operations and is used for combining existing atomic label data and / or existing combined label data to improve combination efficiency.

[0093] It is understandable that if atomic label data is used directly to support queries, the selection action needs to be performed when the query request arrives, that is, the atomic label data is combined to obtain the combined label data when the query demand arrives, and the target subject group corresponding to the combined label data is returned as the query result. In the case of complex selection rules, it will affect the combination efficiency of the atomic label data, thereby reducing the query efficiency. In addition, the combination is only performed on the atomic label data, resulting in poor reusability of some of the same rules in different selection rules.

[0094] Through the implementation of this embodiment, a label data hierarchical system is constructed, and the atomic label data is combined in advance according to the selection rules to obtain the combined label data, and the combined label data is stored in the cache, and the combined label data is used to support the query. On the one hand, when a query request for label data arrives, there is no need to perform a selection action or only a relatively simple selection action is required to obtain the query result, thereby improving the selection efficiency and further improving the query efficiency. On the other hand, the existing atomic combination data and the existing combined label data are also stored in the data warehouse, which facilitates the reuse of the atomic combination data and the combined label data, and further improves the combination efficiency under complex selection rules.

[0095] Figure 5 1 is a flow chart of an embodiment of a tag data query method provided by the present application. Figure 5 As shown, in an embodiment, the tag data query method may include:

[0096] S210: Receive a query request for tag data.

[0097] The execution end of the tag data query method embodiment of the present application can be an electronic device with computing capabilities, including but not limited to mobile phones, microcomputers, servers, laptops, and tablet computers.

[0098] A query request for tag data refers to a request to query the subject through tags.

[0099] S220: Query the cache according to the query request of the tag data, and feed back the corresponding query result.

[0100] The cache stores the tag data according to the tag data storage method provided in any of the above-mentioned embodiments.

[0101] The query request can be parsed to obtain the tag to be queried; the tag to be queried is used to query the corresponding subject in the cache, and the queried subject is returned as the query result.

[0102] Through the implementation of this embodiment, the tag data in the cache is stored according to the tag data storage method provided by any of the aforementioned embodiments, so the query efficiency can be improved.

[0103] In some embodiments, the cache also stores the main body data and supports querying the main body data. For the description of the main body data, please refer to the above embodiments and will not be repeated here.

[0104] In the case where the cache stores both tag data and subject data, a query request is received; it is determined whether the query request is a query request for tag data or a query request for subject data; in response to a query request for tag data, the tag to be queried in the query request is confirmed; a subject corresponding to the tag to be queried is queried; in response to a query request for subject data, the subject to be queried in the query request is confirmed; and the tag corresponding to the subject to be queried is queried.

[0105] The tag data storage method and tag data query method provided in this application can be applied to operational activity scenarios such as content recommendation, message push, and advertising placement.

[0106] For ease of understanding, the following combination Figure 6 , the tag data storage method provided by the present application is described with a specific example:

[0107] Construct atom label data;

[0108] The atomic label data are combined according to the preset configuration rules to obtain the current combined label data.

[0109] Use the bitmap of the current combined tag data and the latest record bitmap of the snapshot table to perform bit operations to obtain the newly added subject and deleted subject corresponding to the tag respectively;

[0110] Synchronize newly added subjects and deleted subjects to cached tag data and subject data;

[0111] Copies the bitmap of the current combined tag data to the snapshot table.

[0112] For ease of understanding, the following combination Figure 7 , an application scenario of the tag data storage method provided by the present application is described with a specific example:

[0113] like Figure 7 As shown, the tag data storage system includes data source, data warehouse, tag system, buffer layer (cache) and operation platform.

[0114] Obtain source data collected through data sources; data sources include business data, platform tracking points, client tracking points, and third-party collection data. The collected source data includes subject basic data, subject-related data, behavioral event data, and detailed data.

[0115] Store source data into the data warehouse.

[0116] The source data is analyzed to obtain atomic label data (A, B, C) and stored in the data warehouse.

[0117] The atomic label data are combined to obtain the combined label data (AB, BC) and stored in the data warehouse.

[0118] Synchronize the difference between the combined tag data and the latest record in the snapshot table to the cached tag data and body data.

[0119] Add the combined tag data to the snapshot table (combined tag data snapshot archive).

[0120] In operational activity scenarios such as advertising, content recommendation, message push, and coupon distribution, query requests for tag data are responded to through cache.

[0121] Figure 8 Schematic diagram of the structure of an embodiment of a tag data storage device provided by the present application. Figure 8 As shown, the tag data storage device 30 may include an acquisition module 31 , a first storage module 32 , and a second storage module 33 .

[0122] The acquisition module 31 is used to obtain the current label data; the first storage module 32 is used to store the current label data in the data warehouse; the second storage module 33 is used to store the current label data in the cache; wherein the data warehouse uses a columnar database and the cache uses a non-relational database; when a query request for label data is received, a query is performed from the cached non-relational database.

[0123] Through the implementation of this embodiment, the storage strategy of the label data is changed. In addition to storing the label data in the data warehouse, the label data is also stored in the cache. The cached label data supports the query of the label data. Thereby, the data warehouse and the label data query business are decoupled. On the one hand, the label data query of the data warehouse is transferred to the cache, and the query of the label data is provided to the outside through the cache. Since the storage structure of the label data in the cached non-relational database is different from that of the columnar database of the data warehouse, it can reduce the query latency, improve the query efficiency, and support high-concurrency query requirements. On the other hand, the query pressure of the data warehouse is significantly reduced, thereby preventing the data warehouse from crashing. On the other hand, storing the label data in the data warehouse can improve the reusability of the label data.

[0124] The functions of each module can be found in the aforementioned tag data storage method embodiment, which will not be described in detail here.

[0125] Fig. 9 Schematic diagram of the structure of an embodiment of the electronic device of the present application. Fig. 9As shown, the electronic device 40 includes a memory 41 and a processor 42, and the processor 42 is used to execute the program instructions stored in the memory 41 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 40 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 40 may also include a carrier device such as a laptop computer and a tablet computer, which is not limited here.

[0126] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps in any of the above method embodiments. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0127] Through the implementation of this embodiment, the storage strategy of the label data is changed. In addition to storing the label data in the data warehouse, the label data is also stored in the cache. The cached label data supports the query of the label data. Thereby, the data warehouse and the label data query business are decoupled. On the one hand, the label data query of the data warehouse is transferred to the cache, and the query of the label data is provided to the outside through the cache. Since the storage structure of the label data in the cached non-relational database is different from that of the columnar database of the data warehouse, it can reduce the query latency, improve the query efficiency, and support high-concurrency query requirements. On the other hand, the query pressure of the data warehouse is significantly reduced, thereby preventing the data warehouse from crashing. On the other hand, storing the label data in the data warehouse can improve the reusability of the label data.

[0128] See also Fig.10 , Fig.10 The computer-readable storage medium 50 stores program instructions 501, which implement the steps in any of the above method embodiments when executed by a processor.

[0129] Through the implementation of this embodiment, the storage strategy of the label data is changed. In addition to storing the label data in the data warehouse, the label data is also stored in the cache. The cached label data supports the query of the label data. Thereby, the data warehouse and the label data query business are decoupled. On the one hand, the label data query of the data warehouse is transferred to the cache, and the query of the label data is provided to the outside through the cache. Since the storage structure of the label data in the cached non-relational database is different from that of the columnar database of the data warehouse, it can reduce the query latency, improve the query efficiency, and support high-concurrency query requirements. On the other hand, the query pressure of the data warehouse is significantly reduced, thereby preventing the data warehouse from crashing. On the other hand, storing the label data in the data warehouse can improve the reusability of the label data.

[0130] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0131] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. In another image position, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0133] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A tag data storage method, characterized in that: include: Get the current tag data; The current tag data is stored in a data warehouse, and the current tag data is stored in a cache; wherein the data warehouse uses a columnar database, and the cache uses a non-relational database; upon receiving a query request for the tag data, a query is performed from the non-relational database in the cache.

2. The method according to claim 1, characterized in that The storing the current tag data in a cache includes: Comparing the current label data with the historical label data in the data warehouse to determine the difference between the current label data and the historical label data; Based on the difference, the historical tag data in the cache is updated.

3. The method according to claim 2, characterized in that The comparing the current label data with the historical label data in the data warehouse to determine the difference between the current label data and the historical label data includes: Acquire bitmap data corresponding to the current tag data and a snapshot bitmap corresponding to the historical tag data; wherein the snapshot bitmap represents the historical tag data in the cache; The bitmap data and the snapshot bitmap are subjected to bitwise operation to obtain the difference portion.

4. The method according to claim 3, characterized in that The tag data represents the subject corresponding to the tag, the difference part includes the newly added subject and the deleted subject corresponding to the tag, and the bit operation of the bitmap data and the snapshot bitmap to obtain the difference part includes: Performing a forward difference operation on the bitmap data and the snapshot bitmap to obtain a newly added subject; and Perform a reverse difference operation on the bitmap data and the snapshot bitmap to obtain a deletion subject.

5. The method according to claim 2, characterized in that: After the historical tag data in the cache is updated based on the difference part, the method further comprises: The historical label data in the data warehouse is updated using the current label data.

6. The method according to claim 2, characterized in that After comparing the current label data with the historical label data in the data warehouse to determine the difference between the current label data and the historical label data, the method further includes: Based on the difference, the historical subject data in the cache is updated, the subject data representing the tag corresponding to the subject.

7. The method according to claim 6, characterized in that Also includes: During the update process, the cache maintains the tag data using the HSET structure and maintains the body data using HASH.

8. The method according to claim 1, characterized in that The current label data includes the current combined label data, and the obtaining of the current label data includes: Acquire existing atomic label data and existing combined label data in the data warehouse, wherein the atomic label data represents a subject corresponding to a single label, and the combined label data represents a subject corresponding to a plurality of the labels; Based on a preset circle selection rule, the existing atomic label data and / or the existing combined label data are combined to obtain the current combined label data.

9. A tag data query method, characterized in that: include: Receive a query request for tag data; According to the query request of the tag data, a query is performed in the cache, and a corresponding query result is fed back; wherein the tag data is stored in the cache according to any one of the methods as claimed in claims 1-8.

10. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 9.

11. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images