A data query method and device based on dynamic switching of data sources

Abstract

The application discloses a data query method based on dynamic switching of data sources, comprising the following steps: obtaining target data and mapping the target data to a first database; calculating and aggregating prepared data in the first database and writing the prepared data into a second database; when the writing into the second database is completed, switching mapping is performed; and the target data is mapped to the second database for ring selection query of the target data. The application further discloses a data query device based on dynamic switching of data sources. The application firstly maps ring selected target data to the first database for storage, and controls switching mapping to be converted to mapping to the second database, at this time, the target data is queried from the second database, and since the switching between the first database and the second database is non-inductive updating switching, data updating is realized, and the new data can be used without the awareness of the querying party.

Description

Technical Field

[0001] This invention relates to the field of data update technology, and specifically to a data query method and apparatus based on dynamically switching data sources. Background Technology

[0002] With the rapid development of enterprises, the volume of data is growing exponentially. Data updates are time-consuming; structural updates require adjustments to code and mappings, increasing the risk of disruption. This leads to a decline in service endpoints and user experience. During data or structural updates, users may encounter defects such as empty, abnormal, or incomplete data, requiring them to wait for completion before returning to normal.

[0003] When users raise new requirements that necessitate updating the prepared data, which may involve hundreds of thousands, millions, or even more data points, it is impossible to update them all at once. In such cases, there will be situations where some data has been updated and some data has not been updated. This data state is called an intermediate state (incomplete update). If selection is made in this state, it will cause errors in the selection results.

[0004] Therefore, we need to develop a data update solution that can update data seamlessly without affecting the user experience. Based on the above requirements, and referencing routing concepts, we achieve a seamless switching effect by switching alias mapping data. Summary of the Invention

[0005] The technical problem this invention aims to solve is to address the shortcomings of the prior art by providing a data query method and apparatus based on dynamically switching data sources. By using abstract aliases to mask data locations, the mapping and switching achieve the purpose of accessing updated data, allowing the querying party to continue using the new data without being aware of it.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A data query method based on dynamically switching data sources includes the following steps:

[0008] Acquire the target data and map it to the first database;

[0009] The prepared data in the first database is calculated and aggregated, and then written into the second database;

[0010] Once the second database write is complete, switch the mapping.

[0011] The target data is mapped to a second database for targeted data selection and querying.

[0012] Furthermore, the step of switching the mapping after the second database is written specifically includes: after the second database is written, using a message queue to notify the mapping switch, switching the mapping from the first database to the second database.

[0013] Furthermore, the step of acquiring target data and mapping the target data to the first database specifically includes: selecting target users by data aliases and mapping the prepared data of the target users to the first database.

[0014] Furthermore, the step of selecting target users by data aliases and mapping the prepared data of the target users to the first database also includes: if new data needs to be added to the selected prepared data, then the first database is directly updated.

[0015] Furthermore, the prepared data consists of behavioral and business data obtained through calculation and aggregation.

[0016] Furthermore, the data aliases are aliases, routes, identifiers, tags, and proxies.

[0017] This invention discloses a data query device based on dynamically switching data sources, comprising:

[0018] The first database is used to obtain the first data after the target data is mapped.

[0019] The second database is used to store the second data formed by calculating and aggregating the prepared data in the first database;

[0020] The switching module is used to convert the first database mapped to the target data into a second database.

[0021] The query module maps data aliases to a second database to perform targeted user queries.

[0022] The beneficial effects of this invention are:

[0023] This invention first maps the selected target data to a first database for storage. Then, it calculates and aggregates the first data in the first database and stores it in a second database. After the second data in the second database is written, the mapping is switched to the second database through control. At this time, the target data can be queried from the second database. Since the switching between the first and second databases is a seamless update, the access data is updated, and the queryer is unaware of the new data and can continue to use it. Attached Figure Description

[0024] Figure 1 This is a flowchart of a data query method based on dynamically switching data sources proposed in this invention;

[0025] Figure 2 This is a structural diagram of a specific embodiment of the present invention;

[0026] Figure 3 This is a structural diagram of a data query device based on dynamically switching data sources proposed in this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0028] The specific embodiments of the present invention are described below with reference to the accompanying drawings and examples:

[0029] like Figure 1 As shown, the present invention discloses a data query method based on dynamically switching data sources, comprising the following steps:

[0030] Step 101: Obtain the target data and map the target data to the first database;

[0031] Step 102: Calculate and aggregate the prepared data in the first database and write it into the second database;

[0032] Step 103: After the second database is written, switch the mapping.

[0033] Step 104: Map the target data to the second database to perform a selection query of the target data.

[0034] In this embodiment of the invention, the selected target data is first mapped to a first database for storage. Then, the first data in the first database is calculated and aggregated and stored in a second database. After the second data in the second database is written, the mapping is switched to the second database through control. At this time, the target data can be queried from the second database. Since the switching between the first and second databases is a seamless update, the access data is updated, and the queryer is unaware of the new data and can continue to use it.

[0035] In step 101, target data is acquired and mapped to the first database; specifically, this includes: selecting target users by data aliases and mapping the prepared data of the target users to the first database.

[0036] The target data is masked by using abstract aliases, thereby mapping and changing the data to achieve the purpose of accessing and updating the data.

[0037] The process includes: selecting target users using data aliases and mapping the prepared data of the target users to a first database; and updating the first database directly if new data needs to be added to the prepared data for selection. When new data is added to the selection, the first database is directly updated. Since updating the first database takes a long time, potentially an hour or more, intermediate states may occur during this period, affecting the selection results. This invention eliminates the need to re-select target data from massive amounts of data, saving time on data updates.

[0038] In step 103, after the second database is written, a mapping switch is performed; specifically, after the second database is written, a message queue is used to notify the mapping switch, and the mapping of the first database is switched to the mapping of the second database.

[0039] The prepared data consists of behavioral and business data derived through calculation and aggregation. Data aliases include aliases, routes, identifiers, tags, and proxies.

[0040] See Figure 2 In this embodiment of the invention, the current selection selects target users by querying data alias A. At this time, the prepared data mapped to A is data 1. If new data needs to be added to the prepared data for selection, data 1 is directly updated, which may take one hour or even longer. During this period, an intermediate state will occur, affecting the selection result. The entire prepared data (including new data) is recalculated and aggregated and written to data 2. At this time, data 1 is still in normal use, and data 2 is not used. After data 2 is prepared, a message queue is used to notify the switching of mappings. Alias ​​A will change from mapping data 1 to mapping data 2. The time from mapping data 1 to mapping data 2 is very short, only a moment. At this time, data 1 becomes unmapped and discarded data, and alias A has been mapped to data 2. (A=1 changes to A=2). Data 1 has become discarded, and the queried data has become the newly updated data 2. Finally, the selection query is for data 2. The alias is still A, but the queried data has been updated, and the user will not notice the existence of the intermediate data.

[0041] See Figure 3 The present invention discloses a data query device based on dynamically switching data sources, comprising:

[0042] The first database 201 is used to obtain the first data after the target data is mapped;

[0043] The second database 202 is used to store the second data formed by calculating and aggregating the prepared data in the first database;

[0044] Switching module 203 is used to convert the first database mapped to the target data into a second database;

[0045] Query module 204 maps data aliases to the second database to perform targeted user selection queries.

[0046] This invention takes user segmentation as an example. It obtains preparatory data for user segmentation, which consists of behavioral and business data derived through calculation and aggregation. Based on the preparatory data and customized segmentation rules, it quickly selects users who meet the rules and makes decisions regarding these target users. After a user submits a rule, the selection calculation is performed, at which point the preparatory data needs to be queried.

[0047] Specific implementation method: Data 1 is aliased. Data 1 is accessed through the alias in the query. Data 1 is calculated and written to data 2. The mapping is switched. At this time, the data alias is changed from mapped data 1 to mapped data 2. The data alias queried by the query party has not changed, but the data access has been changed to data 2.

[0048] This invention uses abstract aliases (including but not limited to aliases, routes, identifiers, tags, and proxies) to mask data locations, thereby mapping and changing access to updated data so that the querying party can continue to use the new data without being aware of it. This solves the problem of intermediate states (such as empty data, abnormal data, incomplete data, etc.) occurring during data updates, switching, and recalculation, allowing users to update data without being noticing.

[0049] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention. These changes involve related technologies well known to those skilled in the art, and all of them fall within the protection scope of the present invention.

[0050] Many other changes and modifications can be made without departing from the concept and scope of this invention. It should be understood that this invention is not limited to the specific embodiments, and the scope of this invention is defined by the appended claims.

Claims

1. A data query method based on dynamic switching of data sources, characterized in that, The method comprises the following steps: acquiring target data and mapping the target data to a first database; performing calculation aggregation on the preparation data in the first database and writing the second data into a second database; performing switching mapping after the second database is written; mapping the target data to the second database for circle selection query of the target data; the switching mapping after the second database is written comprises: using a message queue to notify the switching mapping after the second database is written, and switching the mapping first database to the mapping second database; the acquiring target data and mapping the target data to the first database comprises: selecting a target user through a data alias and mapping preparation data of the target user to the first database; the selecting a target user through a data alias and mapping preparation data of the target user to the first database further comprises: if new data needs to be added to the preparation data for circle selection, directly updating the first database; the preparation data is behavior data and business data obtained through calculation and aggregation, and the data alias is an alias, a route, an identifier, a marker, and an agent.

2. A data query device based on dynamic switching of data sources, characterized in that, The method comprises the following steps: a first database for acquiring first data after the target data is mapped; a second database for storing second data obtained by performing calculation aggregation on the preparation data in the first database; a switching module for converting the first database mapped by the target data into the second database; a query module for mapping a data alias to the second database for circle selection query of a target user.

Images