Data management method, electronic equipment and storage medium

Abstract

The invention discloses a data management method, electronic equipment and a storage medium, and the data management method comprises the steps: obtaining to-be-merged multi-dimensional feature data in a first database, the type of the first database being a relational database; associating and merging the feature data of each dimension into a temporary table in the first database, wherein the data in the temporary table can be periodically cleared; the feature data in the temporary table is pulled into a feature wide table in a second database, the type of the second database is different from that of the first database, and the feature wide table can be used for full-text retrieval. According to the scheme, the feature data of different dimensions can be merged.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a data management method, an electronic device, and a storage medium. Background Art

[0002] Currently, in most databases, the feature data of each dimension is stored in a single table, and the feature data of different dimensions is relatively independent.

[0003] Currently, the feature data of different dimensions can be stored in a relational database and multi-table queries can be performed through associated queries, but the efficiency of such associated queries is too low. Summary of the Invention

[0004] This application provides at least a data management method, an electronic device, and a storage medium.

[0005] This application provides a data management method, including: obtaining the feature data of multiple dimensions to be merged in a first database, where the type of the first database is a relational database; associatively merging the feature data of each dimension into a temporary table in the first database, and the data in the temporary table can be cleared regularly; pulling the feature data in the temporary table into a feature wide table in a second database, where the type of the second database is different from that of the first database, and the feature wide table can be used for full-text retrieval.

[0006] This application provides a data management method device, including: a data acquisition module, an associative merging module, and a data transfer module; the data acquisition module is used to obtain the feature data of multiple dimensions to be merged in a first database, where the type of the first database is a relational database; the associative merging module is used to associatively merge the feature data of each dimension into a temporary table in the first database, and the data in the temporary table can be cleared regularly; the data transfer module is used to pull the feature data in the temporary table into a feature wide table in a second database, where the type of the second database is different from that of the first database, and the feature wide table can be used for full-text retrieval.

[0007] This application provides an electronic device, including a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the above data management method.

[0008] This application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above data management method is implemented.

[0009] In the above solution, after the feature data to be merged in the relational database is associated and merged into a temporary table in the first database, the feature data in the temporary table is pulled into the feature wide table in the second database, thereby realizing the merging of feature data tables in different dimensions, and the feature wide table in the second database can support full-text retrieval. Compared with using the first database for association query, this solution can improve the query efficiency.

[0010] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with this application and, together with the specification, are used to explain the technical solutions of this application.

[0012] Figure 1 is a schematic flowchart of an embodiment of the data management method of this application;

[0013] Figure 2 is Figure 1 a sub - flowchart of step S11 in

[0014] Figure 3 is another schematic flowchart of an embodiment of the data management method of this application;

[0015] Figure 4 is a schematic structural diagram of an embodiment of the data management method device of this application;

[0016] Figure 5 is a schematic structural diagram of an embodiment of the electronic device of this application;

[0017] Figure 6 is a schematic structural diagram of an embodiment of the computer - readable storage medium of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following will describe the solutions of the embodiments of this application in detail with reference to the accompanying drawings of the specification.

[0019] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to thoroughly understand this application.

[0020] As used herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, both A and B exist simultaneously, and B exists alone. Additionally, in this text, the character " / " generally indicates an "or" relationship between the associated objects before and after. Furthermore, the term "multiple" in this text means two or more than two. Additionally, the term "at least one" in this text means any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set composed of A, B, and C.

[0021] In some application scenarios, the data management method provided in this application can be applied to the computer field, etc. The execution entity for implementing the data management method described in this application can be a data management method device, an electronic device, etc. For example, the data management method device can be set in a terminal device, a server, or other processing devices. Among them, the terminal device can be a data management device, an electronic device, a user equipment (UE), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the data management method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0022] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the data management method of this application.

[0023] As Figure 1 shown, the data management method provided by the embodiments of the present disclosure may include the following steps:

[0024] Step S11: Obtain feature data of multiple dimensions to be merged in the first database.

[0025] The executing entity can be a server or a terminal device. The executing entity can establish connections with the first database and the second database, be able to store newly added data into the first database and the second database, and update the feature wide table in the second database through each feature table in the first database. The type of the first database is a relational database. Among them, the feature data in multiple dimensions can include but are not limited to target object information, vehicle information, MAC address (Media Access Control), RFID tag (Radio Frequency Identification), and telephone number, etc. The MAC address can be used for network communication and device identification and is a unique identifier of a network device. The RFID tag can be used to identify and track objects with RFID tags and has wide applications in multiple fields such as logistics, access control, and asset management. Feature data in different dimensions can carry different types of information of the target object. For example, the target object information can carry the facial features of the target object, while the vehicle information can carry the external features of the vehicle. Among them, the target object information and the vehicle information can be stored in an unstructured manner, while the MAC address, RFID tag, and telephone number... Among them, the feature data of each dimension can be stored in a table, or the feature data of the same dimension exists in a set manner.

[0026] Step S12: Correlate and merge the feature data in each dimension into a temporary table in the first database.

[0027] The data in the temporary table can be cleared regularly. That is, it is possible to regularly obtain the feature data in multiple dimensions to be merged in the first database and then perform correlation and merging. After pulling the feature data in the temporary table to the feature wide table in the second database each time of correlation and merging, the feature data in the temporary table can be deleted. The way of correlation and merging can be to use the join method for merging. Exemplarily, taking the feature data in one dimension as vehicle information and the feature data in another dimension as MAC address as an example, the way of merging the feature information in these two dimensions can be: Taking the example that the feature data belonging to the same dimension can be stored in a table, first determine the valid records of each table, that is, the records whose status is not 0. One record can correspond to one feature data. And it is possible to limit the time interval to be merged. For example, it is necessary to correlate and merge the data between January 1, 1970 and January 1, 2018, and the way of correlation and merging can be to perform an associative connection (join) on the identification fields (card_no) of each record in the two feature tables, that is, only keep the records with the same card_no in the two tables. Exemplarily, in the case where the feature data is the MAC address, the card_no can be the device identifier, and the connection between the two tables can be achieved through the identification field. Exemplarily, the specific implementation method can refer to the following example:

[0028]

[0029] Among them, the successfully associated records are merged and written into a temporary table, and the unsuccessfully associated records can also be written into the temporary table. In some application scenarios, two successfully associated records in the temporary table can be written into the temporary table as one record, that is, these records include the feature data of at least two dimensions that are successfully associated. In other application scenarios, if a piece of feature data in a certain dimension is unsuccessfully associated with the feature data of other dimensions, this piece of feature data can also be directly written into the temporary table, so that a certain record in the temporary table may also only contain the feature data of one dimension.

[0030] Step S13: Pull the feature data in the temporary table into the feature wide table in the second database.

[0031] The type of the second database is different from that of the first database, and the feature wide table can be used for full-text retrieval. Exemplarily, the first database can be a Greenplum database with an mpp architecture, while the second database can be an Elasticsearch distributed full-text retrieval engine, and Elasticsearch also has database functions. The pulling method can be pulling one by one, paging pulling, or pulling a preset number of records each time.

[0032] In the above solution, after the feature data to be merged in the relational database is associated and merged into the temporary table in the first database, the feature data in the temporary table is pulled into the feature wide table in the second database, thereby realizing the merging of feature data tables in different dimensions, and the feature wide table in the second database can support full-text retrieval. Compared with using the first database for association query, this solution can improve the query efficiency.

[0033] In some embodiments, please refer to Figure 2 , the above step S11 may include the following steps:

[0034] Step S111: Obtain the merged records of the feature tables in different dimensions in the first database.

[0035] Among them, the feature tables in different dimensions respectively record the feature data corresponding to the dimensions. The merged records may not be stored in the first database. For example, the merged records can be stored in a third database, and the third database can be a MySQL database. For example, after each association and merge, the merged records of each feature table and the merge time can be recorded in the MySQL database for incremental merge the next time.

[0036] Step S112: Determine the incremental data in each feature table based on the merged records of each feature table.

[0037] Specifically, the incremental records in each feature table can be determined by comparing the storage time of each feature data in the feature tables with the latest merge time of each feature table. For example, if the storage time of a certain feature data is later than the latest merge time of the feature table where it is located, then this feature data is regarded as the incremental data in this feature table.

[0038] Among them, the above step S112 can specifically include the following steps: for each feature table, in response to the existence of merge records in the feature table, the new data in the feature table is used as incremental data, or in response to the non-existence of merge records in the feature table, all the data in the feature table is used as new data.

[0039] That is, in the case where there are merge records in the feature table, the feature data with a storage time later than the latest merge time is used as new data. In the case where there are no merge records in the feature table, all the feature data in the feature table is used as new data.

[0040] Step S113: Based on the incremental data in each feature table, obtain the feature data of multiple dimensions to be merged.

[0041] In some application scenarios, the incremental data in each feature table can be directly used as the feature data of multiple dimensions to be merged, that is, the incremental data in each feature table can be respectively associated and merged to obtain the associated merged data and write it into a temporary table. In some application scenarios, for each dimension of the feature table, the feature data of other dimensions to be merged can be determined respectively, so that the incremental data in each dimension table is respectively associated and merged with the incremental feature data in the feature tables of other dimensions, or the incremental data in each dimension table is respectively associated and merged with all the data in the feature tables of other dimensions.

[0042] Among them, the feature data of multiple dimensions to be merged include the multiple multi-dimensional feature data to be merged corresponding to each feature table. On this basis, the above step S113 may include the following steps: For each feature table, the incremental data in the feature table and the full-volume data in other feature tables in the first database are used as the multiple-dimensional feature data to be merged corresponding to the feature table. Other feature tables may be any one or more feature tables other than the feature table where the incremental data is located in the first database. Exemplarily, there are three-dimensional feature tables in the first database, namely feature table a, feature table b, and feature table c. The incremental data in feature table a is a1, the incremental data in feature table b is b1, and the incremental data in feature table c is c1. For feature table a, feature table b and feature table c may be other feature tables. For feature table b, feature table a and feature table c may be other feature tables. For feature table c, feature table a and feature table b may be other feature tables. For feature table a, it is necessary to perform associative merging of a1 with the full-volume data in feature table b and feature table c respectively. The result of the associative merging may be successful association or failed association. For feature table b, it is necessary to perform associative merging of b1 with feature table a and feature table c respectively. The result of the associative merging may be successful association or failed association. For feature table c, it is necessary to perform associative merging of c1 with feature table a and feature table b respectively. The result of the associative merging may be successful association or failed association.

[0043] In some embodiments, the target feature table in the first database may be determined. For this target feature table, all other-dimensional feature tables in the first database may be used as other feature tables. For the feature tables other than the target feature table in the first database, the other feature tables corresponding to this feature table may only refer to the target feature table, that is, the feature tables of each dimension may only be associatively merged with the target feature table. The determination method of the target feature table may include but is not limited to being determined by a specified method or being recommended from each target feature table. Exemplarily, the target feature table for this time may be determined according to the associative merging results among the feature tables of each dimension in the previous associative merging. For example, the association degree between each dimension feature and other feature tables is determined respectively. For each feature table, the association degrees between the feature table and each feature table are weighted and fused to obtain the association value of this feature table. The feature table with a higher association value may be selected as the target feature table. Among them, the feature table corresponding to the highest association value is selected as the target feature table.

[0044] Among them, the priorities of feature tables in different dimensions can be set, and the correlation merging order between each feature table can be determined according to the priorities of feature tables in different dimensions. For example, if the priority of feature table a > the priority of feature table b > the priority of feature table c, then the incremental data in feature table a is preferentially correlated and merged with the full amount data in feature tables b and c, and then the incremental data in feature table b is correlated and merged with the full amount data in feature tables a and c, and finally the incremental data in feature table c is correlated and merged with the full amount data in feature tables a and b. After determining the correlation merging result of a feature table, it is judged whether the correlation merging results of all feature tables are obtained. If the judgment result is no, the correlation merging result of the feature table in the next dimension is continuously determined.

[0045] In the above solution, when performing correlation merging between feature tables, the timed incremental merging method can be adopted, that is, each time of merging, the merging record and the merging time are recorded in the mysql database, and incremental merging is performed according to the record the next time of merging, so as to reduce problems such as poor performance and insufficient memory caused by full amount merging.

[0046] In some embodiments, the above step S13 may include the following steps: pull feature data from the temporary table in pages according to the identifier of the temporary table into the memory; store the feature data in the memory into the feature wide table and delete the feature data that has been stored in the feature wide table in the memory.

[0047] In some application scenarios, the paging method of the temporary table can be determined according to the correlation merging result between the feature tables in each dimension and the memory size, and the temporary table is paged according to the paging method to obtain several pages. Exemplarily, the maximum data volume of the feature data pulled into the memory each time can be determined according to the memory size, and the paging method of the temporary table can be determined according to the maximum data volume and the data volume size of the correlation merging result corresponding to each feature table. Among them, the correlation merging result corresponding to each feature table includes the correlation merging result obtained by correlating and merging the incremental data of the feature table and the full amount data in other feature tables in the first database. For example, if there are a total of n correlation merging results corresponding to feature tables in the temporary table, the division method of each correlation merging result can be determined according to the ratio between the maximum data volume and the data volume of the correlation merging result corresponding to each feature table, and used as the division method of the temporary table. In other application scenarios, the temporary table can be directly paged according to the ratio between the data volume of the obtained temporary table and the maximum data volume. In other application scenarios, the temporary table can also be paged according to other feasible methods. For example, it can also be paged in the way of a preset data volume or paged according to the paging method of a regular document to obtain several paged data.

[0048] Each time, the paged data can be first fetched into the memory of the execution device of the data management method, and then the data in the memory can be fetched into the second database, and then the paged data that has been fetched into the second database in the memory can be deleted. Among them, before fetching the data in the memory into the second database, the paged data in the memory can be first translated or field-mapped, and then the paged data that has been translated or field-mapped can be fetched into the second database. Of course, it can also be translated or field-mapped before the paged data is fetched into the memory.

[0049] When generating the feature wide table in the second database through the feature table of the first database during the creation or update process of the feature wide table provided in this embodiment, a temporary table containing all the fields of the feature table can be created in the first database, and the temporary table needs to set an auto-incrementing id. The feature data in the feature table in the first database is merged into this temporary table, and then the data is fetched page by page into the wide table of the second data through the auto-incrementing ID of the temporary table. In addition, the method of using the temporary table can reduce the poor performance of feature table merging caused by paging problems.

[0050] In some embodiments, the feature data belonging to the same dimension in the first database is stored in the same feature table, and the feature data belonging to the same dimension in the second database is stored in the same feature table. Before performing the above step S11, the following steps may further be included: obtaining the data to be stored in the database; storing the data to be stored in the database into the feature table corresponding to the dimension in the first database; and storing the data to be stored in the database into the feature table corresponding to the dimension in the second database; and performing deduplication processing on the feature tables of each dimension in the first database and the second database.

[0051] Optionally, the method of storing the data to be stored in the database into the feature table corresponding to the dimension in the first database and storing the data to be stored in the database into the feature table corresponding to the dimension in the second database may be: determining the dimension to which the data to be stored in the database belongs, and then storing the data to be stored in the database into the feature table under the corresponding dimension. The method of performing deduplication processing on the feature tables of each dimension in the first database and the second database may include, but is not limited to, determining the similarity of different feature data in the feature table, and performing deduplication processing on the feature data whose similarity reaches a certain threshold. Among them, storing the data to be stored in the database into the first database and the second database at the same time can achieve the effect of mutual backup of the two databases. In the case where one of the databases is abnormal, the data in the other database can be used to restore the data in the abnormal database.

[0052] Among them, the above method of storing the data to be stored in the database into the feature table corresponding to the dimension in the first database may include:

[0053] In response to the data volume of the data to be stored in the database being greater than or equal to the preset data volume, add the data to be stored in the database to the insertion queue. Read the data in the insertion queue and store it in the first database and the second database.

[0054] The preset data volume can be set according to requirements. Among them, the data to be stored in the database may contain feature data of different dimensions, that is, feature data of multiple dimensions can be stored in the database at the same time. If there are a large number of updated data, the insertion queue can be passed through the Message Queue (MQ). Among them, after the data to be stored in the database is successfully stored, the association and merging operation of each feature table is performed at intervals. For example, one hour after the data is stored, the timed association and merging task will overwrite the new feature data into the feature wide table of the second database.

[0055] In some embodiments, please refer to Figure 3 , the data management method may further include the following steps:

[0056] Step S21: Receive a data query request.

[0057] The way to receive the data query request may include but is not limited to obtaining it through the display interface of the execution device, or being sent to the execution device by other devices.

[0058] Step S22: Determine the preset query method corresponding to the data query request.

[0059] Among them, different databases correspond to different preset query methods. The preset query method corresponding to the data query request may include but is not limited to simple query, complex query, and full-text retrieval, etc. The process of determining the preset query method corresponding to the data query request may be to determine the structural characteristics of the data query request, and determine the preset query method according to the structural characteristics. Among them, different structural characteristics correspond to different preset query methods. Exemplarily, the data query requests corresponding to different preset query methods may have different structural characteristics. The structural characteristics may include but are not limited to the connection relationship between the information carried in the data query request and / or the type of information carried.

[0060] Step S23: In response to the preset query method being full-text retrieval, call the interface of the second database to query the feature wide table and obtain the data query result of the data query request.

[0061] In some embodiments, the data management method may further include the following steps: in response to the preset query method being a simple query, call the interface of the second database to query the feature tables of each dimension in the second database and obtain the data query result of the data query request; or, in response to the preset query method being a complex query, call the interface of the first database to query the feature tables of each dimension in the first database and obtain the data query result of the data query request. The complex query includes an association query.

[0062] That is, the database interfaces to be called for different preset query methods may be different. In this solution, for data query, different query schemes and query interfaces are customized for different queries. When querying the feature table, if it is a simple query, directly query the feature table in the second database, thereby improving the query efficiency and avoiding duplicate data. For complex join queries, they can be queried by querying the first database. For the case of full-text retrieval, the feature wide table in the second database can be queried through an externally provided interface to implement the retrieval function.

[0063] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an embodiment of the data management method device of the present application. The data management method device 40 can execute the above data management method. The data management method device 40 includes a data acquisition module 41, a correlation and merging module 42, and a data transfer module 43; the data acquisition module 41 is used to acquire feature data of multiple dimensions to be merged in the first database, and the type of the first database is a relational database; the correlation and merging module 42 is used to correlate and merge the feature data of each dimension into a temporary table in the first database, and the data in the temporary table can be cleared regularly; the data transfer module 43 is used to pull the feature data in the temporary table into the feature wide table in the second database, the type of the second database is different from the type of the first database, and the feature wide table can be used for full-text retrieval.

[0064] In the above solution, after the feature data to be merged in the relational database is correlated and merged into the temporary table in the first database, the feature data in the temporary table is pulled into the feature wide table in the second database, thereby realizing the merging of feature data tables of different dimensions, and the feature wide table in the second database can support full-text retrieval. Compared with using the first database for join queries, this solution can improve the query efficiency.

[0065] Among them, the functions of each module can be referred to in the embodiments of the data management method, and will not be elaborated here.

[0066] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of an embodiment of the electronic device of the present application. The electronic device 50 includes a memory 51 and a processor 52. The processor 52 is used to execute the program instructions stored in the memory 51 to implement the steps in any of the above embodiments of the data management method. In a specific implementation scenario, the electronic device 50 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 50 may also include carrying devices such as a laptop computer, a tablet computer, etc., which are not limited here.

[0067] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps in any of the above-described embodiments of the data management method. The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with the ability to process signals. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 52 may be implemented jointly by integrated circuit chips.

[0068] In the above solution, after the feature data to be merged in the relational database is correlated and merged into a temporary table in the first database, the feature data in the temporary table is pulled into the feature wide table in the second database, thereby realizing the merging of feature data tables in different dimensions, and the feature wide table in the second database can support full-text retrieval. Compared with using the first database for correlation query, this solution can improve the query efficiency.

[0069] Please refer to Figure 6 , Figure 6 is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 60 stores program instructions 601 thereon, and when the program instructions 601 are executed by the processor, the steps in any of the above-described embodiments of the data management method are implemented.

[0070] In the above solution, after the feature data to be merged in the relational database is correlated and merged into a temporary table in the first database, the feature data in the temporary table is pulled into the feature wide table in the second database, thereby realizing the merging of feature data tables in different dimensions, and the feature wide table in the second database can support full-text retrieval. Compared with using the first database for correlation query, this solution can improve the query efficiency.

[0071] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0072] The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments. For their similarities, reference can be made to each other. For the sake of brevity, they will not be elaborated herein again.

[0073] In several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. For another image position, the coupling or direct coupling or communication connection shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0074] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

Claims

1. A data management method, characterized in that, Including: Obtain feature data of multiple dimensions to be merged in the first database, where the type of the first database is a relational database; Associate and merge the feature data of each dimension into a temporary table in the first database, and the data in the temporary table can be cleared regularly; Pull the feature data in the temporary table into a feature wide table in the second database, where the type of the second database is different from that of the first database, and the feature wide table can be used for full-text retrieval.

2. The method according to claim 1, wherein The obtaining the feature data of multiple dimensions to be merged in the first database includes: Obtain the merge records of the feature tables of different dimensions in the first database; Based on the merge records of each feature table, determine the incremental data in each feature table; Based on the incremental data in each feature table, obtain the feature data of multiple dimensions to be merged, and the incremental data in the feature table includes the data that has never been added to the temporary table in the feature table.

3. The method according to claim 2, wherein The determining the incremental data in each feature table based on the merge records of each feature table includes: For each feature table, in response to the existence of merge records in the feature table, use the new data in the feature table as the incremental data, or in response to the non-existence of merge records in the feature table, use the full amount of data in the feature table as the new data; Wherein, the feature data of multiple dimensions to be merged includes the multiple multi-dimensional feature data to be merged corresponding to each feature table, and the obtaining the feature data of multiple dimensions to be merged based on the incremental data in each feature table includes: For each feature table, use the incremental data in the feature table and the full amount of data in other feature tables in the first database as the feature data of multiple dimensions to be merged corresponding to the feature table.

4. The method according to any one of claims 1 to 3, characterized in that The pulling the feature data in the temporary table into the feature wide table in the second database includes: According to the identifier of the temporary table, page and pull the feature data from the temporary table into the memory; Store the feature data in the memory into the feature wide table and delete the feature data that has been stored in the feature wide table in the memory.

5. The method according to any one of claims 1 to 3, characterized in that The feature data belonging to the same dimension in the first database is stored in the same feature table, and the feature data belonging to the same dimension in the second database is stored in the same feature table. Before obtaining the feature data of multiple dimensions to be merged in the first database, the method further includes: Obtain data to be warehoused; Store the data to be warehoused into the feature table corresponding to the dimension in the first database; and store the data to be warehoused into the feature table corresponding to the dimension in the second database; Perform deduplication processing on the feature tables of each dimension in the first database and the second database.

6. The method according to claim 5, wherein The storing the data to be warehoused into the feature table corresponding to the dimension in the first database includes: In response to the data volume of the data to be warehoused being greater than or equal to the preset data volume, add the data to be warehoused to the insertion queue; Read the data in the insertion queue and store it in the first database and the second database.

7. The method according to claim 5, wherein The method further includes: Receive a data query request; Determine the preset query method corresponding to the data query request, where different preset query methods correspond to different databases; In response to the preset query method being full-text retrieval, call the interface of the second database to query the feature wide table, and obtain the data query result of the data query request.

8. The method according to claim 7, wherein The method further includes: In response to the preset query method being a simple query, call the interface of the second database to query the feature tables of each dimension in the second database, and obtain the data query result of the data query request; Or, in response to the preset query method being a complex query, call the interface of the first database to query the feature tables of each dimension in the first database, and obtain the data query result of the data query request, where the complex query includes an association query.

9. An electronic device, characterized in that, It includes a memory and a processor, and the processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images