A data partition archiving method, system, device and medium

Abstract

The application provides a data partition archiving method, system, device and medium, relates to the field of intelligent medical treatment, and comprises the following steps: acquiring a data table to be partitioned, generating attribute features according to data items of the data table to be partitioned, and performing feature matching on the attribute features and partition features of each data partition to determine a target partition to which the data table to be partitioned belongs; creating a shadow table in the target partition, and converting data in the data table to be partitioned into the shadow table; and archiving data in the shadow table based on the attribute features of the data table to be partitioned by linking a table space of the target partition, to obtain a target data archive corresponding to the data table to be partitioned. The application can automatically complete partition archiving storage of the data table to be partitioned, and ensures normal and orderly data services.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of smart medical treatment, and in particular to a data partition archiving method, system, device and medium. BACKGROUND

[0002] In a medical-related business database, the data in the data table gradually increases over time. In order to avoid the increasing data affecting the performance of online medical data processing, the data that is relatively old in time is generally saved to an archive database, and then the corresponding data in the business database is deleted. There is no mature tool in the industry that can realize the partition archiving of a database related to digital medical treatment. Basically, it is operated manually by human beings, which brings the risk of human error that cannot be avoided. At present, there are a large number of data tables in the environment of the business, and if only manual operation is used, a large amount of time of the operation and maintenance personnel will be consumed. SUMMARY

[0003] In view of the problems existing in the prior art, the present application provides a data partition archiving method, system, device and medium, which mainly solves the problem that the existing data partition archiving relies on manual operation, has a large workload and has the risk of human error.

[0004] In order to achieve the above-mentioned purpose and other purposes, the technical scheme adopted by the present application is as follows.

[0005] The present application provides a data partition archiving method, comprising:

[0006] obtaining a data table to be partitioned, generating attribute features according to data items of the data table to be partitioned, and performing feature matching on the attribute features and pre-set partition features of each data partition to determine a target partition to which the data table to be partitioned belongs;

[0007] creating a shadow table in the target partition, and converting data in the data table to be partitioned into the shadow table;

[0008] linking a table space of the target partition, archiving data in the shadow table based on the attribute features of the data table to be partitioned, and obtaining a target data archive corresponding to the data table to be partitioned.

[0009] In an embodiment of the present application, obtaining a data table to be partitioned comprises:

[0010] obtaining the access frequency and the latest access time of the data table in each data partition;

[0011] calculating the hotness value of the corresponding data table according to the access frequency and the latest access time;

[0012] If the heat value of the data table is greater than the preset heat upper limit of the data partition to which the data table belongs, the corresponding data table is taken as the data table to be partitioned.

[0013] In an embodiment of the present application, after the heat value of the corresponding data table is calculated according to the access frequency and the latest access time, the method further comprises:

[0014] If the heat value of the data table is less than the preset heat lower limit of the data partition to which the data table belongs, the corresponding data table is taken as the data table to be partitioned.

[0015] In an embodiment of the present application, before the attribute feature is matched with the preset partition feature of each data partition to determine the target partition to which the data table to be partitioned belongs, the method further comprises:

[0016] The heat value of the data table to be partitioned is compared with the preset heat range of each data partition, and if the heat value of the data table to be partitioned is within the preset heat range, the corresponding data partition is included in the first set, so that the data partitions in the first set are used for feature matching to obtain the target partition.

[0017] In an embodiment of the present application, the attribute feature is generated according to the data items of the data table to be partitioned, comprising:

[0018] All data items in the data table to be partitioned are obtained, and data items matched with the basic features in the preset basic feature library are removed, and the attribute feature is obtained by constructing a data sequence from the remaining data items, wherein the basic feature library records general data items as basic features.

[0019] In an embodiment of the present application, before the shadow table is created in the target partition, the method further comprises:

[0020] Each data item of the data table to be partitioned is compared with each data item of each data table in the target partition, and if the data items of the data table to be partitioned are not completely the same as the data items of each data table in the target partition, the shadow table is created.

[0021] In an embodiment of the present application, after the data in the shadow table is archived based on the attribute feature of the data table to be partitioned, the method further comprises:

[0022] A feature word library corresponding to the preset partition feature of the target partition is obtained.

[0023] The multiple sub-features contained in the attribute feature are compared with the feature words in the feature word library in similarity, if the similarity between the sub-feature and the feature word is lower than a preset threshold, the sub-feature is used to update the feature word library, to generate a new partition feature based on the updated feature word library, wherein each sub-feature corresponds to a data item.

[0024] The application further provides a data partition archiving system, comprising:

[0025] A partition determination module is configured to acquire a data table to be partitioned, generate an attribute feature according to data items of the data table to be partitioned, and perform feature matching between the attribute feature and a preset partition feature of each data partition to determine a target partition to which the data table to be partitioned belongs.

[0026] A shadow table creation module is configured to create a shadow table in the target partition, and convert data in the data table to be partitioned into the shadow table.

[0027] A data archiving module is configured to archive data in the shadow table based on the attribute feature of the data table to be partitioned by linking a table space of the target partition, to obtain a target data archive corresponding to the data table to be partitioned.

[0028] The application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data partition archiving method when executing the computer program.

[0029] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the data partition archiving method.

[0030] As described above, the data partition archiving method, system, device and medium provided by the application have the following beneficial effects.

[0031] The application acquires a to-be-partitioned data table, generates attribute features according to data items of the to-be-partitioned data table, and performs feature matching on the attribute features and partition features preset for each data partition to determine a target partition to which the to-be-partitioned data table belongs; a shadow table is created in the target partition, and data in the to-be-partitioned data table is converted into the shadow table; by linking a table space of the target partition, data in the shadow table is archived based on the attribute features of the to-be-partitioned data table, and a target data archive corresponding to the to-be-partitioned data table is obtained. The application determines a data partition that needs to be exchanged according to attribute features of a to-be-partitioned data table, and creates a shadow table to exchange data, thereby automatically completing partition archiving storage of the to-be-partitioned data table, reducing dependence on manual partition archiving, improving data stability and reliability, and ensuring normal and orderly data services. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 An optional architecture diagram of a data partition archiving system in an embodiment of the application.

[0033] Figure 2 A structure diagram of a terminal in an embodiment of the application.

[0034] Figure 3 A flow diagram of a data partition archiving method in an embodiment of the application.

[0035] Figure 4 A module diagram of a data partition archiving system in an embodiment of the application.

[0036] Figure 5 A structure diagram of a device in an embodiment of the application. DETAILED DESCRIPTION

[0037] The embodiments of the application will be described in detail with specific reference to the drawings. Based on the description of the application made herewith, one skilled in the art will easily perceive other advantages and purposes of the application. The application can be implemented or applied in other different embodiments, and the details in the description can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0038] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the application, and the diagrams only show the components related to the application without showing the number, shape and size of the components in actual implementation. The shape, number and proportion of the components in actual implementation can be randomly changed, and the layout pattern of the components can be more complex.

[0039] Referring toFigure 1 , Figure 1 is an optional architecture diagram of the data partition archiving system provided by the embodiment of the present application. The terminal 400 (exemplarily showing the terminal 400-1) is connected to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0040] The terminal 400-1 is a service terminal, and can interact with a patient or a user having a medical consultation demand through the terminal 400-1. The terminal 400-1 provides a business database of a hospital, a drugstore or a pharmaceutical enterprise, and the data in the business database exists in the form of a table. The business database can be divided into multiple data partitions, and a user can interact with the business system on the terminal 400-1 to call data in a corresponding data partition in the business database. Different data partitions can exchange data based on different access frequencies, and data with lower frequency or less frequently used in a data partition can be uploaded to the server 200 through the network 300. The terminal 400-1 can be selected according to the business scene demand, which is not limited here.

[0041] The server 200 is configured to receive data transmitted by the terminal 400-1 and perform archiving storage of the data according to the attribute characteristics of the data table. A user can initiate a data access request to the server 200 through the terminal 400-1 to remotely call the archived data in the server-side database.

[0042] In some embodiments, the server 200 can be a stand-alone physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart voice interaction device, a smart home appliance, a vehicle terminal, and the like, but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiment of the present application.

[0043] Referring to Figure 2 , Figure 2 is a structural diagram of the terminal 400-1 provided by the embodiment of the present application, Figure 2The illustrated terminal 400-1 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components of terminal 400-1 are coupled together by a bus system 440, which is configured to permit communication between the components. The bus system 440 includes a power bus, a control bus, and a status bus, among others. For purposes of illustration, the bus system 440 is shown in the illustrated implementation as including three busses, although it will be appreciated that the bus system 440 can include more busses, less busses, or different busses. Figure 2 The various busses are shown in the illustrated implementation as including a data bus, a control bus, and a status bus, although it will be appreciated that the bus system 440 can include more busses, less busses, or different busses.

[0044] The processor 410 can be an integrated circuit chip, such as a general purpose processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that has the processing capability to perform a variety of tasks.

[0045] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432 that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons and controls.

[0046] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, and the like. The memory 450 optionally includes one or more storage devices remotely located from the processor 410 in a physical location.

[0047] The memory 450 includes volatile memory or nonvolatile memory, or both. Nonvolatile memory can be read only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or the like. Volatile memory can include random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), fast page mode DRAM (FPM DRAM), extended data output RAM (EDO RAM), extended data output dual data rate RAM (EDO DDR RAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), direct

[0048] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or superset thereof, which are exemplarily illustrated below.

[0049] The operating system 451 includes a system program for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks.

[0050] a network communication module 452 for communicating to other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), etc.;

[0051] a presentation module 453 for enabling presentation of information via one or more output devices 431 associated with the user interface 430 (e.g., a display screen, a speaker, etc.) (e.g., a user interface for operating the peripheral device and displaying content and information);

[0052] an input processing module 454 for detecting and interpreting one or more user inputs or interactions from one or more input devices 432.

[0053] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in software, Figure 2 A data partition archiving system 455 stored in the memory 450 is shown, which can be in the form of software such as programs and plug-ins, including the following software modules: a partition determination module 4551, a shadow table creation module 4552, and a data archiving module 4553. These modules are logical, and thus can be combined or further split according to the implemented functions.

[0054] The functions of the various modules will be described below.

[0055] In some embodiments, the apparatus provided by the embodiments of the present application can be implemented in software,

[0056] In some embodiments, the terminal or server can implement the data partition archiving method provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or a software module in the operating system; can be a native application program (APP), i.e., a program that needs to be installed in the operating system to run, such as a social application APP or a message sharing APP; can also be a small program, i.e., a program that only needs to be downloaded into a browser environment to run; can also be a small program or a web client program that can be embedded into any APP. In summary, the above computer program can be any form of application program, module or plug-in.

[0057] Please refer to Figure 3 , Figure 3 is a flowchart of a data partition archiving method in an embodiment of the present application. The present application provides a data partition archiving method, which comprises the following steps:

[0058] Step S300, obtaining a data table to be partitioned, generating attribute features according to data items of the data table to be partitioned, and performing feature matching between the attribute features and pre-set partition features of each data partition to determine a target partition to which the data table to be partitioned belongs;

[0059] Step S310, creating a shadow table in the target partition, and converting data in the data table to be partitioned into the shadow table;

[0060] Step S320, archiving data in the shadow table based on the attribute features of the data table to be partitioned by linking table spaces of the target partition, to obtain a target data archive corresponding to the data table to be partitioned.

[0061] In step S300, a to-be-partitioned data table is acquired, attribute features are generated according to data items of the to-be-partitioned data table, and the attribute features are matched with partition features preset for each data partition to determine a target partition to which the to-be-partitioned data table belongs. Specifically, business data usually exists in a business database in the form of data tables. If the data volume of a single table is too large, the read-write performance will be affected. Database and table partitioning can be used to solve the performance problem of a single table. A partitioned table of Oracle divides a large table into several smaller tables in a physical manner, and is still a complete table in a logical manner. In this way, each DML (Data Manipulation Language) operation can only consider one partitioned table. The partitioning function can further subdivide tables, indexes, or index-organized tables into segments, thereby enabling more accurate management and access of these database objects. The segments of these database objects are called data partitions. Each data partition can also contain multiple data tables, and different remote databases can be used as table spaces of the data partitions to archive and store data corresponding to the data partitions.

[0062] In an embodiment, a feature word library is constructed for each data item contained in each data partition. Specifically, all data items contained in the corresponding data partition can be extracted to obtain feature words of each data item, and the feature words are stored in a preset database to obtain a feature word library corresponding to each data partition.

[0063] In an embodiment, data items of all data partitions can be traversed to record the number of data partitions covered by the same data item and the occurrence frequency, and a data item with a number of covered data partitions exceeding a preset number threshold or an occurrence frequency higher than a preset frequency threshold is regarded as a common data item. A basic feature library is constructed based on the common data items as basic features. Of course, the basic features in the basic feature library can also be configured by target objects with specific permissions. The basic features contained in the feature library can be adjusted according to different application scenarios, which is not limited here. The feature word library of each data partition is filtered based on the basic feature library, and the feature words corresponding to the common data items in the feature word library are removed, and only the distinguishing feature words are retained. The partition features of the corresponding data partition are generated based on the distinguishing feature words. The partition features can include all or part of the feature words in the feature word library.

[0064] In an embodiment, a to-be-partitioned data table is acquired, including the following steps:

[0065] In step S301, the access frequency and the latest access time of the data table in each data partition are acquired.

[0066] In an embodiment, the access records of each data partition data table can be counted periodically or at a certain frequency, wherein the access records include the time node of the last access to the data table, the time interval of the access to the data table, and the frequency of the access to the data table in a preset time period, etc.

[0067] In step S302, the hotness value of the corresponding data table is calculated according to the access frequency and the recent access time.

[0068] In an embodiment, the time difference between the recent access time and the current time node can be calculated, and the ratio of the time difference to the preset time interval is further calculated as a first ratio. The ratio of the access frequency to the preset access number threshold is taken as a second ratio, and the first ratio and the second ratio are weighted to obtain the hotness value of the data table. Other access data such as the access time interval can also be used to calculate the hotness value of the data table, and the specific calculation process is similar to the foregoing steps, which is not limited here.

[0069] In step S303, if the hotness value of the data table is greater than the preset hotness upper limit of the data partition to which it belongs, the corresponding data table is taken as the data table to be partitioned.

[0070] In an embodiment, the hotness range corresponding to each data partition can be set. Whether the data of different data partitions needs to be exchanged is determined based on the hotness of the data table and the hotness range of the partition. Specifically, when the data table is in a low hotness data partition, if the hotness value of the data table obtained by counting exceeds the hotness upper limit of the current data partition to which it belongs, the corresponding data table can be taken as the data table to be partitioned, and the data table to be partitioned can be exchanged to a data partition with higher hotness. The business database can only locally retain the data partition data with a hotness value, and the data partition with a low hotness value can be archived and stored through a remote database. When data is called, the corresponding data can be obtained by accessing the remote database. Therefore, by exchanging the partition data, the low hotness data partition data is converted to a high hotness data partition, which can ensure fast response speed of the data with high user usage frequency, enhance user experience, and at the same time, some data with low hotness can be stored through a medium with slower response speed, thereby reducing system cost.

[0071] In an embodiment, after the hotness value of the corresponding data table is calculated according to the access frequency and the recent access time, the method further includes:

[0072] If the hotness value of the data table is less than the preset hotness lower limit of the data partition to which it belongs, the corresponding data table is taken as the data table to be partitioned.

[0073] In an embodiment, for a data table of a high-hotness data partition, after obtaining the hotness of the data table by the foregoing statistical method, if the hotness value of the data table is lower than the hotness lower limit of the data partition to which the data table belongs, it can be considered that the frequency of use of the data table is low, and the corresponding data table should be converted to a low-hotness data partition for archival storage, so as to avoid long-term temporary use of the storage space of the business database and affect the response speed and overall use experience of the business data. The data table to be exchanged is taken as a data table to be partitioned.

[0074] In an embodiment, after obtaining the data table to be partitioned by the foregoing steps, the attribute feature of the data table to be partitioned can be generated according to the data items in the data table to be partitioned. The target partition to which the data in the data table to be partitioned needs to be switched is determined based on the attribute feature.

[0075] In an embodiment, the attribute feature is generated according to the data items in the data table to be partitioned, comprising:

[0076] All data items in the data table to be partitioned are obtained, and the data items matched with the basic features in the preset basic feature library are removed. The attribute feature is obtained by constructing a data sequence from the remaining data items, wherein the general data items are recorded in the basic feature library as basic features.

[0077] In an embodiment, the data items in the data table to be partitioned can be filtered according to the basic feature library constructed by the foregoing steps. Specifically, the data items in the data table to be partitioned can be compared with the basic features in the basic feature library by item-by-item comparison. If the similarity between the data items in the data table to be partitioned and the basic features reaches a preset similarity threshold, it is considered that the data items are general data items, and the data items can be filtered. After all the data items in the data table to be partitioned are filtered by the foregoing steps, the remaining data items are used to sequentially concatenate to form a one-dimensional vector as the data sequence corresponding to the data items. The attribute feature corresponding to the data table to be partitioned can be obtained by encoding the data sequence.

[0078] In an embodiment, before the attribute feature is matched with the preset partition features of each data partition to determine the target partition to which the data table to be partitioned belongs, it further comprises:

[0079] The hotness value of the data table to be partitioned is compared with the preset hotness range of each data partition. If the hotness value of the data table to be partitioned is within the preset hotness range, the corresponding data partition is included in the first set, and the data partitions in the first set are used for feature matching to obtain the target partition.

[0080] In an embodiment, after obtaining the hotness value of the data table to be partitioned according to the foregoing steps, the hotness value of the data table to be partitioned can be compared with preset hotness ranges of other data partitions. Specifically, the hotness value of the data table to be partitioned can be compared with upper limits or lower limits of the preset hotness ranges of the data partitions to determine whether the hotness value of the data table to be partitioned falls within the preset hotness ranges of the data partitions. If the hotness value of the data table to be partitioned falls within the preset hotness ranges of multiple data partitions, the corresponding multiple data partitions are included in the first set. The data table to be partitioned can select one data partition from the data partitions in the first set as the target partition. Specifically, the attribute feature of the data table to be partitioned can be matched with the partition features of the data partitions in the first set to determine the target partition for which the data table to be partitioned needs to complete data exchange. The similarity between the attribute feature and the partition features can be calculated by using cosine similarity or norm distance, and if the similarity reaches a preset similarity threshold, it is determined that the corresponding partition feature and the attribute feature are matched. When there are multiple partition features reaching the preset similarity threshold, the partition features can be sorted according to the similarity values from high to low, and the data partition corresponding to the partition feature with the highest similarity is selected as the target partition. If there are more than two partition features at the front of the sorting with the same similarity value, one of the data partitions corresponding to the partition features can be randomly selected as the target partition.

[0081] In step S310, a shadow table is created in the target partition, and data in the data table to be partitioned is converted into the shadow table.

[0082] In an embodiment, the data in the data table to be partitioned can be converted into a shadow table in the target partition by creating the shadow table in the target partition, and the user can directly access the shadow table in the target partition to obtain the data of the data table to be partitioned.

[0083] In an embodiment, before creating the shadow table in the target partition, the method further includes:

[0084] Each data item of the data table to be partitioned is compared with each data item of each data table in the target partition, and the shadow table is created when the data items of the data table to be partitioned and the data items of each data table in the target partition are not completely identical.

[0085] In an embodiment, before creating the shadow table, it is necessary to verify the uniqueness of the to-be-partitioned data table. Specifically, the data items of the to-be-partitioned data table can be extracted, and the data items of the to-be-partitioned data table are compared with each data item in the target partition one by one to determine whether all data items in the to-be-partitioned data table are contained in the target partition. If one or more data items in the to-be-partitioned data table are not contained in the target partition, it is determined that the to-be-partitioned data table does not exist in the target partition, and the shadow table of the to-be-partitioned data table can be created in the target partition. After completing the data conversion of the to-be-partitioned data table to the shadow data table, the to-be-partitioned data table can be deleted from the original data partition to which it belongs. Only the shadow data table in the target partition is retained.

[0086] In step S320, by linking the table space of the target partition, the data in the shadow table is archived based on the attribute features of the to-be-partitioned data table, and the target data archive corresponding to the to-be-partitioned data table is obtained.

[0087] In an embodiment, when the target partition is a low-heat data partition, a database link of the target partition can be generated, the remote database is connected through the database link, and the data in the shadow table is transmitted to the remote database corresponding to the target partition for data archiving. By connecting the remote database through the database link, the data can be directly inserted into the archive through the network, and the database link is initiated by the remote database. The remote database serves as a backup library and does not affect the business access and other operations of the business database. Each archive in the remote database is represented by a feature, and the similarity between the attribute features and the library categories can be compared. Based on the similarity, the data in the shadow table is classified into the corresponding cluster category, and data archiving is completed. When calling remote data, data can be searched based on the keywords of the cluster category to improve the efficiency of remote data access. Further, the attribute features can be used as subcategory features under the cluster category to form a hierarchical archiving database based on the attribute features.

[0088] In an embodiment, after archiving the data in the shadow table based on the attribute features of the to-be-partitioned data table, the method further comprises:

[0089] Obtaining a feature word library corresponding to a preset partition feature of the target partition;

[0090] Comparing the plurality of sub-features contained in the attribute features with the feature words in the feature word library for similarity. If the similarity between the sub-features and the feature words is lower than a preset threshold, the sub-features are used to update the feature word library, so as to generate a new partition feature based on the updated feature word library, wherein each sub-feature corresponds to a data item.

[0091] In an embodiment, since the corresponding feature library is set for each data partition in the foregoing step, the partition feature of the corresponding data partition is generated based on the feature library. The feature library of the target partition can be obtained. The attribute feature is a feature sequence obtained by combining the features of multiple data items, and the attribute feature can be split into multiple sub-features, each of which corresponds to a data item of the data table to be partitioned. Each sub-feature is compared with the feature words in the feature library of the target partition in terms of similarity, and the sub-feature with a similarity lower than a preset threshold in the attribute feature is selected. The feature word corresponding to the sub-feature is input into the feature library for updating the feature library. The feature library of the target partition is updated each time the feature library is updated, and the partition feature is regenerated based on the updated feature library to update the partition feature, so as to ensure the accuracy of the partition and the archived data during subsequent partition archiving, facilitate accurate data searching or retrieval, and provide reliable data support for business data.

[0092] Referring to Figure 4 , Figure 4 FIG. 1 is a module diagram of data partition archiving in an embodiment of the present application. The system comprises a partition determination module 4551 configured to obtain a data table to be partitioned, generate an attribute feature according to a data item of the data table to be partitioned, and perform feature matching on the attribute feature and a preset partition feature of each data partition to determine a target partition to which the data table to be partitioned belongs; a shadow table creation module 4552 configured to create a shadow table in the target partition and convert data in the data table to be partitioned into the shadow table; and a data archiving module 4553 configured to archive data in the shadow table based on the attribute feature of the data table to be partitioned by linking a table space of the target partition, to obtain a target data archive corresponding to the data table to be partitioned.

[0093] In an embodiment, the partition determination module 4551 is further configured to obtain a data table to be partitioned, including: obtaining an access frequency and a latest access time of a data table in each data partition; calculating a hotness value of the corresponding data table according to the access frequency and the latest access time; and if there is a data table whose hotness value is greater than a preset upper limit of the hotness of the data partition to which the data table belongs, regarding the corresponding data table as the data table to be partitioned.

[0094] In an embodiment, the partition determination module 4551 is further configured to calculate a hotness value of a corresponding data table according to the access frequency and the latest access time, and further comprising: if there is a data table whose hotness value is less than a preset lower limit of the hotness of the data partition to which the data table belongs, regarding the corresponding data table as the data table to be partitioned.

[0095] In one embodiment, the partition determination module 4551 is also used to perform feature matching between the attribute characteristics and the preset partition characteristics of each data partition to determine the target partition to which the data table to be partitioned belongs, and also includes: comparing the heat value of the data table to be partitioned with the preset heat range of each data partition; if the heat value of the data table to be partitioned is within the preset heat range, the corresponding data partition is classified into the first set, so that the data partition in the first set is used for feature matching to obtain the target partition.

[0096] In one embodiment, the partition determination module 4551 is also used to generate attribute features based on the data items in the data table to be partitioned, including: obtaining all data items in the data table to be partitioned, eliminating data items that match the basic features in a preset basic feature library, and constructing a data sequence through the remaining data items to obtain the attribute features, wherein the basic feature library records common data items as basic features.

[0097] In one embodiment, the shadow table creation module 4552 is also used to, before creating the shadow table for the target partition, also include: comparing each data item of the data table to be partitioned with the data items of each data table in the target partition item by item, and creating the shadow table when the data items of the data table to be partitioned are not completely the same as the data items of each data table in the target partition.

[0098] In one embodiment, the data archiving module 4553 is also used to archive the data in the shadow table based on the attribute characteristics of the data table to be partitioned, and also includes: obtaining a feature vocabulary corresponding to the preset partition characteristics of the target partition; performing a similarity comparison between multiple sub-features contained in the attribute characteristics and feature words in the feature vocabulary; if the similarity between the sub-features and the feature words is lower than a preset threshold, the sub-features are used to update the feature vocabulary to generate new partition characteristics based on the updated feature vocabulary, wherein each sub-feature corresponds to a data item.

[0099] The data partition archiving system can be implemented in the form of a computer program. Figure 5 The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0100] Each module in the aforementioned data partition archiving system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the terminal's memory in hardware form, or stored in the terminal's memory in software form, so that a processor can call and execute the corresponding operations of each module. The processor can be a central processing unit (CPU), a microprocessor, a single-chip microcomputer, etc.

[0101] As shown in Figure 5 An embodiment of the internal structure of a computer device is shown. A computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the following steps when executing the computer program: obtaining a data table to be partitioned, generating attribute features according to data items of the data table to be partitioned, and performing feature matching between the attribute features and partition features preset for each data partition to determine a target partition to which the data table to be partitioned belongs; creating a shadow table in the target partition, converting data in the data table to be partitioned into the shadow table; and archiving data in the shadow table based on the attribute features of the data table to be partitioned by linking table spaces of the target partition, to obtain a target data archive corresponding to the data table to be partitioned.

[0102] In an embodiment, when the processor executes, the obtaining of the data table to be partitioned comprises: obtaining access frequencies and recent access times of data tables in each data partition; calculating a hotness value of a corresponding data table according to the access frequencies and the recent access times; and if there is a data table whose hotness value is greater than a preset upper limit of hotness of a data partition to which the data table belongs, taking the corresponding data table as the data table to be partitioned.

[0103] In an embodiment, when the processor executes, after the calculating of the hotness value of the corresponding data table according to the access frequencies and the recent access times, the method further comprises: if there is a data table whose hotness value is less than a preset lower limit of hotness of a data partition to which the data table belongs, taking the corresponding data table as the data table to be partitioned.

[0104] In an embodiment, when the processor executes, before the performing of the feature matching between the attribute features and the partition features preset for each data partition to determine the target partition to which the data table to be partitioned belongs, the method further comprises: comparing the hotness value of the data table to be partitioned with preset hotness ranges of each data partition, and if the hotness value of the data table to be partitioned is within the preset hotness range, taking the corresponding data partition into a first set, so as to use data partitions in the first set for the feature matching to obtain the target partition.

[0105] In an embodiment, when the processor executes, the generating of the attribute features according to the data items of the data table to be partitioned comprises: obtaining all data items in the data table to be partitioned, eliminating data items matching with basic features in a preset basic feature library, and constructing a data sequence by using the remaining data items to obtain the attribute features, wherein the basic feature library records general data items as basic features.

[0106] In one embodiment, when the above-mentioned processor is executed, before creating the shadow table for the target partition, the implementation also includes: comparing each data item of the data table to be partitioned with the data items of each data table in the target partition item by item, and creating the shadow table when the data items of the data table to be partitioned are not completely the same as the data items of each data table in the target partition.

[0107] In one embodiment, when the above-mentioned processor is executed, after archiving the data in the shadow table based on the attribute characteristics of the data table to be partitioned, it also includes: obtaining a feature vocabulary corresponding to the preset partition characteristics of the target partition; performing a similarity comparison between multiple sub-features contained in the attribute characteristics and feature words in the feature vocabulary; if the similarity between the sub-features and the feature words is lower than a preset threshold, the sub-features are used to update the feature vocabulary to generate new partition features based on the updated feature vocabulary, wherein each sub-feature corresponds to a data item.

[0108] In one embodiment, the above-mentioned computer device can be used as a server, including but not limited to an independent physical server, or a server cluster composed of multiple physical servers. The computer device can also be used as a terminal, including but not limited to a mobile phone, tablet computer, personal digital assistant or smart device. Figure 5 As shown, the computer device includes a processor, a non-volatile storage medium, an internal memory, a display screen, and a network interface connected via a system bus.

[0109] Among them, the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The non-volatile storage medium of the computer device stores an operating system and a computer program. The computer program can be executed by the processor to implement a data partition archiving method provided in each of the above embodiments. The internal memory in the computer device provides a high-speed cache operating environment for the operating system and computer program in the non-volatile storage medium. The display interface can display data through a display screen. The display screen can be a touch screen, such as a capacitive screen or an electronic screen, which can generate corresponding instructions by receiving a click operation on the control displayed on the touch screen.

[0110] Those skilled in the art will understand that Figure 5 The structure of the computer device shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0111] In one embodiment, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the following steps: obtaining a data table to be partitioned, generating attribute features according to data items of the data table to be partitioned, and performing feature matching between the attribute features and partition features of each data partition to determine a target partition to which the data table to be partitioned belongs; creating a shadow table in the target partition, and converting data in the data table to be partitioned into the shadow table; and archiving data in the shadow table based on the attribute features of the data table to be partitioned by linking a table space of the target partition, to obtain a target data archive corresponding to the data table to be partitioned.

[0112] In one embodiment, when the computer program is executed by the processor, the obtaining of the data table to be partitioned comprises: obtaining access frequencies and recent access times of data tables in each data partition; calculating a hotness value of each data table according to the access frequencies and the recent access times; and if there is a data table whose hotness value is greater than a preset upper limit of hotness of a data partition to which the data table belongs, regarding the data table as the data table to be partitioned.

[0113] In one embodiment, when the computer program is executed by the processor, after the calculating of the hotness value of each data table according to the access frequencies and the recent access times, the method further comprises: if there is a data table whose hotness value is less than a preset lower limit of hotness of a data partition to which the data table belongs, regarding the data table as the data table to be partitioned.

[0114] In one embodiment, when the computer program is executed by the processor, before the performing of the feature matching between the attribute features and the partition features of each data partition to determine the target partition to which the data table to be partitioned belongs, the method further comprises: comparing the hotness value of the data table to be partitioned with preset hotness ranges of each data partition, and if the hotness value of the data table to be partitioned is within the preset hotness range, regarding a data partition corresponding to the preset hotness range as a first set, and using data partitions in the first set for the feature matching to obtain the target partition.

[0115] In one embodiment, when the computer program is executed by the processor, the generating of the attribute features according to the data items of the data table to be partitioned comprises: obtaining all data items in the data table to be partitioned, removing data items matching with basic features in a preset basic feature library, and constructing a data sequence by using remaining data items to obtain the attribute features, wherein the basic feature library records general data items as basic features.

[0116] In an embodiment, when the computer program is executed by the processor, the method further comprises: comparing each data item of the to-be-partitioned data table with each data item of each data table in the target partition one by one, and creating the shadow table when each data item of the to-be-partitioned data table is not completely identical with each data item of each data table in the target partition.

[0117] In an embodiment, when the instruction is executed by the processor, the method further comprises: obtaining a feature word library corresponding to a preset partition feature of the target partition; performing similarity comparison between a plurality of sub-features included in the attribute feature and feature words in the feature word library, and updating the sub-features in the feature word library when the similarity between the sub-features and the feature words is lower than a preset threshold, so as to generate a new partition feature based on the updated feature word library, wherein each sub-feature corresponds to a data item.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The program can be stored in a non-volatile computer readable storage medium, and the program can include the processes of the above-mentioned embodiments when executed.

[0119] The above-mentioned embodiments are only illustrative of the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A data partition archiving method, characterized in that: include: Obtaining a data table to be partitioned, generating attribute features based on data items of the data table to be partitioned, and performing feature matching on the attribute features with partition features preset for each data partition to determine a target partition to which the data table to be partitioned belongs; before performing feature matching on the attribute features with partition features preset for each data partition to determine the target partition to which the data table to be partitioned belongs, further comprising: comparing a heat value of the data table to be partitioned with a preset heat range of each data partition; if the heat value of the data table to be partitioned is within the preset heat range, classifying the corresponding data partition into a first set, so that the data partitions in the first set are used for feature matching to obtain the target partition; Create a shadow table in the target partition, and convert the data in the to-be-partitioned data table into the shadow table; By linking the table space of the target partition, the data in the shadow table is archived based on the attribute characteristics of the data table to be partitioned to obtain a target data archive corresponding to the data table to be partitioned; after archiving the data in the shadow table based on the attribute characteristics of the data table to be partitioned, the method further includes: obtaining a feature vocabulary corresponding to a preset partition feature of the target partition; performing a similarity comparison between multiple sub-features contained in the attribute feature and feature words in the feature vocabulary; if the similarity between the sub-feature and the feature word is lower than a preset threshold, using the sub-feature to update the feature vocabulary to generate a new partition feature based on the updated feature vocabulary, wherein each sub-feature corresponds to a data item.

2. The data partition archiving method according to claim 1, characterized in that: Get the data table to be partitioned, including: Get the access frequency and most recent access time of the data table in each data partition; Calculate the popularity value of the corresponding data table according to the access frequency and the most recent access time; If there is a data table whose heat value is greater than the preset heat upper limit of the data partition to which it belongs, the corresponding data table will be used as the data table to be partitioned.

3. The data partition archiving method according to claim 2, characterized in that: After calculating the popularity value of the corresponding data table according to the access frequency and the most recent access time, the method further includes: If there is a data table whose heat value is less than the preset heat lower limit of the data partition to which it belongs, the corresponding data table is used as the data table to be partitioned.

4. The data partition archiving method according to claim 1, characterized in that: Generating attribute features according to the data items of the data table to be partitioned includes: Obtain all data items in the data table to be partitioned, eliminate data items that match basic features in a preset basic feature library, and construct a data sequence through the remaining data items to obtain the attribute features, wherein the basic feature library records common data items as basic features.

5. The data partition archiving method according to claim 1, characterized in that: Before creating a shadow table for the target partition, the method further includes: Each data item of the data table to be partitioned is compared with the data items of each data table in the target partition item by item, and when the data items of the data table to be partitioned are not completely the same as the data items of each data table in the target partition, the shadow table is created.

6. A data partition archiving system for executing the data partition archiving method according to any one of claims 1 to 5, characterized in that: include: A partition determination module is configured to obtain a data table to be partitioned, generate attribute features based on data items in the data table to be partitioned, and perform feature matching between the attribute features and the partition features preset for each data partition to determine a target partition to which the data table to be partitioned belongs; A shadow table creation module is used to create a shadow table in the target partition and convert the data in the to-be-partitioned data table into the shadow table; The data archiving module is used to archive the data in the shadow table based on the attribute characteristics of the data table to be partitioned by linking the table space of the target partition, so as to obtain the target data archive corresponding to the data table to be partitioned.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data partition archiving method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data partition archiving method according to any one of claims 1 to 5 are implemented.

Images