Data processing method and device

By monitoring the database data status information and transferring it to different performance and security partitions, the problem of crashing in large-scale data processing by the coal field knowledge base management system is solved, and storage performance and security are improved.

CN118170762BActive Publication Date: 2024-11-01CCTEG COAL MINING RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410317420.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-19
Publication Date
2024-11-01
Estimated Expiration
2044-03-19

AI Technical Summary

Technical Problem

In the field of coal, the knowledge base management system often experiences abnormal states such as crashes when calling large-scale data, which affects the normal use of the system.

Method used

By monitoring the data status information in the database, data is transferred to partitions with different read and write performance and security, such as mirrored areas and high-precision areas, based on read volume, modification volume and precision types, to optimize data storage.

Benefits of technology

Improves the storage performance and data reading performance of the database, ensures the normal operation of the system, and meets the data reading performance and security requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118170762B_ABST
    Figure CN118170762B_ABST
Patent Text Reader

Abstract

The present application proposes a data processing method and device, wherein the method includes: monitoring the status information of data in a first database; transferring the data in the first database to a corresponding partition according to the status information; wherein the first database includes multiple partitions, and each partition has corresponding read and write performance and security. This solution can transfer the data in the first database to the corresponding partition based on the status of the data in the first database to meet the requirements for data reading performance and security, improve the storage performance of the first database, and also improve the reading performance and security of the data in the first database to ensure the normal operation of the relevant system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device. Background Art

[0002] In the coal field, it is often necessary to collect data related to coal knowledge and store it in a database. The background knowledge base management system can call the data in the database for data management.

[0003] However, due to the large amount of data, the knowledge base management system often freezes or experiences other abnormal conditions when calling data in the database, affecting the normal use of the system. Summary of the invention

[0004] In order to solve the above problems, the present application provides a data processing method and device.

[0005] According to a first aspect of the present application, a data processing method is provided, comprising:

[0006] Monitoring status information of data in the first database;

[0007] According to the status information, the data in the first database is transferred to the corresponding partition; wherein the first database includes multiple partitions, and each partition has corresponding read-write performance and security.

[0008] The status information includes at least one of the following:

[0009] The amount of data read;

[0010] The amount of data modification;

[0011] The precision type of the data.

[0012] In some embodiments of the present application, transferring data in the first database to a corresponding partition according to the state information includes:

[0013] Determine the partition transfer index value of the data according to the status information;

[0014] According to the partition transfer index value, the data in the first database is transferred to the corresponding partition.

[0015] As an example, the state information includes the amount of data read and the amount of data modified; determining the partition transfer index value of the data according to the state information includes:

[0016] Determine the partition transfer index value of the data based on the amount of data read and modified.

[0017] As a possible implementation manner, according to the partition transfer index value, the data in the first database is transferred to the corresponding partition, including:

[0018] If the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or,

[0019] If the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database; wherein the read and write speeds of the mirror area are higher than those of the high-precision area, and the security of the mirror area is lower than that of the high-precision area.

[0020] In some other embodiments of the present application, the state information includes a data precision type; and partitioning the data in the first database is transferred according to the state information, including:

[0021] If the precision type of the third data in the first database is a precision data type, the third data is transferred to a high-precision area of ​​the first database.

[0022] In some embodiments of the present application, the method further comprises:

[0023] Get the data to be stored;

[0024] From the data to be stored, determine first data to be stored whose data type is a multimedia data type, and second data to be stored whose data type is a text type;

[0025] Storing the first data to be stored in a non-check area of ​​the first database; wherein the non-check area is a partition with the highest read / write performance and the lowest security among the multiple partitions;

[0026] The second data to be stored is stored in the second database.

[0027] As a possible implementation manner, storing the first data to be stored in the non-verification area of ​​the first database includes:

[0028] Storing the first data to be stored in the first sub-partition of the non-verification area; wherein the first sub-partition is stored in a solid state hard disk;

[0029] Monitoring the storage time of the first data to be stored in the first sub-partition;

[0030] If the storage time of the fourth data in the first data to be stored in the first sub-partition is greater than or equal to the first time threshold, the fourth data is transferred to the second sub-partition in the non-verification area; wherein the second sub-partition adopts disk storage.

[0031] Wherein, the first database adopts network attached storage.

[0032] According to a second aspect of the present application, there is provided a data processing device, comprising:

[0033] A monitoring module, used to monitor the status information of the data in the first database;

[0034] The transfer module is used to transfer the data in the first database to the corresponding partition according to the status information; wherein the first database includes multiple partitions, and each partition has corresponding read and write performance and security.

[0035] The status information includes at least one of the following:

[0036] The amount of data read;

[0037] The amount of data modification;

[0038] The precision type of the data.

[0039] In some embodiments of the present application, the transfer module is specifically used to:

[0040] Determine the partition transfer index value of the data according to the status information;

[0041] According to the partition transfer index value, the data in the first database is transferred to the corresponding partition.

[0042] As an example, the state information includes the amount of data read and the amount of data modified; the transfer module is specifically used for:

[0043] Determine the partition transfer index value of the data based on the amount of data read and modified.

[0044] As a possible implementation method, the transfer module is specifically used for:

[0045] If the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or,

[0046] If the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database; wherein the read and write speeds of the mirror area are higher than those of the high-precision area, and the security of the mirror area is lower than that of the high-precision area.

[0047] In some other embodiments of the present application, the state information includes the data precision type; the transfer module is further used for:

[0048] If the precision type of the third data in the first database is a precision data type, the third data is transferred to a high-precision area of ​​the first database.

[0049] In some embodiments of the present application, the device further comprises:

[0050] An acquisition module, used to acquire data to be stored;

[0051] A determination module, used for determining, from the data to be stored, first data to be stored whose data type is a multimedia data type, and second data to be stored whose data type is a text type;

[0052] A first storage module, used to store the first data to be stored in a non-check area of ​​the first database; wherein the non-check area is a partition with the highest read / write performance and the lowest security among the multiple partitions;

[0053] The second storage module is used to store the second data to be stored in the second database.

[0054] As a possible implementation manner, the first storage module is specifically used for:

[0055] Storing the first data to be stored in the first sub-partition of the non-verification area; wherein the first sub-partition is stored in a solid state hard disk;

[0056] Monitoring the storage time of the first data to be stored in the first sub-partition;

[0057] If the storage time of the fourth data in the first data to be stored in the first sub-partition is greater than or equal to the first time threshold, the fourth data is transferred to the second sub-partition in the non-verification area; wherein the second sub-partition adopts disk storage.

[0058] Wherein, the first database adopts network attached storage.

[0059] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0060] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0061] According to the technical solution of the present application, by monitoring the status information of the data in the first database, the data in the first database is transferred to the corresponding partition according to the status information, wherein the first database includes multiple partitions, and each partition has corresponding read and write performance and security. In other words, this solution can transfer the data in the first database to the corresponding partition based on the status of the data in the first database to meet the requirements for data reading performance and security, improve the storage performance of the first database, and also improve the reading performance and security of the data in the first database to ensure the normal operation of the relevant system.

[0062] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0064] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0065] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application;

[0066] Figure 3 A flowchart of another data processing method provided in an embodiment of the present application;

[0067] Figure 4 A structural block diagram of a data processing device provided in an embodiment of the present application;

[0068] Figure 5 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0070] It should be noted that in the coal field, it is often necessary to collect data related to coal knowledge and store it in a database. The background knowledge base management system can call the data in the database for data management. However, due to the large amount of data, the knowledge base management system often freezes and other abnormal conditions when calling the data in the database, affecting the normal use of the system.

[0071] In order to solve the above problems, the present application provides a data processing method and device.

[0072] Figure 1 This is a flow chart of a data processing method provided in an embodiment of the present application. It should be noted that the data processing method in the embodiment of the present application can be applied to the data processing device in the embodiment of the present application, and the device can be configured in an electronic device. Among them, the electronic device can be a server, a terminal device, etc. Figure 1 As shown, the method may include:

[0073] Step 101: monitor status information of data in a first database.

[0074] The first database may be a database storing multimedia data such as images and videos, or a database storing texts and multimedia data, etc. The first database may be a database using NAS (Network Attached Storage). NAS refers to a device connected to the network with data storage function, also known as network storage, which is a dedicated data storage server. NAS is data-centric, completely separates storage devices from servers, and centrally manages data, thereby releasing bandwidth, improving performance, and reducing costs.

[0075] In some embodiments of the present application, the data in the first database may be coal knowledge data collected based on the collection system. The collection system may be used to collect coal knowledge data from various data channels. As an example, the knowledge base management system is deployed in the background intranet, and multiple collection systems are deployed in the extranet, and multiple clusters are deployed to push relevant coal knowledge data to the knowledge base management system through a multi-threaded asynchronous interface. For the number of collection systems, it is preferred to use a number that can meet the data push efficiency requirements.

[0076] Among them, the status information of the data in the first database is used to characterize the importance of the data. For example, the status information can characterize the frequency of use of the data, and can also characterize the accuracy information of the data. As an example, the status information of the data in the first database may include at least one of the following: the amount of data read, the amount of data modified, and the accuracy type of the data. Among them, the amount of data read refers to the number of times the data is read, the amount of data modified refers to the number of times the data is modified, and the data accuracy type refers to whether the data is a precise data type or a non-precision data type. It should be noted that the status information of the data can be the status information of each stored data in the first database, and the status information of the data can be updated based on the usage of the data through the knowledge base management system.

[0077] Step 102: transfer the data in the first database to the corresponding partition according to the status information; wherein the first database includes a plurality of partitions, and each partition has corresponding read / write performance and security.

[0078] In some embodiments of the present application, the first database may include multiple different partitions, wherein different partitions may correspond to different storage performances, and the storage performance here may include read-write performance and security.

[0079] That is to say, according to the data status information, the data is transferred to a partition with corresponding read / write performance and security to meet the requirements for data reading and security and improve the storage performance of the first database.

[0080] As a possible implementation method, if the status information includes the data reading volume, the reading volume of each data in the first database can be compared with the reading volume threshold of each partition to determine the partition corresponding to the reading volume of each data, and the data can be transferred to the corresponding partitions.

[0081] As another possible implementation method, if the status information includes the precision type of the data, the first database includes a high-precision area and a low-precision area. If the precision type of the data is a precise data type, the data is transferred to the high-precision area; if the precision type of the data is a non-precision data type, the data is transferred to the low-precision area.

[0082] As another possible implementation method, if the status information includes the read amount and modification amount of the data, based on the preset read amount range and modification amount range corresponding to each partition, the partition corresponding to the read amount and modification amount of each data is determined, and the data is transferred to the corresponding partition.

[0083] It should be noted that not all data in the first database needs to be transferred. If the partition where a certain data is located is different from the partition determined based on the status information, the data needs to be transferred to the partition determined based on the status information.

[0084] According to the data processing method of the embodiment of the present application, by monitoring the status information of the data in the first database, the data in the first database is transferred to the corresponding partition according to the status information, wherein the first database includes multiple partitions, and each partition has corresponding read and write performance and security. In other words, this solution can transfer the data in the first database to the corresponding partition based on the status of the data in the first database to meet the requirements for data reading performance and security, improve the storage performance of the first database, and also improve the reading performance and security of the data in the first database to ensure the normal operation of the relevant system.

[0085] Next, the specific implementation process of transferring the data in the first database to the corresponding partition according to the status information will be introduced.

[0086] Figure 2 This is a flow chart of another data processing method provided by an embodiment of the present application. Figure 2 As shown, based on the above embodiments, Figure 1 The implementation process of step 102 may include:

[0087] Step 201, determining a partition transfer index value of data according to state information.

[0088] The partition transfer index value is the basis for data partition transfer, which is used to characterize the importance of data and can also be the basis for dividing partitions.

[0089] As a possible implementation manner, if the status information includes the data read amount, the partition transfer index value of each piece of data may be determined according to the read amount of each piece of data based on the relationship between the preset partition transfer index value and the read amount.

[0090] As another possible implementation, if the status information includes the amount of data read and the amount of data modified, the partition transfer index value of the data can be determined based on the following formula (1):

[0091] Ni=lgn 1i +lgn 2i (1)

[0092] Where Ni is the partition transfer index value of data i; n 1i is the reading amount of data i; n 2i is the modification amount of data i.

[0093] Step 202: transfer the data in the first database to the corresponding partition according to the partition transfer index value.

[0094] Since the partition transfer index value represents the importance of the data, the data can be divided into corresponding partitions according to the partition transfer index value.

[0095] In some embodiments of the present application, the first database may include a non-check area, a mirror area, and a high-precision area. Among them, the non-check area can continuously divide data in bits or bytes, and read and write in parallel on multiple disks, so it has a very high data transmission rate, but there is no data redundancy, which meets the functions of fast, large-scale, and high-performance read and write storage. The mirror area can achieve data redundancy through disk data mirroring, and generate data that backs up each other on paired independent disks. When the original data fails, the system can automatically switch to reading and writing on the mirror disk. The data in the high-precision area is divided into bits or bytes and stored in each disk, and the domestic SM2 algorithm is used for encryption verification code. The SM2 encryption machine is used to provide error checking and recovery using verification redundant information, which is extremely safe.

[0096] Among them, in storage systems with high reliability requirements, RAID (Redundant Arrays of Independent Disks) is generally used to perform redundant processing on data to achieve redundancy when some disks are damaged or data blocks are damaged. Since the acquisition system collects a large amount of data and not all information is used in the knowledge base management system, the NAS storage used by the system is divided into three areas, non-verification area, mirror area and high-precision area. The read and write performance of the non-verification area is better than that of the mirror area, the read and write performance of the mirror area is better than that of the high-precision area, the security of the high-precision area is better than that of the mirror area, the security of the mirror area is better than that of the non-verification area, the read and write speed of the non-verification area is better than that of the mirror area, and the read and write speed of the mirror area is better than that of the high-precision partition. Among them, the read and write performance of the non-verification area is much higher than that of the other two areas.

[0097] In some embodiments of the present application, the data collected by the collection system can be stored in the non-verification area by default, and the read amount and modification amount of the data initially stored in the system can be defaulted to 0, and then the knowledge base management system updates the status information of the data in the database. Based on the updated status information, the data in the first database is partitioned and transferred.

[0098] As a possible implementation, according to the partition transfer index value, the implementation method of transferring the data in the first database to the corresponding partition may include: if the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or, if the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database. As an example, if the partition transfer index value is obtained based on the above formula (1), the first index threshold can be 0, the second index threshold can be 0.8, and the third index threshold can be 1, that is, if the partition transfer index value of the first data is greater than or equal to 0 and less than or equal to 0.8, the first data is transferred to the mirror area, and if the partition transfer index value of the second data is greater than 0.8 and less than or equal to 1, the first data is transferred to the high-precision area. It should be noted that the first data is any one or more data in the first database, and the second data is also any one or more data in the first database.

[0099] In some other embodiments of the present application, if the state information also includes the precision type of the data, then Figure 1 The implementation process of step 102 in may also include:

[0100] Step 203: if the precision type of the third data in the first database is a precision data type, the third data is transferred to a high-precision area of ​​the first database.

[0101] In some embodiments of the present application, the precision type of data can be set by relevant staff through the management system, or it can be updated based on a preset judgment logic. For example, if the data is judged to be high-authority data, it will be changed to a precision data type. Since the high-precision area has the highest security, the third data of the precision data type can be transferred to the high-precision area.

[0102] According to the data processing method of the embodiment of the present application, the partition transfer index value of the data is determined according to the state information, and the data in the first database is transferred to the mirror area or the high-precision area according to the partition transfer index value. If the precision type of the third data in the first database is a precision data type, the third data is transferred to the high-precision area. This solution can greatly improve the storage performance of data, and while ensuring the read and write performance, redundant settings are made for data with a higher degree of importance to ensure the accuracy and security of the data.

[0103] Figure 3 Flow chart of another data processing method provided in the embodiment of the present application. Figure 3 As shown, based on the above embodiment, the method may further include the following steps:

[0104] Step 301, obtaining data to be stored.

[0105] The data to be stored may be coal knowledge data collected by the collection system.

[0106] Step 302: Determine, from the data to be stored, first data to be stored whose data type is a multimedia data type, and second data to be stored whose data type is a text type.

[0107] The text type may be a text type, that is, the second data to be stored may be text type data, and the first data to be stored may be image, video and other types of data.

[0108] Step 303: store the first data to be stored in the non-check area of ​​the first database; wherein the non-check area is a partition with the highest read / write performance and the lowest security among the multiple partitions.

[0109] In some embodiments of the present application, the first data to be stored is stored in the NAS mount, and enters the non-check area by default, and the data read and modified are both defaulted to 0. The non-check area continuously divides the data in bits or bytes, and reads / writes in parallel on multiple disks, has a high data transmission rate, can achieve high-performance and fast storage, has no data redundancy, and once data errors or missing occur, it cannot be restored by disk redundancy. Due to the low error rate, when missing occurs, it is necessary to rely on the data collection source field, return to the collection system, and re-collect.

[0110] Step 304: store the second data to be stored in the second database.

[0111] The second database may be a Jincang database. Since the Jincang database has a large performance bottleneck for storing a large number of pictures and videos, it cannot process the existing millions of articles and about 10TB of picture and video data, and often crashes. Therefore, only storing text data with a smaller storage capacity in the second database can effectively improve data storage performance.

[0112] In some embodiments of the present application, the implementation process of step 303 may also include: storing the first data to be stored in the first sub-partition of the non-check area; wherein the first sub-partition is stored in a solid state hard disk; monitoring the storage time of the first data to be stored in the first sub-partition; if the storage time of the fourth data in the first data to be stored in the first sub-partition is greater than or equal to the first time threshold, transferring the fourth data to the second sub-partition of the non-check area; wherein the second sub-partition is stored in a disk. When the fourth data is transferred to the second sub-partition, the modification amount of the fourth data is updated.

[0113] As an example, the first sub-partition and the second sub-partition in the non-verification area can be the new push area and the old push area, respectively. The storage size ratio of the new push area and the old push area can be 2:8. When data is pushed to the non-verification partition, the newly pushed data is stored in the new push area by default, the time when the data is stored is recorded, and the above storage duration is calculated. A threshold N can be set here. When the time reaches N, if a certain data is still stored in the new push area, a replication algorithm is used to transfer the data to the old push area, and the data modification amount is +1. The storage method of the old push area adopts traditional disk storage, while the new push area adopts SSD storage, and no redundant verification is performed to ensure high-speed storage efficiency.

[0114] According to the data processing method of the embodiment of the present application, when storing, the first data to be stored of the multimedia data type is stored in the non-check area of ​​the first database, and the second data to be stored of the text type is stored in the second database. This solution stores data of different data types and different storage amounts in the first database and the second database, and performs partition storage management on the data with larger storage amounts, which can further improve the data storage performance.

[0115] In order to implement the above embodiments, the present application proposes a data processing device.

[0116] Figure 4 This is a structural block diagram of a data processing device provided in an embodiment of the present application. Figure 4 As shown, the device may include:

[0117] Monitoring module 401, used to monitor the status information of data in the first database;

[0118] The transfer module 402 is used to transfer the data in the first database to the corresponding partition according to the state information; wherein the first database includes multiple partitions, and each partition has corresponding read / write performance and security.

[0119] The status information includes at least one of the following:

[0120] The amount of data read;

[0121] The amount of data modification;

[0122] The precision type of the data.

[0123] In some embodiments of the present application, the transfer module 402 is specifically used to:

[0124] Determine the partition transfer index value of the data according to the status information;

[0125] According to the partition transfer index value, the data in the first database is transferred to the corresponding partition.

[0126] As an example, the state information includes the amount of data read and the amount of data modified; the transfer module 402 is specifically used for:

[0127] Determine the partition transfer index value of the data based on the amount of data read and modified.

[0128] As a possible implementation manner, the transfer module 402 is specifically used for:

[0129] If the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or,

[0130] If the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database; wherein the read and write speeds of the mirror area are higher than those of the high-precision area, and the security of the mirror area is lower than that of the high-precision area.

[0131] In some other embodiments of the present application, the state information includes the data precision type; the transfer module 402 is further used to:

[0132] If the precision type of the third data in the first database is a precision data type, the third data is transferred to a high-precision area of ​​the first database.

[0133] In some embodiments of the present application, the device further comprises:

[0134] The acquisition module 403 is used to acquire the data to be stored;

[0135] A determination module 404 is used to determine, from the data to be stored, first data to be stored whose data type is a multimedia data type, and second data to be stored whose data type is a text type;

[0136] The first storage module 405 is used to store the first data to be stored in the non-check area of ​​the first database; wherein the non-check area is a partition with the highest read / write performance and the lowest security among the multiple partitions;

[0137] The second storage module 406 is used to store the second data to be stored in the second database.

[0138] As a possible implementation manner, the first storage module 405 is specifically used for:

[0139] Storing the first data to be stored in the first sub-partition of the non-verification area; wherein the first sub-partition is stored in a solid state hard disk;

[0140] Monitoring the storage time of the first data to be stored in the first sub-partition;

[0141] If the storage time of the fourth data in the first data to be stored in the first sub-partition is greater than or equal to the first time threshold, the fourth data is transferred to the second sub-partition in the non-verification area; wherein the second sub-partition adopts disk storage.

[0142] Wherein, the first database adopts network attached storage.

[0143] According to the data processing device of the embodiment of the present application, by monitoring the status information of the data in the first database, according to the status information, the data in the first database is transferred to the corresponding partition, wherein the first database includes multiple partitions, and each partition has corresponding read and write performance and security. In other words, this solution can transfer the data in the first database to the corresponding partition based on the status of the data in the first database, so as to meet the requirements for data reading performance and security, improve the storage performance of the first database, and also improve the reading performance and security of the data in the first database, so as to ensure the normal operation of the relevant system.

[0144] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0145] Figure 5 A block diagram of an electronic device for implementing the data processing method of the above-mentioned embodiment provided by the embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0146] like Figure 5As shown, the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 501 is taken as an example.

[0147] The memory 502 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor so that the at least one processor executes the data processing method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to cause a computer to execute the data processing method provided in the present application. The computer program product of the present application includes a computer program, which, when executed by the processor 501, implements the data processing method proposed in the present application.

[0148] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the data processing method in the embodiment of the present application. The processor 501 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 502, that is, implements the data processing method in the above method embodiment.

[0149] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device processed by the voice recognition, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely arranged relative to the processor 501, and these remote memories may be connected to the electronic device of the data processing method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0150] The electronic device of the data processing method may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0151] The input device 503 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the speech recognition processing method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 504 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0152] In order to implement the above embodiments, the present disclosure also proposes a storage medium.

[0153] When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the data processing method as described above.

[0154] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0155] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0156] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0157] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0158] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0159] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0160] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0161] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A data processing method, characterized in that: include: Monitoring status information of data in the first database; According to the state information, the data in the first database is transferred to the corresponding partition, wherein the state information includes the read amount and the modified amount of the data; wherein the first database includes a plurality of partitions, and each partition has corresponding read and write performance and security; The transferring the data in the first database to a corresponding partition according to the state information includes: Determining a partition transfer index value of the data according to the read amount and the modified amount of the data; According to the partition transfer index value, the data in the first database is transferred to the corresponding partition; wherein, according to the partition transfer index value, the data in the first database is transferred to the corresponding partition, including: If the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or, If the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database; wherein the read and write speed of the mirror area is higher than the read and write speed of the high-precision area, and the security of the mirror area is lower than the security of the high-precision area.

2. The method according to claim 1, characterized in that The state information includes the precision type of the data; and transferring the data in the first database to a corresponding partition according to the state information includes: If the precision type of the third data in the first database is a precision data type, the third data is transferred to a high-precision area of ​​the first database.

3. The method according to claim 1, characterized in that Also includes: Get the data to be stored; From the data to be stored, determine first data to be stored whose data type is a multimedia data type, and second data to be stored whose data type is a text type; Storing the first data to be stored in a non-check area of ​​the first database; wherein the non-check area is a partition with the highest read / write performance and the lowest security among the multiple partitions; The second data to be stored is stored in a second database.

4. The method according to claim 3, characterized in that The storing the first data to be stored in the non-verification area of ​​the first database includes: storing the first data to be stored in a first sub-partition of the non-check area; wherein the first sub-partition is stored in a solid state hard disk; Monitoring the storage time of the first data to be stored in the first sub-partition; If the storage time of the fourth data in the first data to be stored in the first sub-partition is greater than or equal to the first time threshold, the fourth data is transferred to the second sub-partition of the non-verification area; wherein the second sub-partition adopts disk storage.

5. The method according to any one of claims 1 to 4, characterized in that The first database uses a network attached storage.

6. A data processing device, characterized in that: include: A monitoring module, used to monitor the status information of the data in the first database; A transfer module, configured to transfer the data in the first database to a corresponding partition according to the state information, wherein the state information includes the read amount and the modified amount of the data; wherein the first database includes a plurality of partitions, and each of the partitions has corresponding read and write performance and security; The transferring the data in the first database to a corresponding partition according to the state information includes: Determining a partition transfer index value of the data according to the read amount and the modified amount of the data; According to the partition transfer index value, the data in the first database is transferred to the corresponding partition; wherein, according to the partition transfer index value, the data in the first database is transferred to the corresponding partition, including: If the partition transfer index value of the first data in the first database is greater than or equal to the first index threshold and less than or equal to the second index threshold, the first data is transferred to the mirror area of ​​the first database; or, If the partition transfer index value of the second data in the first database is greater than the second index threshold and less than or equal to the third index threshold, the second data is transferred to the high-precision area of ​​the first database; wherein the read and write speed of the mirror area is higher than the read and write speed of the high-precision area, and the security of the mirror area is lower than the security of the high-precision area.