Data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a data processing method and device, electronic equipment and storage medium, the method comprises: when there are at least two target nodes write data information to the unified database, the data information in the target node is cached to its backup database, and the data information in the backup database of at least two target nodes is synchronized to the unified database one by one. In the embodiments of the present application, a distributed file system is deployed in the NAS cluster, so that each node can access the unified database under the distributed file system; meanwhile, a backup database is configured in each node, when the unified database exists concurrent write data information, the data information in each target node is cached to its backup database, and then the data information in the backup database of each target node is synchronized to the unified database one by one, solving the problem of concurrent write of the unified database, thereby avoiding data information write failure or data information loss.

Description

Technical Field

[0001] This invention relates to the field of server technology, and in particular to a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. Background Technology

[0002] SQLite is a lightweight, embedded file database that operates within a process and does not natively support network access. As a lightweight database, it is stored on a single machine in a single file; a single .sqlite file contains everything required for the SQLite database. Data in a NAS (Network Attached Storage) cluster can be stored entirely in an SQLite database. Other databases, such as MySQL, require a running server, and clients and servers access each other over a network. To reduce resource consumption in a NAS cluster, SQLite can be used to handle data storage and access.

[0003] However, as a standalone file database, SQLite cannot be accessed over the network and has database locks (only one process can access it at a time). Therefore, when using SQLite database in a NAS cluster, there are issues such as nodes in the cluster being unable to access the SQLite database and write problems to the SQLite database under high concurrency. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, electronic device, and storage medium to solve the problems of nodes in a NAS cluster being unable to access the SQLite database and the write operations of the SQLite database under high concurrency when using the SQLite database.

[0005] This invention discloses a data processing method applied to a distributed file system deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node of the NAS cluster is configured with a backup database. Both the unified database and the backup databases are SQLite databases. The method includes:

[0006] When at least two target nodes write data to the unified database, the data in the target nodes is cached in their backup database.

[0007] The data information in the backup databases of at least two of the target nodes is synchronized to the unified database one by one.

[0008] Optionally, it also includes:

[0009] When a target node writes data information to the unified database, the data information in the target node is written into the unified database.

[0010] Optionally, caching the data information in the target node to its backup database includes:

[0011] When there are at least two processes writing data to the backup database in the target node, the timeliness condition of the process is obtained;

[0012] The data information is written into the backup database according to the timeliness conditions.

[0013] Optionally, writing the data information into the backup database according to the timeliness conditions includes:

[0014] When it is determined that at least two of the processes can be completed one by one within the time limit, at least two of the processes are executed one by one to write the data information into the backup database.

[0015] Optionally, writing the data information into the backup database according to the timeliness conditions includes:

[0016] When it is determined that at least two of the processes cannot be completed one by one within the time limit, a temporary database process is established.

[0017] Migrate the data information from at least two of the processes to the temporary database process, and release at least two of the processes;

[0018] The temporary database process is executed to write the data information into the backup database.

[0019] Optionally, the temporary database process is a single-threaded process, and executing the temporary database process to write the data information into the backup database includes:

[0020] The temporary database process is executed to write the data information into the backup database one by one.

[0021] Optionally, caching the data information in the target node to its backup database includes:

[0022] When there is a process in the target node that writes data information to the backup database, the process is executed to write the data information to the backup database.

[0023] This invention also discloses a data processing device applied to a distributed file system deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node of the NAS cluster is configured with a backup database. Both the unified database and the backup database are SQLite databases, comprising:

[0024] The data caching module is used to cache the data information in the target nodes to their backup database when at least two target nodes write data information to the unified database.

[0025] The data synchronization module is used to synchronize the data information in the backup databases of at least two target nodes to the unified database one by one.

[0026] Optionally, it also includes:

[0027] The data writing module is used to write the data information in the target node into the unified database when a target node writes data information to the unified database.

[0028] Optionally, the data caching module includes:

[0029] The timeliness acquisition submodule is used to acquire the timeliness conditions of the process when there are at least two processes writing data information to the backup database in the target node.

[0030] The first data writing submodule is used to write the data information into the backup database according to the timeliness conditions.

[0031] Optionally, the first data writing submodule includes:

[0032] The first data writing unit is used to execute at least two processes one by one to write the data information into the backup database when it is determined that at least two processes can be completed one by one within the time limit.

[0033] Optionally, the first data writing submodule includes:

[0034] A database process creation unit is used to create a temporary database process when it is determined that at least two processes cannot be completed one by one within the time limit conditions.

[0035] A data migration unit is used to migrate data information from at least two of the processes to the temporary database process and release at least two of the processes;

[0036] The second data writing unit is used to execute the temporary database process to write the data information into the backup database.

[0037] Optionally, the temporary database process is a single-threaded process, and the second data writing unit includes:

[0038] The data writing subunit is used to execute the temporary database process to write the data information into the backup database one by one.

[0039] Optionally, the data caching module includes:

[0040] The second data writing submodule is used to execute the process to write the data information into the backup database when there is a process writing data information to the backup database in the target node.

[0041] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0042] The memory is used to store computer programs;

[0043] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0044] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0045] The embodiments of the present invention have the following advantages: The data processing method deploys a distributed file system in the NAS cluster. Through the network management deployment of the distributed file system itself, each node in the NAS cluster can access the unified database under the distributed file system. At the same time, a backup database is configured in each node. When there is concurrent writing of data information in the unified database, the data information in each target node is cached in its backup database and is no longer directly written to the unified database. Then, the data information in the backup database of each target node is synchronized to the unified database under the distributed file system one by one, which solves the problem of concurrent writing of the unified database and avoids data writing failure or data loss. Attached Figure Description

[0046] Figure 1 This is a flowchart of the steps of a data processing method provided in an embodiment of the present invention;

[0047] Figure 2 This is a structural block diagram of a data processing device provided in an embodiment of the present invention;

[0048] Figure 3This is a block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] SQLite is a file-based database that does not natively support network access. As a lightweight database, it is stored on a single machine in a single file; a single .sqlite file contains everything in the SQLite database. All data in the NAS cluster can be stored in the SQLite database. Other databases, such as MySQL, require a running server, and clients and servers access each other over the network. To reduce resource consumption in a NAS cluster, SQLite can be used for data storage and access. However, as a single-machine file database, SQLite cannot be accessed over the network. Since the nodes in the NAS cluster need to access each other, it is necessary to solve the SQLite cluster access problem and the SQLite write problem under high concurrency.

[0051] To address the aforementioned issues, this invention proposes a data processing method, apparatus, electronic device, and storage medium. A distributed file system is used to store SQLite database files. Each node places its SQLite database file in the distributed file system directory to form a unified database. Through the network management and deployment of the distributed file system itself, all nodes in the cluster can access the unified database, ensuring that there are no network restrictions on database access between nodes. Each node is also configured with a backup database. In the event of high concurrency, the high-concurrency data is retained in the backup database. Finally, the data in the backup database is synchronized one by one to the unified database, thereby solving the concurrent write problem in a NAS cluster.

[0052] Reference Figure 1 The diagram illustrates a flowchart of a data processing method provided in an embodiment of the present invention. The data processing method is applied to a distributed file system deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node of the NAS cluster is configured with a backup database. Both the unified database and the backup database are SQLite databases. The steps include:

[0053] Step 101: When at least two target nodes write data information to the unified database, cache the data information in the target nodes to their backup database.

[0054] Deploying a distributed file system in the NAS cluster ensures that every node in the NAS cluster can access the file system, thus completing the network deployment.

[0055] SQLite is a file-based database. It can only be created and saved on each node. Because it does not support network access, each node can only access the database within its own node and cannot access the data stored in the databases of other nodes. Therefore, each node places the SQLite database file in a distributed file system directory. Through the network management and deployment of the distributed file system itself, each node can access the SQLite database, thus ensuring that each node can access the database information.

[0056] The distributed file system can be GlusterFS (Gluster File System), but it can also be other distributed file systems; this embodiment of the invention does not limit this. For example, it can also be Lustre, Hadoop, or other distributed file systems.

[0057] Specifically, when two or more target nodes in a NAS cluster need to write data to a unified database in a distributed file system, concurrent writing is not supported because SQLite databases have database locks. Therefore, concurrent writing issues can easily lead to data writing failures and data loss.

[0058] Therefore, when two or more target nodes in a NAS cluster need to write data to the unified database in the distributed file system, the data in each target node is cached in its backup database and is no longer written directly to the unified database.

[0059] For example, if there are 5 target nodes in a NAS cluster that need to write data to a unified database in a distributed file system, this is considered a concurrent write situation. However, SQLite databases have database locks, which do not support concurrent writes. Therefore, the data from the 5 target nodes is first cached in their backup database.

[0060] Step 102: Synchronize the data information in the backup databases of at least two target nodes to the unified database one by one.

[0061] Specifically, after caching the data information in each target node to its backup database, the data information in the backup database of each target node is synchronized to the unified database under the distributed file system one by one, thereby completing the writing of data information to the unified database. At the same time, the target node itself will not fail to write to the database due to other target nodes writing to the database, thus solving the database read and write failure caused by the concurrency problem of network access between nodes.

[0062] For example, the data information of the five target nodes in the NAS cluster is first cached in their backup databases. Then, the five target nodes are sorted, and the data information in the backup databases of each target node is synchronized to the unified database one by one. This completes the writing of data information into the unified database. At the same time, the target node itself will not fail to write to the database due to the writing of other target nodes, thus solving the problem of database read and write failures caused by the concurrency of network access between nodes.

[0063] In this embodiment of the invention, by deploying a distributed file system in the NAS cluster and managing the network of the distributed file system itself, each node in the NAS cluster can access the unified database under the distributed file system. At the same time, a backup database is configured in each node. When there is concurrent writing of data information to the unified database, the data information in each target node is cached in its backup database instead of being directly written to the unified database. Then, the data information in the backup database of each target node is synchronized to the unified database under the distributed file system one by one, which solves the problem of concurrent writing to the unified database and avoids data writing failure or data loss.

[0064] This ensures that SQLite databases can be used for data updates and storage within the NAS cluster, avoiding the use of large databases like MySQL, which require server and client deployment, thus reducing resource consumption in the NAS cluster.

[0065] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0066] In an optional embodiment of the present invention, the method further includes: when a target node writes data information to the unified database, writing the data information in the target node into the unified database.

[0067] Specifically, when only one target node is writing data to the unified database under the distributed file system, there is no need to cache the data in the backup database on the target node. The data in the target node can be directly written to the unified database, thus improving work efficiency.

[0068] In an optional embodiment of the present invention, caching the data information in the target node to its backup database includes: when there are at least two processes in the target node writing data information to the backup database, obtaining the timeliness conditions of the processes; and writing the data information to the backup database according to the timeliness conditions.

[0069] Among them, the time limit refers to the time limit for the process to complete. For example, some processes are limited to completing the processing within 20 minutes, while others are limited to completing the processing within 1 day. Therefore, when multiple processes write data information to the backup database at the same time, the time limit of the process needs to be considered.

[0070] Specifically, for the backup database in the target node, since the backup database is also a SQLite database, it has database locks and cannot support concurrent writes. Therefore, when there are at least two processes writing data information to the backup database in the target node, the timeliness conditions of each process writing data information are obtained, and the data information is written to the backup database according to the timeliness conditions of each process.

[0071] For example, when there are 5 processes that need to write data to the backup database, the timeliness conditions of the 5 processes are obtained, and the data information of the 5 processes is written to the backup database according to the timeliness conditions of the 5 processes.

[0072] In an optional embodiment of the present invention, writing the data information into the backup database according to the timeliness condition includes: when it is determined that at least two processes can be completed one by one within the timeliness condition, executing at least two processes one by one to write the data information into the backup database.

[0073] Specifically, when it is determined that at least two processes can be completed one by one within the time limit, the at least two processes are sorted and executed one by one, so as to complete the execution of each process within the time limit and write the data information into the backup database.

[0074] For example, if 100 scheduled tasks are triggered at the same time, and these scheduled tasks all need to synchronize data information to the backup database, if the timeliness of the scheduled tasks is not important, such as needing to complete 100 scheduled tasks within one day, and the estimated total completion time of the 100 scheduled tasks is less than one day, a file blocking lock can be used to ensure the writing order of the backup database. The 100 scheduled tasks can be sorted and executed one by one to solve the problem of concurrent writing to the backup database, thereby avoiding data writing failure or data loss.

[0075] In an optional embodiment of the present invention, the step of writing the data information into the backup database according to the time limit condition includes: when it is determined that at least two processes cannot be completed one by one within the time limit condition, establishing a temporary database process; migrating the data information in at least two processes to the temporary database process and releasing at least two processes; and executing the temporary database process to write the data information into the backup database.

[0076] Specifically, when it is determined that at least two processes cannot be completed one by one within the time limit, a temporary database process is established, the data information in the at least two processes is migrated to the temporary database process, the at least two processes are released, and the temporary database process is executed to write the data information into the backup database.

[0077] For example, if 100 scheduled tasks are triggered at the same time, and these scheduled tasks all need to synchronize data information to the backup database, and if the timeliness of the scheduled tasks is strong, such as needing to complete 100 scheduled tasks within 1 hour, but the estimated total completion time of the 100 scheduled tasks is greater than 1 hour, the 100 data information can be saved in a temporary database process first, and then the 100 data information can be written to the backup database by executing the temporary database process. This solves the problem of concurrent writing to the backup database, thereby avoiding data writing failure or data loss.

[0078] In an optional embodiment of the present invention, the temporary database process is a single-threaded process, and executing the temporary database process to write the data information into the backup database includes: executing the temporary database process to write the data information into the backup database one by one.

[0079] Specifically, the temporary database process is a single-threaded process. Therefore, when the temporary database process is executed, data information is written to the backup database one by one.

[0080] For example, if 100 scheduled tasks are triggered at the same time, and these scheduled tasks all need to synchronize data information to the backup database, if the timeliness of the scheduled tasks is strong, such as needing to complete 100 scheduled tasks within 1 hour, and the estimated total completion time of the 100 scheduled tasks is greater than 1 hour, the 100 data information can be saved in a temporary database process first, and then the temporary database sorts these 100 data information and writes them into the backup database one by one.

[0081] In an optional embodiment of the present invention, caching the data information in the target node to its backup database includes: when there is a process in the target node that writes data information to the backup database, executing the process to write the data information to the backup database.

[0082] Specifically, when there is only one process writing data to the backup database in the target node, the process can be executed directly to write the data directly to the backup database without considering timeliness conditions.

[0083] To better understand the embodiments of the present invention, an example is given below.

[0084] Deploying the GlusterFS file system in a NAS cluster first allows each node in the NAS cluster to access the GlusterFS file system, thereby accessing the files within the file system.

[0085] A unified SQLite database (unified database) is established under the GlusterFS file system, and all data information is synchronized to this database. Then, a database file (backup database) is created for each node, which is only related to itself. When each node is running, the data information is first synchronized to the backup database of its own node, and then synchronized to the unified database one by one. This ensures that updates between database nodes do not affect each other, and completes the database update.

[0086] When a scheduled task is triggered, a temporary database process is first created to save the data that needs to be updated to the temporary database. After all the data from the scheduled tasks is saved to the temporary database process, the data is then uniformly updated and synchronized to the backup database through the temporary database process. By synchronizing all concurrent data to a unified database at once, concurrent read and write issues are reduced, and the SQLite database read and write problems under high concurrency are solved.

[0087] In the above embodiments, the SQLite database can be deployed using the GlusterFS distributed file system, ensuring that all nodes in the cluster can access the SQLite database. Furthermore, by creating backup databases for each node to store backups, the problem of database access failures between nodes is resolved. Scheduled tasks running within the nodes are uniformly updated through the backup databases, addressing the issue of high-concurrency database updates. This ensures that SQLite databases can be used for data updates and storage within the NAS cluster, avoiding the use of large databases like MySQL, which require server and client deployments and reduces resource consumption in the NAS cluster.

[0088] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0089] Reference Figure 2 This diagram illustrates a structural block diagram of a data processing device provided in an embodiment of the present invention. The data processing device 200 is applied to a distributed file system, which is deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node of the NAS cluster is configured with a backup database. Both the unified database and the backup database are SQLite databases.

[0090] The data caching module 201 is used to cache the data information in the target nodes to their backup database when at least two target nodes write data information to the unified database.

[0091] The data synchronization module 202 is used to synchronize the data information in the backup databases of at least two target nodes to the unified database one by one.

[0092] In an optional embodiment of the present invention, it further includes:

[0093] The data writing module is used to write the data information in the target node into the unified database when a target node writes data information to the unified database.

[0094] In an optional embodiment of the present invention, the data caching module includes:

[0095] The timeliness acquisition submodule is used to acquire the timeliness conditions of the process when there are at least two processes writing data information to the backup database in the target node.

[0096] The first data writing submodule is used to write the data information into the backup database according to the timeliness conditions.

[0097] In an optional embodiment of the present invention, the first data writing submodule includes:

[0098] The first data writing unit is used to execute at least two processes one by one to write the data information into the backup database when it is determined that at least two processes can be completed one by one within the time limit.

[0099] In an optional embodiment of the present invention, the first data writing submodule includes:

[0100] A database process creation unit is used to create a temporary database process when it is determined that at least two processes cannot be completed one by one within the time limit conditions.

[0101] A data migration unit is used to migrate data information from at least two of the processes to the temporary database process and release at least two of the processes;

[0102] The second data writing unit is used to execute the temporary database process to write the data information into the backup database.

[0103] In an optional embodiment of the present invention, the temporary database process is a single-threaded process, and the second data writing unit includes:

[0104] The data writing subunit is used to execute the temporary database process to write the data information into the backup database one by one.

[0105] In an optional embodiment of the present invention, the data caching module includes:

[0106] The second data writing submodule is used to execute the process to write the data information into the backup database when there is a process writing data information to the backup database in the target node.

[0107] In the above embodiments, the data processing device deploys a distributed file system in the NAS cluster. Through the network management deployment of the distributed file system itself, each node in the NAS cluster can access the unified database under the distributed file system. At the same time, a backup database is configured in each node. When there is concurrent writing of data information to the unified database, the data information in each target node is cached in its backup database and is no longer directly written to the unified database. Then, the data information in the backup database of each target node is synchronized to the unified database under the distributed file system one by one, which solves the problem of concurrent writing to the unified database and avoids data writing failure or data loss.

[0108] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0109] In addition, this invention also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described data acquisition method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0110] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described data acquisition method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0111] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0112] The electronic device 300 includes, but is not limited to, components such as: a radio frequency unit 301, a network module 302, an audio output unit 303, an input unit 304, a sensor 305, a display unit 306, a user input unit 307, an interface unit 308, a memory 309, a processor 310, and a power supply 311. Those skilled in the art will understand that... Figure 3 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. In embodiments of the present invention, the electronic device includes, but is not limited to, mobile phones, tablet computers, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.

[0113] It should be understood that, in this embodiment of the invention, the radio frequency unit 301 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 310; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 301 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 301 can also communicate with networks and other devices through a wireless communication system.

[0114] The electronic device provides users with wireless broadband internet access through network module 302, such as helping users send and receive emails, browse web pages, and access streaming media.

[0115] The audio output unit 303 can convert audio data received by the radio frequency unit 301 or the network module 302 or stored in the memory 309 into audio signals and output them as sound. Furthermore, the audio output unit 303 can also provide audio output related to specific functions performed by the electronic device 300 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 303 includes a speaker, a buzzer, and a receiver, etc.

[0116] Input unit 304 is used to receive audio or video signals. Input unit 304 may include a graphics processing unit (GPU) 3041 and a microphone 3042. The GPU 3041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 306. The image frames processed by GPU 3041 can be stored in memory 309 (or other storage media) or transmitted via radio frequency unit 301 or network module 302. Microphone 3042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 301 in telephone call mode.

[0117] The electronic device 300 also includes at least one sensor 305, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 3061 according to the ambient light level, and the proximity sensor can turn off the display panel 3061 and / or backlight when the electronic device 300 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 305 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.

[0118] The display unit 306 is used to display information input by the user or information provided to the user. The display unit 306 may include a display panel 3061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0119] User input unit 307 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 307 includes a touch panel 3071 and other input devices 3072. Touch panel 3071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 3071). Touch panel 3071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 310, which receives and executes commands from the processor 310. In addition, touch panel 3071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to touch panel 3071, user input unit 307 may also include other input devices 3072. Specifically, other input devices 3072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0120] Furthermore, the touch panel 3071 can cover the display panel 3061. When the touch panel 3071 detects a touch operation on or near it, it transmits the information to the processor 310 to determine the type of touch event. Subsequently, the processor 310 provides corresponding visual output on the display panel 3061 based on the type of touch event. Although in Figure 3 In this embodiment, the touch panel 3071 and the display panel 3061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 3071 and the display panel 3061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.

[0121] Interface unit 308 serves as an interface for connecting external devices to electronic device 300. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 307 can be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 300, or it can be used to transmit data between electronic device 300 and external devices.

[0122] The memory 309 can be used to store software programs and various data. The memory 309 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 309 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0123] The processor 310 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 309, and by calling data stored in the memory 309, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 310 may include one or more processing units; preferably, the processor 310 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 310.

[0124] The electronic device 300 may also include a power supply 311 (such as a battery) for supplying power to various components. Preferably, the power supply 311 can be logically connected to the processor 310 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0125] In addition, the electronic device 300 includes some functional modules not shown, which will not be described in detail here.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0128] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0129] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0130] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0131] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0134] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0135] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is applied to a distributed file system deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node in the NAS cluster is configured with a backup database. Both the unified database and the backup databases are SQLite databases. Through network management and deployment of the distributed file system, each node can access the SQLite databases. The method includes: When at least two target nodes write data to the unified database, the data from the target nodes is cached in their backup database; the unified database is formed by each node placing its SQLite database file in the directory of the distributed file system. The data information in the backup databases of at least two of the target nodes is synchronized to the unified database one by one; The step of caching the data information in the target node to its backup database includes: When there are at least two processes writing data to the backup database in the target node, the timeliness condition of the process is obtained; the timeliness condition refers to the time limit for the process to complete execution. The data information is written into the backup database according to the timeliness conditions. The step of writing the data information into the backup database according to the timeliness conditions includes: When it is determined that at least two of the processes can be completed one by one within the time limit, at least two of the processes are executed one by one to write the data information into the backup database; The step of writing the data information into the backup database according to the timeliness conditions includes: When it is determined that at least two of the processes cannot be completed one by one within the time limit, a temporary database process is established. Migrate the data information from at least two of the processes to the temporary database process, and release at least two of the processes; Execute the temporary database process to write the data information into the backup database; The temporary database process is a single-threaded process. Executing the temporary database process to write the data information into the backup database includes: The temporary database process is executed to write the data information into the backup database one by one.

2. The method according to claim 1, characterized in that, Also includes: When a target node writes data information to the unified database, the data information in the target node is written into the unified database.

3. The method according to claim 1, characterized in that, The step of caching the data information in the target node to its backup database includes: When there is a process in the target node that writes data information to the backup database, the process is executed to write the data information to the backup database.

4. A data processing apparatus, characterized in that, This is applied to a distributed file system deployed in a NAS cluster. The distributed file system is configured with a unified database, and each node in the NAS cluster is configured with a backup database. Both the unified database and the backup databases are SQLite databases. Through the network management deployment of the distributed file system, each node can access the SQLite databases; including: A data caching module is used to cache the data information in the target nodes to their backup database when at least two target nodes are writing data information to the unified database; the unified database is formed by each node placing SQLite database files in the directory of the distributed file system; The data synchronization module is used to synchronize the data information in the backup databases of at least two target nodes to the unified database one by one; The data caching module includes: The timeliness acquisition submodule is used to acquire the timeliness conditions of a process when there are at least two processes writing data information to the backup database in the target node; the timeliness conditions refer to the time limit for the process to complete execution. The first data writing submodule is used to write the data information into the backup database according to the timeliness conditions; The first data writing submodule includes: The first data writing unit is used to execute at least two processes one by one to write the data information into the backup database when it is determined that at least two processes can be completed one by one within the time limit. The first data writing submodule includes: A database process creation unit is used to create a temporary database process when it is determined that at least two processes cannot be completed one by one within the time limit conditions. A data migration unit is used to migrate data information from at least two of the processes to the temporary database process and release at least two of the processes; The second data writing unit is used to execute the temporary database process to write the data information into the backup database; The temporary database process is a single-threaded process, and the second data writing unit includes: The data writing subunit is used to execute the temporary database process to write the data information into the backup database one by one.

5. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-3.

Images