Data reading and writing method and device, electronic equipment and readable storage medium

Abstract

The application provides a data read-write method and device, electronic equipment and a readable storage medium, comprising: receiving a first input, the first input being first information of client uploaded data; in response to the first input, calling a shard information corresponding to a system table of a warehousing script verification system, selecting a target database for storing the first information, and switching traffic of reading data to a non-target database; receiving a second input, the second input being shard reading information of the target database; in response to the second input, storing the first information in a target shard in a non-reading state, and switching the traffic of reading data back to the target database after storage is completed. The application avoids the occurrence of burr phenomenon, reduces the influence on data retrieval during data writing, improves the stability of the system, and realizes fast full update of data files.

Description

Technical Field

[0001] This invention relates to the field of computer network communication technology, and in particular to a data reading and writing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] As business scales and data volume increases, database read and write operations rise, putting increasing pressure on the database. To alleviate this pressure, existing methods separate read and write operations through physical or network separation, reducing database load and improving performance and stability. Furthermore, separating read and write operations also prevents the primary database from being overwhelmed by numerous write operations, thus improving its performance and availability.

[0003] However, existing separation methods increase network latency because query requests and responses need to be sent and received separately. Furthermore, if read-write separation is not appropriate, write requests may consume too many database server resources, preventing other query requests from obtaining sufficient resources. This also limits the database's concurrent processing capabilities and increases the database's maintenance costs. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data read and write method to solve the glitches in online retrieval when data is entered in batches, and to achieve rapid full update of data files.

[0005] According to one aspect of the present invention, a data read / write method is provided, comprising:

[0006] Receive the first input, which is the first information of the data uploaded by the client;

[0007] In response to the first input, the database entry script is invoked to verify the sharding information corresponding to the system table, the target database for storing the first information is selected, and the data reading traffic is switched to a non-target database.

[0008] Receive a second input, which is the fragmented read information of the target database, including read status and non-read status;

[0009] In response to the second input, the first information is stored in the target shard in a non-read state, and after storage is completed, the traffic for reading data is switched back to the target database.

[0010] Optionally, the step of storing the first information in a target fragment in a non-read state in response to the second input includes:

[0011] Add an update label to the file name storing the first information.

[0012] Optionally, after responding to the second input by storing the first information in the target shard in a non-read state and switching the data reading traffic back to the target database after storage is complete, the method further includes:

[0013] Switch traffic from the non-target database to the target database;

[0014] The non-target database updates the file data corresponding to the target database according to the update annotation;

[0015] Distribute the data reading traffic to the non-target database.

[0016] Optionally, after responding to the second input by storing the first information in the target shard in a non-read state and switching the data reading traffic back to the target database after storage is complete, the method further includes:

[0017] According to the update annotations, the old data information in the corresponding files is cleared.

[0018] According to a second aspect of the present invention, a data read / write device is provided, comprising:

[0019] The first receiving module is used to receive the first input, which is the first information of the data uploaded by the client.

[0020] The first switching module, in response to the first input, calls the database entry script to verify the sharding information corresponding to the system table, selects the target database for storing the first information, and switches the data reading traffic to a non-target database.

[0021] The second receiving module is used to receive a second input, which is the fragmented reading information of the target database, including reading status and non-reading status;

[0022] The first update module, in response to the second input, stores the first information in the target shard in a non-read state, and switches the data reading traffic back to the target database after storage is completed.

[0023] Optionally, the first update module includes:

[0024] The annotation module is used to add update annotations to the file name storing the first information.

[0025] Optionally, the data read / write device further includes:

[0026] The second switching module is used to switch traffic from the non-target database to the target database;

[0027] The second update module is used to update the file data corresponding to the target database in the non-target database according to the update annotation;

[0028] The third switching module is used to distribute the data reading traffic to the non-target database.

[0029] Optionally, the data read / write device further includes:

[0030] The clearing module is used to clear old data information in the corresponding file according to the update annotation.

[0031] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0032] Processor; and

[0033] Stored program memory,

[0034] The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of the first aspects of the invention.

[0035] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are configured to cause a computer to perform the method according to any one of the first aspects of the present invention.

[0036] One or more technical solutions provided in this application embodiment switch the traffic of the selected target database to a non-target database when data is being imported. The non-target database receives the read request and stores the data in the target shard in the non-read state according to the read status of the shard in the target database. This achieves read-write separation when data is imported in batches, avoids the occurrence of spikes, reduces the impact of data writing on data retrieval, improves system stability, and enables fast full updates of data files. Attached Figure Description

[0037] Further details, features, and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0038] Figure 1 A schematic diagram of an example system in which the various methods described herein may be implemented according to exemplary embodiments of the present invention is shown;

[0039] Figure 2 A flowchart of a data read / write method according to an exemplary embodiment of the present invention is shown;

[0040] Figure 3 A schematic diagram of the data flow in a data read / write method according to an exemplary embodiment of the present invention is shown;

[0041] Figure 4 A schematic diagram of database switching in a data read / write method according to an exemplary embodiment of the present invention is shown;

[0042] Figure 5 A schematic diagram illustrating data clearing in a data read / write method according to an exemplary embodiment of the present invention is shown.

[0043] Figure 6 A schematic block diagram of a data read / write apparatus according to an exemplary embodiment of the present invention is shown;

[0044] Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0045] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0046] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0047] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this invention are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0048] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0049] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0050] The present invention will now be described with reference to the accompanying drawings. The technical solutions provided by the embodiments of this application will be explained in detail through specific examples and application scenarios.

[0051] NoSQL generally refers to non-relational databases. Online NoSQL systems, due to their low latency, do not inherently incorporate computational logic. As upper-layer business logic becomes increasingly complex, especially for business units that are algorithm strategy platforms, requiring the retrieval of large amounts of feature data from key-value storage systems for online prediction, overall performance is significantly impacted. Here, a key-value storage system refers to a database that stores data using key-value pairs.

[0052] If some computational modules could be moved to the core of the key-value storage system, allowing for localized computation, retrieval performance could be significantly improved. On the other hand, various algorithmic strategies, such as financial risk control models, require frequent batch updates of relevant feature data, and the data volume is growing explosively, ultimately putting significant pressure on online storage resources. These factors all present new challenges to existing distributed key-value storage systems.

[0053] As business scales and data volume increases, database read and write operations rise, putting increasing pressure on the database. To alleviate this pressure, existing technologies have introduced a data read / write separation architecture. By separating read and write operations, the database load can be reduced, improving performance and stability. Simultaneously, separating read and write operations also prevents the main database from being overwhelmed by numerous write operations, thus improving its performance and availability.

[0054] Different distributed database read / write splitting methods are suitable for different scenarios and business needs, and should be selected and optimized according to specific circumstances. At the same time, read / write splitting also requires attention to data security and privacy protection. Common distributed database read / write splitting methods include the following:

[0055] (1) Physical separation: Read and write operations are placed on different physical disks, such as placing read operations on local disks and writing operations on cloud storage. This method can reduce latency and bandwidth consumption between read and write operations, and improve the performance and stability of the database.

[0056] (2) Network separation: Read operations are placed on the primary network interface, while write operations are placed on the secondary network interface. This reduces network latency and bandwidth consumption, and improves database performance and stability.

[0057] (3) Database configuration: In a distributed database, strategies such as read / write splitting switches and read / write splitting sorting can be used to control the degree and method of read / write splitting. For example, read / write splitting switches can be used to control the read / write ratio of the database, and read / write splitting sorting can be used to arrange the data writing order based on the data's read operation history.

[0058] (4) Cache optimization: In distributed databases, caching mechanisms can be used to cache frequently used data in memory, reducing the number of database read and write operations and improving database performance and stability.

[0059] (5) Application Configuration: Applications can also be customized for read / write separation based on business needs and actual conditions. For example, the database read / write ratio can be adjusted according to the data read and write frequency, and the database caching mechanism can be configured according to the data access method.

[0060] However, the main drawbacks of current distributed database read / write separation are as follows:

[0061] (1) Increased network latency. Read-write separation increases network latency because the database server needs to send and receive query requests and responses separately, which may cause some query requests to wait for responses from other query requests.

[0062] (2) Increased database load. If read-write separation is not handled properly, write requests may consume too many database server resources, causing other query requests to be unable to obtain sufficient resources.

[0063] (3) It may increase the database's concurrent processing capacity. If read-write separation is not appropriate, query requests may consume too many resources of the database server, causing other query requests to be unable to obtain enough resources, thereby limiting the database's concurrent processing capacity.

[0064] (4) It may increase the maintenance cost of the database. If read and write separation is not done properly, it may cause communication delays and data inconsistencies between database servers, thus increasing the maintenance cost of the database.

[0065] In one embodiment of this application, Eggroll-rianbow experiences numerous offline update operations after the decision platform goes live. These batch update operations, performed independently by multiple business units, are difficult to control inbound behavior, impacting online retrieval. Therefore, a traffic switching interface is designed to regulate inbound behavior during the inbound process. Eggroll is a distributed NoSQL storage and computational key-value database, while rainbow is a storage engine that supports batch updates and real-time read-only operations.

[0066] With the development of internet technology, big data processing and cloud computing technologies are being applied more and more widely. Many business applications need to correlate and integrate data from different sources in order to better extract data value. Faced with massive amounts of user data and rich feature dimensions, decision-making platforms often have key-value query requirements for hundreds of terabytes of persistent data in their online prediction service scenarios, and expect offline data to be synchronized online on T+1 days to accelerate model iteration.

[0067] In one embodiment of this application, such as Figure 1 As shown, the data read / write method provided in this application mainly includes the following stages: executing MR task, executing data entry task, executing data activation task, and clearing old data task.

[0068] In the MR task phase, where MR stands for MapReduce, a parallel computing model and method for large-scale data processing, analysis revealed that the decision-making platform's model data is obtained offline from various data warehouses and stored in HDFS. This means that only retrieval is a real-time requirement online, while updates are not real-time but rather periodically in batches. HDFS stands for Hadoop Distributed File System, a distributed file system designed to run on commodity hardware. By combining this with the MapReduce mechanism, customized Map and Reduce tasks consistent with online sharding can be used to quickly generate the data format required for online loading offline. Then, the data to be updated is loaded onto the online system via storage nodes, achieving rapid batch updates.

[0069] During the database ingestion phase, due to periodic batch updates, the system periodically scans for tasks awaiting ingestion to determine if the limit has been reached. If not, a new ingestion task is added, and its status is updated to "ingestion in progress." Data is then retrieved from the data collection system (DGS) and stored locally. During data retrieval, the current file (`current`) in the shard directory is verified. The `current` file records the index value of the currently retrieved file, which toggles between 0 and 1. Newly ingested data files are renamed based on this index, thus separating online retrieved files from newly written files. Finally, the ingestion status of each shard is reported to the master node. The master node is responsible for creating system tables, maintaining metadata, and performing heartbeat checks for ingestion operations. The proxy module is responsible for fetching metadata from the master node and reading / writing data to other nodes.

[0070] During the execution phase of the activation task, after the decision platform went live, there were many offline batch update operations. Because these batch update operations were performed individually by the business side, it was difficult to control their data entry behavior, frequently causing retrieval disruptions online. Therefore, traffic switching was needed to standardize the data entry behavior during the data entry process, achieving strict read-write separation and improving the stability of the online system. After all shards of the table were successfully entered into the database, traffic was sequentially switched between the primary and standby databases. When switching to the standby database, the primary database index was updated and took effect; when switching back to the primary database, the standby database index was updated and took effect.

[0071] During the old data cleanup task phase, after the table switchover is successfully completed, the old data is cleared. The current index value of each shard is queried for each node, and the data files to be cleared are updated automatically based on this current index value. Here, a node is a database storage node responsible for data storage, heartbeat reporting, etc.

[0072] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a data read / write method provided in an embodiment of this application. The method may include the following steps S201 to S204:

[0073] S201, Receive the first input, where the first input is the first information of the data uploaded by the client.

[0074] S202, in response to the first input, call the database entry script to verify the sharding information corresponding to the system table, select the target database for storing the first information, and switch the data reading traffic to a non-target database.

[0075] In this embodiment, the target database is the primary database. An index is built using a traffic switching interface to standardize data insertion behavior, thereby achieving strict read-write separation and improving system stability. When data is uploaded from the client, traffic switching is used to redirect connections to the primary and backup databases to the backup database (i.e., a non-target database). In this state, all traffic reading from the database is distributed to the non-target database, ensuring that data insertion to the target database is not interfered with, while maintaining read access to the non-target database.

[0076] S203, Receive a second input, the second input being the fragmented read information of the target database, the fragmented read information including read status and non-read status.

[0077] In this embodiment, after the traffic switch during data ingestion, the status of each shard in the database is determined by the shard read information to avoid updating the data being read, which would cause interference with data read and write operations. Each table in the database has a current file corresponding to its shard. The current file records the current search index (0, 1). When there is a data ingestion task, the index is created in the current file. 0 represents a non-reading state, and 1 represents a reading state.

[0078] S204, in response to the second input, the first information is stored in the target shard in the non-read state, and after storage is completed, the traffic for reading data is switched back to the target database.

[0079] In this embodiment, the first information uploaded by the client is stored in a non-read shard of the target database. After storage, the 0 in the current file is switched to 1 to update the database entry status. At this point, traffic is switched back to the target database, allowing data from both the target and non-target databases to be read. This achieves strict read-write separation, thereby improving the stability of the online system.

[0080] In one embodiment of this application, such as Figure 3 As shown, when an inbound request is received, the inbound script is called to notify the database master node to verify the sharding information corresponding to the system table, notify the database storage node to pull the sharded data, select the target database, and store the inbound data in the target shard in a non-read state. After storage is completed, the database master node notifies the database storage node to switch the retrieval index in the current file.

[0081] In one embodiment of this application, storing the first information in a target fragment in a non-read state in response to the second input includes:

[0082] S2041, Add an update label to the file name storing the first information.

[0083] In this embodiment, by setting an update label for the file name storing the first information, the updated file can be distinguished from other files.

[0084] In one embodiment of this application, after both the target database and non-target databases have completed batch updates of the same data, the set update markers are removed, so that the target file is unified with other files.

[0085] In one embodiment of this application, after responding to the second input, storing the first information in a target shard in a non-read state, and switching the data reading traffic back to the target database after storage is complete, the method further includes:

[0086] S205, switch the traffic from the non-target database to the target database;

[0087] S206, the non-target database updates the file data corresponding to the target database according to the update annotation;

[0088] S207, distribute the data reading traffic to the non-target database.

[0089] In this embodiment, as Figure 4 As shown, the target database is the primary database, and the non-target database is the standby database. The files in the primary and standby databases correspond to each other. When the primary database completes the data update, the traffic from the non-target database is switched to the target database, which then handles the data reading. The non-target database performs batch data updates. At this time, the standby database updates the files corresponding to the primary database according to the update markers. When both the target and non-target databases have completed the same data update, the data reading traffic is shared by both the target and non-target databases.

[0090] In one embodiment of this application, after responding to the second input, storing the first information in a target shard in a non-read state, and switching the data reading traffic back to the target database after storage is complete, the method further includes:

[0091] S208, according to the update annotation, clear the old data information in the corresponding file.

[0092] In this embodiment, as Figure 5 As shown, once the target database or non-target database successfully completes the traffic switch, the old data before the update is cleared. The database storage node queries the current file of the current shard and updates the data files that need to be cleared according to the update annotation.

[0093] In one embodiment of this application, the data read / write method further includes:

[0094] 1. A scheduled task scans for pending data regeneration (MR) tasks and checks if the MR task limit has been reached. If not, additional tasks are added, and the MR task status is updated. Once data production is complete, the MR task status is updated to "completed".

[0095] 2. A scheduled task scans for tasks to be inserted into the database and checks if the task limit has been reached. If not, additional tasks are added, and the task status is updated to "inserting." Simultaneously, the insertion script is invoked to begin the insertion process. The insertion script verifies the existence of the shard folder corresponding to the table. If the folder does not exist, an insertion failure is reported to the master. If the folder exists, data is pulled from HDFS to the local machine. HDFS is the Hadoop Distributed File System, designed to run on commodity hardware. During data retrieval, the progress is continuously queried to check for any failed shard pulls. If any are found, the insertion process is terminated until completion. Before pulling data, the shard folder is checked to ensure it contains a `current` file, which records the current shard's query file information. When pulling data from HDFS to the local machine, the data filename is updated between 0 and 1 based on the information in the `current` file. Finally, the insertion status of each shard is reported to the master for the current node.

[0096] 3.a. A scheduled task scans for tasks awaiting activation and checks if the current activation limit has been reached. If not, additional tasks are added. b. It determines whether a primary / backup traffic switch is needed. If so, all proxy locks are held before switching traffic, and the task status is updated to "in progress." c. The activation interface is called, and the system iterates through all shards of the table awaiting activation to determine if they are ready (i.e., all have been successfully installed). d. The current file value is queried for each node, and the shard indexes under each node are updated (0 and 1 are updated). The current file value is also updated. If a failure occurs, a rollback is performed. After all operations are successful, the shard installation status is set to 0, and all proxy locks are released. After a successful table switch on one side, traffic is switched back, and the same operation is repeated on the other side.

[0097] 4. After the table switch is successfully completed, the deletion interface is called to delete the old data. First, the table and node to be deleted are verified. Then, the status of all shards of the table is checked in a loop to see if the table switch is successful. If there is no error, the current value of each node is queried. Then, the old data of each shard in the previous database is cleared under each node. This operation is based on the current value.

[0098] The data read / write method provided in this application embodiment switches the traffic of the selected target database to a non-target database when data is being imported. The non-target database receives the read request and stores the data in the target shard in the non-read state according to the read status of the shard in the target database. This achieves read / write separation when data is imported in batches, avoids the occurrence of spikes, reduces the impact of data writing on data retrieval, improves system stability, and enables fast full updates of data files.

[0099] Corresponding to the above embodiments, see [link to relevant documentation]. Figure 6 This application also provides a data read / write device 600, comprising:

[0100] The first receiving module 601 is used to receive a first input, wherein the first input is the first information of the data uploaded by the client.

[0101] The first switching module 602, in response to the first input, calls the database entry script to verify the sharding information corresponding to the system table, selects the target database for storing the first information, and switches the data reading traffic to a non-target database.

[0102] The second receiving module 603 is used to receive a second input, which is the fragmented reading information of the target database, including reading status and non-reading status;

[0103] The first update module 604, in response to the second input, stores the first information in the target shard in a non-read state, and switches the data reading traffic back to the target database after storage is completed.

[0104] Optionally, the first update module 604 includes:

[0105] The annotation module 6041 is used to add update annotations to the file name storing the first information.

[0106] Optionally, the data read / write device 600 further includes:

[0107] The second switching module 605 is used to switch the traffic from the non-target database to the target database;

[0108] The second update module 606 is used to update the file data corresponding to the target database in the non-target database according to the update annotation;

[0109] The third switching module 607 is used to distribute the data reading traffic to the non-target database.

[0110] Optionally, the data read / write device 600 further includes:

[0111] The clearing module 608 is used to clear old data information in the corresponding file according to the update annotation.

[0112] The data read / write device provided in this embodiment switches the traffic of the selected target database to a non-target database when data is being imported. The non-target database receives the read request and stores the data in the target shard in the non-read state according to the read status of the shard in the target database. This achieves read / write separation when data is imported in batches, avoids glitches, reduces the impact of data writing on data retrieval, improves system stability, and enables fast full updates of data files.

[0113] An exemplary embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of the present invention.

[0114] An exemplary embodiment of the present invention also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0115] An exemplary embodiment of the present invention also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method according to an embodiment of the present invention.

[0116] refer to Figure 7 The present invention will now be described in the form of a structural block diagram of an electronic device 700 that can serve as a server or client of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0117] like Figure 7As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0118] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0119] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the auditing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 may be configured to perform the auditing method by any other suitable means (e.g., by means of firmware).

[0120] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0122] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0125] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A data read / write method, characterized in that, include: Receive the first information of the data uploaded by the client; Call the database entry script to verify the sharding information corresponding to the system table, select the target database for storing the first information, and switch the traffic connecting to the target database to read the database to a non-target database; Based on the current file under the shard corresponding to each table in the target database, obtain the shard read information of the target database. The current file records the current search index (0, 1). 0 in the current search index (0, 1) indicates that the shard read information of the corresponding shard is in a non-read state, and 1 in the current search index (0, 1) indicates that the shard read information of the corresponding shard is in a read state. Based on the sharding read information of the target database, the first information is stored in the target shard in the non-reading state, and after storage is completed, the traffic for reading data is switched back to the target database, and an update label is added to the file name storing the first information. Switch traffic from the non-target database to the target database; The non-target database updates the file data corresponding to the target database according to the update annotation; Distribute the data reading traffic to the non-target database.

2. The data read / write method according to claim 1, characterized in that, The step of storing the first information in the target shard in a non-reading state according to the shard reading information of the target database, and switching the data reading traffic back to the target database after storage is completed, further includes: According to the update annotations, the old data information in the corresponding files is cleared.

3. A data read / write device, characterized in that, include: The first receiving module is used to receive the first information of the data uploaded by the client. The first switching module calls the database entry script to verify the sharding information corresponding to the system table, selects the target database storing the first information, and switches the traffic connecting to the target database to read the database to a non-target database. The second receiving module is used to obtain the shard read information of the target database according to the current file under the shard corresponding to each table in the target database. The current file records the current search index (0, 1). 0 in the current search index (0, 1) indicates that the shard read information of the corresponding shard is in a non-reading state, and 1 in the current search index (0, 1) indicates that the shard read information of the corresponding shard is in a reading state. The first update module is used to store the first information in the target shard in a non-reading state according to the shard reading information of the target database, and switch the data reading traffic back to the target database after storage is completed, and add an update label to the file name storing the first information. The second switching module is used to switch traffic from the non-target database to the target database; The second update module is used to update the file data corresponding to the target database in the non-target database according to the update annotation; The third switching module is used to distribute the data reading traffic to the non-target database.

4. The data read / write device according to claim 3, characterized in that, The data reading and writing device further includes: The clearing module is used to clear old data information in the corresponding file according to the update annotation.

5. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-2.

6. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-2.

Images