Data storage method and apparatus

Abstract

The application relates to the technical field of data storage, and provides a data storage method and device.The method comprises the following steps: obtaining additional data of message data according to the message data sent by a calling party; storing the message data into a first data table and storing the additional data into a second data table; wherein the message data is used for sending to an associated party corresponding to the calling party; and the additional data comprises a time of first sending of the message data to the associated party, a time of next sending of the message data to the associated party, a remaining sending frequency and a response result of the associated party to the message data.The data storage method provided by the embodiment of the application can reduce the load of a database when data is extracted from the database subsequently, and improve the query efficiency of the data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, in particular to a data storage method and device. BACKGROUND

[0002] In the research and development of gateway applications, the message data of the notification type usually has the requirement of storage forwarding. The storage forwarding refers to that the gateway sends the message data of the notification type to the calling party, and the gateway forwards the message data to the associated party after receiving the message data, and stores the message data and the additional data of the message data. This process is the storage forwarding. If the associated party does not respond to the message data within the predetermined time, the gateway will send the stored message data to the associated party to notify the associated party until the associated party responds to the message data or the number of notifications reaches the preset number.

[0003] In this process, the message data and the additional data need to be stored in the database. Since the data volume of the message data and the additional data is large, how to effectively store the message data and the additional data to reduce the load of the database when extracting data from the database subsequently and improve the query efficiency of the data is a technical problem that needs to be solved at present. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art. To this end, the present application provides a data storage method, which can reduce the load of the database when extracting data from the database subsequently and improve the query efficiency of the data.

[0005] The present application also provides a data storage device.

[0006] The present application also provides an electronic device.

[0007] The present application also provides a computer readable storage medium.

[0008] The data storage method according to the first aspect of the present application comprises:

[0009] According to the message data sent by the calling party, the additional data of the message data is obtained;

[0010] The message data is stored in a first data table, and the additional data is stored in a second data table;

[0011] The message data is used to send to the associated party corresponding to the calling party;

[0012] The additional data includes the time of the first sending of the message data to the associated party, the time of the next sending of the message data to the associated party, the remaining sending number, and the response result of the associated party to the message data.

[0013] The data storage method provided by the embodiments of the present application stores the message data sent by the calling party to the associated party into a first data table, and stores the additional data of the message data into a second data table, so that the message data and the additional data of the message data required to be stored in the storage forwarding process are stored separately into different data tables, so that when the message data is extracted and retransmitted subsequently, only the first data table needs to be queried, the amount of data to be queried is reduced, a large amount of I / O operation is avoided when the message data is extracted subsequently, the load of the database when the data is extracted from the database is reduced, and the query efficiency of the data is improved.

[0014] According to an embodiment of the present application, the message data is stored into the first data table, comprising:

[0015] When it is determined that the associated party does not respond to the message data within a preset time period when the message data is sent to the associated party for the first time, the message data is stored into the first data table.

[0016] According to an embodiment of the present application, further comprising:

[0017] When it is determined that the associated party responds to the message data within a preset time period when the message data is sent to the associated party for the first time, the message data is deleted.

[0018] According to an embodiment of the present application, further comprising:

[0019] According to the message ID of the message data, a data index of the first data table is generated; and,

[0020] According to each preset field of the additional data, a data index of the second data table is generated.

[0021] According to an embodiment of the present application, after the message data is stored into the first data table, further comprising:

[0022] The message data is acquired from the first data table at a time;

[0023] When it is determined that the associated party responds to the message data, the message data is deleted from the first data table.

[0024] According to an embodiment of the present application, further comprising:

[0025] When it is determined that the storage time length of the additional data in the second data table reaches a preset time length, the additional data is deleted from the second data table.

[0026] According to one embodiment of the present application, the preset time length is determined according to a service type of the message data corresponding to the additional data.

[0027] According to the data storage device of the second aspect of the present application, comprising:

[0028] The data acquisition module is configured to acquire, according to the message data sent by the invoker, additional data of the message data.

[0029] The data storage module is configured to store the message data to a first data table and store the additional data to a second data table.

[0030] The message data is used to be sent to an associated party corresponding to the invoker.

[0031] The additional data includes a time of first sending the message data to the associated party, a time of next sending the message data to the associated party, a remaining sending number, and a response result of the associated party to the message data.

[0032] According to the electronic device of the third aspect of the present application, comprising a processor and a memory storing a computer program, the processor executes the computer program to realize the data storage method of any one of the above embodiments.

[0033] According to the computer readable storage medium of the fourth aspect of the present application, a computer program is stored thereon, and the computer program is executed by a processor to realize the data storage method of any one of the above embodiments.

[0034] According to the computer program product of the fifth aspect of the present application, comprising: the computer program is executed by a processor to realize the data storage method of any one of the above embodiments.

[0035] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0036] By storing the message data sent by the invoker to the associated party to the first data table and storing the additional data of the message data to the second data table, the message data and the additional data of the message data required to be stored in the storage forwarding process are separated and stored in different data tables, so that subsequent extraction and retransmission of the message data only need to be queried in the first data table, the amount of data to be queried is reduced, a large amount of I / O operation is avoided in subsequent extraction of the message data, and the load of the database in subsequent extraction of data from the database is reduced, and the query efficiency of the data is improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0038] Figure 1 is a flowchart of a data storage method provided by an embodiment of the application;

[0039] Figure 2 is a flowchart of the packet data storage of the data storage method provided by an embodiment of the application; Figure 1

[0040] Figure 3 is a flowchart of the data storage to the data table in an embodiment of the application;

[0041] Figure 4 is a flowchart of a data storage method provided by another embodiment of the application;

[0042] Figure 5 is a structural diagram of a data storage device provided by an embodiment of the application;

[0043] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be clearly and completely described in the following with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all the other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the application.

[0045] In the following, the data storage method and device provided by the embodiments of the application will be introduced and described in detail through several specific embodiments.

[0046] ​In an embodiment, a data storage method is provided, which is applied to a gateway device for data storage. The gateway device, also known as an inter-network connector or a protocol converter, is an intermediate device connecting an internal network of the device to other networks on the Internet, and is used to realize interconnection between the associated party and the calling party. The gateway device can include a gateway server, which can be a standalone server or a server cluster composed of multiple servers, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence sampling point device.

[0047] As shown in Figure 1 The data storage method provided by the embodiment includes the following steps:

[0048] In step 101, additional data of the message data sent by the calling party is obtained according to the message data.

[0049] In step 102, the message data is stored in a first data table, and the additional data is stored in a second data table.

[0050] The message data is used to be sent to the associated party corresponding to the calling party.

[0051] The additional data includes the time of the first sending of the message data to the associated party, the time of the next sending of the message data to the associated party, the remaining sending times, and the response result of the associated party to the message data.

[0052] By storing the message data sent by the calling party to the associated party in the first data table, and storing the additional data of the message data in the second data table, the message data and the additional data of the message data required to be stored in the storage and forwarding process are separated and stored in different data tables, so that subsequent extraction and retransmission of the message data only need to be queried in the first data table, reducing the amount of data to be queried, avoiding a large amount of I / O operation when extracting the message data, and further reducing the load of the database when extracting data from the database, thereby improving the query efficiency of the data.

[0053] In an embodiment, the invoker can be a terminal device sending the message data. When the gateway device receives the message data from any invoker, or extracts the message data sent by the invoker from the first data table, the message data is forwarded to the corresponding associated party of the invoker, and the additional data corresponding to the message data is obtained according to the message data. For example, after the message data is forwarded to the corresponding associated party of the invoker, the time when the message data is first sent to the associated party is obtained according to the message ID of the message data, and the response result of the invoker to the message data after the message data is forwarded to the invoker this time is recorded.

[0054] In addition, the gateway device is preset with a sending time interval and a total sending number corresponding to the message data of each data type. For example, if the data type is a financial notification message, the corresponding sending time interval can be 30S, and the total sending number is 5. In this way, when the data type of the obtained message data is a financial notification message, the corresponding sending time interval can be 60S, and the total sending number is 5. At this time, the next sending time of the message data to the associated party can be determined according to the timestamp when the message data is sent to the associated party. For example, if the timestamp when the message data is sent to the associated party is 8:00 a.m., the next sending time of the message data to the associated party is 8:01 a.m. At the same time, the remaining sending number of the message data can be determined according to the cumulative sending number of the current message data and the total sending number of the message data. For example, assuming that the cumulative sending number of the message data is 1 after sending the message data this time, and the total sending number of the message data is 5, the remaining sending number of the message data can be determined to be 4.

[0055] After obtaining the additional data of the message data, the message data and its additional data can be stored separately, so as to store the message data in the first data table and store the additional data in the second data table different from the first data table. In order to reduce the data storage amount of the first data table and reduce the data amount to be traversed when querying the message data in the first data table in the future, improve the query efficiency of the message data in the future, in an embodiment, whether the message data is stored in the first data table can be detected first. If the message data is stored in the first data table, the message data is not stored again; otherwise, the message data is stored in the first data table. In this way, a large amount of repeated message data can be avoided in the first data table, so as to reduce the message data to be stored in the first data table, and further reduce the data amount to be traversed when querying the message data in the first data table in the future, reduce the resource consumption when accessing the first data table, and improve the query efficiency of the message data in the future.

[0056] For the additional data of the message data, no deduplication processing is performed, and the additional data can be directly stored in the second data table to facilitate the query of the sending result of each message data and avoid omission.

[0057] When the gateway device receives the message data of the calling party each time, if the message data is the message data sent by the calling party for the first time, the message data does not exist in the first data table, and the message data is directly inserted into the first data table. However, considering that the data volume of the message data is usually large, even if only the message data sent by the calling party for the first time is stored, when the calling party sends too much message data or there are too many calling parties, the data volume of the message data stored in the first data table is also large. Therefore, in order to further reduce the storage pressure of the first data table and further reduce the access consumption of the first data table, in an embodiment, as shown in the following table, the message data is stored in the first data table, including: Figure 2

[0058] In step 201, when the message data is sent to the associated party for the first time, the associated party does not respond to the message data within a preset time period, and the message data is stored in the first data table.

[0059] In an embodiment, when the message data is the message data sent by the calling party and is not the message data extracted from the first data table, it can be detected whether the message data is the message data sent to the associated party for the first time. For example, the message data can be matched with each message data in the first data table, and if the message data does not match each message data in the first data table, it indicates that the message data is the message data sent to the associated party for the first time. After it is determined that the message data is the message data sent to the associated party for the first time, it is detected whether the associated party responds to the message data within a preset time period, such as 1 minute, for example, whether the response data sent by the associated party in response to the message data is received within the preset time period. If it is detected that the associated party does not respond to the message data within the preset time period, the message data is stored in the first data table, so that only the message data sent to the associated party for the first time and not responded to by the associated party within the preset time period is stored in the first data table, and the data volume stored in the first data table can be greatly reduced, and the access consumption of the first data table can be reduced.

[0060] In an embodiment, if it is determined that the message data is not the message data sent to the associated party for the first time, it indicates that the message data is already stored in the first data table, and the message data can be ignored at this time. If it is determined that the associated party responds to the message data within the preset time period when the message data is sent to the associated party for the first time, it indicates that the associated party is informed successfully, and the message data can be deleted at this time, so that the message data that does not need to be sent to the associated party is not stored in the first data table, and the data volume stored in the first data table is reduced.

[0061] ​In consideration of some services must store the message data from the calling party as soon as received, such as the financial notification message, the message data from the calling party is first received, and the service type corresponding to the message data is detected. If the service type corresponding to the message data is a pre-specified service type, such as the financial notification message, the message data is directly stored into the first data table; if the service type corresponding to the message data is not the pre-specified service type, whether the message data is stored into the first data table is determined according to whether the associated party responds to the message data within a preset time period.

[0062] Similarly, for the storage of the additional data, whether the additional data of the message data from the calling party is stored into the second data table is also determined according to whether the associated party responds to the message data within a preset time period. If the associated party responds to the message data within the preset time period, it indicates that the notification of the associated party is successful, and the additional data of the message data can be deleted to reduce the amount of data stored in the second data table. If the associated party does not respond to the message data within the preset time period, the additional data of the message data is stored into the second data table.

[0063] After the message data is stored into the first data table and the message data is stored into the second data table, in order to further improve the data query efficiency, in an embodiment, as shown in Figure 3 , the method further comprises:

[0064] Step 301, generating a data index of the first data table according to the message ID of the message data; and,

[0065] Step 302, generating a data index of the second data table according to each pre-specified field of the additional data.

[0066] In an embodiment, after the message data is stored into the first data table, the message ID of the message data is extracted to generate a data index of the first data table. Thus, the corresponding message data can be accurately found from the first data table according to the message ID, thereby improving the query efficiency.

[0067] For the second data table, although the data index can also be generated by the message ID, since there can be multiple pieces of additional data for the same message data, if the data index is generated by the message ID, a large amount of additional data can be queried by the same message ID. Therefore, in order to accurately retrieve the additional data from the second data table, each preset field in the additional data can be extracted, such as a first time field for recording the time when the message data is first sent to the associated party, a first time field for recording the time when the message data is next sent to the associated party, a number field for recording the remaining number of times of sending the message data, and a result field for recording the response result of the associated party to the message data, to generate the data index of the second data table one by one, so that subsequent complex queries can be performed on the second data table according to each preset field to accurately find the corresponding additional data, thereby improving the query efficiency.

[0068] In order to further reduce the storage pressure of the database, in an embodiment, as shown in Figure 4 After the message data is stored in the first data table, the method further includes:

[0069] Step 401, periodically obtaining the message data sent to the associated party from the first data table;

[0070] Step 402, determining the response of the associated party to the message data, and deleting the message data from the first data table.

[0071] In an embodiment, after the message data is stored in the first data table, a storage forwarding mechanism is triggered periodically to extract the message data from the first data table and forward it to the associated party. After the message data is extracted from the first data table and forwarded to the associated party each time, it is detected whether the response data fed back by the associated party in response to the message data is received. If the response data is received, it indicates that the associated party has responded to the message data, i.e., the message data has successfully notified the associated party, and at this time the message data can be deleted from the first data table. Therefore, the first data table maintains the minimum necessary storage as much as possible, reducing access consumption.

[0072] In addition, when the number of times of forwarding the message data from the first data table to the associated party reaches a preset number, i.e., the remaining number of times of sending the message data is 0, it indicates that the message data cannot be sent, and at this time the message data can also be deleted from the first data table, thereby reducing the storage amount of the first data table as much as possible.

[0073] For the second data table, a periodic cleaning method is used to maintain a smaller amount of data in the second data table. Specifically, the storage duration of the additional data stored in the second data table can be detected. If it is detected that the storage duration of the additional data in the second data table reaches a preset duration, such as 24 hours, the additional data is deleted from the second data table.

[0074] Considering that the importance of the additional data of the message data of different service types is different, the additional data of the message data of the financial type is usually more important. Therefore, different preset time lengths can be set for the additional data of the message data of different service types. When detecting the storage time length of a certain additional data, the preset time length corresponding to the additional data can be acquired according to the data type of the service data corresponding to the additional data, and then whether the storage time length of the additional data in the second data table reaches the preset time length is detected, so that the additional data is deleted from the second data table when the storage time length of the additional data in the second data table reaches the preset time length.

[0075] The data storage apparatus provided in the present application is described below, and the data storage apparatus described below can be referred to in correspondence with the data storage method described above.

[0076] In an embodiment, as shown in Figure 5 a data storage apparatus is provided, comprising:

[0077] The data acquisition module 210 is configured to acquire additional data of the message data according to the message data sent by the invoker.

[0078] The data storage module 220 is configured to store the message data to a first data table and store the additional data to a second data table.

[0079] The message data is used to be sent to an associated party corresponding to the invoker.

[0080] The additional data includes the time of the first sending of the message data to the associated party, the time of the next sending of the message data to the associated party, the remaining sending times, and the response result of the associated party to the message data.

[0081] By storing the message data sent by the invoker to the associated party to the first data table and storing the additional data of the message data to the second data table, the message data and the additional data of the message data required to be stored in the storage forwarding process are separated and stored in different data tables, so that subsequent extraction and retransmission of the message data only need to be queried in the first data table, the amount of data to be queried is reduced, a large amount of I / O operation is avoided when the message data is extracted subsequently, the load of the database when the data is extracted from the database is reduced, and the query efficiency of the data is improved.

[0082] In an embodiment, the data storage module 220 is specifically configured to:

[0083] When the message data is first sent to the associated party, and the associated party does not respond to the message data within a preset time period, the message data is stored in the first data table.

[0084] In one embodiment, the data storage module 220 is further configured to:

[0085] When the message data is first sent to the associated party, the associated party responds to the message data within a preset time period and deletes the message data.

[0086] In one embodiment, the data storage module 220 is further configured to:

[0087] Generate a data index for the first data table based on the message ID of the message data; and,

[0088] A data index for the second data table is generated based on each preset field of the additional data.

[0089] In one embodiment, the data storage module 220 is further configured to:

[0090] After storing the message data in the first data table, the message data is periodically retrieved from the first data table and sent to the associated party;

[0091] If the associated party responds to the message data, the message data is deleted from the first data table.

[0092] In one embodiment, the data storage module 220 is further configured to:

[0093] Once the storage duration of the additional data in the second data table reaches a preset duration, the additional data is deleted from the second data table.

[0094] In one embodiment, the preset duration is determined based on the service type of the message data corresponding to the additional data.

[0095] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call a computer program in the memory 830 to execute a data storage method, such as including:

[0096] Based on the message data sent by the caller, obtain the additional data of the message data;

[0097] store the message data to a first data table and store the additional data to a second data table;

[0098] The message data is used to send to an associated party corresponding to the calling party.

[0099] The additional data includes a first sending time of the message data to the associated party, a next sending time of the message data to the associated party, a remaining sending number of the message data to the associated party, and a response result of the associated party to the message data.

[0100] Further, the logic instructions in the memory 830 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0101] On the other hand, the embodiments of the present application also provide a processor readable storage medium, which stores a computer program. The computer program is used to enable a processor to execute the method provided by the above-mentioned embodiments, for example, including:

[0102] According to the message data sent by the calling party, additional data of the message data is acquired;

[0103] store the message data to a first data table and store the additional data to a second data table;

[0104] The message data is used to send to an associated party corresponding to the calling party.

[0105] The additional data includes a first sending time of the message data to the associated party, a next sending time of the message data to the associated party, a remaining sending number of the message data to the associated party, and a response result of the associated party to the message data.

[0106] The processor-readable storage medium can be any available medium or data storage device that can be accessed by a processor including, but not limited to, a magnetic storage (e.g., floppy diskette, hard disk, tape, MO, etc.), an optical storage (e.g., CD, DVD, BD, HVD, etc.), and a semiconductor storage (e.g., ROM, EPROM, EEPROM, NAND FLASH, SSD, etc.), etc.

[0107] The apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0109] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data storage method, characterized in that, include: Based on the message data sent by the caller, obtain the additional data of the message data; The message data is stored in a first data table, and the additional data is stored in a second data table; The message data is used to send to the associated party corresponding to the caller; The additional data includes the time when the message data was first sent to the associated party, the time when it was sent to the associated party again, the remaining number of transmissions, and the associated party's response to the message data.

2. The data storage method according to claim 1, characterized in that, Storing the message data in the first data table includes: When the message data is first sent to the associated party, and the associated party does not respond to the message data within a preset time period, the message data is stored in the first data table.

3. The data storage method according to claim 2, characterized in that, Also includes: When the message data is first sent to the associated party, the associated party responds to the message data within a preset time period and deletes the message data.

4. The data storage method according to any one of claims 1-3, characterized in that, Also includes: Generate a data index for the first data table based on the message ID of the message data; and, A data index for the second data table is generated based on each preset field of the additional data.

5. The data storage method according to claim 1, characterized in that, After storing the message data in the first data table, the method further includes: The message data is periodically retrieved from the first data table and sent to the associated party; If the associated party responds to the message data, the message data is deleted from the first data table.

6. The data storage method according to claim 1 or 5, characterized in that, Also includes: Once the storage duration of the additional data in the second data table reaches a preset duration, the additional data is deleted from the second data table.

7. The data storage method according to claim 6, characterized in that, The preset duration is determined based on the service type of the message data corresponding to the additional data.

8. A data storage device, characterized in that, include: The data acquisition module is used to acquire additional data from the message data sent by the caller. A data storage module is used to store the message data in a first data table and the additional data in a second data table; The message data is used to send to the associated party corresponding to the caller; The additional data includes the time when the message data was first sent to the associated party, the time when it was sent to the associated party again, the remaining number of transmissions, and the associated party's response to the message data.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the data storage method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data storage method according to any one of claims 1 to 7.

Images