Data access method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a data access method, device, equipment and storage medium, which relates to the field of data processing technology. The specific implementation scheme is: receiving a data access instruction sent by a user terminal; sending a data acquisition request for the data to be accessed to a target server; receiving a response result corresponding to the data acquisition request fed back by the target server; if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to a second type of server in the database cluster to which the target server belongs, and returns to the step of sending a data acquisition request for the data to be accessed to the target server; if the response result is a result indicating that the data acquisition is successful, the response result is fed back to the user terminal. It can be seen that through this scheme, the stability of the data access service in the production environment can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data access method, device, equipment and storage medium. Background Art

[0002] The system environment can be divided into test environment, pre-production environment and production environment. Among them, the test environment is for testers to test, the pre-production environment is for simulation testing before the product service goes online, and the production environment is a real online environment for daily operation of product services.

[0003] In the related art, when a user terminal needs to access user data in a production environment, the user terminal can access a database server in the production environment through a service scheduling server, so that the database server feeds back the required user data to the user terminal through the service scheduling server.

[0004] However, considering that the data that the user terminal frequently accesses is the data within a recent period of time, the data stored by the database server is also based on the data of the most recent specified time period set in the recent period of time. In this way, when the data that the user terminal needs to access is historical data that exceeds the storage period of the database server, it will not be able to access the data, thereby affecting the stability of the data access service in the production environment. Especially for fields such as stocks and funds that have high requirements for the stability of data access services, the solutions provided by the relevant technologies are obviously not applicable.

[0005] Therefore, there is an urgent need for a data access method that can ensure the stability of data access services in a production environment. Summary of the invention

[0006] The purpose of the embodiments of the present invention is to provide a data access method, apparatus, device and storage medium to ensure the stability of data access services in a production environment. The specific technical solution is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a data access method, the method comprising:

[0008] Receiving a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs;

[0009] Sending a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster further includes: a second-class server; the first-class server is a database server storing user data of a recent specified time period, and the second-class server is a database server storing full user data;

[0010] Receive a response result corresponding to the data acquisition request fed back by the target server;

[0011] If the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to the second type of server in the database cluster to which the target server belongs, and the step of sending a data acquisition request for the data to be accessed to the target server is returned;

[0012] If the response result is a result indicating successful data acquisition, the response result is fed back to the user end.

[0013] Optionally, the method further comprises:

[0014] According to a predetermined detection cycle, detecting whether a database server currently communicating with the service scheduling server fails;

[0015] If a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster;

[0016] The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong in the second database cluster in the pre-production environment and the third database cluster in the test environment, and the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced;

[0017] The second database cluster and the third database cluster both include the first-category server and the second-category server, and the designated server is a database server of the same category as the target server before switching.

[0018] Optionally, the first-category server and the second-category server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-category server is quantitatively cleared at a preset period so that the user data of the most recent specified time period is stored;

[0019] Different database clusters correspond to different application server clusters.

[0020] Optionally, any application server cluster includes a master application server and a backup application server operating in a dual-host hot standby mode;

[0021] The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

[0022] In a second aspect, an embodiment of the present invention provides a data access system, including a business scheduling server, and a database cluster in a production environment, wherein a first database cluster in the production environment includes a first type of server and a second type of server, wherein the first type of server and the second type of server use different storage methods, wherein the first type of server is a database server that stores user data of a recent specified time period, and the second type of server is a database server that stores full user data;

[0023] A business scheduling server is used to receive a data access instruction sent by a user terminal; send a data acquisition request for the data to be accessed to a target server; receive a response result corresponding to the data acquisition request fed back by the target server; if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, switch the database server communicating with the business scheduling server to the second-type server in the database cluster to which the target server belongs; and return to the step of sending the data acquisition request for the data to be accessed to the target server; if the response result is a result indicating successful data acquisition, feed back the response result to the user terminal; wherein the data access instruction carries the target time range to which the data to be accessed belongs; the target server is the database server currently communicating with the business scheduling server; the target server defaults to the first-type server in the first database cluster under the production environment;

[0024] The target server is used to respond to the data acquisition request when receiving the data acquisition request, and feed back the response result to the service scheduling server.

[0025] Optionally, the system further includes a second database cluster in a pre-production environment and a third database cluster in a test environment, wherein the second database cluster and the third database cluster both include the first type of server and the second type of server; the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced;

[0026] The service scheduling server is further used for:

[0027] According to a predetermined detection cycle, detecting whether a database server currently communicating with the service scheduling server fails;

[0028] If a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster;

[0029] The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong, in the second database cluster in the pre-production environment and the third database cluster in the test environment;

[0030] The designated server is a database server of the same type as the target server before switching

[0031] Optionally, the system further comprises: an application server cluster corresponding to each database cluster; different database clusters correspond to different application server clusters;

[0032] The first-class server and the second-class server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-class server is quantitatively cleared at a preset period so as to store the user data of the most recent specified time;

[0033] Each application server cluster is used to read metadata in the basic database server, and perform specified processing on the read metadata to obtain processing results, and the processing results are stored in the first type of servers and the second type of servers in the corresponding database cluster.

[0034] Optionally, any application server cluster includes a master application server and a backup application server operating in a dual-host hot standby mode;

[0035] The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

[0036] In a third aspect, an embodiment of the present invention provides a data access device, which is applied to a service scheduling server, and the device includes:

[0037] A first receiving module is used to receive a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs;

[0038] A request module is used to send a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster also includes: a second-class server; the first-class server is a database server that stores user data of a recent specified time period, and the second-class server is a database server that stores full user data;

[0039] A second receiving module is used to receive a response result corresponding to the data acquisition request fed back by the target server;

[0040] A first response module is used for, if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, switching the database server communicating with the business scheduling server to the second type of server in the database cluster to which the target server belongs, and returning to the step of sending a data acquisition request for the data to be accessed to the target server;

[0041] The second response module is used to feed back the response result to the user end if the response result is a result indicating successful data acquisition.

[0042] Optionally, the device further comprises:

[0043] A detection module, used to detect whether a database server currently communicating with the service scheduling server fails according to a predetermined detection period;

[0044] A switching module, used to switch the database server currently communicating with the business scheduling server to a designated server in a designated cluster if a failure occurs;

[0045] The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong in the second database cluster in the pre-production environment and the third database cluster in the test environment, and the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced;

[0046] The second database cluster and the third database cluster both include the first-category server and the second-category server, and the designated server is a database server of the same category as the target server before switching.

[0047] Optionally, the first-category server and the second-category server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-category server is quantitatively cleared at a preset period so that the user data of the most recent specified time period is stored;

[0048] Different database clusters correspond to different application server clusters.

[0049] Optionally, any application server cluster includes a master application server and a backup application server operating in a dual-host hot standby mode;

[0050] The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

[0051] In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0052] Memory, used to store computer programs;

[0053] The processor is used to implement the steps of any of the above-mentioned data access methods when executing the program stored in the memory.

[0054] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned data access methods are implemented.

[0055] Beneficial effects of the embodiments of the present invention:

[0056] The data access method provided by the embodiment of the present invention is that when the business scheduling server receives the data access instruction for the data to be accessed within the target time range sent by the user terminal, it sends a data acquisition request for the data to be accessed to the target server, and receives the response result corresponding to the data acquisition request fed back by the target server. The target server is the first type server in the first database cluster in the production environment by default. If the response result is a failed acquisition result that indicates that the target time range does not belong to the time coverage range of the data stored in the target server, the second type server in the database cluster to which the target server belongs is used as the new target server, and the step of sending the data acquisition request for the data to be accessed to the target server is returned, so that when the business scheduling server fails to obtain the data to be accessed from the first type server in the first database cluster, it can switch to the second type server in the first database cluster to obtain the data to be accessed. Since the second type server is a database server that stores the full amount of user data, the data to be accessed within the target time range can be obtained from the second type server, so that the business scheduling server can receive the response result indicating that the data acquisition is successful, and the response result is fed back to the user terminal to ensure the stability of the data access service in the production environment.

[0057] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0059] Figure 1 A flow chart of a data access method provided by an embodiment of the present invention;

[0060] Figure 2 Another flow chart of the data access method provided by the embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a data migration process for implementing an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of a data access process for implementing an embodiment of the present invention;

[0063] Figure 5 A schematic diagram of the structure of a data access system provided by an embodiment of the present invention;

[0064] Figure 6A schematic diagram of the structure of a data access device provided by an embodiment of the present invention;

[0065] Figure 7 The present invention is a block diagram of an electronic device for implementing the data access method in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0067] The following first introduces the professional terms involved in the embodiments of the present invention:

[0068] Application server: A server where application programs are deployed, such as the server where Python or Java (computer programming language) code developed by developers is deployed.

[0069] Database server: A server that stores data, such as a SQL (Structured Query Language) database server.

[0070] In order to ensure the stability of data access services in a production environment, embodiments of the present invention provide a data access method, apparatus, device, and storage medium.

[0071] Below, a data access method provided by an embodiment of the present invention is first introduced.

[0072] A data access method provided by an embodiment of the present invention is applied to a business scheduling server. After receiving a data access instruction sent by a user terminal, the business scheduling server is used to send a data acquisition request for the data to be accessed to a target server currently communicating with the business scheduling server, and feed back a response result corresponding to the data acquisition request fed back by the target server to the user terminal.

[0073] The data access method provided by the embodiment of the present invention may include the following steps:

[0074] Receiving a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs;

[0075] Sending a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster further includes: a second-class server; the first-class server is a database server storing user data of a recent specified time period, and the second-class server is a database server storing full user data;

[0076] Receive a response result corresponding to the data acquisition request fed back by the target server;

[0077] If the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to the second type of server in the database cluster to which the target server belongs, and the step of sending a data acquisition request for the data to be accessed to the target server is returned;

[0078] If the response result is a result indicating successful data acquisition, the response result is fed back to the user end.

[0079] It should be noted that the business scheduling server and target server involved in the embodiments of the present invention can be applied to the fields of stocks, funds, etc. The embodiments of the present invention are described using the stock field as the application field. At this time, the business scheduling server is used to schedule data access instructions for stock data, and the target server is used to store stock-related data.

[0080] The solution provided by the embodiment of the present invention is that when the business scheduling server receives a data access instruction for the data to be accessed within the target time range sent by the user end, it sends a data acquisition request for the data to be accessed to the target server, and receives a response result corresponding to the data acquisition request fed back by the target server. The target server is by default a first-class server in the first database cluster in the production environment. If the response result is a failed acquisition result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the second-class server in the database cluster to which the target server belongs is used as a new target server, and the step of sending a data acquisition request for the data to be accessed to the target server is returned, so that when the business scheduling server fails to obtain the data to be accessed from the first-class server in the first database cluster, it can switch to the second-class server in the first database cluster to obtain the data to be accessed. Since the second-class server is a database server that stores all user data, the data to be accessed within the target time range can be obtained from the second-class server, so that the business scheduling server can receive a response result indicating that the data acquisition is successful, and the response result is fed back to the user end to ensure the stability of the data access service in the production environment.

[0081] The data access method provided by the embodiment of the present invention is introduced below in conjunction with the accompanying drawings.

[0082] like Figure 1 As shown, the data access method provided by the embodiment of the present invention may include steps S101-S105:

[0083] S101, receiving a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs;

[0084] In this embodiment, when the user terminal needs to obtain user data stored in the database server, it can send a data access instruction to the business scheduling server, so that after receiving the access instruction, the business scheduling server requests to obtain the user data from the database server and feeds back the user data fed back by the database server to the user terminal, thereby realizing the data access process.

[0085] In actual application scenarios, the user terminal can be a mobile phone terminal, a computer terminal, etc.; the user data can be the data generated by daily operation after the product service is launched. For example, if the product service is a stock trading application for recommending stocks to users, then the user data can be the result data obtained by analyzing the trading market data or capital flow data generated on the stock trading day after the stock trading application is launched, such as the average of the opening price of the stock in the last 3 days or the last week, etc. The data access instruction carries the target time range to which the data to be accessed belongs, so that the business scheduling server can request the user data within the target time range from the database server according to the target time range.

[0086] Exemplarily, in a specific implementation method, user A can use the stock trading application in the mobile terminal to enter the target time range of the data to be accessed, such as the most recent week, and initiate a data access instruction to obtain user data within the time range of the most recent week. The business scheduling server receives the data access instruction for the user data of the most recent week and executes subsequent data access processes.

[0087] S102, sending a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster further includes: a second-class server; the first-class server is a database server storing user data of a recent specified time period, and the second-class server is a database server storing full user data;

[0088] In this embodiment, the first database cluster in the production environment may include a first type of server and a second type of server, the first type of server is a database server that stores user data of the most recent specified time, and the second type of server is a database server that stores full user data. Exemplarily, the most recent specified time may be the most recent six months, the most recent year, and so on. The most recent specified time may be determined by the developer based on the frequency of user access to data of different periods. For example, if the data that the user often accesses is user data within the most recent three years, and the user data before the most recent three years is basically not accessed, then the most recent specified time may be the most recent three years, so as to save data storage costs while ensuring that stable data access is provided to the user. It should be noted that the embodiment of the present invention does not limit the most recent specified time. In addition, the full amount of user data is all the user data generated after the product service is launched. For example, if the stock trading application is launched on May 7, 2010, then all the user data generated after May 7, 2010 is the full amount of user data corresponding to the application.

[0089] It is understandable that, since the database server is a server for storing user data, the user data generated by the product service after going online will be stored in the database server in the production environment. Therefore, after receiving the data access instruction sent by the user terminal, the business scheduling server can send a data acquisition request to the database server that communicates with the business scheduling server, that is, send a data acquisition request to the target server. The target server defaults to the first type of server in the first database cluster in the production environment, that is, the business scheduling server establishes a communication connection with the first type of server in the first database cluster by default, and preferentially sends a data acquisition request for the data to be accessed to the first type of server in the first database cluster. It is understandable that, since the first type of server is a database server that stores the user data of the most recent specified time, and the user's access frequency to the user data of the most recent specified time is high, therefore, the first type of server in the production environment is used as the target server, so that when querying data, the amount of data in the queried database is less, thereby improving the response speed of user access.

[0090] Exemplarily, in one implementation, after receiving the data access instruction sent by the user terminal, the service scheduling server can directly send the data access instruction to the target server. At this time, the data access instruction can be used as the data acquisition request, and the data acquisition request carries the target time range to which the data to be accessed belongs. Exemplarily, in another implementation, after receiving the data access instruction sent by the user terminal, the service scheduling server can use the target time range carried in the data access instruction to regenerate a data acquisition request for the data to be accessed in the target time range, and send the data acquisition request to the target server.

[0091] S103, receiving a response result corresponding to the data acquisition request fed back by the target server;

[0092] In this embodiment, after sending the data acquisition request to the target server in step S102, the target server receives the data acquisition request, searches for data belonging to the target time range from its own stored data, and generates a response result for the data acquisition request. Exemplarily, if the target server receives a data acquisition request for user data of the last two years sent by the business scheduling server, the target server searches for user data belonging to the last two years from its own stored data. If the target server only stores user data of the last year, a response result indicating data acquisition failure is generated; if the target server stores user data of the last three years, a response result carrying the user data of the last two years is generated.

[0093] S104, if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to the second type of server in the database cluster to which the target server belongs, and the step of sending a data acquisition request for the data to be accessed to the target server is returned;

[0094] It is understandable that if the response result is an acquisition failure, the user data in the target time range cannot be successfully accessed. At this time, if the response result is fed back to the user end, the user end cannot obtain the user data in the target time range, which will lead to poor stability of the data access service in the production environment.

[0095] In order to ensure the stability of the data access service in the production environment, in this embodiment, if the response result fed back by the target server is an acquisition failure result, the database server communicating with the business scheduling server is switched to the second-class server in the database cluster to which the target server belongs, that is, a communication connection is established between the second-class server and the business scheduling server, that is, the second-class server in the database cluster to which the target server belongs is used as the new target server, and the step of sending a data acquisition request for the data to be accessed to the target server in step S103 is returned. It can be understood that since the second-class server is a database server that stores the full amount of user data, the data stored in the second-class server includes the user data in the target time range. Therefore, the second-class server is used as the new target server, and the business scheduling server can send a data acquisition request to the new target server to obtain a response result carrying the user data in the target time range.

[0096] In addition, it should be noted that since the target server is by default the first type of server in the first database cluster in the production environment, when the database server communicating with the business scheduling server is switched from the first type of server to the second type of server, and the second type of server completes the response to the data acquisition request sent by the business scheduling server, the business scheduling server can also switch back to communicating with the first type of server to improve the response speed of user access. Exemplarily, in a specific implementation, when the second type of server feeds back the response result of the data acquisition request to the business scheduling server, the business scheduling server disconnects the communication connection with the second type of server after receiving the response result, and re-establishes the communication connection with the first type of server.

[0097] S105: If the response result is a result indicating successful data acquisition, the response result is fed back to the user end.

[0098] It can be understood that if the response result is a result that represents successful data acquisition, the response result can be fed back to the user end so that the user end can receive the user data of the target time range carried in the response result, thereby realizing access to the data to be accessed within the target time range and ensuring the stability of the data access service in the production environment.

[0099] The solution provided in this embodiment is that when the business scheduling server receives a data access instruction for the data to be accessed within the target time range sent by the user end, it sends a data acquisition request for the data to be accessed to the target server, and receives a response result corresponding to the data acquisition request fed back by the target server. The target server is by default a first-class server in the first database cluster in the production environment. If the response result is a failed acquisition result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the second-class server in the database cluster to which the target server belongs is used as a new target server, and the step of sending a data acquisition request for the data to be accessed to the target server is returned, so that when the business scheduling server fails to obtain the data to be accessed from the first-class server in the first database cluster, it can switch to the second-class server in the first database cluster to obtain the data to be accessed. Since the second-class server is a database server that stores all user data, the data to be accessed within the target time range can be obtained from the second-class server, so that the business scheduling server can receive a response result indicating that the data acquisition is successful, and the response result is fed back to the user end to ensure the stability of the data access service in the production environment.

[0100] Optionally, in another embodiment of the present invention, Figure 1 Based on the embodiment shown, Figure 2 As shown, the data access method further includes steps S201-S202:

[0101] S201, detecting whether a database server currently communicating with the service scheduling server fails according to a predetermined detection cycle;

[0102] In this embodiment, the business scheduling server can detect whether the database server communicating with it is faulty by sending a heartbeat packet according to a predetermined detection period. The predetermined detection period can be every 30 seconds, 1 minute, and so on. Exemplarily, the business scheduling server can send a heartbeat packet to the database server every 1 minute, and the database server replies with a fixed message after receiving the heartbeat packet. If the business scheduling server does not receive the fixed message replied by the database server, it is considered that the database server has failed. It should be noted that the predetermined detection period can be set by the relevant staff according to the needs, and the embodiment of the present invention does not limit this.

[0103] S202, if a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster; wherein the designated cluster is a cluster of the next level of the cluster to which the currently communicating database server belongs, in the second database cluster in the pre-production environment and the third database cluster in the test environment, and the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced; the second database cluster and the third database cluster both include the first type of server and the second type of server, and the designated server is a database server of the same category as the target server before the switch.

[0104] It is understandable that if the database server currently communicating with the business scheduling server fails, the database server cannot feed back the response result corresponding to the data acquisition request to the business scheduling server, which will affect the stability of the data access service. Therefore, in order to further ensure the stability of the data access service, if the database server currently communicating with the business scheduling server fails, the database server currently communicating with the business scheduling server can be switched to a designated server in a designated cluster, so that the business scheduling server can initiate a data acquisition request to the designated server in the designated cluster. Among them, the designated cluster is the next level cluster of the cluster to which the database server currently communicating with the business scheduling server belongs.

[0105] It should be noted that, in this embodiment, the level of the first database cluster in the production environment is higher than the level of the second database cluster in the pre-production environment, and the level of the second database cluster in the pre-production environment is higher than the third database cluster in the test environment. Exemplarily, if the business scheduling server communicates with the designated server in the first database cluster, if the designated server in the first database cluster fails, the business scheduling server switches the communication connection and communicates with the designated server in the second database cluster. In addition, the data in the first type of servers in the pre-production environment and the test environment at least includes the data in the first type of servers in the production environment, and the data in the second type of servers in the pre-production environment and the test environment at least includes the data in the second type of servers in the production environment, so as to ensure that when the target server fails, the database server in the next level cluster of the cluster to which the target server belongs and of the same category as the target server can provide data access services to the user terminal.

[0106] In addition, the second database cluster and the third database cluster both include the first-class server and the second-class server, and the designated server is a database server of the same class as the target server before the switch. Exemplarily, if the database server communicating with the business scheduling server is a first-class server in the first database cluster, then the first-class server in the first database cluster fails, and the business scheduling server switches the communication connection to communicate with the first-class server in the second database cluster. It can be understood that since the data stored in the first-class server and the second-class server are not exactly the same, if the first-class server in the production environment fails, it can be switched to the first-class server in the pre-production environment to use the data stored in the first-class server in the pre-production environment to complete the data access service provided to the user end.

[0107] It can be seen that through this solution, if it is detected that the database server currently communicating with the business scheduling server fails, the database server currently communicating with the business scheduling server will be switched to a designated server in a designated cluster to utilize the data stored in the designated server in the designated cluster to complete the data access service provided to the user end, thereby further ensuring the stability of the data access service.

[0108] Optionally, in another embodiment of the present invention, the first type of server and the second type of server in any database cluster are used to store: the processing results of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first type of server is quantitatively cleared at a preset period so that user data of the most recent specified time period is stored; different database clusters correspond to different application server clusters.

[0109] In this embodiment, the basic database server stores unprocessed metadata. When the service product is launched, the application server cluster processes the metadata, generates processing results and stores them in the first type of server and the second type of server in the database cluster corresponding to the application server cluster. In addition, different database clusters correspond to different application server clusters, that is, the database set in each system environment corresponds to the application server cluster in the system environment. Exemplarily, the first database cluster in the production environment corresponds to the application server cluster in the production environment. The application server cluster in the production environment processes the metadata in the basic database server and stores the processing results in the first type of server and the second type of server in the database cluster in the production environment.

[0110] Exemplarily, for the stock trading scenario, the metadata may include basic information of listed companies (such as stock code, stock name, listing date, industry sector), daily trading market data (such as opening price, closing price, rise and fall, turnover rate, transaction amount) and capital flow data (such as margin trading data, northbound capital flow), etc. Exemplarily, when the stock trading application is launched, the application server cluster corresponding to the first database cluster in the production environment can process the metadata stored in the basic database server. For example, after the end of each stock trading day, the average capital flow of the capital flow data stored in the basic database server for the last 3 days, the last week, and the last month is calculated. The average capital flow value is stored as the processing result of the metadata in the first type of server and the second type of server in the first database cluster for access by the user. In addition, it should be noted that this solution does not limit the specific processing method for the metadata.

[0111] It can be understood that since the first-class server and the second-class server in each database cluster are used to store: the processing results of the application server cluster corresponding to the database cluster for the metadata in the basic database server, the first-class servers and the second-class servers in different system environments store the processing results for the metadata in the basic database server. Therefore, when the first-class server or the second-class server currently communicating with the business scheduling server fails, the first-class server or the second-class server currently communicating with the business scheduling server can be switched to the first-class server or the second-class server in the next-level cluster of the cluster to which the currently communicating database server belongs, and the required data can be obtained from the switched first-class server or the second-class server, thereby ensuring the stability of the data access service in the production environment.

[0112] In addition, since the first type of server stores the data of the most recent specified time, when the processing results for the metadata are stored in the first type of server and the second type of server at the same time, the first type of server can quantitatively delete the data stored in it at a preset period to ensure that it stores the data of the most recent specified time. For example, the first type of server can use a quarter as a preset period, and delete the data belonging to the earliest quarter among the data stored in it at the end of each quarter.

[0113] Optionally, in one implementation, any application server cluster includes a primary application server and a backup application server operating in a hot standby manner;

[0114] In this implementation, a scheduled task can be deployed in both the main application server and the backup application server, and the scheduled task is used to start the execution of the metadata in the basic database server at a scheduled time, and store the processing results for the metadata in the first type of server and the second type of server in the corresponding database cluster. It can be understood that since hot standby is a working method that uses fault point transfer to ensure business continuity, the recovery of its business is not on the original server, but on the backup server. Therefore, if the main application server working in hot standby mode fails, the backup application server can be used to generate the processing results for the metadata and store the processing results in the database cluster.

[0115] It should be noted that the process of the application server cluster obtaining metadata from the basic database, processing the metadata and storing it in the database cluster corresponding to the application server is a data migration process. It is understandable that during the data migration process, if the application server fails, it cannot generate the processing results for the metadata and store the processing results in the database cluster. At this time, if the user accesses the data in the database cluster, it may not be able to access the latest processing results of the metadata. The application server cluster of this solution includes a main application server and a backup application server working in a hot standby mode, which can ensure that when the main application server cluster fails, the backup application server can still generate the processing results for the metadata and store the processing results in the database cluster, so as not to affect the user's access to the data, so as to realize the dynamic migration of data. In addition, for the fields of stocks, funds, etc., if the user cannot obtain data from the database server on the trading day, it may cause a large economic loss. Therefore, this solution can reduce the risk of economic loss by ensuring the stability of the data access service during the data migration process.

[0116] It can be seen that through this solution, the first-class server and the second-class server in any database cluster are used to store: the processing results of the application server cluster corresponding to the database cluster for the metadata in the basic database server, so that when the first-class server or the second-class server currently communicating with the business scheduling server fails, the required data can be obtained from the switched first-class server or the second-class server to ensure the stability of data access services in the production environment.

[0117] In order to better understand this solution, Figure 3 and Figure 4 A specific example of an embodiment of the present invention is described.

[0118] Figure 3 A schematic diagram of a data migration process for implementing an embodiment of the present invention is shown. Figure 3As shown, the application server cluster includes a main application server and a standby application server working in a hot standby mode, and a scheduled task is deployed in the main application server and the standby application server. When the scheduled task is started, the steps of obtaining metadata from the basic database server, processing the metadata and storing the processing results in the database server DB_OLD (corresponding to the second type of server mentioned above) and the database server DB_NEW (corresponding to the first type of server mentioned above) are executed.

[0119] In this example, the primary and standby application servers perform a one-time migration of all existing data, so that the primary and standby application servers can update the existing data synchronously. For the incremental data added on each stock trading day, the primary and standby application servers process the incremental data on each stock trading day and generate processing results for the incremental data. The database server DB_OLD and the database server DB_NEW store data in the form of data tables, which are divided into two types of data tables: full update and incremental update. For the data table with full update, the primary and standby application servers will first delete all data in the data table on each stock trading day, and then rewrite the result data containing the latest trading day into the data table. For the data table with incremental update, the latest date of the data table is read first, and then compared with the current date. If the latest date is earlier than the current date, the startup scheduled task of the primary and standby application servers generates incremental data greater than the latest date and stores it synchronously in the database server DB_OLD and the database server DB_NEW.

[0120] When the main application server fails, it can seamlessly connect to the backup application server to perform hot switching of database read and write, and store the latest data in the database server DB_OLD and database server DB_NEW, thereby enhancing the fault tolerance performance of the server system cluster.

[0121] It is understandable that in order to achieve physical isolation of data in the related technology, data will be artificially placed in different IP address segments. For example, the server cluster in the test environment, the server cluster in the pre-production environment, and the server cluster in the production environment all have their corresponding IP address segments. If you want to modify the data in the database server corresponding to one of the IP address segments, the data in the database servers in the other two IP address segments will not be automatically modified. In this way, during the data migration process in the production environment, if the database server to which the data is to be migrated fails, the user end will not be able to obtain the required data from the database server in the pre-production environment. This solution uses the database server in each system environment to store: the processing results of the application server corresponding to the database server for the metadata in the basic database server, so that if the database server in the production environment fails, the required data can be obtained from the database server in the pre-production environment, thereby ensuring the stability of the data access service in the production environment.

[0122] Figure 4 A schematic diagram of a data access process for implementing an embodiment of the present invention is shown. Figure 4 As shown, the user terminal initiates a data access instruction for the target time range; after receiving the data access instruction, the business scheduling server initiates a data acquisition request to the database server DB_NEW that communicates with it; the database server DB_NEW searches for the data of the target time range from the data stored in itself, and if the data is stored, the response result carrying the target time range data is fed back to the business scheduling server. After receiving the response result, the business scheduling server feeds back the response result to the user terminal to complete a data access process.

[0123] If the data stored in the database server DB_NEW does not contain the data in the target time range, the database server DB_NEW that communicates with the business scheduling server is switched to the database server DB_OLD, and a data acquisition request is initiated to the database server DB_OLD. Since the database server DB_OLD stores all the user data, the response result carrying the target time range data can be fed back to the business scheduling server. After receiving the response result, the business scheduling server feeds back the response result to the user end to complete another data access process.

[0124] It can be seen that through this solution, in the dynamic migration of data in the production environment, the hot switching of the master and standby application servers and the hot switching of the read and write of the database server are realized, ensuring the consistency, integrity and synchronization of the data in the production environment; the application server cluster includes the master application server and the standby application server working in hot standby mode, which enhances the fault tolerance performance of the server system cluster; and supports the dynamic migration of full, incremental, batch and real-time database tables.

[0125] Corresponding to the above method embodiment, the present invention also provides a data access system, such as Figure 5 As shown, the system includes a business scheduling server 510, and a database cluster 520 in a production environment, wherein the first database cluster in the production environment includes a first type of server and a second type of server, wherein the first type of server and the second type of server use different storage methods, wherein the first type of server is a database server that stores user data of a recent specified time period, and the second type of server is a database server that stores full user data;

[0126] The business scheduling server 510 is used to receive a data access instruction sent by a user terminal; send a data acquisition request for the data to be accessed to a target database server; receive a response result corresponding to the data acquisition request fed back by the target database server; if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target database server, the database server communicating with the business scheduling server is switched to the second type of server in the database cluster to which the target server belongs; and return to the step of sending the data acquisition request for the data to be accessed to the target database server; if the response result is a result indicating successful data acquisition, the response result is fed back to the user terminal; wherein the data access instruction carries the target time range to which the data to be accessed belongs; the target server is the database server currently communicating with the business scheduling server; the target server defaults to the first type of server in the first database cluster in the production environment;

[0127] The target database server is used to respond to the data acquisition request when receiving the data acquisition request, and feed back the response result to the business scheduling server.

[0128] Optionally, the system further includes a second database cluster in a pre-production environment and a third database cluster in a test environment, wherein the second database cluster and the third database cluster both include the first type of server and the second type of server; the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced;

[0129] The service scheduling server is further used for:

[0130] According to a predetermined detection cycle, detecting whether a database server currently communicating with the service scheduling server fails;

[0131] If a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster;

[0132] The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong, in the second database cluster in the pre-production environment and the third database cluster in the test environment;

[0133] The designated server is a database server of the same category as the target server before the switch.

[0134] Optionally, the system further comprises: an application server cluster corresponding to each database cluster; different database clusters correspond to different application server clusters;

[0135] The first-class server and the second-class server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-class server is quantitatively cleared at a preset period so as to store the user data of the most recent specified time;

[0136] Each application server cluster is used to read metadata in the basic database server, perform specified processing on the read metadata, obtain processing results, and store the processing results in the first type of server and the second type of server in the corresponding database cluster.

[0137] Optionally, any application server cluster includes a master application server and a backup application server operating in a dual-host hot standby mode;

[0138] The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

[0139] Corresponding to the above method embodiment, the embodiment of the present invention also provides a data access device, which is applied to a service scheduling server, such as Figure 6 As shown, the device comprises:

[0140] The first receiving module 610 is used to receive a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs;

[0141] The request module 620 is used to send a data acquisition request for the data to be accessed to the target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster also includes: a second-class server; the first-class server is a database server that stores user data of a recent specified time period, and the second-class server is a database server that stores full user data;

[0142] The second receiving module 630 is used to receive a response result corresponding to the data acquisition request fed back by the target server;

[0143] The first response module 640 is used to switch the database server communicating with the business scheduling server to the second type of server in the database cluster to which the target server belongs, and return to the step of sending a data acquisition request for the data to be accessed to the target server if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server;

[0144] The second response module 650 is configured to feed back the response result to the user end if the response result is a result indicating successful data acquisition.

[0145] Optionally, the device further comprises:

[0146] A detection module, used to detect whether a database server currently communicating with the service scheduling server fails according to a predetermined detection period;

[0147] A switching module, used to switch the database server currently communicating with the business scheduling server to a designated server in a designated cluster if a failure occurs;

[0148] The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong in the second database cluster in the pre-production environment and the third database cluster in the test environment, and the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced;

[0149] The second database cluster and the third database cluster both include the first-category server and the second-category server, and the designated server is a database server of the same category as the target server before switching.

[0150] Optionally, the first-category server and the second-category server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-category server is quantitatively cleared at a preset period so that the user data of the most recent specified time period is stored;

[0151] Different database clusters correspond to different application server clusters.

[0152] Optionally, any application server cluster includes a master application server and a backup application server operating in a dual-host hot standby mode;

[0153] The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

[0154] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0155] Memory 703, used for storing computer programs;

[0156] The processor 701 is used to implement the steps of any of the above data access methods when executing the program stored in the memory 703.

[0157] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0158] The communication interface is used for communication between the above electronic device and other devices.

[0159] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0160] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0161] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above data access methods are performed.

[0162] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes any one of the data access methods in the above embodiments.

[0163] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0164] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0165] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A data access method, characterized in that: Applied to a service scheduling server, the method includes: Receiving a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs; Sending a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster further includes: a second-class server; the first-class server is a database server storing user data of a recent specified time period, and the second-class server is a database server storing full user data; Receive a response result corresponding to the data acquisition request fed back by the target server; If the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to the second type of server in the database cluster to which the target server belongs, and the step of sending a data acquisition request for the data to be accessed to the target server is returned; If the response result is a result indicating successful data acquisition, the response result is fed back to the user end.

2. The method according to claim 1, characterized in that The method further comprises: According to a predetermined detection cycle, detecting whether a database server currently communicating with the service scheduling server fails; If a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster; The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong in the second database cluster in the pre-production environment and the third database cluster in the test environment, and the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced; The second database cluster and the third database cluster both include the first-category server and the second-category server, and the designated server is a database server of the same category as the target server before switching.

3. The method according to claim 1 or 2, characterized in that: The first type of server and the second type of server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; The data stored in the first type of server is quantitatively cleared at a preset period so that the user data of the most recent specified time period is stored; Different database clusters correspond to different application server clusters.

4. The method according to claim 3, characterized in that Any application server cluster includes a main application server and a backup application server working in a dual-host hot standby mode; The main application server and the backup application server are both deployed with scheduled tasks. When the scheduled tasks are started, the metadata in the basic database server is processed and the processing results of the metadata are stored in the first type of server and the second type of server in the corresponding database cluster.

5. A data access system, characterized in that: It includes a business scheduling server and a database cluster in a production environment, wherein the first database cluster in the production environment includes a first type of server and a second type of server, the first type of server and the second type of server use different storage methods, wherein the first type of server is a database server that stores user data of a recent specified time period, and the second type of server is a database server that stores all user data; A business scheduling server is used to receive a data access instruction sent by a user terminal; send a data acquisition request for the data to be accessed to a target server; receive a response result corresponding to the data acquisition request fed back by the target server; if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, the database server communicating with the business scheduling server is switched to the second-type server in the database cluster to which the target server belongs; and return to the step of sending the data acquisition request for the data to be accessed to the target server; if the response result is a result indicating successful data acquisition, the response result is fed back to the user terminal; wherein the data access instruction carries the target time range to which the data to be accessed belongs; the target server is the database server currently communicating with the business scheduling server; the target server defaults to the first-type server in the first database cluster in the production environment; The target server is used to respond to the data acquisition request when receiving the data acquisition request, and feed back the response result to the service scheduling server.

6. The system according to claim 5, characterized in that The system further includes a second database cluster in a pre-production environment and a third database cluster in a test environment, wherein the second database cluster and the third database cluster both include the first type of server and the second type of server; the levels of the first database cluster, the second database cluster and the third database cluster are gradually reduced; The service scheduling server is further used for: According to a predetermined detection cycle, detecting whether a database server currently communicating with the service scheduling server fails; If a failure occurs, the database server currently communicating with the business scheduling server is switched to a designated server in a designated cluster; The designated cluster is a cluster of the next level of the cluster to which the currently communicating database servers belong, in the second database cluster in the pre-production environment and the third database cluster in the test environment; The designated server is a database server of the same category as the target server before the switch.

7. The system according to claim 5 or 6, characterized in that: The system further includes: an application server cluster corresponding to each database cluster; different database clusters correspond to different application server clusters; The first-class server and the second-class server in any database cluster are used to store: the processing result of the application server cluster corresponding to the database cluster on the metadata in the basic database server; the data stored in the first-class server is quantitatively cleared at a preset period so as to store the user data of the most recent specified time; Each application server cluster is used to read metadata in the basic database server, and perform specified processing on the read metadata to obtain processing results, and the processing results are stored in the first type of servers and the second type of servers in the corresponding database cluster.

8. A data access device, characterized in that: Applied to a service scheduling server, the device comprises: A first receiving module is used to receive a data access instruction sent by a user terminal; wherein the data access instruction carries a target time range to which the data to be accessed belongs; A request module is used to send a data acquisition request for the data to be accessed to a target server; wherein the target server is a database server currently communicating with the business scheduling server; the target server is a first-class server in a first database cluster in a production environment by default, and the first database cluster also includes: a second-class server; the first-class server is a database server that stores user data of a recent specified time period, and the second-class server is a database server that stores full user data; A second receiving module is used to receive a response result corresponding to the data acquisition request fed back by the target server; A first response module is used for, if the response result is: an acquisition failure result indicating that the target time range does not belong to the time coverage range of the data stored in the target server, switching the database server communicating with the business scheduling server to the second type of server in the database cluster to which the target server belongs, and returning to the step of sending a data acquisition request for the data to be accessed to the target server; The second response module is used to feed back the response result to the user end if the response result is a result indicating successful data acquisition.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 4 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images