A data processing method and apparatus

Abstract

The application discloses a data processing method and device, and relates to the technical field of computers. A specific implementation manner of the method comprises the following steps: in response to a data processing request initiated by a service server, a storage instance list is acquired; wherein the data processing request comprises a data reading request and / or a data writing request; it is judged whether the storage instance list comprises a plurality of storage instances; wherein a storage instance indicates a corresponding relationship between the service server and a storage device; if yes, a storage key value of to-be-processed data corresponding to the data processing request is determined, storage configuration information is acquired, a target storage instance is determined according to the storage key value and the storage configuration information, and the data processing request is sent to a storage device indicated by the target storage instance, so that the storage device performs corresponding data processing. The implementation manner can balance the loads of the storage devices, shorten the data processing time, reduce the system risk, improve the data processing efficiency, and improve the user experience.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data processing method and apparatus. Background Technology

[0002] Distributed caching systems, supporting multiple data types and scalability, are often used to resolve bottlenecks between database servers and web servers. However, with the continuous growth of internet services, a single distributed caching system cluster often cannot meet business needs, requiring expansion to two or even more clusters. How to dynamically route requests from business servers to the appropriate distributed caching system for data processing is a pressing problem that needs to be solved.

[0003] The existing technology has at least the following problems:

[0004] In existing data processing methods, access requests from business servers need to be sent uniformly to a certain storage device, or polled from each storage device separately before being sent. This results in uneven load capacity of storage devices, long processing time, high system risk, easy downtime, low data processing efficiency, and poor user experience. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a data processing method and apparatus that can balance the load of each storage device, shorten data processing time, reduce system risk, improve data processing efficiency, and enhance user experience.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a data processing method is provided, comprising:

[0007] In response to a data processing request initiated by the business server, obtain a list of storage instances; where the data processing request includes a data read request and / or a data write request.

[0008] Determine whether the storage instance list includes multiple storage instances; where a storage instance indicates the correspondence between a business server and a storage device.

[0009] If so, determine the storage key value of the data to be processed corresponding to the data processing request, obtain the storage configuration information, determine the target storage instance based on the storage key value and the storage configuration information, and send the data processing request to the storage device indicated by the target storage instance so that the storage device can perform the corresponding data processing.

[0010] Furthermore, determining the storage key value of the data to be processed corresponding to the data processing request also includes: obtaining business requirements and determining the storage key value based on the business requirements and the data type of the data to be processed.

[0011] Furthermore, sending the data processing request to the storage device indicated by the storage instance also includes:

[0012] Based on the business server number corresponding to the data processing request and the storage instance, determine the appropriate storage device to which the data processing request is sent.

[0013] Furthermore, the storage configuration information includes at least one of the following: business server configuration information, storage key-value configuration information, and storage device configuration information.

[0014] Furthermore, if the storage instance list includes multiple storage instances, determining the target storage instance based on the storage key value and storage configuration information also includes:

[0015] Set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information;

[0016] The storage key value is matched with multiple pieces of information in the storage configuration information according to their matching priority to determine the target storage instance.

[0017] Furthermore, the method also includes: obtaining the current load capacity of multiple storage devices, determining the target storage device based on the current load capacity, and sending the data processing request to the target storage device.

[0018] Furthermore, the storage device is a distributed caching system cluster, and the method also includes:

[0019] Adjust the distributed cache system corresponding to the data processing request based on the load capacity of multiple distributed cache systems within the distributed cache system cluster.

[0020] According to a second aspect of the present invention, a data processing apparatus is provided, comprising:

[0021] The acquisition module is used to obtain a list of storage instances in response to data processing requests initiated by the business server; wherein, the data processing requests include data read requests and / or data write requests;

[0022] The judgment module is used to determine whether the storage instance list includes multiple storage instances; where storage instances indicate the correspondence between business servers and storage devices.

[0023] The data processing module, if the storage instance list includes multiple storage instances, is used to determine the storage key value of the data to be processed corresponding to the data processing request, obtain storage configuration information, determine the target storage instance based on the storage key value and storage configuration information, and send the data processing request to the storage device indicated by the target storage instance so that the storage device can perform the corresponding data processing.

[0024] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0025] One or more processors;

[0026] Storage device for storing one or more programs.

[0027] When one or more programs are executed by one or more processors, the one or more processors implement any of the data processing methods described above.

[0028] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the data processing methods described above.

[0029] One embodiment of the above invention has the following advantages or beneficial effects: By employing a technical means of responding to a data processing request initiated by a business server, obtaining a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determining whether the storage instance list includes multiple storage instances; wherein the storage instance indicates the correspondence between the business server and the storage device; if so, determining the storage key value of the data to be processed corresponding to the data processing request, obtaining storage configuration information, determining the target storage instance based on the storage key value and storage configuration information, and sending the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing, the invention overcomes the technical problems in existing data processing methods, such as high system risk, easy downtime, low data processing efficiency, and poor user experience caused by uneven load capacity of storage devices and long processing time. This achieves the technical effects of balancing the load of each storage device, shortening data processing time, reducing system risk, improving data processing efficiency, and enhancing user experience.

[0030] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0031] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0032] Figure 1 This is a schematic diagram of the main flow of the data processing method provided according to the first embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the main flow of the data processing method provided according to the second embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the main modules of a data processing apparatus provided according to an embodiment of the present invention;

[0035] Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0036] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0037] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0038] Figure 1 This is a schematic diagram of the main flow of the data processing method provided according to the first embodiment of the present invention; as shown Figure 1 As shown, the data processing method provided in this embodiment of the invention mainly includes:

[0039] Step S101: In response to a data processing request initiated by the business server, obtain a list of storage instances; wherein, the data processing request includes a data read request and / or a data write request.

[0040] Specifically, according to embodiments of the present invention, when a large number of users send data read requests or data write requests based on the business server, in the face of high-concurrency user requests, a storage instance list is obtained, wherein the storage instance list records the correspondence between each business server and storage device, so as to match the corresponding storage device for the user request, which helps to improve data processing efficiency and enhance user experience.

[0041] Step S102: Determine whether the storage instance list includes multiple storage instances; where a storage instance indicates the correspondence between the business server and the storage device. If yes, that is, the storage instance list includes multiple (two or more) storage instances, proceed to step S103; if no, that is, the storage instance list includes at most one storage instance, proceed to step S104.

[0042] Specifically, according to embodiments of the present invention, if the storage instance list includes only one storage instance, meaning that for a business server, it corresponds to only one storage device, then upon receiving a data processing request from the business server, it is only necessary to send the data processing request to the single storage device, without requiring additional routing calls. If the storage instance list includes multiple (two or more) storage instances, meaning that for a business server, it corresponds to at least two storage devices, then it is necessary to match the target storage device for the data processing request sent by the business server according to subsequent storage configuration information to complete the data processing, thereby improving data processing efficiency and enhancing user experience.

[0043] Step S103: If the storage instance list includes multiple storage instances, determine the storage key value of the data to be processed corresponding to the data processing request, obtain the storage configuration information, determine the target storage instance based on the storage key value and the storage configuration information, and send the data processing request to the storage device indicated by the target storage instance so that the storage device can perform the corresponding data processing.

[0044] Specifically, according to an embodiment of the present invention, the step of determining the storage key value of the data to be processed corresponding to the data processing request further includes: obtaining business requirements, and determining the storage key value according to the business requirements and the data type of the data to be processed.

[0045] By setting up the above settings, different storage key values ​​can be set for the data types of the data to be processed according to business needs. This helps to match the target storage device with the data processing request based on the storage key value, and helps to optimize the load balancing between storage devices.

[0046] Furthermore, according to embodiments of the present invention, the above-mentioned storage configuration information includes at least one of the following: business server configuration information, storage key-value configuration information, and storage device configuration information.

[0047] Among them, the configuration information of the business server mainly refers to the configuration information of the same data center, that is, the storage device configured for the business server in the same data center.

[0048] Storage key-value configuration information refers to setting the level of data to be processed based on storage key values. Storage key values ​​can be key values ​​within a specific range, or they can be set in the form of specific prefixes or suffixes to distinguish data to be processed at different levels. Then, the target storage device is matched according to the storage key values ​​to optimize the load balancing between storage devices and write data of the same level to the same storage device for easy subsequent reading and other operations.

[0049] Storage device configuration information refers to adjusting the data processing capacity of a storage device based on its current load. In other words, it matches data processing requests with storage devices that have a lower workload.

[0050] Preferably, according to an embodiment of the present invention, if the storage instance list includes multiple storage instances, the step of determining the target storage instance based on the storage key value and storage configuration information further includes:

[0051] Set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information;

[0052] The storage key value is matched with multiple pieces of information in the storage configuration information according to their matching priority to determine the target storage instance.

[0053] By setting priorities for multiple pieces of information in the storage configuration information, the routing and scheduling of access requests are further improved, load balancing is achieved, system risks are reduced, data processing efficiency is improved, and user experience is enhanced.

[0054] Furthermore, according to an embodiment of the present invention, the above method further includes: obtaining the current load capacity of multiple storage devices, determining a target storage device based on the current load capacity, and sending a data processing request to the target storage device.

[0055] By setting up the above, the target storage device is determined based on the current load capacity of multiple storage devices. This achieves load balancing, avoids the risk of downtime, avoids multiple time-consuming queries, and improves data processing efficiency.

[0056] Preferably, according to an embodiment of the present invention, the storage device is a distributed caching system cluster, and the method further includes:

[0057] Adjust the distributed cache system corresponding to the data processing request based on the load capacity of multiple distributed cache systems within the distributed cache system cluster.

[0058] With the above settings and the use of a distributed caching system cluster, we can handle high-concurrency user requests, expand business service capabilities and the applicable scenarios of the above data processing methods.

[0059] Step S104: If the storage instance list includes only one storage instance, send the data processing request to the storage device indicated by the storage instance so that the storage device can perform the corresponding data processing.

[0060] Specifically, according to an embodiment of the present invention, the step of sending the data processing request to the storage device indicated by the storage instance further includes:

[0061] Based on the business server number corresponding to the data processing request and the storage instance, determine the appropriate storage device to which the data processing request is sent.

[0062] According to a specific embodiment of the present invention, storage devices located in the same data center are preferentially matched with business servers, and storage instances are generated for this matching relationship and stored in a storage instance list. This setting facilitates subsequent data management.

[0063] According to the technical solution of the present invention, by responding to a data processing request initiated by a business server, obtaining a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determining whether the list of storage instances includes multiple storage instances; wherein the storage instance indicates the correspondence between the business server and the storage device; if so, determining the storage key value of the data to be processed corresponding to the data processing request, obtaining storage configuration information, determining the target storage instance based on the storage key value and the storage configuration information, and sending the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing, the technical means overcome the technical problems in the existing data processing methods, such as high system risk, easy downtime, low data processing efficiency, and poor user experience caused by the unbalanced load capacity of storage devices and long processing time. This achieves the technical effects of balancing the load of each storage device, shortening data processing time, reducing system risk, improving data processing efficiency, and enhancing user experience.

[0064] Figure 2 This is a schematic diagram of the main flow of the data processing method provided according to the second embodiment of the present invention; as shown Figure 2 As shown, the data processing method provided in this embodiment of the invention mainly includes:

[0065] Step S201: Obtain business requirements and determine storage key values ​​based on business requirements and the data type of the data to be processed.

[0066] According to embodiments of the present invention, data importance levels can be classified based on business needs. For data with higher importance levels, the corresponding stored values ​​are set with a fixed prefix. It should be noted that any method that achieves the effect of differentiation is acceptable; a fixed suffix or other specific numerical value can also be used.

[0067] By setting different storage key values ​​for the data types to be processed according to business needs, it is helpful to match the target storage device with the data processing request based on the storage key value. For example, storing data of higher importance in a specific storage device can help optimize the load balancing between storage devices while meeting user needs.

[0068] Step S202: In response to a data processing request initiated by the business server, obtain a list of storage instances; wherein, the data processing request includes a data read request and / or a data write request.

[0069] Specifically, according to an embodiment of the present invention, the storage instance list records the correspondence between each business server and storage device, and the corresponding storage device is matched for user requests based on this, which helps to improve data processing efficiency and enhance the technical effect of user experience.

[0070] Step S203: Determine the number of storage instances included in the storage instance list; where a storage instance indicates the correspondence between the business server and the storage device. If the storage instance list includes only one storage instance, proceed to step S204; if the storage instance list includes multiple storage instances, proceed to step S205.

[0071] Specifically, according to embodiments of the present invention, if the storage instance list includes only one storage instance, meaning that for a business server, it corresponds to only one storage device, then upon receiving a data processing request from the business server, it is only necessary to send the data processing request to the single storage device, without requiring additional routing calls. If the storage instance list includes multiple (two or more) storage instances, meaning that for a business server, it corresponds to at least two storage devices, then it is necessary to match the target storage device for the data processing request sent by the business server according to subsequent storage configuration information to complete the data processing, thereby improving data processing efficiency and enhancing user experience.

[0072] Step S204: Based on the business server number corresponding to the data processing request and the storage instance, determine the corresponding storage device and send the data processing request to the storage device.

[0073] According to a specific embodiment of the present invention, storage devices located in the same data center are preferentially matched with business servers, and storage instances are generated for this matching relationship and stored in a storage instance list. This setting facilitates subsequent data management.

[0074] Step S205: If the storage instance list includes multiple storage instances, determine whether the business server configuration information indicates a target storage instance. If yes, that is, the business server configuration information indicates a target storage instance, proceed to step S206; if no, that is, the business server configuration information does not indicate a target storage instance, proceed to step S207.

[0075] Specifically, the configuration information of the business server mainly refers to the configuration information of the same data center, that is, the storage device configured for the business server in the same data center.

[0076] Step S206: Send the data processing request to the storage device corresponding to the target storage instance.

[0077] Step S207: Determine whether the stored value of the data to be processed is a specific value based on the storage key-value configuration information. If yes, that is, the stored value of the data to be processed is a specific value, proceed to step S209; if no, that is, the stored value of the data to be processed is not a specific value, proceed to step S208.

[0078] Specifically, storage key-value configuration information refers to setting the level of data to be processed based on storage key values. Storage key values ​​can be key values ​​in a specific range, or they can be set in the form of specific prefixes or suffixes to distinguish data to be processed in different levels. Then, the target storage device is matched according to the storage key values ​​to optimize the load balancing between storage devices and write data of the same level to the same storage device to facilitate subsequent reading and other operations.

[0079] Step S208: Send the data processing request to the storage device corresponding to the specific value.

[0080] Step S209: Obtain the current load of multiple storage devices and determine whether the storage device with the current high load is the same as the storage device in the same data center as the business server. If yes, that is, the storage device with the current high load is the same as the storage device in the same data center as the business server, proceed to step S210; if no, that is, the storage device with the current high load is not the same as the storage device in the same data center as the business server, proceed to step S211.

[0081] Specifically, storage device configuration information refers to adjusting the data processing capacity of a storage device based on its current load. In other words, it matches data processing requests with storage devices that have a lower workload.

[0082] Preferably, according to an embodiment of the present invention, the above-mentioned business server configuration information, storage key-value configuration information, and storage device configuration information are all storage configuration information. If the storage instance list includes multiple storage instances, the above method further includes:

[0083] Set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information;

[0084] The storage key value is matched with multiple pieces of information in the storage configuration information according to their matching priority to determine the target storage instance.

[0085] By setting priorities for multiple pieces of information in the storage configuration information, the routing and scheduling of access requests are further improved, load balancing is achieved, system risks are reduced, data processing efficiency is improved, and user experience is enhanced.

[0086] Step S210: Send the data processing request to the storage device in the same data center as the business server.

[0087] Step S211: Determine the target storage device based on the current load capacity and send the data processing request to the target storage device.

[0088] Furthermore, according to an embodiment of the present invention, the above method further includes: obtaining the current load capacity of multiple storage devices, determining a target storage device based on the current load capacity, and sending a data processing request to the target storage device.

[0089] By setting up the above, the target storage device is determined based on the current load capacity of multiple storage devices. This achieves load balancing, avoids multiple time-consuming queries, and improves data processing efficiency.

[0090] Step S212: Perform corresponding data processing using the storage device.

[0091] Preferably, according to an embodiment of the present invention, the storage device is a distributed caching system cluster, and the method further includes:

[0092] Adjust the distributed cache system corresponding to the data processing request based on the load capacity of multiple distributed cache systems within the distributed cache system cluster.

[0093] With the above settings and the use of a distributed caching system cluster, we can handle high-concurrency user requests, expand business service capabilities and the applicable scenarios of the above data processing methods.

[0094] According to the technical solution of the present invention, by responding to a data processing request initiated by a business server, obtaining a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determining whether the list of storage instances includes multiple storage instances; wherein the storage instance indicates the correspondence between the business server and the storage device; if so, determining the storage key value of the data to be processed corresponding to the data processing request, obtaining storage configuration information, determining the target storage instance based on the storage key value and the storage configuration information, and sending the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing, the technical means overcome the technical problems in the existing data processing methods, such as high system risk, easy downtime, low data processing efficiency, and poor user experience caused by the unbalanced load capacity of storage devices and long processing time. This achieves the technical effects of balancing the load of each storage device, shortening data processing time, reducing system risk, improving data processing efficiency, and enhancing user experience.

[0095] Figure 3 This is a schematic diagram of the main modules of a data processing apparatus provided according to an embodiment of the present invention; as shown Figure 3 As shown, the data processing device 300 provided in this embodiment of the invention mainly includes:

[0096] The acquisition module 301 is used to obtain a list of storage instances in response to a data processing request initiated by the business server; wherein the data processing request includes a data read request and / or a data write request.

[0097] Specifically, according to embodiments of the present invention, when a large number of users send data read requests or data write requests based on the business server, in the face of high-concurrency user requests, a storage instance list is obtained, wherein the storage instance list records the correspondence between each business server and storage device, so as to match the corresponding storage device for the user request, which helps to improve data processing efficiency and enhance user experience.

[0098] The judgment module 302 is used to determine whether the storage instance list includes multiple storage instances; wherein, the storage instance indicates the correspondence between the business server and the storage device.

[0099] Specifically, according to embodiments of the present invention, if the storage instance list includes only one storage instance, meaning that for a business server, it corresponds to only one storage device, then upon receiving a data processing request from the business server, it is only necessary to send the data processing request to the single storage device, without requiring additional routing calls. If the storage instance list includes multiple (two or more) storage instances, meaning that for a business server, it corresponds to at least two storage devices, then it is necessary to match the target storage device for the data processing request sent by the business server according to subsequent storage configuration information to complete the data processing, thereby improving data processing efficiency and enhancing user experience.

[0100] The data processing module 303, if the storage instance list includes only one storage instance, is used to send a data processing request to the storage device indicated by the storage instance so that the storage device can perform the corresponding data processing; if the storage instance list includes multiple storage instances, it is used to determine the storage key value of the data to be processed corresponding to the data processing request, obtain storage configuration information, determine the target storage instance based on the storage key value and storage configuration information, and send the data processing request to the storage device indicated by the target storage instance so that the storage device can perform the corresponding data processing.

[0101] Specifically, according to an embodiment of the present invention, the data processing module 303 is further configured to:

[0102] Based on the business server number corresponding to the data processing request and the storage instance, determine the appropriate storage device to which the data processing request is sent.

[0103] According to a specific embodiment of the present invention, storage devices located in the same data center are preferentially matched with business servers, and storage instances are generated for this matching relationship and stored in a storage instance list. This setting facilitates subsequent data management.

[0104] Furthermore, according to an embodiment of the present invention, the data processing module 303 is further configured to: obtain business requirements, and determine storage key values ​​based on the business requirements and the data type of the data to be processed.

[0105] By setting up the above settings, different storage key values ​​can be set for the data types of the data to be processed according to business needs. This helps to match the target storage device with the data processing request based on the storage key value, and helps to optimize the load balancing between storage devices.

[0106] Preferably, according to an embodiment of the present invention, the above-mentioned storage configuration information includes at least one of the following: business server configuration information, storage key-value configuration information, and storage device configuration information.

[0107] Among them, the configuration information of the business server mainly refers to the configuration information of the same data center, that is, the storage device configured for the business server in the same data center.

[0108] Storage key-value configuration information refers to setting the level of data to be processed based on storage key values. Storage key values ​​can be key values ​​within a specific range, or they can be set in the form of specific prefixes or suffixes to distinguish data to be processed at different levels. Then, the target storage device is matched according to the storage key values ​​to optimize the load balancing between storage devices and write data of the same level to the same storage device for easy subsequent reading and other operations.

[0109] Storage device configuration information refers to adjusting the data processing capacity of a storage device based on its current load. In other words, it matches data processing requests with storage devices that have a lower workload.

[0110] Furthermore, according to an embodiment of the present invention, if the storage instance list includes multiple storage instances, the data processing module 303 is further configured to:

[0111] Set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information;

[0112] The storage key value is matched with multiple pieces of information in the storage configuration information according to their matching priority to determine the target storage instance.

[0113] By setting priorities for multiple pieces of information in the storage configuration information, the routing and scheduling of access requests are further improved, load balancing is achieved, system risks are reduced, data processing efficiency is improved, and user experience is enhanced.

[0114] For example, according to an embodiment of the present invention, the data processing apparatus 300 further includes an adjustment module, configured to: obtain the current load capacity of a plurality of storage devices, determine a target storage device based on the current load capacity, and send a data processing request to the target storage device.

[0115] By setting up the above, the target storage device is determined based on the current load capacity of multiple storage devices. This achieves load balancing, avoids the risk of storage device downtime, avoids multiple time-consuming queries, and improves data processing efficiency.

[0116] Preferably, according to an embodiment of the present invention, the storage device is a distributed caching system cluster, and the adjustment module is further configured to:

[0117] Adjust the distributed cache system corresponding to the data processing request based on the load capacity of multiple distributed cache systems within the distributed cache system cluster.

[0118] With the above settings and the use of a distributed caching system cluster, we can handle high-concurrency user requests, expand business service capabilities and the applicable scenarios of the above data processing methods.

[0119] According to the technical solution of the present invention, by responding to a data processing request initiated by a business server, obtaining a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determining whether the list of storage instances includes multiple storage instances; wherein the storage instance indicates the correspondence between the business server and the storage device; if so, determining the storage key value of the data to be processed corresponding to the data processing request, obtaining storage configuration information, determining the target storage instance based on the storage key value and the storage configuration information, and sending the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing, the technical means overcome the technical problems in the existing data processing methods, such as high system risk, easy downtime, low data processing efficiency, and poor user experience caused by the unbalanced load capacity of storage devices and long processing time. This achieves the technical effects of balancing the load of each storage device, shortening data processing time, reducing system risk, improving data processing efficiency, and enhancing user experience.

[0120] Figure 4 An exemplary system architecture 400 is shown that can be applied to the data processing method or data processing apparatus of the present invention.

[0121] like Figure 4As shown, system architecture 400 may include terminal devices 401, 402, and 403, network 404, and server 405 (this architecture is merely an example; the components included in a specific architecture may be adjusted according to the specific application). Network 404 serves as the medium for providing a communication link between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0122] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as data processing applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0123] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0124] Server 405 can be a server that provides various services, such as servers that process data for users using terminal devices 401, 402, and 403 (for example only). This server can analyze and process received data processing requests and other data, and feed back the processing results (e.g., target storage instance - for example only) to the terminal devices.

[0125] It should be noted that the data processing method provided in the embodiments of the present invention is generally executed by server 405, and correspondingly, the data processing device is generally located in server 405.

[0126] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0127] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing terminal devices or servers of the present invention. Figure 5 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0128] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0130] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0131] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0133] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an acquisition module, a judgment module, and a data processing module. The names of these modules do not necessarily limit the module itself; for example, the acquisition module can also be described as "a module for acquiring a list of storage instances in response to a data processing request initiated by a business server."

[0134] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to: respond to a data processing request initiated by a business server, obtain a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determine whether the list of storage instances includes multiple storage instances; wherein the storage instances indicate a correspondence between the business server and a storage device; if yes, if no, determine the storage key value of the data to be processed corresponding to the data processing request, obtain storage configuration information, determine a target storage instance based on the storage key value and the storage configuration information, and send the data processing request to the storage device indicated by the target storage instance, so that the storage device performs the corresponding data processing.

[0135] According to the technical solution of the present invention, by responding to a data processing request initiated by a business server, obtaining a list of storage instances; wherein the data processing request includes a data read request and / or a data write request; determining whether the list of storage instances includes multiple storage instances; wherein the storage instance indicates the correspondence between the business server and the storage device; if so, determining the storage key value of the data to be processed corresponding to the data processing request, obtaining storage configuration information, determining the target storage instance based on the storage key value and the storage configuration information, and sending the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing, the technical means overcome the technical problems in the existing data processing methods, such as high system risk, easy downtime, low data processing efficiency, and poor user experience caused by the unbalanced load capacity of storage devices and long processing time. This achieves the technical effects of balancing the load of each storage device, shortening data processing time, reducing system risk, improving data processing efficiency, and enhancing user experience.

[0136] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, include: In response to a data processing request initiated by the business server, a list of storage instances is obtained; wherein, the data processing request includes a data read request and / or a data write request; Determine whether the storage instance list includes multiple storage instances; wherein, the storage instance indicates the correspondence between the business server and the storage device; If so, determine the storage key value of the data to be processed corresponding to the data processing request, obtain storage configuration information, determine the target storage instance based on the storage key value and the storage configuration information, and send the data processing request to the storage device indicated by the target storage instance so that the storage device performs the corresponding data processing; wherein, the storage configuration information includes at least one of the following: business server configuration information, storage key value configuration information, and storage device configuration information; the business server configuration information refers to the same data center configuration information, the storage key value configuration information refers to the level of the data to be processed set according to the storage key value, and the storage device configuration information refers to adjusting the corresponding data processing volume according to the current load of the storage device; If the storage instance list includes multiple storage instances, the step of determining the target storage instance based on the storage key value and the storage configuration information further includes: Set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information; The storage key value is matched with multiple pieces of information in the storage configuration information according to their matching priority to determine the target storage instance; The method further includes: obtaining the current load capacity of multiple storage devices, determining a target storage device based on the current load capacity, and sending the data processing request to the target storage device.

2. The data processing method according to claim 1, characterized in that, The step of determining the storage key value of the data to be processed corresponding to the data processing request further includes: obtaining business requirements and determining the storage key value based on the business requirements and the data type of the data to be processed.

3. The data processing method according to claim 1, characterized in that, The step of sending the data processing request to the storage device indicated by the storage instance further includes: Based on the business server number corresponding to the data processing request and the storage instance, a corresponding storage device is determined so that the data processing request can be sent to that storage device.

4. The data processing method according to claim 1, characterized in that, The storage device is a distributed caching system cluster, and the method further includes: The distributed caching system corresponding to the data processing request is adjusted according to the load capacity of multiple distributed caching systems within the distributed caching system cluster.

5. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire a list of storage instances in response to a data processing request initiated by the business server; wherein the data processing request includes a data read request and / or a data write request. The determination module is used to determine whether the storage instance list includes multiple storage instances; wherein, the storage instance indicates the correspondence between the business server and the storage device; A data processing module, if the storage instance list includes multiple storage instances, is used to determine the storage key value of the data to be processed corresponding to the data processing request, obtain storage configuration information, determine the target storage instance based on the storage key value and the storage configuration information, and send the data processing request to the storage device indicated by the target storage instance, so that the storage device performs corresponding data processing; wherein, the storage configuration information includes at least one of the following: business server configuration information, storage key value configuration information, and storage device configuration information; the business server configuration information refers to the same data center configuration information, the storage key value configuration information refers to the level of the data to be processed set according to the storage key value, and the storage device configuration information refers to adjusting the corresponding data processing volume according to the current load of the storage device; If the storage instance list includes multiple storage instances, the data processing module is further configured to: set matching priorities for business server configuration information, storage key-value configuration information, and storage device configuration information; and sequentially match the storage key-value with the matching priorities of multiple pieces of information in the storage configuration information to determine the target storage instance. The current load capacity of multiple storage devices is obtained, the target storage device is determined based on the current load capacity, and the data processing request is sent to the target storage device.

6. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images