Method and device for eliminating hotspot data

Abstract

A method and device for eliminating hotspot data, used to detect and eliminate hotspot data in a cluster. The method includes: obtaining the performance impact factor of each server in the cluster; the performance impact factor includes access frequency and garbage collection time; determining the hotspot server in the cluster according to the garbage collection time of each server; the hotspot server is a server in the hotspot data set; if the performance impact factor of the hotspot server meets the trigger condition, the data in the hotspot server is distributed to other servers in the cluster.

Description

Technical Field

[0001] The present application relates to the field of big data cluster technology, and in particular to a method and device for eliminating hotspot data. Background Art

[0002] With the rapid development of the Internet and mobile devices, more and more data is generated in people's daily life and work. In order to analyze and process large-scale data, companies and enterprises need to build big data clusters. Hadoop, as an open source distributed data processing framework, has been widely used in the field of big data analysis and processing.

[0003] HBase is a distributed database in the Hadoop cluster. It has high reliability, high performance, column orientation and scalability, and can provide fast random access to massive data. However, when accessing HBase, when one or more data tables in the RegionServer are frequently accessed, hot spots will appear. A large number of accesses will cause the performance of the RegionServer where the hot data is located to decline. For the cluster as a whole, uneven data access will lead to unbalanced HBase load, which will in turn reduce the overall performance of the cluster.

[0004] Therefore, there is an urgent need for a solution to detect and eliminate hotspot data in a cluster. Summary of the invention

[0005] The present application provides a method and device for eliminating hotspot data, which are used to detect and eliminate hotspot data in a cluster.

[0006] In a first aspect, an embodiment of the present application provides a method for eliminating hotspot data, the method comprising: obtaining a performance impact factor of each server in a cluster; the performance impact factor comprises an access frequency and a garbage collection time; determining a hotspot server in the cluster according to the garbage collection time of each server; the hotspot server is a server in a hotspot data set; if the performance impact factor of the hotspot server meets a trigger condition, the data in the hotspot server is distributed to other servers in the cluster.

[0007] In the above technical solution, the hotspot server is determined based on the garbage collection time of each server in the cluster. When the hotspot server meets the trigger condition, the data in the currently detected hotspot server is distributed to other servers in the cluster by utilizing the high data availability characteristics of the cluster itself, so that the read and write loads of each server in the cluster are dynamically balanced, thereby continuously maintaining the high performance of the entire cluster.

[0008] In one possible design, determining the hotspot server in the cluster based on the garbage collection time of each server includes: sorting the garbage collection time of all servers in the cluster, and determining the server with the longest garbage collection time as the hotspot server in the cluster.

[0009] In the above technical solution, garbage collection of memory requires consumption of CPU and other resources. The time consumption of garbage collection of the server has a great impact on the user experience. Therefore, the server with the longest garbage collection time can be determined as the hotspot server in the cluster.

[0010] In one possible design, if the performance impact factor of the hotspot server meets the trigger condition, including if the product of the inverse of the access frequency and the garbage collection time is greater than a preset threshold, then the trigger condition is met.

[0011] In the above technical solution, when the data in the hotspot server is distributed to other servers, the hotspot server and the data in the hotspot server cannot be accessed. When a certain hotspot server is accessed a particularly large number of times per minute, reallocating the data on it will affect a large number of users. Therefore, it is necessary to distribute the data on the hotspot server when the access frequency of the hotspot server is relatively small.

[0012] In one possible design, the data in the hotspot server is distributed to other servers in the cluster, including: sending an instruction to take the hotspot server offline to a management server in the cluster; and the management server distributes the data in the hotspot server to other servers in the cluster according to a load balancing strategy.

[0013] In the above technical solution, after detecting the hotspot server in the data access concentration, an instruction to shut down the hotspot server is sent to the management server in the cluster. By utilizing the high availability of the cluster's own data, the data in the server (i.e., the hotspot server) that is instructed to go offline is distributed to other servers in the cluster.

[0014] In one possible design, the method further includes: sending an instruction to the monitoring server in the cluster to take the hotspot server offline; the monitoring server deletes the hotspot server from a registration list so that the hotspot server does not provide services to the outside world.

[0015] In the above technical solution, after detecting the hotspot server in the data access concentration, an instruction to the hotspot server to go offline is sent to the monitoring server in the cluster, instructing the monitoring server to delete the hotspot server from the registration list, so as to avoid the situation where the hotspot server continues to be read and written due to the fact that the hotspot server is not known to be offline.

[0016] In a possible design, after the management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy, it also includes: the management server brings the hotspot server online; the monitoring server adds the hotspot server to the registration list, so that the hotspot server resumes providing services to the outside world.

[0017] In the above technical solution, after the management server distributes the data in the hotspot server, the hotspot server is put back online to re-enable the hotspot server; and the hotspot server is added to the registration list to inform the hotspot server that it can continue to provide services externally.

[0018] In one possible design, the method further includes: detecting a state of the cluster; if the state of the cluster is abnormal, suspending acquisition of a performance impact factor of each server in the cluster until it is detected that the state of the cluster returns to normal.

[0019] In the above technical solution, when an abnormal cluster state is detected, the above step of eliminating hotspot data is suspended to avoid data confusion in the process of allocating data in the hotspot server to other servers in the cluster due to the abnormal cluster state.

[0020] In a second aspect, an embodiment of the present application provides a device for eliminating hotspot data, including:

[0021] An acquisition module is used to acquire the performance impact factor of each server in the cluster; the performance impact factor includes access frequency and garbage collection time;

[0022] A determination module, used to determine a hotspot server in the cluster according to the garbage collection time of each server; the hotspot server is a server in the hotspot data set;

[0023] The processing module is used to distribute the data in the hotspot server to other servers in the cluster if the performance impact factor of the hotspot server meets the trigger condition.

[0024] In one possible design, the determination module is further used to sort the garbage collection times of all the servers in the cluster, and determine the server with the longest garbage collection time as the hotspot server in the cluster.

[0025] In one possible design, if the performance impact factor of the hotspot server meets the trigger condition, including if the product of the inverse of the access frequency and the garbage collection time is greater than a preset threshold, then the trigger condition is met.

[0026] In one possible design, the processing module is also used to send an instruction to the management server in the cluster to shut down the hotspot server; the management server distributes the data in the hotspot server to other servers in the cluster according to a load balancing strategy.

[0027] In a possible design, the processing module is also used to send an instruction to the monitoring server in the cluster to shut down the hotspot server; the monitoring server deletes the hotspot server from the registration list, so that the hotspot server does not provide services to the outside world.

[0028] In a possible design, after the management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy, it also includes: the management server brings the hotspot server online; the monitoring server adds the hotspot server to the registration list, so that the hotspot server resumes providing services to the outside world.

[0029] In one possible design, the device also includes a detection module for detecting the state of the cluster; the processing module is also used to suspend obtaining the performance impact factor of each server in the cluster if the state of the cluster is abnormal until it is detected that the state of the cluster returns to normal.

[0030] In a third aspect, an embodiment of the present application further provides a computing device, including:

[0031] A memory for storing program instructions;

[0032] A processor is used to call the program instructions stored in the memory and execute the method described in any possible design of the first aspect according to the obtained program instructions.

[0033] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer-readable instructions are stored. When a computer reads and executes the computer-readable instructions, the method described in any possible design of the above-mentioned first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A schematic diagram of the HBase system architecture provided in an embodiment of the present application;

[0036] Figure 2 A flowchart of a method for eliminating hotspot data provided in an embodiment of the present application;

[0037] Figure 3 A line graph of the garbage collection time of each server in the cluster before executing the solution provided in the embodiment of the present application;

[0038] Figure 4 A line graph of the garbage collection time of each server in the cluster after executing the solution provided in the embodiment of the present application;

[0039] Figure 5 A schematic diagram of a specific process for eliminating hotspot data provided in an embodiment of the present application;

[0040] Figure 6 A schematic diagram of the structure of a device for eliminating hotspot data provided in an embodiment of the present application;

[0041] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0043] In the embodiments of the present application, a plurality refers to two or more than two. The words "first", "second" and the like are only used for the purpose of distinguishing descriptions and cannot be understood as indicating or implying relative importance or order.

[0044] As an open source distributed data processing framework, Hadoop has been widely used in the field of big data processing and analysis, and has become an indispensable option for companies and enterprises to build big data clusters. As the scale of data that needs to be analyzed and processed grows rapidly, the number of servers in the big data cluster is also increasing rapidly, and applications are reading and writing data to the big data cluster more and more frequently. The problem that follows is that the servers in the cluster become abnormal or even crash due to frequent access, which will bring huge losses to companies and enterprises. Therefore, the stability and robustness of the operation of the big data cluster has become an issue that cannot be ignored.

[0045] HBase is the abbreviation of Hadoop Database. It is a distributed column-oriented database built on the Hadoop file system. It has the characteristics of high reliability, high performance, column orientation and scalability, and can provide fast random access to massive data.

[0046] Figure 1 The schematic diagram of the HBase system architecture is shown as an example. The HBase is managed uniformly through the big data cluster CDH (cloudera's distribution including apache hadoop) integrated package. HBase runs on HDFS and uses HDFS as the basic storage facility. From the HBase architecture diagram, it can be seen that the components in HBase include Client, Master, Zookeeper cluster and Region Server cluster.

[0047] The Client contains an interface for accessing HBase. HBase accesses the HBase database through the application program interface (API) provided by the Client to complete data writing and reading.

[0048] The Master manages all Region Servers in the Region Server cluster, monitors the status of each Region Server, allocates Regions to each Region Server, and implements load balancing and failover of the Region Server cluster.

[0049] The Zookeeper cluster consists of multiple Zookeepers, which store the status information of the Region Server cluster and the addressing entry of each Region in the Region Server. The Master and Region Server will register with Zookeeper when they start. Zookeeper can ensure that there is only one Master in the cluster at any time and monitor the RegionServer.

[0050] A Region Server cluster consists of multiple Region Servers. A Region Server is used to maintain the Region assigned to it by the Master. Region is the smallest unit of distributed storage and load balancing in HBase.

[0051] It should be noted that the above Figure 1The HBase system architecture shown is only an example and is not specifically limited in the embodiments of the present application.

[0052] In the prior art, there are two ways to distribute data tables to each Region Server. One way is to use the load balancer that comes with HBase to distribute data tables according to the average upper and lower limit algorithm to ensure that the number of regions served by each Region Server is balanced. This method does not consider factors such as hot data access, underlying file distribution, and physical resources of each node when distributing data tables, so it is easy to generate read and write hot spots and cannot achieve ideal load balance.

[0053] Another way is to use commands to scatter a single data table to multiple Region Servers, but the scattering rule of this method is just a simple average distribution. For example, a data table is evenly distributed to 10 specified Region Servers. Since hot data and hot Region Servers cannot be predicted in advance, after running for a period of time, due to the uneven read and write access of the application, hot data and hot Region Servers will randomly appear in the Hbase cluster. As the usage time increases, it is easy to cause the data distribution in the Region Server cluster to gradually deteriorate, thereby reducing the performance of the entire cluster.

[0054] Based on the above description, the present application provides a method for eliminating hotspot data, so as to detect and eliminate hotspot data in a cluster.

[0055] Figure 2 The flowchart of a method for eliminating hotspot data provided by an embodiment of the present application is exemplarily shown as follows: Figure 2 As shown, the method comprises the following steps:

[0056] Step 201: Obtain the performance impact factor of each server (Region Server) in the cluster.

[0057] Among them, performance influencing factors include access frequency and garbage collection (GC) time. Access frequency is the number of times the Region Server is accessed per unit time. Garbage collection time is used to recycle and release unnecessary memory space to avoid unlimited memory growth leading to out of memory (OOM) situation.

[0058] Step 202: Determine the hotspot servers in the cluster according to the garbage collection time of each server.

[0059] In the embodiment of the present application, the hotspot server is a Region Server in the hotspot data set. When garbage collection is performed on the memory, resources such as CPU are consumed. The time taken for garbage collection of the Region Server has a great impact on the user experience. Therefore, the hotspot Region Server in the cluster can be determined according to the garbage collection time of each Region Server.

[0060] For example, when determining the hot RegionServer in the cluster according to the garbage collection time of each RegionServer, the garbage collection times of all RegionServers in the cluster may be sorted, and the RegionServer with the longest garbage collection time may be determined as the hot RegionServer in the cluster.

[0061] Step 203: If the performance impact factor of the hotspot server meets the trigger condition, the data in the hotspot server is distributed to other servers in the cluster.

[0062] In one example, when determining whether the hotspot Region Server performance impact factor meets the trigger condition, the product of the reciprocal of the access frequency and the garbage collection time being greater than a preset threshold can be used as a trigger condition. If the product of the reciprocal of the access frequency and the garbage collection time is greater than the preset threshold, the trigger condition is met, and the data in the hotspot Region Server is distributed to other Region Servers in the cluster; if the product of the reciprocal of the access frequency and the garbage collection time is not greater than the preset threshold, the trigger condition is not met, and the above steps 501 to 503 are repeated.

[0063] It should be noted that the preset threshold can be determined based on the actual garbage collection time range of the Region Server in the cluster and the range of the number of accesses per minute of the Region Server. For example, if the actual garbage collection time of the Region Server in a cluster ranges from 40s to 180s, and the number of accesses per minute of the Region Server ranges from 10 to 300, then the preset threshold can be set to 1, that is, the trigger condition is met when the garbage collection time / access frequency of the hotspot Region Server is greater than 1, and the unit of the garbage collection time here is seconds.

[0064] In the above technical solution, when the hotspot Region Server is taken offline and the data in the hotspot Region Server is allocated to other servers, the hotspot Region Server and the data in the hotspot Region Server cannot be accessed. When the determined hotspot Region Server is accessed particularly many times per minute, the reallocation of the data on it will affect more users. Therefore, it is necessary to allocate the data on the hotspot Region Server when the access frequency of the hotspot Region Server is relatively small. For example, the current garbage collection time of the hotspot Region Server is 180s. When the range of the number of accesses to the Region Server per minute is less than 180 times, it is considered that taking the hotspot Region Server offline has little impact on users, and the data in the hotspot Region Server can be allocated to other servers. When the range of the number of accesses to the Region Server per minute is greater than or equal to 180 times, it is considered that taking the hotspot Region Server offline has a greater impact on users, and the data in the hotspot Region Server will not be processed at this time.

[0065] It is understandable that the above trigger condition may also be that the product of the access frequency and the inverse of the garbage collection time is less than a preset threshold.

[0066] In another example, when determining whether the hotspot Region Server performance impact factor meets the trigger condition, the access frequency less than the preset threshold can be used as the trigger condition. If the access frequency is less than the preset threshold, it is considered that the impact of taking the hotspot Region Server offline on the user is small, the trigger condition is met, and the data in the hotspot Region Server is distributed to other Region Servers in the cluster; if the access frequency is not less than the preset threshold, it is considered that taking the hotspot RegionServer offline has a large impact on the user, the trigger condition is not met, and the above steps 501 to 503 are repeated. For example, the actual garbage collection time of the Region Server in the cluster ranges from about 40s to 180s, and the number of accesses to the Region Server per minute ranges from about 10 to 300 times, then the preset threshold can be set to 180, that is, the access frequency of the hotspot RegionServer is less than 180 times to meet the trigger condition.

[0067] After determining that the trigger conditions are met, the hotspot Region Server offline instruction can be sent to the Master and Zookeeper in the cluster through the API interface. After the Master receives the hotspot Region Server offline instruction, it distributes the data in the hotspot Region Server to other Region Servers in the cluster according to the load balancing strategy. After Zookeeper receives the hotspot Region Server offline instruction, it deletes the hotspot Region Server from the registration list, so that the hotspot Region Server does not provide external services.

[0068] It is understandable that, in actual execution, the Master distributes the data in the hotspot Region Server to other Region Servers in the cluster. It does not distribute the data actually stored in the hotspot Region Server, but restores the copy of the hotspot Region Server and distributes it to other Region Servers in the cluster.

[0069] After the Master distributes the data in the hotspot Region Server to other Region Servers in the cluster, the Master brings the hotspot Region Server online, and Zookeeper adds the hotspot server to the registration list, allowing the hotspot Region Server to provide services to the outside world again.

[0070] After the above operation, the data on the Region Server with the most concentrated hot data can be distributed to other Region Servers in the cluster. Repeating the above operation can make the garbage collection time of each Region Server in the cluster close to the same, that is, the difference between the longest garbage collection time and the shortest garbage collection time is small, so that the Region Servers of the entire cluster can achieve real-time load balancing. For example, Figure 3 This is a line graph of the garbage collection time of each server in the cluster before executing this solution; Figure 4 This is a line graph of the garbage collection time of each server in the cluster after executing this solution. Figure 3The garbage collection time of the top Region Server is significantly longer than that of other Region Servers in the cluster. As the usage time increases, the data distribution in the Region Server cluster is likely to deteriorate, which in turn reduces the performance of the entire cluster. After multiple detections of hotspot Region Servers and dispersing the data in the hotspot Region Servers to other Region Servers in the cluster, such as Figure 4 As shown in the figure, the garbage collection time of each Region Server in the cluster is close to the same.

[0071] In a possible implementation, before obtaining the performance impact factor of each server in the cluster, the state of the cluster is detected. If the state of the cluster is abnormal, the acquisition of the performance impact factor of each server in the cluster is suspended until it is detected that the state of the cluster returns to normal. This avoids data confusion in the process of allocating data in the hotspot server to other servers in the cluster due to abnormal cluster state.

[0072] The present application provides a method for eliminating hotspot data. According to the garbage collection time of each Region Server in the cluster, the Region Server with the longest garbage collection time is determined as the hotspot Region Server. When the hotspot RegionServer meets the trigger condition, the data in the currently detected hotspot RegionServer is distributed to other Region Servers in the cluster by utilizing the high availability characteristics of the cluster's own data, so that the read and write loads of each Region Server in the cluster are dynamically balanced, thereby continuously maintaining the high performance of the entire Hbase cluster.

[0073] In order to better explain the embodiments of this application, Figure 5 A schematic diagram of a specific process for eliminating hotspot data provided in an embodiment of the present application is exemplarily shown, and the process includes the following steps.

[0074] Step 501: Obtain cluster status information.

[0075] Get cluster status information through the API interface.

[0076] Step 502: Determine whether the cluster status is normal.

[0077] It is determined whether the cluster is running normally according to the acquired cluster status information; if the cluster status is normal, step 503 is executed; if the cluster status is abnormal, step 508 is executed.

[0078] Step 503: Determine the hotspot Region Server in the cluster according to the garbage collection time of each Region Server in the cluster.

[0079] Specifically, the garbage collection times of all Region Servers in the cluster may be sorted, and the Region Server with the longest garbage collection time may be determined as the hotspot Region Server in the cluster.

[0080] Step 504: Determine whether the garbage collection time / access frequency is greater than 1.

[0081] Determine whether the product of the inverse of the access frequency of the hotspot Region Server and the garbage collection time is greater than 1; if greater than 1, execute step 505; otherwise, execute step 501.

[0082] The access frequency is the number of times the hotspot Region Server is accessed per minute, and the garbage collection time is in seconds.

[0083] Step 505: Send an instruction to the Master and Zookeeper in the cluster to shut down the hotspot Region Server.

[0084] Step 506: The Master distributes the data in the hotspot Region Server to other Region Servers in the cluster according to the load balancing strategy.

[0085] Zookeeper deletes the hotspot Region Server from the registration list, so that the hotspot Region Server does not provide services to the outside world.

[0086] Step 507: The Master brings the hotspot Region Server online.

[0087] Zookeeper adds the hotspot Region Server to the registration list, and the hotspot Region Server resumes providing services to the outside world.

[0088] After executing step 507, step 501 is repeated.

[0089] Step 508: tentatively set for 5 minutes.

[0090] After pausing for 5 minutes, step 501 is repeated.

[0091] Based on the same technical concept, Figure 6 An exemplary embodiment of the present application provides a device for eliminating hotspot data. Figure 6 As shown, the device 600 includes:

[0092] The acquisition module 601 is used to acquire the performance impact factor of each server in the cluster; the performance impact factor includes access frequency and garbage collection time;

[0093] A determination module 602 is used to determine a hotspot server in the cluster according to the garbage collection time of each server; the hotspot server is a server in the hotspot data set;

[0094] The processing module 603 is configured to distribute the data in the hotspot server to other servers in the cluster if the performance impact factor of the hotspot server meets a trigger condition.

[0095] In a possible design, the determination module 602 is further used to sort the garbage collection times of all the servers in the cluster, and determine the server with the longest garbage collection time as the hotspot server in the cluster.

[0096] In one possible design, if the performance impact factor of the hotspot server meets the trigger condition, including if the product of the inverse of the access frequency and the garbage collection time is greater than a preset threshold, then the trigger condition is met.

[0097] In one possible design, the processing module 603 is also used to send an instruction to the management server in the cluster to shut down the hotspot server; the management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy.

[0098] In a possible design, the processing module 603 is also used to send an instruction to the monitoring server in the cluster to shut down the hotspot server; the monitoring server deletes the hotspot server from the registration list, so that the hotspot server does not provide services to the outside world.

[0099] In a possible design, after the management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy, it also includes: the management server brings the hotspot server online; the monitoring server adds the hotspot server to the registration list, so that the hotspot server resumes providing services to the outside world.

[0100] In one possible design, the device also includes a detection module 604 for detecting the state of the cluster; the processing module 603 is also used to suspend obtaining the performance impact factor of each server in the cluster if the state of the cluster is abnormal until it is detected that the state of the cluster returns to normal.

[0101] Based on the same technical concept, the embodiment of the present application provides a computing device, such as Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to the at least one processor. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 For example, the processor 701 and the memory 702 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0102] In the embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701, and the at least one processor 701 can execute the above-mentioned method for eliminating hotspot data by executing the instructions stored in the memory 702.

[0103] The processor 701 is the control center of the computing device, and can use various interfaces and lines to connect various parts of the computer device, and perform resource settings by running or executing instructions stored in the memory 702 and calling data stored in the memory 702.

[0104] Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0105] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0106] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 702 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0107] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute any of the methods listed in the above-mentioned method to eliminate hotspot data.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0112] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0113] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for eliminating hotspot data, characterized in that: The method comprises: Obtain the performance impact factor of each server in the cluster; the performance impact factor includes access frequency and garbage collection time; the garbage collection time is used to recycle and release unnecessary memory space to avoid unlimited memory growth resulting in exceeding available memory; Determine a hotspot server in the cluster according to the garbage collection time of each server; the hotspot server is a server in the hotspot data set; If the performance impact factor of the hotspot server meets the trigger condition, the data in the hotspot server is distributed to other servers in the cluster; If the performance impact factor of the hotspot server meets a trigger condition, it includes: If the product of the inverse of the access frequency and the garbage collection time is greater than a preset threshold, the trigger condition is met.

2. The method according to claim 1, characterized in that The step of determining the hotspot server in the cluster according to the garbage collection time of each server includes: The garbage collection times of all servers in the cluster are sorted, and the server with the longest garbage collection time is determined as a hotspot server in the cluster.

3. The method according to claim 1, characterized in that Distributing the data in the hotspot server to other servers in the cluster includes: Sending an instruction to the management server in the cluster to take the hotspot server offline; The management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy.

4. The method according to claim 3, characterized in that The method further comprises: Sending an instruction to the monitoring server in the cluster to take the hotspot server offline; The monitoring server deletes the hotspot server from the registration list, so that the hotspot server does not provide services to the outside.

5. The method according to claim 4, characterized in that After the management server distributes the data in the hotspot server to other servers in the cluster according to the load balancing strategy, the method further includes: The management server brings the hotspot server online; The monitoring server adds the hotspot server to the registration list, so that the hotspot server provides services to the outside again.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: detecting a status of the cluster; If the state of the cluster is abnormal, the acquisition of the performance impact factor of each server in the cluster is suspended until it is detected that the state of the cluster returns to normal.

7. A device for eliminating hotspot data, characterized in that: include: The acquisition module is used to obtain the performance impact factor of each server in the cluster; The performance influencing factors include access frequency and garbage collection time; Garbage collection time is used to reclaim and release unnecessary memory space to avoid unlimited memory growth that causes the available memory to exceed the limit. A determination module, used to determine the hotspot server in the cluster according to the garbage collection time of each server; The hotspot server is a server in the hotspot data set; A processing module, configured to distribute the data in the hotspot server to other servers in the cluster if the performance impact factor of the hotspot server meets a trigger condition; If the performance impact factor of the hotspot server meets a trigger condition, it includes: If the product of the inverse of the access frequency and the garbage collection time is greater than a preset threshold, the trigger condition is met.

8. A computing device, characterized in that include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 6 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions, and when a computer reads and executes the computer-readable instructions, the method according to any one of claims 1 to 6 is implemented.

Images