Method and device for identifying hotspot data

Abstract

The invention discloses a hotspot data identification method and device, and relates to the technical field of big data and electronic commerce. According to one specific embodiment, the method comprises the steps that in response to an identification request of hotspot data, to-be-identified data and a working node list are obtained, the working node list comprises at least one working node, and the at least one working node has an arrangement sequence; determining a target working node corresponding to the to-be-identified data according to a preset distribution rule and the arrangement sequence, and sending the access record of the to-be-identified data to the target working node; and performing aggregation processing on the access record of the to-be-identified data through the target working node, and performing hotspot data identification on the to-be-identified data according to an aggregation processing result. According to the embodiment, the access record of the to-be-identified data is acquired through the working node, and the hotspot data identification is performed on the to-be-identified data, so that the pressure of the server can be reduced, the cost is reduced, the access request processing capability of the server is improved, and the traffic peak condition is dealt with.

Description

Technical Field

[0001] The present invention relates to the technical field of big data and e-commerce, and in particular to a method and device for identifying hotspot data. Background Art

[0002] During the peak traffic period of promotional activities, a large number of access requests are made to the same data, which will cause great pressure on the server. The current solution is to increase double-write random reads or increase the server's storage cache to handle access requests during peak traffic.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] The cost of adding double-write random reads or increasing the server's storage cache is high, and the ability to process access requests is limited. The server is still under great pressure and cannot cope with traffic peaks. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for identifying hot data. Access records of data to be identified are obtained through working nodes, and hot data identification is performed on the data to be identified. This can reduce the pressure on the server, reduce costs, and improve the server's ability to process access requests, thereby coping with traffic peaks.

[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for identifying hotspot data is provided.

[0007] A method for identifying hot data comprises: in response to a request for identifying hot data, obtaining data to be identified and a list of working nodes, wherein the list of working nodes comprises at least one working node, and the at least one working node has an arrangement order; according to a preset distribution rule, determining a target working node corresponding to the data to be identified according to the arrangement order, and sending access records of the data to be identified to the target working node; aggregating the access records of the data to be identified through the target working node, and identifying hot data for the data to be identified according to the aggregation processing result.

[0008] Optionally, the target working node corresponding to the data to be identified is determined according to the arrangement order in accordance with a preset distribution rule, including: calculating a hash value of the data to be identified, and performing a modulo operation on the number of all working nodes in the target working node based on the hash value; and determining the target working node corresponding to the data to be identified according to the modulo result and the arrangement order.

[0009] Optionally, the data to be identified has a corresponding application; sending the access record of the data to be identified to the target working node includes: establishing a long connection between the application corresponding to the data to be identified and the target working node; monitoring the access record of the data to be identified through the long connection, and sending the access record to the target working node according to a preset period.

[0010] Optionally, the access records of the data to be identified are aggregated through the target working node, and hot spot data is identified for the data to be identified based on the aggregation processing results, including: aggregating the access records through a preset sliding time window to obtain the number of accesses of the data to be identified within the sliding time window; when the number of accesses is greater than a preset threshold, identifying the data to be identified as hot spot data.

[0011] Optionally, the data to be identified has a corresponding application; after identifying the data to be identified as hot data, it includes: saving the data to be identified to a preset cache of the application to execute an access request for the data to be identified based on the preset cache.

[0012] Optionally, after saving the data to be identified in the preset cache of the application, it also includes: in response to the cache duration of the data to be identified exceeding a preset cache duration threshold, deleting the data to be identified from the preset cache; or, in response to the usage capacity of the preset cache reaching a preset capacity threshold, deleting the data to be identified from the preset cache according to a preset elimination strategy.

[0013] Optionally, the method also includes: monitoring the abnormal status of each of the working nodes by means of a heartbeat service; in the event of an abnormality in the working node, for the data to be identified, re-determining the target working node corresponding to the data to be identified according to the distribution rule and the arrangement order, and sending the access record of the data to be identified to the target working node.

[0014] According to another aspect of an embodiment of the present invention, a device for identifying hotspot data is provided.

[0015] A device for identifying hot data, comprising: a working node list acquisition module, used to respond to a request for identifying hot data, obtain data to be identified and a working node list, the working node list including at least one working node, and the at least one working node having an arrangement order; a target working node determination module, used to determine the target working node corresponding to the data to be identified according to the arrangement order in accordance with a preset distribution rule, and send the access record of the data to be identified to the target working node; a hot data identification module, used to aggregate the access record of the data to be identified through the target working node, and perform hot data identification on the data to be identified according to the aggregation processing result.

[0016] Optionally, the target working node determination module is also used to: calculate the hash value of the data to be identified, and perform a modulo operation on the number of all working nodes in the target working node based on the hash value; and determine the target working node corresponding to the data to be identified based on the modulo result and the arrangement order.

[0017] Optionally, the data to be identified has a corresponding application; the target working node determination module is also used to: establish a long connection between the application corresponding to the data to be identified and the target working node; through the long connection, monitor the access records of the data to be identified, and send the access records to the target working node according to a preset period.

[0018] Optionally, the hot data identification module is also used to: aggregate the access records through a preset sliding time window to obtain the number of accesses of the data to be identified within the sliding time window; when the number of accesses is greater than a preset threshold, identify the data to be identified as hot data.

[0019] Optionally, the data to be identified has a corresponding application; the device also includes a hot data processing module, which is used to: save the data to be identified to a preset cache of the application, so as to execute an access request for the data to be identified based on the preset cache.

[0020] Optionally, the hot data processing module is also used to: in response to the cache length of the data to be identified exceeding a preset cache length threshold, delete the data to be identified from the preset cache; or, in response to the usage capacity of the preset cache reaching a preset capacity threshold, delete the data to be identified from the preset cache according to a preset elimination strategy.

[0021] Optionally, the target working node determination module is also used to: monitor the abnormal status of each of the working nodes through a heartbeat service; when an abnormality occurs in the working node, for the data to be identified, re-determine the target working node corresponding to the data to be identified according to the distribution rule and the arrangement order, and send the access record of the data to be identified to the target working node.

[0022] According to yet another aspect of the embodiments of the present invention, an electronic device is provided.

[0023] An electronic device includes: one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the hotspot data identification method provided by an embodiment of the present invention.

[0024] According to yet another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0025] A computer-readable medium stores a computer program, which, when executed by a processor, implements a method for identifying hotspot data provided by an embodiment of the present invention.

[0026] An embodiment of the above invention has the following advantages or beneficial effects: by responding to the identification request of hot spot data, the data to be identified and the list of working nodes are obtained, the list of working nodes includes no less than one working node, and no less than one working node has an arrangement order; according to the preset distribution rule, the target working node corresponding to the data to be identified is determined according to the arrangement order, and the access record of the data to be identified is sent to the target working node; the access record of the data to be identified is aggregated by the target working node, and the hot spot data identification is performed on the data to be identified according to the aggregation processing result. The technical scheme obtains the access record of the data to be identified through the working node, and the hot spot data identification is performed on the data to be identified, which can reduce the pressure on the server, reduce costs, and improve the server's ability to process access requests, thereby coping with traffic peaks.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0029] Figure 1 is a schematic diagram of the main steps of a method for identifying hotspot data according to an embodiment of the present invention;

[0030] Figure 2is a schematic diagram of the architecture of a method for identifying hotspot data according to an embodiment of the present invention;

[0031] Figure 3 is a schematic flow chart of a method for identifying hotspot data according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of main modules of a device for identifying hotspot data according to an embodiment of the present invention;

[0033] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0034] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0037] Figure 1 4 is a schematic diagram of the main steps of a method for identifying hotspot data according to an embodiment of the present invention.

[0038] like Figure 1 As shown, a method for identifying hotspot data according to an embodiment of the present invention mainly includes the following steps S101 to S103.

[0039] Step S101: in response to a request for identifying hotspot data, obtaining data to be identified and a list of working nodes, wherein the list of working nodes may include no less than one working node, and no less than one working node has an arrangement order.

[0040] Specifically, the architecture of the hotspot data identification method according to an embodiment of the present invention is as follows: Figure 2As shown, the data to be identified may have a corresponding application. Multiple docker instances (i.e., container instances created based on containers) may be deployed in the application. The detection tool is pre-connected to each docker instance in the application. When connected for the first time, the unique identifier of the application (e.g., application name, appName) is registered in the distributed storage, and an appName cluster corresponding to the appName is created in the worker cluster (i.e., working cluster). The appName cluster may include multiple working nodes, and the working nodes have an arrangement order, such as Figure 2 worker-1, worker-2, ..., worker-k in the distributed storage. The distributed storage may be ETCD, which is a distributed key-value storage developed by CoreOS (a lightweight operating system) based on the Raft (distributed consensus algorithm) protocol. It can be used for service discovery, shared configuration, and consistency assurance. The detection tool may be a hot key detection framework.

[0041] Pull pre-configured rule data from distributed storage. The rule data may include the key field to be matched, matching method (such as regular matching, prefix matching, etc.), local cache duration, reporting cycle, and preset threshold. According to the matching method, match the key field in each docker instance of the application, and use the matched key field key as the key of the data to be identified.

[0042] In another embodiment, a default cluster may be created in advance, and when there is no appName cluster corresponding to the application appName in the worker cluster, hotspot data identification may be performed on the data to be identified of the application through the default cluster.

[0043] Step S102: According to the preset distribution rule, the target working node corresponding to the data to be identified is determined according to the arrangement order, and the access record of the data to be identified is sent to the target working node.

[0044] In one embodiment, according to a preset distribution rule, determining the target working node corresponding to the data to be identified according to the arrangement order may include: calculating the hash value of the data to be identified, and performing a modulo operation on the number of all working nodes in the target working node based on the hash value; and determining the target working node corresponding to the data to be identified according to the modulo result and the arrangement order.

[0045] Specifically, for each key of the data to be identified, a hash value is taken for the key of the data to be identified, and then a modulo operation is performed on the number of all working nodes in the target working node. The modulo result calculation method can be specifically: Hash(key)%count(worker node list), and the worker node corresponding to the modulo result is used as the target working node corresponding to the data to be identified in the order of arrangement. Among them, Hash(key) means taking a hash value for the key of the data to be identified, % means a modulo operation, and count(worker node list) means obtaining the number of all working nodes in the target working node.

[0046] In one embodiment, sending access records of the data to be identified to the target working node may include: establishing a long connection between the application corresponding to the data to be identified and the target working node; monitoring the access records of the data to be identified through the long connection, and sending the access records to the target working node according to a preset period.

[0047] Specifically, for each data to be identified, a long connection can be established between the docker instance where the data to be identified is located and the target working node, and then the detection tool is used to obtain the access records in each reporting cycle according to the reporting cycle, and the access records are sent to the target working node through the long connection. The reporting can be performed synchronously or asynchronously, and the reporting cycle can be set to 500 milliseconds.

[0048] In one embodiment, the abnormal status of each working node can be monitored by means of a heartbeat service; in the event of an abnormality in a working node, for the data to be identified, the target working node corresponding to the data to be identified is determined according to the distribution rules and the arrangement order, and the access record of the data to be identified is sent to the target working node.

[0049] Specifically, the detection tool can be used to monitor the heartbeat service list of the working node. For example, a heartbeat notification is sent every 3 seconds, indicating that the server is working normally. If the heartbeat of a working node is not received, it can be determined that the working node is abnormal. If a working node is abnormal, or a working node is added or deleted, the number of working nodes in the working node list changes. The target working node corresponding to the data to be identified can be determined according to the distribution rules and the arrangement order, and the access record of the data to be identified is sent to the target working node.

[0050] Step S103: performing aggregation processing on the access records of the data to be identified through the target working node, and identifying hot data of the data to be identified according to the aggregation processing result.

[0051] In one embodiment, the access records of the data to be identified are aggregated by the target working node, and hot spot data is identified for the data to be identified based on the aggregation processing results, which may include: aggregating the access records through a preset sliding time window to obtain the number of accesses of the data to be identified within the sliding time window; when the number of accesses is greater than a preset threshold, identifying the data to be identified as hot spot data.

[0052] Specifically, for each data to be identified, the access records within the sliding time window are obtained in the worker cluster through the preset sliding time window, and then the access records are aggregated to obtain the number of accesses of the data to be identified within the sliding time window. When the number of accesses is greater than the preset threshold, the data to be identified is identified as hot data, communicates with the application through a long connection, and marks the data to be identified as hot data in all docker instances in the application.

[0053] In one embodiment, after identifying the data to be identified as hot data, the method may include: saving the data to be identified in a preset cache of the application, so as to execute an access request for the data to be identified based on the preset cache. In response to the cache duration of the data to be identified exceeding a preset cache duration threshold, deleting the data to be identified from the preset cache; or, in response to the usage capacity of the preset cache reaching a preset capacity threshold, deleting the data to be identified from the preset cache according to a preset elimination strategy.

[0054] Specifically, the data to be identified is marked as hot data and saved in the preset cache of the application (such as the application's memory). When an access request for the data to be identified is received, the data to be identified can be directly returned through the preset cache, thereby reducing the pressure on the server and improving the server's ability to process access requests.

[0055] The preset cache may pre-set a cache duration threshold. When the duration of the data to be identified in the preset cache reaches the cache duration threshold, the data to be identified is deleted from the preset cache. The preset cache may also have a pre-set capacity threshold. When the usage capacity of the preset cache reaches the preset capacity threshold, the data to be identified may be deleted from the preset cache according to a preset elimination strategy. The elimination strategy may be the least recently used elimination algorithm (Least Recently Used) or the least frequently used elimination algorithm (Least Frequently Used).

[0056] Figure 3 It is a flowchart of a method for identifying hotspot data according to an embodiment of the present invention.

[0057] like Figure 3As shown, in one embodiment, the data to be identified and the corresponding working node list of the application are obtained, and the hash value of the data to be identified is calculated according to the preset distribution rules. The modulus operation is performed on the number of all working nodes in the target working node based on the hash value, and the target working node corresponding to the data to be identified is determined according to the modulus result and the arrangement order. A long connection is established between the application corresponding to the data to be identified and the target working node. Through the long connection, the access records of the data to be identified are monitored, and the access records are sent to the target working node according to the preset period. Through the preset sliding time window, the access records are aggregated to obtain the number of accesses to the data to be identified within the sliding time window. When the number of accesses is greater than the preset threshold, the data to be identified is identified as hot data, and the data to be identified is saved in the preset cache of the application, so as to execute the access request of the data to be identified based on the preset cache.

[0058] The embodiment of the present invention proposes a lightweight hotspot data identification solution, which realizes unified management of the data to be identified in the application dimension, obtains the access records of the data to be identified through the working nodes, and identifies the hotspot data of the data to be identified, which can reduce the pressure on the server, reduce costs, and improve the server's ability to process access requests, thereby coping with traffic peaks.

[0059] Figure 4 4 is a schematic diagram of main modules of a device for identifying hotspot data according to an embodiment of the present invention.

[0060] like Figure 4 As shown, a hotspot data identification device 400 according to an embodiment of the present invention mainly includes: a working node list acquisition module 401 , a target working node determination module 402 , and a hotspot data identification module 403 .

[0061] The working node list acquisition module 401 is used to respond to the identification request of the hot data and obtain the data to be identified and the working node list. The working node list may include no less than one working node, and no less than one working node has an arrangement order.

[0062] The target working node determination module 402 is used to determine the target working node corresponding to the data to be identified according to the preset distribution rule and the arrangement order, and send the access record of the data to be identified to the target working node.

[0063] The hot data identification module 403 is used to perform aggregation processing on the access records of the data to be identified through the target working node, and to identify the hot data of the data to be identified according to the aggregation processing result.

[0064] In one embodiment, the target working node determination module 402 is specifically used to: calculate the hash value of the data to be identified, perform a modulo operation on the number of all working nodes in the target working node based on the hash value; and determine the target working node corresponding to the data to be identified based on the modulo result and the arrangement order.

[0065] In one embodiment, the data to be identified may have a corresponding application; the target working node determination module 402 is specifically used to: establish a long connection between the application corresponding to the data to be identified and the target working node; through the long connection, monitor the access records of the data to be identified, and send the access records to the target working node according to a preset period.

[0066] In one embodiment, the hot data identification module 403 is specifically used to: aggregate the access records through a preset sliding time window to obtain the number of accesses of the data to be identified within the sliding time window; when the number of accesses is greater than a preset threshold, identify the data to be identified as hot data.

[0067] In one embodiment, the data to be identified may have a corresponding application; the device may also include a hot data processing module (not shown in the figure) for: saving the data to be identified to a preset cache of the application to execute access requests for the data to be identified based on the preset cache.

[0068] In one embodiment, the hot data processing module is specifically used to: in response to the cache length of the data to be identified exceeding a preset cache length threshold, delete the data to be identified from the preset cache; or, in response to the usage capacity of the preset cache reaching a preset capacity threshold, delete the data to be identified from the preset cache according to a preset elimination strategy.

[0069] In one embodiment, the target working node determination module 402 is specifically used to: monitor the abnormal status of each working node through the heartbeat service; when an abnormality occurs in a working node, for the data to be identified, re-determine the target working node corresponding to the data to be identified according to the distribution rules and the arrangement order, and send the access record of the data to be identified to the target working node.

[0070] In addition, the specific implementation content of the hotspot data identification device in the embodiment of the present invention has been described in detail in the above hotspot data identification method, so the repeated content will not be described again here.

[0071] Figure 5 An exemplary system architecture 500 is shown to which the method for identifying hotspot data or the apparatus for identifying hotspot data according to the embodiment of the present invention can be applied.

[0072] like Figure 5As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0073] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as hot data identification applications, access applications, shopping applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0074] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0075] The server 505 may be a server that provides various services, such as a background management server that supports identification websites of hotspot data browsed by users using terminal devices 501, 502, and 503 (for example only). The background management server may respond to the identification request of the hotspot data and other data received, obtain the data to be identified and the list of working nodes, the list of working nodes including no less than one working node, and no less than one working node having an arrangement order; determine the target working node corresponding to the data to be identified according to the arrangement order according to the preset distribution rules, and send the access record of the data to be identified to the target working node; aggregate the access record of the data to be identified through the target working node, and perform hotspot data identification and other processing on the data to be identified according to the aggregation processing result, and feed back the processing result (such as the identification result of the hotspot data - for example only) to the terminal device.

[0076] It should be noted that the hotspot data identification method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the hotspot data identification device is generally disposed in the server 505 .

[0077] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0078] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 6The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0079] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0080] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0081] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0082] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0083] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor includes a working node list acquisition module, a target working node determination module, and a hotspot data identification module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the working node list acquisition module may also be described as a "module for obtaining data to be identified and a working node list in response to a request for identifying hotspot data."

[0085] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: in response to a request for identification of hotspot data, obtaining data to be identified and a list of working nodes, the list of working nodes includes no less than one working node, and no less than one working node has an arrangement order; according to a preset distribution rule, determining the target working node corresponding to the data to be identified according to the arrangement order, and sending the access record of the data to be identified to the target working node; performing aggregation processing on the access record of the data to be identified through the target working node, and performing hotspot data identification on the data to be identified according to the aggregation processing result.

[0086] According to the technical solution of the embodiment of the present invention, in response to the identification request of the hot data, the data to be identified and the list of working nodes are obtained, the list of working nodes includes no less than one working node, and no less than one working node has an arrangement order; according to the preset distribution rule, the target working node corresponding to the data to be identified is determined according to the arrangement order, and the access record of the data to be identified is sent to the target working node; the access record of the data to be identified is aggregated by the target working node, and the hot data of the data to be identified is identified according to the aggregation processing result. Obtaining the access record of the data to be identified by the working node and identifying the hot data of the data to be identified can reduce the pressure on the server, reduce costs, and improve the ability of the server to process access requests, thereby coping with traffic peaks.

[0087] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying hotspot data, characterized in that: include: In response to a request for identifying hotspot data, obtaining data to be identified and a working node list, wherein the working node list includes at least one working node, and the at least one working node has an arrangement order; According to the preset distribution rule, the target working node corresponding to the data to be identified is determined according to the arrangement order, and the access record of the data to be identified is sent to the target working node; The access records of the data to be identified are aggregated by the target working node, and hotspot data of the data to be identified are identified according to the aggregation processing result.

2. The method according to claim 1, characterized in that: The step of determining the target working node corresponding to the to-be-identified data according to the arrangement order according to the preset distribution rule includes: Calculate a hash value of the data to be identified, and perform a modulo operation on the number of all working nodes in the target working node based on the hash value; The target working node corresponding to the data to be identified is determined according to the remainder result and the arrangement order.

3. The method according to claim 1, characterized in that The data to be identified has a corresponding application; The step of sending the access record of the to-be-identified data to the target working node includes: Establishing a long connection between the application corresponding to the data to be identified and the target working node; The access records of the data to be identified are monitored through the long connection, and the access records are sent to the target working node according to a preset period.

4. The method according to claim 1, characterized in that: The aggregating the access records of the data to be identified by the target working node, and identifying hotspot data of the data to be identified according to the aggregation processing result, includes: Aggregating the access records through a preset sliding time window to obtain the number of accesses of the to-be-identified data within the sliding time window; When the number of accesses is greater than a preset threshold, the data to be identified is identified as hotspot data.

5. The method according to claim 4, characterized in that The data to be identified has a corresponding application; After the data to be identified is identified as hotspot data, the method includes: The data to be identified is saved in a preset cache of the application, so as to execute an access request for the data to be identified based on the preset cache.

6. The method according to claim 5, characterized in that After the data to be identified is saved in the preset cache of the application, the method further includes: In response to the cache duration of the data to be identified exceeding a preset cache duration threshold, deleting the data to be identified from the preset cache; Alternatively, in response to the usage capacity of the preset cache reaching a preset capacity threshold, the to-be-identified data is deleted from the preset cache according to a preset elimination strategy.

7. The method according to claim 1, characterized in that The method further comprises: Monitor the abnormal status of each working node by means of heartbeat service; In the event that an exception occurs in the working node, for the data to be identified, the target working node corresponding to the data to be identified is determined again according to the distribution rule and the arrangement order, and the access record of the data to be identified is sent to the target working node.

8. A device for identifying hotspot data, characterized in that: include: A working node list acquisition module, used for acquiring the data to be identified and the working node list in response to the identification request of the hotspot data, wherein the working node list includes at least one working node, and the at least one working node has an arrangement order; A target working node determination module is used to determine the target working node corresponding to the data to be identified according to the arrangement order according to a preset distribution rule, and send the access record of the data to be identified to the target working node; The hot data identification module is used to aggregate the access records of the data to be identified through the target working node, and identify the hot data of the data to be identified according to the aggregation processing result.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images