Processing method and device based on hotspot data, storage medium and electronic device

Abstract

The application discloses a kind of based on hotspot data processing method, device, storage medium and electronic equipment.It relates to the field of financial technology or other related technical fields.Among them, the method comprises: based on the first preset rule, M piece of to-be-detected data is filtered, and N piece of to-be-detected data is obtained, wherein the first preset rule is used to filter hotspot data on the client;Each to-be-detected data is coded and processed to obtain N piece of target data;Based on long connection channel, N piece of target data is sent to target server, wherein target server filters N piece of target data based on the second preset rule, and obtains S piece of target data;S piece of target data fed back by target server is received, and S piece of target data is loaded to local cache.The application solves the technical problem that in related art, by increasing the size of server cluster, the high availability of system is ensured when traffic bursts, resulting in high server redundancy when traffic data is small.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of financial technology or other related technical fields, in particular, to a hot data processing method and device, storage medium and electronic equipment based on hot data. BACKGROUND

[0002] In the Internet application, the distributed instantaneous surge of a small number of hot data flow will cause great impact on the stability of the application system. In related technologies, in order to cope with the situation of traffic explosion, the following methods are usually used to ensure the high availability of the system.

[0003] Method one: increase the cluster size to trade off the throughput of the overall system, but simply increasing the cluster size restricts flexibility, for example, when the traffic is small, it will produce too much server redundancy, waste cost, and when the traffic is up, too few servers will put the platform service on the brink of collapse. Therefore, the method of adjusting the cluster size will bring the rise of cost and affect the stability of the system.

[0004] Method two: improve the throughput of single node, such as introducing Redis (remote dictionary service) and other high-performance memory database to improve the processing efficiency of the system, but through the memory database such as Redis can improve the efficiency, which is also an external dependence. With the expansion of application servers, the number of server connections increases, and the system performance will decrease.

[0005] Method three: data preheating, for large-scale operation activities, the activity data will be loaded in advance, but the traffic of the Internet is uncertain, for example, the emergence of a hot search cannot be predicted, and the applicability of data preheating has certain limitations.

[0006] In view of the above problems, no effective solution has been proposed. SUMMARY

[0007] The embodiments of the present application provide a hot data processing method and device, storage medium and electronic equipment based on hot data, to at least solve the technical problem that the method of increasing the server cluster size is used to ensure the high availability of the system when the traffic is up, resulting in high server redundancy when the traffic data is small.

[0008] According to an aspect of an embodiment of the present application, a processing method based on hotspot data is provided, comprising: screening M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data at a client, the hotspot data comprises data with a number of visits greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M; performing encoding processing on each piece of the to-be-detected data to obtain N pieces of target data; sending the N pieces of target data to a target server based on a long connection channel between the client and the target server, wherein the target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client, S is a positive integer less than N; receiving the S pieces of target data fed back by the target server, and loading the S pieces of target data to a local cache, wherein the local cache is used to respond to a target access request, and the target access request comprises an access request for accessing the target data.

[0009] Further, after loading the S pieces of target data to the local cache, the method further comprises: in response to the target access request, calling a target interface, traversing data stored in the local cache, reading the target data from the local cache in a case where the target data exists in the local cache; and reading the target data from a target storage space in a case where the target data is not read from the local cache, wherein the target storage space comprises at least one of a database for storing data of the client and a database for storing data of the target server.

[0010] Further, after loading the S pieces of target data to the local cache, the method further comprises: receiving a target operation instruction, wherein the target operation instruction comprises at least one of a refresh instruction for periodically refreshing data in the local cache, an addition instruction for adding new data in the local cache, and a deletion instruction for deleting specified data in the local cache; and performing the target operation instruction to update data in the local cache.

[0011] Further, screening M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data comprises: matching each piece of the to-be-detected data with the first preset rule to determine the to-be-detected data meeting the first preset rule; adding the to-be-detected data meeting the first preset rule to a target message queue, and reading the to-be-detected data in the target message queue to obtain N pieces of to-be-detected data.

[0012] Further, before the M pieces of to-be-detected data are filtered based on the first preset rule to obtain N pieces of to-be-detected data, the method further includes: registering a node representing the client and a node representing the target server to a target service system; and setting the first preset rule and the second preset rule in the target service system.

[0013] Further, after receiving the S pieces of target data sent by the target server and loading the S pieces of target data to a local cache, the method further includes: transmitting the S pieces of target data to the target service system, and after the target service system receives the S pieces of target data, displaying the S pieces of target data, first registration information representing the node of the client, and second registration information representing the node of the target server on a target screen.

[0014] According to another aspect of the embodiments of the present application, a processing method based on hotspot data is also provided, including: receiving, by a target server, N pieces of target data sent by a client, wherein the client is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, encode each piece of to-be-detected data to obtain N pieces of target data, and send the N pieces of target data to the target server based on a long connection channel between the client and the target server, the first preset rule is used to filter hotspot data at the client, the hotspot data includes data with a number of access times greater than a preset number threshold within a target time period, M and N are positive integers, and N is less than or equal to M; filtering, based on a second preset rule, the N pieces of target data to obtain S pieces of target data, and returning the S pieces of target data to the client, wherein the second preset rule is used to filter hotspot data at the target server, and the client is further configured to load the S pieces of target data to a local cache after receiving the S pieces of target data sent by the target server, and the local cache is used to respond to a target access request, the target access request includes an access request for accessing the target data, and S is a positive integer less than N.

[0015] According to another aspect of the embodiments of the present application, a hotspot data processing device is also provided, which comprises: a first screening unit configured to screen M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data on a client side, the hotspot data comprises data with a number of visits greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M; an encoding unit configured to perform encoding processing on each piece of the to-be-detected data to obtain N pieces of target data; a first sending unit configured to send the N pieces of target data to a target server based on a long connection channel between the client side and the target server, wherein the target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client side, S is a positive integer less than N; and a processing unit configured to receive the S pieces of target data fed back by the target server, and load the S pieces of target data to a local cache, wherein the local cache is used to respond to a target access request, and the target access request comprises an access request for accessing the target data.

[0016] Further, the hotspot data processing device further comprises: a first reading unit configured to, after loading the S pieces of target data to the local cache, respond to the target access request, call a target interface, traverse data stored in the local cache, and read the target data from the local cache in a case where the target data exists in the local cache; and a second reading unit configured to, in a case where the target data is not read from the local cache, read the target data from a target storage space, wherein the target storage space comprises at least one of a database for storing data of the client side and a database for storing data of the target server.

[0017] Further, the hotspot data processing device further comprises: an instruction receiving unit configured to, after loading the S pieces of target data to the local cache, receive a target operation instruction, wherein the target operation instruction comprises at least one of a refresh instruction for periodically refreshing data in the local cache, an addition instruction for adding new data in the local cache, and a deletion instruction for deleting specified data in the local cache; and an execution unit configured to execute the target operation instruction to update data in the local cache.

[0018] Further, the first screening unit comprises a determining sub-unit, configured to match each of the to-be-detected data with the first preset rule, and determine the to-be-detected data meeting the first preset rule; and a processing sub-unit, configured to add the to-be-detected data meeting the first preset rule into a target message queue, and read the to-be-detected data in the target message queue to obtain N pieces of to-be-detected data.

[0019] Further, the hotspot data-based processing apparatus further comprises a registration unit, configured to register a node representing the client and a node representing the target server to a target service system before screening M pieces of to-be-detected data based on the first preset rule to obtain N pieces of to-be-detected data; and a setting unit, configured to set the first preset rule and the second preset rule in the target service system.

[0020] Further, the hotspot data-based processing apparatus further comprises a transmission unit, configured to transmit S pieces of target data to the target service system after receiving the S pieces of target data sent by the target server and loading the S pieces of target data to a local cache, and display the S pieces of target data, first registration information representing the node of the client, and second registration information representing the node of the target server on a target screen after the target service system receives the S pieces of target data.

[0021] According to another aspect of the embodiment of the present application, a hotspot data-based processing apparatus is further provided, comprising: a second sending unit, configured to receive N pieces of target data sent by a client through a target server, wherein the client is configured to screen M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, perform encoding processing on each of the to-be-detected data to obtain N pieces of target data, and send the N pieces of target data to the target server based on a long connection channel between the client and the target server, the first preset rule is used to screen hotspot data at the client, the hotspot data comprises data with a number of access times greater than a preset number threshold within a target time period, M and N are positive integers, and N is less than or equal to M; and a second screening unit, configured to screen N pieces of target data based on a second preset rule to obtain S pieces of target data, and return the S pieces of target data to the client, wherein the second preset rule is used to screen hotspot data at the target server, and the client is further configured to load S pieces of target data to a local cache after receiving the S pieces of target data sent by the target server, the local cache is used to respond to a target access request, the target access request comprises an access request for accessing the target data, and S is a positive integer less than N.

[0022] According to another aspect of the embodiments of the present application, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the hotspot data based processing method of any one of the above by executing the executable instructions.

[0023] According to another aspect of the embodiments of the present application, a computer readable storage medium storing a computer program is also provided, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to perform the hotspot data based processing method of any one of the above when the computer program is running.

[0024] In the present application, based on the first preset rule, M pieces of to-be-detected data are screened to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data on the client side, the hotspot data includes data with a number of visits greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M; each piece of to-be-detected data is coded to obtain N pieces of target data; based on a long connection channel between the client and a target server, the N pieces of target data are sent to the target server, wherein the target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client, S is a positive integer less than N; the S pieces of target data fed back by the target server are received, and the S pieces of target data are loaded to a local cache, wherein the local cache is used to respond to a target access request, and the target access request includes an access request for accessing the target data. Thus, the technical problem of high server redundancy when the traffic data is small in the related art is solved by increasing the size of the server cluster to ensure the high availability of the system when the traffic bursts.

[0025] In the present application, according to the preset rule, the hotspot data is screened on the client side and the server side, and the hotspot data is stored in the local cache, so that the hotspot data is read from the local cache when the hotspot data is accessed, avoiding the increase of the size of the server cluster in the related art to cope with traffic bursts, and achieving the purpose of ensuring the high availability of the application system without increasing the size of the server cluster, thereby realizing the technical effect of reducing the size of the server cluster and saving costs. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application. In the drawings:

[0027] Figure 1 is a flowchart of an optional hotspot data based processing method according to an embodiment of the present application;

[0028] Figure 2 is a multi-level cache architecture diagram according to an embodiment of the present application;

[0029] Figure 3 is a flow chart of another optional hotspot data based processing method according to an embodiment of the present application;

[0030] Figure 4 is a schematic diagram of another optional hotspot data based automatic detection framework according to an embodiment of the present application;

[0031] Figure 5 is a schematic diagram of an optional hotspot data based processing apparatus according to an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of another optional hotspot data based processing apparatus according to an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the persons skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the persons skilled in the art without creative work should belong to the protection scope of the present application.

[0035] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0036] It should be noted that the processing method, device, storage medium and electronic equipment based on hotspot data in the present application can be used in the field of financial technology to process hotspot data and save hotspot data in local cache. In addition to the field of financial technology, the processing method, device, storage medium and electronic equipment based on hotspot data in the present application can also be used in any field to process hotspot data and save hotspot data in local cache. The application field of the processing method, device, storage medium and electronic equipment based on hotspot data in the present application is not limited.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.), collected information and data (including but not limited to data for analysis, stored data, displayed data, hotspot data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in the relevant region, take necessary security measures, do not violate public order and good customs, and provide corresponding operation portal for user selection authorization or refusal.

[0038] Embodiment one

[0039] According to the embodiment of the present application, an optional processing method based on hotspot data is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in different order from here.

[0040] Figure 1 is a flowchart of an optional processing method based on hotspot data according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0041] Step S101, based on the first preset rule, M pieces of to-be-detected data are screened to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data on the client side, the hotspot data includes data with access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M.

[0042] The to-be-detected data described above can include associated data in the client of the financial institution. The first preset rule described above can include a rule for screening hotspot data of the client by screening the data of the client. N pieces of to-be-detected data can be composed of all data meeting the first preset rule in M pieces of to-be-detected data of the client, so as to gather the hotspot data on the client side once.

[0043] Step S102, encoding processing is performed on each piece of to-be-detected data to obtain N pieces of target data.

[0044] In order to avoid excessive traffic of data transmission, in the embodiment, each piece of to-be-detected data in the N pieces of to-be-detected data can be encoded by a specified encoding algorithm or by a self-defined protocol, so as to compress the N pieces of to-be-detected data to obtain N pieces of target data.

[0045] Step S103, based on the long connection channel between the client and the target server, the N pieces of target data are sent to the target server, wherein the target server filters the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client, S being a positive integer less than N.

[0046] The target server described above can include one of a plurality of servers in a service cluster. In order to improve the transmission efficiency of messages, in the embodiment, the N pieces of target data can be sent to the target server based on the long connection channel between the client and the target server. Since each client can establish a connection with each server in a plurality of servers (for example, a plurality of servers), a specific strategy can be selected, such as Hash (hash) according to the key (for example, the identification information of each target data) of each target data, determining the target server from the plurality of servers according to the obtained hash value, and reporting each target data to the designated server (i.e., the target server), thereby improving the processing capacity of the system, and further ensuring the timeliness of hot key data (i.e., hot data) detection.

[0047] After the target server receives the N pieces of target data, the target server can filter the N pieces of target data based on a second preset rule for filtering hot data by the target server, so as to perform secondary aggregation of hot data by the target server to obtain S pieces of target data. After obtaining the S pieces of target data, the S pieces of target data can be returned to the client.

[0048] Step S104, receiving the S pieces of target data fed back by the target server, and loading the S pieces of target data to a local cache, wherein the local cache is used to respond to a target access request, and the target access request includes an access request for accessing the target data.

[0049] The local cache can include a JVM cache (a memory area for storing temporary data), and the local cache can provide high-concurrency access capability. After the client receives S pieces of target data fed back by the target server, in order to avoid the situation that the system pressure increases when the traffic bursts, the target data can be loaded into the local cache, so that the target data can be provided by the local cache in the first time when the client receives an access request for the target data, and the situation that the system stability is affected by reading data from a server database is avoided, thereby achieving the purpose of guaranteeing the stability of the system without increasing the size of the cluster and introducing a high-performance memory database.

[0050] In the embodiment, by the above steps, the hot data is screened on the client and the server according to the preset rule, and the hot data is stored in the local cache, so that the hot data is read from the local cache in priority when the hot data is accessed, and the situation that the size of the server cluster is increased to cope with the traffic burst in the related art is avoided, thereby achieving the purpose of guaranteeing the high availability of the application system without increasing the size of the server cluster, and thereby realizing the technical effect of reducing the size of the server cluster and saving the cost. Furthermore, the technical problem that the high availability of the system is guaranteed by increasing the size of the server cluster in the related art when the traffic bursts, and the server redundancy is high when the traffic data is small is solved.

[0051] Optionally, after the S pieces of target data are loaded into the local cache, the method further includes: in response to a target access request, calling a target interface, and traversing the data stored in the local cache, in a case that the target data exists in the local cache, reading the target data from the local cache, and in a case that the target data does not exist in the local cache, reading the target data from a target storage space, wherein the target storage space includes at least one of a database for storing data of the client and a database for storing data of the target server.

[0052] In order to avoid the situation that the system pressure is too large due to the traffic burst when the hot data is accessed, in the embodiment, the target data can be read from the local cache when the target access request is received, the target data is directly read from the local cache in a case that the target data exists in the local cache, and the target data is read from the target storage space in a case that the target data does not exist in the local cache, thereby realizing the technical effect of reducing the pressure of the application system.

[0053] Optionally, after the S pieces of target data are loaded into the local cache, the method further includes: receiving a target operation instruction, wherein the target operation instruction includes at least one of a refresh instruction for periodically refreshing the data in the local cache, an addition instruction for adding new data in the local cache, and a deletion instruction for deleting specified data in the local cache; and executing the target operation instruction to update the data in the local cache.

[0054] In order to ensure the timeliness of the hotspot data in the local cache, in the embodiment, a group of threads can be added, and a Redis (remote dictionary service) message subscription mode is used, the preheating (i.e. adding) instructions, refreshing instructions, and eliminating instructions (corresponding to the deleting instructions) of the local cache are received in real time through the group of threads, and then the target operation instructions are executed to add, refresh, or eliminate the data in the local cache, so as to update the local cache, achieve the purpose of real-time intervention on the refreshing period and effectiveness of the local cache, and realize the technical effect of ensuring the timeliness of the local cache data.

[0055] For example, in the related art, in order to ensure that the system can cope with the impact of instantaneous traffic, obviously, neither Mysql (database) nor HTTP (hypertext transfer protocol) interface calling can meet the demand, and the introduction of the Redis cache to improve efficiency will cause IO (input / output) consumption, and with the increase of the deployment terminals, the number of Redis connections limits the horizontal expansion ability of the application to some extent, therefore, in the embodiment, in order to ensure the timeliness of the interface, a layer of local cache is added on the basis of the general Redis cache, and a multi-level cache architecture is obtained.

[0056] Figure 2 is an optional multi-level cache architecture diagram according to the embodiment of the application, as shown in Figure 2 The multi-level cache architecture includes terminal request, JVM cache, Redis, HTTP calling, Mysql, cache elimination, cache preheating, cache refreshing, and publishing and subscription, and the multi-level cache architecture is described as follows: after a user enters the system, the user can retrieve aggregated information from multiple data sources, such as user information, wealth information, and qualification information, the data sources of the information are different, and the information can be stored in Mysql and Redis, and the data in the local cache can be obtained in real time through the HTTP interface, in the embodiment, a plurality of groups of local caches are designed according to the data change frequency, and are used to store different contents. In order to ensure the timeliness of the cache, the preheating instructions, refreshing instructions, and eliminating instructions of the cache can be received in real time through the Redis message subscription mode.

[0057] It should be noted that updating the data in the local cache can be applied to the scenario of explicitly updating the data, for example, a user information retrieval service, when a user submits a request for updating personal information, after the operation personnel approve, the local caches distributed in various terminal nodes are actively updated.

[0058] Optionally, based on the first preset rule, the M pieces of to-be-detected data are screened to obtain N pieces of to-be-detected data, including: matching each piece of to-be-detected data with the first preset rule to determine to-be-detected data meeting the first preset rule; adding the to-be-detected data meeting the first preset rule to the target message queue, and reading the to-be-detected data in the target message queue to obtain N pieces of to-be-detected data.

[0059] In the embodiment, by encapsulating the public hotspot detection tool, all keys (for example, identifiers of to-be-detected data meeting the first preset rule) of data meeting the detection rule (that is, the first preset rule) and needing to be detected are determined, and the keys are written into the cache queue (corresponding to the target message queue) of the client. After simple aggregation, the to-be-detected data in the cache queue is read, and N pieces of to-be-detected data can be obtained. The technical effect of improving the hotspot data extraction efficiency is achieved.

[0060] Optionally, before the M pieces of to-be-detected data are screened based on the first preset rule to obtain N pieces of to-be-detected data, the method further includes: registering a node representing the client and a node representing the target service end to a target service system; and setting the first preset rule and the second preset rule in the target service system.

[0061] In the embodiment, each node participating in the distributed hot key automatic detection calculation, that is, each node participating in the hotspot data calculation, for example, the node representing the client and the node representing the target service end, can register its own information to the target service system (ZK for short) when the application is started. The first preset rule and the second preset rule can be configured in the target service center, and the technical effect of improving the flexibility of setting the hotspot data screening rule is achieved.

[0062] Optionally, after receiving the S pieces of target data sent by the target service end and loading the S pieces of target data to the local cache, the method further includes: transmitting the S pieces of target data to the target service system, and after the target service system receives the S pieces of target data, displaying the S pieces of target data, the first registration information of the node representing the client, and the second registration information of the node representing the target service end on the target screen.

[0063] The target screen can include a visual large screen for maintaining hotspot data. For example, the target screen can display the current hotspot information of the application system, maintain the access of the distributed hotspot calculation node, configure the hotspot detection rule, and the like. In the embodiment, the S pieces of target data, the first registration information of the node representing the client, and the second registration information of the node representing the target service end can be displayed on the target screen, so as to facilitate the timely monitoring of the hotspot data by the management personnel.

[0064] Embodiment Two

[0065] According to an embodiment of the present application, another alternative method embodiment of the hotspot data-based processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0066] Figure 3 is a flowchart of an alternative hotspot data-based processing method according to an embodiment of the present application, as shown in Figure 3 , the method comprises the following steps:

[0067] Step S301, receiving N pieces of target data sent by a client through a target server.

[0068] Among them, the client is used to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, encode each piece of to-be-detected data to obtain N pieces of target data, and send the N pieces of target data to the target server based on a long connection channel between the client and the target server. The first preset rule is used to filter hotspot data at the client. The hotspot data includes data with access times greater than a preset number threshold in a target time period. M and N are positive integers, and N is less than or equal to M.

[0069] The to-be-detected data described above can include data associated in the client of a financial institution. The first preset rule described above can include a rule for filtering data of the client to filter out hotspot data of the client. N pieces of to-be-detected data can be composed of all data meeting the first preset rule from M pieces of to-be-detected data of the client, so as to gather the hotspot data at the client once.

[0070] In order to reduce the traffic of data transmission, in the present embodiment, each piece of to-be-detected data in the N pieces of to-be-detected data can be encoded by a specified encoding algorithm or by a self-defined protocol, so as to compress the N pieces of to-be-detected data to obtain N pieces of target data.

[0071] The target server can include one of the plurality of servers in the service cluster. In order to improve the transmission efficiency of the message, in an embodiment, the N pieces of target data can be sent to the target server based on the long connection channel between the client and the target server. Since each client can establish a connection with each server in the plurality of servers (e.g., servers), a specific strategy can be selected, such as performing Hash on the key (e.g., identification information) of each target data, determining the target server from the plurality of servers according to the obtained Hash value, and reporting each target data to the designated server (i.e., the target server), thereby improving the processing capacity of the system and ensuring the timeliness of the hot key data (i.e., the hotspot data) detection.

[0072] In step S302, the N pieces of target data are filtered based on a second preset rule to obtain S pieces of target data, and the S pieces of target data are returned to the client.

[0073] The second preset rule is used to filter the hotspot data at the target server. After receiving the S pieces of target data sent by the target server, the client is further configured to load the S pieces of target data into the local cache. The local cache is used to respond to a target access request. The target access request includes an access request for accessing the target data. S is a positive integer less than N.

[0074] After receiving the N pieces of target data, the target server can filter the N pieces of target data based on the second preset rule for filtering the hotspot data at the target server, so as to perform secondary aggregation of the hotspot data at the target server to obtain S pieces of target data. After obtaining the S pieces of target data, the S pieces of target data can be returned to the client.

[0075] The local cache can include a JVM cache (a memory area for storing temporary data). The local cache can provide high-concurrency access capability. After receiving the S pieces of target data fed back by the target server, the client can load the target data into the local cache to provide the target data in the first time through the local cache when the client receives an access request for the target data, so as to avoid reading data from the server database and affecting the stability of the system, thereby achieving the purpose of ensuring the stability of the system without increasing the size of the cluster and introducing a high-performance memory database.

[0076] In the embodiment, according to a preset rule, hotspot data is screened on the client and the server, and the hotspot data is stored to a local cache, so that when the hotspot data is accessed, the hotspot data is read from the local cache preferentially, the condition of increasing the server cluster size to cope with traffic burst in the related art is avoided, the purpose of guaranteeing the high availability of the application system without increasing the server cluster size is achieved, and the technical effect of reducing the server cluster size and saving cost is achieved. Further, the technical problem that the high availability of the system when traffic bursts is guaranteed by increasing the server cluster size in the related art, and the server redundancy is high when the traffic data is small is solved.

[0077] Embodiment three

[0078] The embodiment three of the application provides a kind of optional hotspot data processing system, and a kind of optional hotspot data processing system can be used to execute the hotspot data processing method provided in the embodiment one or embodiment two of the present application, the hotspot data processing system includes: access layer, application layer, service layer and data layer.

[0079] Wherein, (1) access layer: access layer is mainly accessed to the business system corresponding to each level of user of application system, wherein, application system can include: multiple business systems, each business system can be used to realize at least one business function.

[0080] (2) application layer: application layer mainly includes each core business system of system, can be abstracted as user service, data service and hotspot large screen service. Wherein, hotspot large screen can cover current hotspot information display of application system, distributed hotspot computing node access maintenance, hotspot detection rule configuration etc.

[0081] (3) service layer: the core of service layer is the automatic detection of hot key (i.e. hotspot data), mainly by the registration of computing node, task engine calculation, rule engine self-update, long connection information communication, scalable data compression protocol module composition. Each node participating in distributed hot key automatic detection calculation will register the information of itself in ZK when application system starts, and computing node can be divided into Server end (i.e. server end) and Client end (i.e. client end).

[0082] In the embodiment, the connection can be established and the communication can be maintained through the long connection channel, then the Client end can perform preliminary aggregation on the keys meeting the detection condition (for example, the first preset rule) according to the detection rule in the rule engine, and transmit the aggregation result to the target Server end (corresponding to the target service end) through a specific transmission protocol at a time, and the target Server end performs secondary aggregation, and transmits the keys meeting the hot key condition (for example, the second preset rule) to the Client end through the long connection framework, and the Client end receives the information, and loads the related keys into the JVM memory (for example, the local cache with a specific valid time), so as to complete the whole process of detection, aggregation, identification and processing of the hot key. The retrieved hot key can be used for various purposes, for example, data access.

[0083] (4) Data layer: the data layer can mainly include three dimensions of information: storing data in ZK, storing data in the database of the Client end and the Server end, and storing data in the local cache. Among them, the registration information of the distributed computing node can be stored in ZK, the real-time detection rule has timeliness, and the Server end and the Client end are frequently read and written, therefore, the data can be stored in Redis, and written into Mysql synchronously, the hot key historical detection data mechanism is different, and the traffic is high, therefore, the data is stored in the database.

[0084] Figure 4 is another optional schematic diagram of the automatic detection framework based on hot data according to the embodiment of the application, as shown in Figure 4 The application service (application service 1 and application service 2) includes: dataId_xxx (data identifier), userId_xxx (user identifier) and award_xxx (reward identifier), and the application service further includes: rule matching, local aggregation and the like, and the hot spot receiving and task reporting between the application service and the computing node can be performed through the long connection channel (that is, the long connection channel) for data transmission, the computing node further includes: sliding interval 1 and sliding interval 2, the sliding interval can be used for further aggregation of the data reported by each Client end based on the time window algorithm, the application service and the computing node can perform node registration and data listening in ZK, and the automatic detection framework based on hot data can maintain the data in ZK through the hot data control large screen.

[0085] The embodiment also provides a multi-level cache architecture, mainly including a multi-level cache self-updating module, a distributed hotspot detection module, a rule engine configuration module, a hotspot data visual display module and the like. Relying on these modules, real-time mining of hotspot data of an application system can be realized, and the hotspot data is transmitted to each terminal node in real time, high-concurrency access capability is provided through a JVM local cache, and a visual large screen is used to provide real-time running conditions of an online system for development and operation personnel.

[0086] In the embodiment, the hotspot detection main process can include:

[0087] (1) End-to-end data collection: through encapsulation of a public hotspot detection tool, all data to be detected and meeting a detection rule are first written into a cache queue of a client, after simple aggregation, information is sent to a target server through a long connection established in advance, and in order to reduce traffic, data is encoded through a self-defined protocol before being sent. It should be noted that the connection between the client and the server can be one-to-many, and the client can select a specific strategy before reporting data, such as reporting to a specified server according to a key, so as to improve the processing capacity of the system, and thus ensure the timeliness of the hot key data detection.

[0088] (2) Hotness calculation and matching: after the target server receives the data from the client, the data reported by each client can be further aggregated based on a time window algorithm, and matched with a preconfigured rule.

[0089] (3) Hotspot data distribution and JVM local cache maintenance: the key triggering the hotness rule is distributed to each client through a long connection, and each client receives the hot key push and executes corresponding business logic according to the key attribute.

[0090] It should be noted that in the embodiment, by constructing a multi-level cache architecture, the hotspot data is preloaded in the JVM cache according to the hotspot update frequency and other information, so as to maximize the response capability of the system, reduce the dependence of the system on Redis and other middleware, and have a certain degree of disaster recovery. When the Redis fails in the past, the impact on the customer is minimized, but in the past, the data cache needs to be triggered actively. By constructing an automatic hotspot detection mechanism, this shortcoming can be effectively made up, the hotspot recognition capability is greatly improved, and finally the automatic loading and updating of the hotspot data are realized, and the timeliness and accuracy are combined.

[0091] In the embodiment, a high-efficiency data communication protocol can be used for data transmission between the client and the server, reducing the impact of data reporting on bandwidth traffic. For example, in combination with the customization and compression of the message protocol, the message body can be reduced by 30% in a single reporting. A distributed deployment mode is used to provide one-to-one and one-to-many deployment modes, which can ensure that the node is deployed on the server and the traffic is adjusted at any time according to the business concurrency, and the client-side pre-aggregation can greatly reduce the consumption of data communication on the traffic bandwidth and improve the efficiency of hotspot data detection. In addition, the multi-level cache architecture has strong scalability, and can be divided into a task work engine and a message communication engine. Through the pre-aggregation of the client, a task processing capacity of 10W+ per second can be realized. The framework can be applied to interface flow limiting, illegal user account identification and other application scenarios, greatly improving the security of the system.

[0092] Embodiment four

[0093] The embodiment four of the application provides a kind of optional processing device based on hotspot data, each implementation unit in the processing device is used to implement each implementation step in embodiment one.

[0094] Figure 5 It is a schematic diagram of an optional processing device based on hotspot data according to the embodiment of the application, as Figure 5 As shown in the figure, the processing device includes a first screening unit 51, an encoding unit 52, a first sending unit 53 and a processing unit 54.

[0095] Specifically, the first screening unit 51 is used to screen M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data on the client side, the hotspot data includes data with a number of access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M.

[0096] The encoding unit 52 is used to perform encoding processing on each piece of to-be-detected data to obtain N pieces of target data.

[0097] The first sending unit 53 is used to send the N pieces of target data to a target server based on a long connection channel between the client and the target server, wherein the target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client, and S is a positive integer less than N.

[0098] The processing unit 54 is used to receive the S pieces of target data fed back by the target server, and load the S pieces of target data to a local cache, wherein the local cache is used to respond to a target access request. The target access request includes an access request for accessing the target data.

[0099] In the hotspot data-based processing apparatus provided in the fourth embodiment of the present application, the first screening unit 51 can screen the M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used to screen hotspot data at the client side, the hotspot data includes data with a number of visits greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M. The encoding unit 52 encodes each piece of to-be-detected data to obtain N pieces of target data. The first sending unit 53 sends the N pieces of target data to the target server based on a long connection channel between the client and the target server. The target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client. S is a positive integer less than N. The processing unit 54 receives the S pieces of target data fed back by the target server, and loads the S pieces of target data to a local cache. The local cache is used to respond to a target access request, and the target access request includes an access request for accessing target data. Thus, the technical problem that, in the related art, increasing the size of a server cluster to ensure the high availability of a system in the case of traffic explosion results in high server redundancy in the case of less traffic data is solved.

[0100] In the present embodiment, hotspot data is screened at the client side and the server side according to a preset rule, and the hotspot data is stored to a local cache, so that the hotspot data is read from the local cache in priority when the hotspot data is accessed, thereby avoiding the case that, in the related art, the size of a server cluster is increased to cope with traffic explosion, and achieving the purpose of ensuring the high availability of an application system without increasing the size of the server cluster, thereby realizing the technical effect of reducing the size of the server cluster and saving cost.

[0101] Optionally, in the hotspot data-based processing apparatus provided in the fourth embodiment of the present application, the hotspot data-based processing apparatus further includes a first reading unit configured to, after loading the S pieces of target data to the local cache, respond to a target access request, call a target interface, traverse data stored in the local cache, and read target data from the local cache in the case that the target data exists in the local cache; and a second reading unit configured to, in the case that the target data does not exist in the local cache, read the target data from a target storage space, wherein the target storage space includes at least one of a database configured to store data of the client and a database configured to store data of the target server.

[0102] Optionally, in the hotspot data-based processing apparatus provided in Embodiment Four of the present application, the hotspot data-based processing apparatus further comprises an instruction receiving unit configured to receive a target operation instruction after the S pieces of target data are loaded into the local cache, wherein the target operation instruction comprises at least one of a refresh instruction for periodically refreshing the data in the local cache, an addition instruction for adding new data in the local cache, and a deletion instruction for deleting specified data in the local cache; and an execution unit configured to execute the target operation instruction to update the data in the local cache.

[0103] Optionally, in the hotspot data-based processing apparatus provided in Embodiment Four of the present application, the first screening unit comprises a determination sub-unit configured to match each piece of to-be-detected data with the first preset rule to determine the to-be-detected data meeting the first preset rule; and a processing sub-unit configured to add the to-be-detected data meeting the first preset rule into the target message queue, and read the to-be-detected data in the target message queue to obtain the N pieces of to-be-detected data.

[0104] Optionally, in the hotspot data-based processing apparatus provided in Embodiment Four of the present application, the hotspot data-based processing apparatus further comprises a registration unit configured to register the node representing the client and the node representing the target service end to the target service system before screening the M pieces of to-be-detected data based on the first preset rule to obtain the N pieces of to-be-detected data; and a setting unit configured to set the first preset rule and the second preset rule in the target service system.

[0105] Optionally, in the hotspot data-based processing apparatus provided in Embodiment Four of the present application, the hotspot data-based processing apparatus further comprises a transmission unit configured to transmit the S pieces of target data to the target service system after receiving the S pieces of target data sent by the target service end and loading the S pieces of target data into the local cache, and display the S pieces of target data, the first registration information representing the node of the client, and the second registration information representing the node of the target service end on the target screen after the target service system receives the S pieces of target data.

[0106] The hotspot data-based processing apparatus described above can further comprise a processor and a memory, and the first screening unit 51, the encoding unit 52, the first sending unit 53, and the processing unit 54 described above are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0107] The processor includes a core, and the core calls corresponding program units in the memory. The core can be one or more, and the core parameters are adjusted according to preset rules to filter hotspot data on the client and the server, and the hotspot data is stored in the local cache. When the hotspot data is accessed, the hotspot data is read from the local cache, avoiding the increase of the server cluster size in the related art to cope with the traffic surge, and achieving the purpose of guaranteeing the high availability of the application system without increasing the server cluster size, thereby realizing the technical effect of reducing the server cluster size and saving the cost.

[0108] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0109] Embodiment five

[0110] Embodiment four of the application provides another optional hotspot data-based processing device, and each implementation unit in the processing device corresponds to each implementation step in embodiment two.

[0111] Figure 6 is a schematic diagram of an optional hotspot data-based processing device according to an embodiment of the application, as shown in Figure 6 The processing device includes a second sending unit 61 and a second filtering unit 62.

[0112] Specifically, the second sending unit 61 is configured to receive N pieces of target data sent by a client through a target server, wherein the client is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, perform encoding processing on each piece of to-be-detected data to obtain N pieces of target data, and send the N pieces of target data to the target server based on a long connection channel between the client and the target server, the first preset rule is configured to filter hotspot data on the client, the hotspot data includes data with a number of access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M.

[0113] The second filtering unit 62 is configured to filter the N pieces of target data based on a second preset rule to obtain S pieces of target data, and return the S pieces of target data to the client, wherein the second preset rule is configured to filter hotspot data on the target server, and the client is further configured to load the S pieces of target data to a local cache after receiving the S pieces of target data sent by the target server, the local cache is configured to respond to a target access request, the target access request includes an access request for accessing target data, and S is a positive integer less than N.

[0114] In the hotspot data-based processing apparatus provided in the fifth embodiment of the present application, the second sending unit 61 is configured to receive N pieces of target data sent by the client from the target server, wherein the client is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, encode each piece of to-be-detected data to obtain a piece of target data, and send the N pieces of target data to the target server based on a long connection channel between the client and the target server, the first preset rule is used to filter hotspot data at the client, the hotspot data includes data with a number of access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than or equal to M, the second filtering unit 62 is configured to filter the N pieces of target data based on a second preset rule to obtain S pieces of target data, and return the S pieces of target data to the client, wherein the second preset rule is used to filter hotspot data at the target server, and the client is further configured to load the S pieces of target data to a local cache after receiving the S pieces of target data sent by the target server, the local cache is used to respond to a target access request, the target access request includes an access request for target data, and S is a positive integer less than N. Thus, the technical problem that, in the related art, the server cluster size is increased to ensure the high availability of the system when the traffic bursts, resulting in a high server redundancy when the traffic data is small, is solved.

[0115] In this embodiment, the hotspot data is filtered at the client and the server according to the preset rule, and the hotspot data is stored in the local cache, so that the hotspot data is read from the local cache when the hotspot data is accessed, which avoids the situation that, in the related art, the server cluster size is increased to cope with the traffic burst, and achieves the purpose of ensuring the high availability of the application system without increasing the server cluster size, thereby realizing the technical effect of reducing the server cluster size and saving the cost.

[0116] The hotspot data-based processing apparatus described above can further include a processor and a memory, and the second sending unit 61 and the second filtering unit 62 and the like described above are stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0117] The processor described above includes a kernel, and the kernel calls the corresponding program units from the memory. The kernel can be one or more, and the kernel parameters are adjusted to filter the hotspot data at the client and the server according to the preset rule, and the hotspot data is stored in the local cache, so that the hotspot data is read from the local cache when the hotspot data is accessed, which avoids the situation that, in the related art, the server cluster size is increased to cope with the traffic burst, and achieves the purpose of ensuring the high availability of the application system without increasing the server cluster size, thereby realizing the technical effect of reducing the server cluster size and saving the cost.

[0118] The memory can include non-persistent memory in a computer readable medium, forms of random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory, including at least one memory chip.

[0119] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to perform any of the above-mentioned hotspot data based processing methods by executing the executable instructions.

[0120] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer program, when executed, controls a device where the computer readable storage medium is located to perform any of the above-mentioned hotspot data based processing methods.

[0121] Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present application, as Figure 7 shown, the embodiments of the present application provide an electronic device 70, which includes a processor, a memory and a program stored in the memory and executable on the processor, and the processor implements any of the above-mentioned hotspot data based processing methods when executing the program.

[0122] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0123] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0124] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between units or modules, which can be electrical or other forms.

[0125] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0126] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0127] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical scheme of the present application or the part of the present application which contributes to the prior art or the whole or part of the technical scheme can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0128] The above is only the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A processing method based on hotspot data, characterized by, The method comprises the following steps: screening M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used for screening hot data on a client side, the hot data comprises data with a number of visits greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than M; performing encoding processing on each piece of to-be-detected data to obtain N pieces of target data; sending the N pieces of target data to a target server based on a long connection channel between the client and the target server, wherein the target server screens the N pieces of target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of target data to the client, S is a positive integer less than N; receiving the S pieces of target data fed back by the target server and loading the S pieces of target data into a local cache, wherein the local cache is used to respond to a target access request, and the target access request comprises an access request for accessing the target data.

2. The treatment method according to claim 1, characterized in that, After loading the S pieces of target data into the local cache, the method further comprises the following steps: in response to the target access request, calling a target interface, traversing data stored in the local cache, and reading the target data from the local cache in the case that the target data exists in the local cache; in the case that the target data is not read from the local cache, reading the target data from a target storage space, wherein the target storage space comprises at least one of a database for storing data of the client and a database for storing data of the target server.

3. The treatment method of claim 1, wherein After loading the S pieces of target data into the local cache, the method further comprises the following steps: receiving a target operation instruction, wherein the target operation instruction comprises at least one of a refresh instruction for periodically refreshing data in the local cache, an addition instruction for adding new data in the local cache, and a deletion instruction for deleting specified data in the local cache; executing the target operation instruction to update the data in the local cache.

4. The treatment method of claim 1, wherein Screening M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data comprises the following steps: matching each piece of to-be-detected data with the first preset rule to determine the to-be-detected data meeting the first preset rule; adding the to-be-detected data meeting the first preset rule to a target message queue and reading the to-be-detected data in the target message queue to obtain N pieces of to-be-detected data.

5. The treatment method of claim 1, wherein Before screening M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, the method further comprises the following steps: registering a node representing the client and a node representing the target server to a target service system; setting the first preset rule and the second preset rule in the target service system.

6. The treatment method according to claim 5, characterized in that, After receiving the S pieces of target data sent by the target server and loading the S pieces of target data into the local cache, the method further comprises the following steps: S pieces of the target data are transmitted to the target service system, and after the target service system receives the S pieces of the target data, the S pieces of the target data, first registration information representing a node of the client, and second registration information representing a node of the target service end are displayed on a target screen.

7. A processing method based on hotspot data, characterized by, Comprise: N pieces of target data are received by a target service end, wherein the client is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, encode each piece of the to-be-detected data to obtain N pieces of the target data, and send the N pieces of the target data to the target service end based on a long connection channel between the client and the target service end, the first preset rule is used to filter hot data at the client, the hot data includes data with a number of access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than M; N pieces of the target data are filtered based on a second preset rule to obtain S pieces of target data, and the S pieces of the target data are returned to the client, wherein the second preset rule is used to filter hot data at the target service end, and the client is further configured to load the S pieces of the target data into a local cache after receiving the S pieces of the target data sent by the target service end, the local cache is used to respond to a target access request, the target access request includes an access request for accessing the target data, and S is a positive integer less than N.

8. A processing device based on hotspot data, characterized by, Comprise: A first filtering unit is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, wherein the first preset rule is used to filter hot data at the client, the hot data includes data with a number of access times greater than a preset number threshold in a target time period, M and N are positive integers, and N is less than M; An encoding unit is configured to encode each piece of the to-be-detected data to obtain N pieces of target data; A first sending unit is configured to send the N pieces of the target data to a target service end based on a long connection channel between the client and the target service end, wherein the target service end filters N pieces of the target data based on a second preset rule to obtain S pieces of target data, and returns the S pieces of the target data to the client, S is a positive integer less than N; A processing unit is configured to receive S pieces of the target data fed back by the target service end, and load the S pieces of the target data into a local cache, wherein the local cache is used to respond to a target access request, and the target access request includes an access request for accessing the target data.

9. A processing device based on hotspot data, characterized by, Comprise: The second sending unit is configured to receive N pieces of target data sent by a client through a target server, wherein the client is configured to filter M pieces of to-be-detected data based on a first preset rule to obtain N pieces of to-be-detected data, perform encoding processing on each piece of the to-be-detected data to obtain N pieces of the target data, and send the N pieces of the target data to the target server based on a long connection channel between the client and the target server, the first preset rule is used to filter hot data at the client, the hot data includes data with a number of access times greater than a preset number threshold in a target time period, and M and N are positive integers and N is less than M; The second filtering unit is configured to filter the N pieces of the target data based on a second preset rule to obtain S pieces of target data, and return the S pieces of the target data to the client, wherein the second preset rule is used to filter hot data at the target server, and the client is further configured to load the S pieces of the target data into a local cache after receiving the S pieces of the target data sent by the target server, the local cache is used to respond to a target access request, the target access request includes an access request for accessing the target data, and S is a positive integer less than N.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program runs, controls a device where the computer readable storage medium is located to execute the hot data based processing method in any one of claims 1 to 6.

11. An electronic device, comprising: The device includes one or more processors and a memory, and the memory is configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the hot data based processing method in any one of claims 1 to 6.

Images