Data storage method and device, electronic equipment and storage medium
By periodically obtaining, classifying and sorting terminal event information in an enterprise-level wireless management system, and using message queues and hash operations to ensure sequential storage of events, the out-of-order storage problem caused by terminal device movement is solved, and data accuracy and consistency are achieved.
Patent Information
- Application Number
- CN202410039800.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-09
- Publication Date
- 2025-07-11
AI Technical Summary
In enterprise-level wireless management systems, due to the movement of terminal devices, the communication events reported by network devices are out of order, which may cause the network administrator to exist on multiple network devices at the same time when viewing terminal events, resulting in inaccurate data.
Terminal event information is obtained periodically through the server, and then stored in the database according to the terminal identification classification and timestamp sorting. Message queue and hashing operations are used to ensure that events are updated in the correct order and avoid unintended storage.
Ensures the accuracy of data when network administrators view terminal events and avoids error records when multiple network devices are connected to the same terminal at the same time. Although there is a certain delay, the accuracy of data is ensured.
Smart Images

Figure CN120296019A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technologies. Specifically, this application relates to a data storage method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Art
[0002] Currently, many large and medium-sized enterprises deploy enterprise-level wireless management systems, including servers for wireless management deployed in the cloud (or in their own computer rooms) (hereinafter referred to as servers) and network devices distributed in various office areas. The server combines with the network devices to provide services such as authentication, authorization, and monitoring for enterprise wireless network access.
[0003] In the related art, the server needs to save all network devices and the list of terminals connected to each network device in the database, and update the terminal list in the database after receiving the terminal event information reported by the terminal.
[0004] Since the carrier of the terminal moves within the enterprise, the terminal will also roam on different network devices as the carrier moves. However, due to network link reasons, when the enterprise network administrator views the event information, it is possible to see a situation where a terminal exists on two network devices at the same time. In extreme cases, if the terminal moves quickly across regions, there may be more wireless devices that contain the terminal. Summary of the Invention
[0005] Embodiments of this application provide a data storage method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can solve the above problems of the prior art. The technical solutions are as follows:
[0006] In a first aspect, an embodiment of this application provides a data storage method, which is applied to a server cluster. The server cluster includes at least one server. The method is executed by a first server, and the first server is any one of the servers in the server cluster. The method includes:
[0007] Periodically obtain an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes a communication event between a terminal and a network device, a timestamp indicating the moment when the communication event occurs, and a terminal identifier of the corresponding terminal;
[0008] Classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier;
[0009] Sort the terminal event information in each information cluster according to the timestamps in the terminal event information to obtain the sorted information cluster corresponding to each information cluster;
[0010] Store the terminal event information in each sorted information cluster into the database.
[0011] In a second aspect, an embodiment of the present application provides a data storage device, which is applied to a server cluster. The server cluster includes at least one server. The method is executed by a first server, and the first server is any one of the servers in the server cluster. The device includes:
[0012] An information set acquisition module, configured to periodically acquire an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes a communication event between a terminal and a network device, a timestamp for characterizing the occurrence time of the communication event, and a terminal identifier of the corresponding terminal;
[0013] A classification module, configured to classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier;
[0014] A sorting module, configured to sort the terminal event information in each information cluster according to the timestamp in the terminal event information to obtain the sorted information cluster corresponding to each information cluster;
[0015] A storage module, configured to store the terminal event information in each sorted information cluster into the database.
[0016] Based on the above embodiments, as an optional embodiment, the information set corresponding to the first server is cached in a first storage area in advance. The first storage area is a storage area corresponding to the first server among multiple pre-created storage areas. The terminal event information cached in each storage area includes a terminal identifier corresponding to the storage area;
[0017] The information set acquisition module includes:
[0018] A region determination unit, configured to determine the first storage area from the multiple storage areas;
[0019] A periodic acquisition unit, configured to periodically acquire the cached terminal event information from the first storage area.
[0020] Based on the above embodiments, as an alternative embodiment, the storage area is a partition in a preset first message queue. The first message queue is used to use each network device as a producer, the first server as a consumer, the terminal event information uploaded by the producer as a message, and cache the message to the partition corresponding to the terminal identifier of the message.
[0021] Based on the above embodiments, as an alternative embodiment, the first message queue caches the message to the partition corresponding to the terminal identifier of the message in the following manner:
[0022] For each message, perform a hash operation on the terminal identifier of the message to obtain a hash value, use the hash value as the key of the message, and send the message to the partition corresponding to the key through a partitioner.
[0023] Based on the above embodiments, as an alternative embodiment, the storage area is a second message queue among multiple preset second message queues. The second message queue is used to use a preset processing device as a producer, the server corresponding to the second message queue as a consumer, and cache the terminal event information uploaded by the producer as a message;
[0024] Among them, the processing device is used to receive the terminal event information uploaded by each network device, and send the terminal event information to the corresponding second message queue according to the terminal identifier in the terminal event information.
[0025] Based on the above embodiments, as an alternative embodiment, the storage module includes:
[0026] A splitting unit, configured to split each sorted information cluster with a preset time window length to obtain an information subset corresponding to each time window;
[0027] A merging unit, configured to start from the information subset corresponding to the first time window for each sorted information cluster, merge adjacent and non-empty information subsets to obtain at least one target set, where the target final set includes at least one non-empty information subset;
[0028] A storage unit, configured to store each target set corresponding to each information cluster set into a database.
[0029] Based on the above embodiments, as an alternative embodiment, the merging unit is specifically configured to:
[0030] Use each information subset as a subset to be processed, and sequentially perform the following operations on each of the subsets to be processed in the order of the time windows to obtain at least one target set:
[0031] If the information subset to be processed is an empty information subset, skip the information subset to be processed;
[0032] If the information subset to be processed is a non-empty information subset and meets the first condition, create a new target terminal event set and add the information subset to be processed to the target set;
[0033] If the information subset to be processed is a non-empty information subset and does not meet the first condition, add the information subset to be processed to the target set where the previous information subset is located;
[0034] Among them, the meeting of the first condition includes: the information subset to be processed is the first information subset, or the previous information subset of the information subset to be processed is an empty information subset.
[0035] Based on the above embodiments, as an optional embodiment, the type of any of the communication events is a connection event or a disconnection event. The connection event refers to an event in which a terminal is connected to a network device, and the disconnection event refers to an event in which a terminal is disconnected from a network device;
[0036] The storage unit is further configured to: for the terminal event information of the same type in each information subset, only retain the terminal event information with the latest timestamp in the terminal event information of the same type.
[0037] In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the above data storage method.
[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above data storage method is implemented.
[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the above method is implemented.
[0040] The technical solution provided by the embodiment of the present application is applied to a server cluster. The server cluster includes at least one server. The method is executed by a first server, and the first server is any one of the servers in the server cluster. To avoid the situation where communication events are stored in a disorderly manner and the situation where multiple network devices are recorded as being connected to a terminal at the same time, the present application periodically obtains an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes, in addition to the communication event between the terminal and the network device, a timestamp for characterizing the occurrence time of the communication event and the terminal identifier of the corresponding terminal. After receiving each terminal event information, the server does not process it immediately, but first caches it and processes it in batches at regular intervals. Because the event carries a timestamp, the events can be updated to the database in the correct order during batch processing, so as to ensure that the network management will not see incorrect data when viewing the communication events of the terminal. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1a It is a schematic diagram of the architecture of the wireless management system provided by the embodiment of the present application;
[0042] Figure 1b It is a schematic diagram of the server updating the communication events in the database provided by the embodiment of the present application;
[0043] Figure 2 It is a schematic flow diagram of a data storage method provided by the embodiment of the present application;
[0044] Figure 3 It is a schematic diagram of the relationship between the terminal event information uploaded by the network device and the storage area provided by the embodiment of the present application;
[0045] Figure 4 It is a schematic diagram of caching the terminal event information uploaded by the network device through the kafka message queue provided by the embodiment of the present application;
[0046] Figure 5 It is a schematic diagram of an application scenario provided by the embodiment of the present application;
[0047] Figure 6 It is a schematic diagram of the architecture of the wireless management system provided by another embodiment of the present application;
[0048] Figure 7 It is a schematic diagram of dividing an information cluster provided by the embodiment of the present application;
[0049] Figure 8 It is a schematic flow diagram of merging each information subset provided by the embodiment of the present application;
[0050] Figure 9A schematic flowchart of a data storage method provided by another embodiment of the present application;
[0051] Figure 10 A schematic structural diagram of a data storage device provided by an embodiment of the present application;
[0052] Figure 11 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners
[0053] The embodiments of the present application will be described below with reference to the accompanying drawings in the present application. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute limitations on the technical solutions of the embodiments of the present application.
[0054] Those skilled in the art of the present technology can understand that unless specifically stated otherwise, the singular forms "a", "an", and "the" used herein may also include the plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components, and / or their combinations supported by the art of the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include a wireless connection or a wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or implemented as "B", or implemented as "A and B".
[0055] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.
[0056] Currently, many medium and large enterprises will deploy enterprise-level wireless management systems, including servers for wireless management deployed in the cloud (or in an on-premises data center) (hereinafter referred to as servers) and network devices distributed in various office areas. The network devices in the embodiments of the present application can be wireless network devices, such as wireless access points (APs), access controllers (ACs), fat access points (Fat APs), etc. The server combines with the network devices to provide services such as wireless network access authentication, authorization, and monitoring for enterprises.
[0057] Please refer to Figure 1a, which exemplarily shows the schematic architecture diagram of the wireless management system provided by the embodiments of the present application. As shown in the figure, the network devices in the embodiments of the present application include AC devices, AP devices, and Fat-AP devices, and multiple network devices can form a hierarchical structure. In the figure, one AC device is connected to two AP devices, and each AP device is respectively connected to one terminal. It should be noted that the embodiments of the present application do not specifically limit the hierarchical structure formed by network devices. The hierarchical structure may include more levels, and each level may include more network devices. Moreover, the embodiments of the present application do not limit the number of terminals connected to one network device. In actual applications, one network device can be connected to more terminals.
[0058] The server is oriented to enterprise network management. The gateway can configure wireless access authentication methods, network access permission control, etc. for enterprise employees, and can also observe the terminal connection status of employees in real time. For example, it can query at any time how many employees' terminals are connected in a certain meeting room.
[0059] A long connection is established between the server and the network device. The downlink of the connection is used to transmit the configuration information of the server for the network device, such as keys, network control records, etc.; the uplink of the connection is used for event reporting of the network device, such as periodic reporting of operating status, network rate, etc. When a terminal accesses or disconnects from the network device, terminal event information will also be reported. The server can obtain the distribution of terminals in the office area at a certain moment through event statistics.
[0060] Please refer to Table 1, which exemplarily shows the communication event table of the embodiments of the present application. Four events are shown in Table 1, namely the event of recording the CPU usage rate of the network device, the event of memory usage rate, the connection event, and the disconnection event. It can be seen that there are certain differences in the content of different events, but all events include the ID of the network device reporting the communication event.
[0061]
[0062] Table 1 Communication Event Table
[0063] The embodiments of the present application focus on terminal events, including the StationOnline event (also known as the connection event) and the StationOffline event (also known as the disconnection event) in Table 1. It should be understood that when a terminal connects to a network device, the network device will report a connection event to the server. If the terminal disconnects from the network device, the network device will also report a disconnection event to the server.
[0064] In the related art, the server needs to store all network devices and the list of terminals connected to each network device in the database. After receiving the communication events reported by the network devices, it updates the terminal list in the database and records the time when the corresponding communication events are received. Please refer to Figure 1b , which exemplarily shows a schematic diagram of the server updating the communication events in the database provided by the embodiment of the present application. As shown in the figure, different storage spaces are set for different network devices in the database. Only the communication events reported by the corresponding network device are stored in one storage space. Two storage spaces are shown in the figure, which are the storage spaces of network device 1 (i.e., Device-1 in the figure) and network device 2 (i.e., Device-1 in the figure) respectively. The storage space of network device 1 only stores each communication event reported by network device 1, and the storage space of network device 2 also only stores each communication event reported by network device 2. In the figure, station1, station2, and station3 respectively represent the terminal event information related to terminal 1, terminal 2, and terminal 3.
[0065] Since the carriers of the terminals will move within the enterprise, the terminals will also roam on different network devices as the carriers move. However, since the communication events of each network device are reported independently, due to network link reasons, the communication events reported by the network devices may arrive at the server out of order. Suppose the terminal roams from network device 1 to network device 2. Obviously, network device 1 generates a disconnection event first, and network device 2 generates a connection event later. However, due to network link reasons, the connection event of network device 2 is received by the server first. At this time, when the enterprise network administrator views the network events, it may be seen that there is a terminal on two network devices at the same time. In an extreme case, if the terminal moves quickly across regions, there may be more wireless devices that include this terminal.
[0066] The data storage method, device, electronic device, computer-readable storage medium, and computer program product provided by the present application aim to solve the above technical problems in the prior art.
[0067] Next, through the description of several exemplary embodiments, the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described. It should be noted that the following embodiments can refer to, draw on, or combine with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.
[0068] In the embodiment of the present application, a data storage method is provided, which is applied to a server cluster. The server cluster can include one server or multiple servers. This data storage method is executed by the first server, and the first server is any one of the servers in the server cluster. For example Figure 2As shown, the method includes:
[0069] S101. Periodically obtain an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes a communication event between a terminal and a network device, a timestamp for characterizing the occurrence time of the communication event, and a terminal identifier of the corresponding terminal.
[0070] In the embodiments of the present application, a correspondence relationship is established between the server and the information set. One server can correspond to at least one information set, but one information set only corresponds to one server. At least one terminal event information is included in one information set. It should be understood that when there is only one server in the server cluster, the server will correspond to all information sets. Thus, the server can completely obtain all terminal event information of the terminal.
[0071] In addition to including the communication event between the terminal and the network device, the terminal event information in the embodiments of the present application also includes a timestamp for characterizing the sending time of the communication event. Such a setting enables the grid to determine the sequence of occurrence of different communication events based on the timestamp when viewing the terminal event information, ensuring that the gateway will not be misled when viewing the terminal event information of the terminal. Considering that due to network link reasons, the terminal event information that occurs first may be received by the server later than the terminal communication event that occurs later, the present application adopts a periodic acquisition method, that is, when the server receives the terminal communication event, it does not process it immediately, but caches it first, and then periodically obtains the information set in batches, so as to ensure that there will be no incorrect data when the network management views the terminal event information of the terminal (in this scenario, the accuracy of the data is more important than the real-time nature of the data. Although the periodic batch processing of events causes a slight delay in data update, it can ensure the accuracy of the events).
[0072] The embodiments of the present application do not limit the duration of the period. For example, it can be several seconds, several minutes, several hours, or several days. In some embodiments, the period in the embodiments of the present application can also be a non-fixed period duration. The network device can count the number of new terminal events uploaded to a certain server. When the preset number is reached, a prompt message is sent to the server and the count is cleared. The server obtains the terminal event information uploaded by the network device according to the prompt message, so as to ensure that the server can dynamically select the terminal event information to be processed according to the busy degree of the network device.
[0073] S102. Classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier.
[0074] The terminal identifiers included in each terminal event information in the information set of the embodiments of the present application may not be the same, that is, the information set includes the terminal event information of multiple terminals. Therefore, the terminal event information of the embodiments of the present application also includes the terminal identifiers of the corresponding terminals. By grouping the information set according to the terminal identifiers, information clusters corresponding to each terminal identifier can be obtained. It can be understood that an information cluster includes at least one terminal event information with the corresponding terminal identifier. By classifying the information set according to the terminal identifiers, the terminal event information of the same terminal is grouped together, laying a foundation for subsequent batch processing of the terminal event information of the same terminal.
[0075] S103. Sort the terminal event information in each information cluster according to the timestamps in the terminal event information to obtain the sorted information cluster corresponding to each information cluster.
[0076] In the embodiments of the present application, by sorting the terminal event information in the information cluster according to the timestamps in the terminal event information, the various terminal event information of a terminal can be arranged in sequence from first to last according to the timing of communication events, laying a foundation for the subsequent network management to view the terminal event information with correct timing.
[0077] S104. Store the terminal event information in each sorted information cluster into a database.
[0078] In the embodiments of the present application, each terminal event information can be stored in the database in units of information clusters, that is, all the terminal event information of the same terminal is centrally stored in a storage area in the database, facilitating network management query.
[0079] Please refer to Table 2, which exemplarily shows the information cluster table of the embodiments of the present application. As shown in Table 2, this information cluster table records 4 terminal event information, corresponding to two connection events and two disconnection events respectively. In the embodiments of the present application, the information cluster is stored in the database in the form of a table, and the gateway can intuitively see each terminal event and the order of occurrence through the table when querying the information cluster.
[0080] Communication event Timestamp Terminal identifier Connection event 2024.1.1.10:21:03 165465132.29 Disconnection event 2024.1.11:25:16 165465132.29 Connection event 2024.1.1.14:25:17 165465132.29 Disconnection event 2024.1.1.14:50:48 165465132.29
[0081] Table 2 Information Cluster Table
[0082] In the embodiments of the present application, the information cluster can be stored in the form of key-value pairs, with the terminal identifier corresponding to the information cluster as the key and the sorted communication events and timestamps of the information cluster as the value for storage. The embodiments of the present application do not make specific limitations.
[0083] The data storage method according to the embodiment of the present application is applied to a server cluster. The server cluster includes at least one server, and the method is executed by a first server, where the first server is any one of the servers in the server cluster. To avoid the situation where communication events are stored in a disordered manner and the situation where multiple network devices are recorded as being connected to a terminal at the same time, the present application periodically obtains an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes, in addition to the communication event between the terminal and the network device, a timestamp indicating the occurrence time of the communication event and the terminal identifier of the corresponding terminal. After receiving each terminal event information, the server does not process it immediately, but first caches it and performs batch storage at regular intervals. Since the event carries an event timestamp, the events can be updated to the database in the correct order during batch processing. In this way, it can be ensured that the network management will not see incorrect data when viewing the communication events of the terminal (in this scenario, the accuracy of the data is more important than the real-time nature of the data. Processing events in batches at regular intervals will cause a delay in data update, but there will be no errors).
[0084] Based on the above embodiments, when the scale of the enterprise accessed by the server is very large, the performance of a single server cannot meet the requirements. At this time, it is necessary to expand the service in a clustered manner to form a server cluster. However, when it is expanded into multiple servers, the method of caching event information in each server and then performing batch processing at regular intervals will not work, because the event information reported by the network device may reach any one of the servers. If each server cannot obtain all the event information of a terminal, the significance of batch processing will be small.
[0085] In view of the above situation, when there are multiple servers in the server cluster according to the embodiment of the present application, as an optional embodiment, the terminal event information corresponding to each server in the server cluster is cached in the storage area corresponding to the server, and the terminal event information cached in each storage area includes the terminal identifier corresponding to the storage area.
[0086] That is to say, on the one hand, according to the terminal identifier recorded in the terminal event information, the embodiment of the present application establishes a corresponding relationship between the terminal identifier and the storage area. Therefore, all the terminal event information corresponding to the same terminal will be stored in the same storage area. On the other hand, a corresponding relationship between the server and the storage area is established. A server can establish a corresponding relationship with at least one storage area, but a storage area is only in a corresponding relationship with one server. In this way, it can be ensured that all the terminal event information corresponding to a terminal will be processed by one server and will not be repeatedly processed by multiple servers.
[0087] Please refer to Figure 3, which exemplarily shows a schematic diagram of the relationship between the terminal event information uploaded by the network device provided in the embodiment of the present application and the storage area. As shown in the figure, each network device is connected to one or more terminals, and uploads the terminal event information corresponding to the terminal to the storage area corresponding to the terminal identifier. That is to say, in the embodiment of the present application, the network device has the ability to upload the terminal event information to the corresponding storage area based on the terminal identifier. The network device can pre-store the corresponding relationship between the terminal identifier and the storage area. It can be seen from the figure that a storage area can store the terminal event information of only one terminal, or can store the terminal event information of multiple terminals. The network device uploads the terminal event information to the corresponding storage area for caching, and the server periodically obtains the terminal event information from the corresponding storage area. It can be seen from the figure that one server is associated with at least one storage area, so that one server can obtain the terminal event information cached on at least one storage area.
[0088] The embodiment of the present application periodically obtains the terminal event information of at least one terminal having a corresponding relationship with the first server, including:
[0089] Determine the first storage area from the multiple storage areas;
[0090] Periodically obtain the cached terminal event information from the first storage area.
[0091] In the embodiment of the present application, by pre-establishing the corresponding relationship between each server and the storage area, the information of the first storage area having a corresponding relationship with the server is stored in each server. Thus, the first server determines the first storage area according to this information, and then periodically obtains the stored terminal event information from the first storage area.
[0092] Based on the above embodiments, as an optional embodiment, the storage area is a partition in a preset first message queue. The message queue includes at least one partition, and each partition has a corresponding relationship with a unique server in the server cluster. The first message queue is used to use each network device as a producer respectively, use the first server as a consumer, use the terminal event information uploaded by the producer as a message, and cache the message to the partition corresponding to the terminal identifier of the message.
[0093] Consumers read messages in Kafka and can consume data of any Topic. Multiple Consumers form a consumer group. Generally, a consumer must have a group name. If not, it will be assigned a default group name.
[0094] The embodiments of the present application introduce a first message queue between the server and the network device. In some embodiments, the first message queue is a Kafka message queue. A Kafka queue may include at least one producer, at least one topic, and a topic may include multiple partitions. Each partition can only be consumed by one consumer. Specifically,
[0095] A producer, as the name implies, is something that produces, that is, sends messages. For each message sent by the producer, there must be a Topic (topic), which can also be said to be the category of the message. The producer continuously sends messages to the Kafka server.
[0096] Each message sent to Kafka has a topic, which can also be called a category, similar to the table name in our traditional database. For example, when sending a message with the topic of "order", there will be multiple messages about orders under this "order". It's just that Kafka calls it a topic, and it's the same principle.
[0097] The message data Topic sent by the producer will be stored in the partition. The concept of this partition is the same as the concept of sharding in ElasticSearch. They both want to divide the data into multiple chunks to achieve load balancing and reasonably distribute the messages on different partitions, so that a large number of messages achieve load balancing. Each Topic can specify multiple partitions, but at least one partition must be specified. The data stored in each partition is ordered, and the data between different partitions is not guaranteed to be ordered. Because if there are multiple partitions, when consuming data, each partition will start independently. Some may consume slowly and some may consume quickly, so the order cannot be guaranteed. Then when it is necessary to ensure the sequential consumption of messages, it can be set to one partition. As long as there is only one partition, only this one partition can be consumed, and naturally the order can be guaranteed.
[0098] In the embodiments of the present application, each terminal event information is a message in the Kafka message queue. Each server is a message consumer of the Fakfa message queue, and all message consumers are in the same consumer group, thus avoiding duplicate consumption.
[0099] The Kafka message queue includes multiple partitions, and a partition can only be consumed by one consumer in the consumer group. And a consumer can consume one or more partitions. Through this partition consumption method, the throughput of the message consumer can be improved, and at the same time, both the publish / subscribe mode and the point-to-point mode of the message can be realized.
[0100] After generating terminal event information, when the network device in the embodiment of the present application writes the terminal event information into the kafka queue, it uses the terminal identifier as the key. At the same time, the kafka message queue is configured to perform a hash operation according to the key, and determine the partition where the terminal event information is located based on the result of the hash operation, and put the terminal event information into the corresponding partition. In this way, it can be ensured that the terminal event information of the same terminal will definitely be assigned to the same partition and will definitely be consumed by the same server.
[0101] Please refer to Figure 4 , which exemplarily shows a schematic diagram of the embodiment of the present application caching the terminal event information uploaded by the network device through the kafka message queue. As shown in the figure, all network devices only need to send the collected terminal event information to the kafka message queue, without caring about which partition the terminal event information is cached in and which server processes it, reducing the workload of the network device. The kafka message queue performs a hash operation on the terminal identifier of the message to obtain a hash value, uses the hash value as the key of the message, and sends the message to the partition corresponding to the key through the partitioner. There are 8 partitions in the figure, where partitions 1-3 correspond to server 1, and server 1 periodically obtains messages from partitions 1-3; partitions 4-6 correspond to server 2, and server 2 periodically obtains messages from partitions 4-6; partitions 7 and 8 correspond to server 3, and server 3 periodically obtains messages from partitions 7 and 8.
[0102] On the basis of the above embodiments, as an optional embodiment, the terminal identifier is the Media Access Control (MAC) address of the terminal.
[0103] The partition where each terminal event information is located is related to the hash value of the MAC address included in the terminal event information.
[0104] On the one hand, the MAC address can uniquely identify a terminal in the network. On the other hand, when the terminal performs data transmission, it also needs to use the MAC address to represent the sender and receiver of the data, which makes the MAC address of the terminal be recorded in the data access record. In the present application, by using the MAC address of the terminal as the terminal identifier and the terminal event information also includes the device identifier of the corresponding network device, it is convenient for the network administrator to locate the terminal event information of the terminal with the MAC address after determining the abnormal data access record. Further, since the terminal event information is sent by the network device and the location of the network device is usually fixed, after determining the terminal event information of the terminal with the abnormal data access record, the location range where the terminal is located can be further determined according to the location of the network device.
[0105] Please refer toFigure 5 , which exemplarily shows a schematic diagram of an application scenario provided by an embodiment of the present application. As shown in the figure, network devices are deployed on the A floor of a company building. A certain lawbreaker connects to the network device on the A floor through a terminal. The network device forms a terminal event information by combining the connection event of the terminal, the terminal identifier, the device identifier, and the timestamp, and sends it to the first message queue. The first message queue further sends the terminal event information to the corresponding partition, and a certain server in the server cluster obtains the terminal event information from the partition and saves it to the database. The lawbreaker accesses the company's internal network through the network device and reads some important data with a relatively high privilege. Since the MAC address of the terminal itself will be recorded in the access record when the terminal accesses important data in the internal network, the company's information security system automatically triggers an alarm by traversing the real-time generated access records, and sends the MAC address to the network administrator. The network administrator finds the corresponding terminal event information in the database according to the MAC address, and further finds that the device is deployed on the A floor according to the device identifier in the terminal event information. The network administrator can quickly search for the lawbreaker on the A floor, ensuring the information security of the company.
[0106] Based on the above embodiments, as an optional embodiment, the embodiments of the present application can also configure multiple second message queues. A second message queue is a storage area that caches the respective terminal event information of at least one corresponding terminal. Each message queue has a corresponding relationship with a unique server in the server cluster. The second message queue is used to use a preset processing device as a producer, use the server having a corresponding relationship with the second message queue as a consumer, and cache the terminal event information uploaded by the producer as a message.
[0107] That is to say, when the message queue adopted does not have the automatic partitioning ability of the kafka message queue, the number of message queues set is multiple. Each second message queue is used to cache the respective terminal event information of at least one corresponding terminal. Each message queue has a corresponding relationship with a unique server in the server cluster. In this way, it can also ensure that the respective terminal event information of one terminal can be processed by a unique server, avoiding the problem of repeated processing and the problem that a server cannot obtain all the terminal event information of a terminal.
[0108] Please refer to Figure 6, which exemplarily shows a schematic architecture diagram of a wireless management system provided by another embodiment of the present application, including a server cluster, a processing device, and a message queue cluster. In the embodiments of the present application, all network devices uniformly upload the collected terminal event information to the processing device, and the processing device sends the terminal event information to the corresponding second message queue in the message queue cluster according to the terminal identifier in the terminal event information. Each server in the server cluster corresponds to at least one second message queue and periodically obtains the terminal event information from the corresponding second message queue.
[0109] Based on the above embodiments, as an optional embodiment, storing the terminal event information in each sorted information cluster into a database includes:
[0110] S201. For each sorted information cluster, divide the sorted information cluster with a preset time window length to obtain an information subset corresponding to each time window;
[0111] S202. For each sorted information cluster, starting from the information subset corresponding to the first time window, merge adjacent and non-empty information subsets to obtain at least one target set, where the target final set includes at least one non-empty information subset;
[0112] S203. Store each target set corresponding to each information cluster set into the database.
[0113] It should be noted that through the processing of the above various embodiments, each server can ensure that it completely receives all the terminal event information of the corresponding terminal. In theory, the terminal event information will be uploaded from the network device far more than enough. The server also needs to have a certain tolerance for the out-of-order of the terminal event information. Therefore, the embodiments of the present application can use stream computing to process the terminal event information.
[0114] Since each sorted information cluster contains all the terminal event information of a terminal within a period of time on the one hand, and on the other hand, these terminal event information have also been arranged based on the order of timestamps, the embodiments of the present application can divide the sorted information cluster based on a preset time window length to obtain an information subset corresponding to each time window. The embodiments of the present application do not limit the size of the time window length. For example, it can be several seconds, several minutes, several hours, etc.
[0115] It should be noted that when the present application divides the information cluster, there is no overlapping part between adjacent time windows. Please refer to Figure 7, for example, a cluster of information includes 7 terminal event information. From the timestamps of each terminal event information, it can be found that the time span of this cluster of information is from 13:10:10 to 13:12:10 on a certain day. If the time window length is 30 seconds, the time span corresponding to the first time window segmented is from 13:10:10 to 13:10:40, the time span corresponding to the second time window is from 13:10:40 to 13:11:10, the time span corresponding to the third time window is from 13:11:10 to 13:11:40, and the time span corresponding to the fourth time window is from 13:11:40 to 13:12:10. Different time windows correspond to different information subsets. It should be understood that there may or may not be terminal event information within a time window. Continuing with the above Figure 5 as an example, it can be seen that there is no terminal event information within the third time window, that is, the information subset corresponding to the third time window is empty.
[0116] In the embodiments of the present application, starting from the information subset corresponding to the first time window, adjacent and non-empty information subsets are merged to obtain at least one target set. It can be understood that each target set includes at least one target terminal set, which includes at least one non-empty information subset. For Figure 5 example, since the information subsets corresponding to the first time window and the second time window both have terminal event information, the information subsets corresponding to the first time window and the second time window can be merged. However, since the information subset corresponding to the third time window is empty, it cannot be continued to be merged with the information subsets corresponding to the first time window and the second time window. Also, since the information subset corresponding to the fourth time window is not empty, two target sets are finally obtained. Target set 1 includes the information subsets corresponding to the first time window and the second time window, and target set 2 includes the information subset corresponding to the fourth time window.
[0117] Based on the above embodiments, as an alternative embodiment, starting from the information subset corresponding to the first time window, adjacent and non-empty information subsets are merged to obtain at least one target set, including:
[0118] Each information subset is used as a subset to be processed, and in the order of the time windows, the following operations are sequentially performed on each of the subsets to be processed to obtain at least one target set:
[0119] If the subset to be processed is an empty information subset, skip the subset to be processed;
[0120] If the subset to be processed is a non-empty information subset and satisfies the first condition, a new target terminal event set is created, and the subset to be processed is added to the target set;
[0121] If the subset to be processed is a non-empty information subset and does not meet the first condition, add the subset to be processed to the target set where the previous information subset is located;
[0122] Among them, meeting the first condition includes: the subset to be processed is the first information subset, or the previous information subset of the subset to be processed is an empty information subset.
[0123] In the embodiments of the present application when merging information subsets, each information subset is used as the subset to be processed, and each subset to be processed is traversed according to the chronological order of time windows. For the currently traversed subset to be processed, if the subset to be processed is empty, then skip the subset to be processed. If the subset to be processed is not empty, and the subset to be processed is the first information subset or its previous information subset is an empty information subset, a new target terminal event set is created, and the subset to be processed is added to the target set. If the subset to be processed is a non-empty information subset, and it is neither the first information subset nor its previous information subset is an empty information subset, it is added to the target set where the previous information subset is located.
[0124] Please refer to Figure 8 , which exemplarily shows a schematic flow diagram for merging each information subset provided by the embodiments of the present application. As shown in the figure, it includes:
[0125] S301. Select the information subset processed in this round as the subset to be processed according to the chronological order of time windows;
[0126] S302. Determine whether the subset to be processed is an empty information subset. If so, return to execute S301. If not, execute S303;
[0127] S303. Determine whether the subset to be processed is the first information subset. If so, execute S304. If not, execute S305;
[0128] S304. Create a new target terminal event set, add the subset to be processed to the target set, and return to execute S301;
[0129] S305. Determine whether the previous information subset of the subset to be processed is an empty information subset. If so, execute S304. If not, execute S306;
[0130] S306. Add the subset to be processed to the target set where the previous information subset is located, and return to S301.
[0131] In the embodiments of the present application, by merging the information subsets of each time window, it is possible to prevent terminal event information from exactly appearing at the critical position of the time window. Therefore, the information subsets of each time window are merged until there is no terminal event information in the next time window. It should be noted that there are two cases where there is no terminal event information in the next time window. One is that the terminal has indeed reached a steady state (steadily associated with a network device or left the wireless area), and the other possibility is that the reporting delay of the terminal event information of the terminal is too large, exceeding the size of a time window. Even for streaming computing, data delay cannot be tolerated infinitely. Therefore, the size of the time window in the embodiments of the present application is the maximum tolerance time of the service, and it is configured according to the specific business scenario when the solution is implemented.
[0132] On the basis of the above embodiments, as an alternative embodiment, the type of any one of the communication events is a connection event or a disconnection event. The connection event refers to an event in which a terminal is connected to a network device, and the disconnection event refers to an event in which a terminal is disconnected from a network device;
[0133] Storing the terminal event information in each sorted information cluster into a database further includes:
[0134] For the terminal event information of the same type in each information subset, only the terminal event information with the latest timestamp among the terminal event information of the same type is retained.
[0135] For example, the terminal event information in an information subset includes a connection event 1, a disconnection event 1, and a connection event 2 in sequence according to the chronological order of the timestamps. Since there are two connection events, the terminal event information corresponding to the connection event 1 with an earlier timestamp is deleted, and the terminal event information corresponding to the connection event 2 with a later timestamp is retained. In the embodiments of the present application, for the communication events of the same type in each information subset, only the terminal event information with the latest timestamp among the terminal event information of the same type is retained. On the one hand, it can reduce the storage amount stored in the database finally. On the other hand, with the time window length as the granularity, the terminal event information in each information subset is deleted, rather than retaining the terminal event information with the latest timestamp in the information cluster, in order to ensure that the terminal event information generated by the terminal in the non-steady state stage can be eliminated, laying a foundation for querying the terminal event information generated by the terminal in the steady state stage subsequently.
[0136] Please refer to Figure 9 , which exemplarily shows a schematic flowchart of a data storage method provided by another embodiment of the present application, including:
[0137] S401. Determine the corresponding first partition from the first message queue;
[0138] S402. Periodically obtain the cached terminal event information from the first partition;
[0139] S403. Classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier.
[0140] S404. Sort the terminal event information in each information cluster according to the time stamp in the terminal event information to obtain a sorted information cluster corresponding to each information cluster.
[0141] S405. For each sorted information cluster, segment the sorted information cluster with a preset time window length to obtain an information subset corresponding to each time window.
[0142] S406. For the terminal event information of the same type in each information subset, only retain the terminal event information with the latest time stamp among the terminal event information of the same type.
[0143] S407. For each sorted information cluster, starting from the information subset corresponding to the first time window, merge adjacent and non-empty information subsets to obtain at least one target set. The target final set includes at least one non-empty information subset.
[0144] S408. Store each target set corresponding to each information cluster set into the database.
[0145] The embodiment of the present application provides a data storage device, as Figure 10 shown, applied to a server cluster. The server cluster includes at least one server. The device is executed by a first server, and the first server is any one of the servers in the server cluster. The device may include: an information set acquisition module 101, a classification module 102, a sorting module 103, and a storage module 104, where
[0146] The information set acquisition module 101 is configured to periodically acquire an information set having a corresponding relationship with the first server. The information set includes at least one terminal event information. Each terminal event information includes a communication event between a terminal and a network device, a time stamp for characterizing the occurrence time of the communication event, and a terminal identifier of the corresponding terminal.
[0147] The classification module 102 is configured to classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier.
[0148] The sorting module 103 is configured to sort the terminal event information in each information cluster according to the time stamp in the terminal event information to obtain a sorted information cluster corresponding to each information cluster.
[0149] A storage module 104 for storing the terminal event information in each sorted information cluster into a database.
[0150] Based on the above embodiments, as an optional embodiment, the information set corresponding to the first server is pre-cached in a first storage area, where the first storage area is a storage area corresponding to the first server among multiple pre-created storage areas, and the terminal event information cached in each storage area includes the terminal identifier corresponding to the storage area.
[0151] An information set acquisition module, including:
[0152] A region determination unit for determining the first storage area from the multiple storage areas.
[0153] A periodic acquisition unit for periodically acquiring the cached terminal event information from the first storage area.
[0154] Based on the above embodiments, as an optional embodiment, the storage area is a partition in a preset first message queue, where the first message queue is used to use each network device as a producer, the first server as a consumer, the terminal event information uploaded by the producer as a message, and cache the message into the partition corresponding to the terminal identifier of the message.
[0155] Based on the above embodiments, as an optional embodiment, the first message queue caches the message into the partition corresponding to the terminal identifier of the message in the following manner:
[0156] For each message, perform a hash operation on the terminal identifier of the message to obtain a hash value, use the hash value as the key of the message, and send the message to the partition corresponding to the key through a partitioner.
[0157] Based on the above embodiments, as an optional embodiment, the storage area is a second message queue among multiple preset second message queues, where the second message queue is used to use a preset processing device as a producer, the server corresponding to the second message queue as a consumer, and cache the terminal event information uploaded by the producer as a message;
[0158] Wherein, the processing device is used to receive the terminal event information uploaded by each network device and send the terminal event information to the corresponding second message queue according to the terminal identifier in the terminal event information.
[0159] Based on the above embodiments, as an optional embodiment, the storage module includes:
[0160] A splitting unit, configured to split each sorted information cluster according to a preset time window length, to obtain an information subset corresponding to each time window;
[0161] A merging unit, configured to, for each sorted information cluster, starting from the information subset corresponding to the first time window, merge adjacent and non-empty information subsets to obtain at least one target set, where the target final set includes at least one non-empty information subset;
[0162] A storage unit, configured to store each target set corresponding to each information cluster set into a database.
[0163] Based on the above embodiments, as an optional embodiment, the merging unit is specifically configured to:
[0164] Take each information subset as a subset to be processed, and sequentially perform the following operations on each of the subsets to be processed in the order of the time windows, to obtain at least one target set:
[0165] If the subset to be processed is an empty information subset, skip the subset to be processed;
[0166] If the subset to be processed is a non-empty information subset and satisfies a first condition, create a new target terminal event set, and add the subset to be processed to the target set;
[0167] If the subset to be processed is a non-empty information subset and does not satisfy the first condition, add the subset to be processed to the target set where the previous information subset is located;
[0168] Wherein, the satisfying the first condition includes: the subset to be processed is the first information subset, or the previous information subset of the subset to be processed is an empty information subset.
[0169] Based on the above embodiments, as an optional embodiment, the type of any one of the communication events is a connection event or a disconnection event, where the connection event refers to an event that a terminal is connected to a network device, and the disconnection event refers to an event that a terminal is disconnected from a network device;
[0170] The storage unit is further configured to: for terminal event information of the same type in each information subset, only retain the terminal event information with the latest timestamp in the terminal event information of the same type.
[0171] The device in the embodiments of the present application can execute the method provided in the embodiments of the present application, and the implementation principle is similar. The actions performed by each module in the device in the embodiments of the present application correspond to the steps in the methods in the embodiments of the present application. For the detailed function descriptions of each module of the device, reference can specifically be made to the descriptions in the corresponding methods shown above, and details are not described herein again.
[0172] In an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory. The processor executes the above computer program to implement the steps of a data storage method. Compared with the related art, it can achieve the following: applied to a server cluster, the server cluster includes at least one server, and the method is executed by a first server, where the first server is any one of the servers in the server cluster. To avoid the situation where communication events are stored in a disorderly manner and the situation where multiple network devices are recorded as being connected to a terminal at the same time, the present application periodically obtains an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes, in addition to the communication event between the terminal and the network device, a timestamp for characterizing the occurrence time of the communication event and the terminal identifier of the corresponding terminal. After receiving each terminal event information, the server does not process it immediately, but first caches it and performs batch storage at regular intervals. Since the event carries an event timestamp, the events can be updated to the database in the correct order during batch processing. In this way, it can be ensured that when the network management checks the communication events of the terminal, there will be no incorrect data (in this scenario, the accuracy of the data is more important than the timeliness of the data. Processing events in batches at regular intervals will cause a delay in data update, but there will be no errors).
[0173] In an alternative embodiment, an electronic device is provided, as Figure 11 shown Figure 11 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004. The transceiver 4004 can be used for data interaction between the electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.
[0174] The processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of this application. The processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0175] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 only a thick line is used to represent it herein, but it does not mean that there is only one bus or one type of bus.
[0176] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.
[0177] The memory 4003 is used to store the computer program for implementing the embodiments of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.
[0178] The embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.
[0179] The embodiments of the present application further provide a computer program product, including a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.
[0180] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification, claims and the above drawings of the present application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than the illustrated or described order.
[0181] It should be understood that although the flowchart of the embodiments of the present application indicates each operation step by an arrow, the execution order of these steps is not limited to the order indicated by the arrow. Unless otherwise clearly stated in this document, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage of these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.
[0182] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.
Claims
1. A data storage method, characterized in that, Applied to a server cluster, the server cluster includes at least one server, the method is executed by a first server, and the first server is any one of the servers in the server cluster. The method includes: Periodically obtain an information set corresponding to the first server. The information set includes at least one terminal event information. Each terminal event information includes a communication event between a terminal and a network device, a timestamp indicating the occurrence time of the communication event, and a terminal identifier of the corresponding terminal; Classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier. The information cluster includes at least one terminal event information with the corresponding terminal identifier; Sort the terminal event information in each information cluster according to the timestamp in the terminal event information to obtain a sorted information cluster corresponding to each information cluster; Store the terminal event information in each sorted information cluster into a database.
2. The method according to claim 1, wherein The information set corresponding to the first server is cached in a first storage area in advance. The first storage area is a storage area corresponding to the first server among multiple pre-created storage areas. The terminal event information cached in each storage area includes a terminal identifier corresponding to the storage area; The periodically obtaining the terminal event information of at least one terminal corresponding to the first server includes: Determine the first storage area from the multiple storage areas; Periodically obtain the cached terminal event information from the first storage area.
3. The method according to claim 2, wherein The storage area is a partition in a preset first message queue. The first message queue is used to use each network device as a producer, the first server as a consumer, the terminal event information uploaded by the producer as a message, and cache the message into the partition corresponding to the terminal identifier of the message.
4. The method according to claim 3, characterized in that, The first message queue caches the message into the partition corresponding to the terminal identifier of the message in the following way: For each message, perform a hash operation on the terminal identifier of the message to obtain a hash value, use the hash value as the key of the message, and send the message to the partition corresponding to the key through a partitioner.
5. The method according to claim 2, wherein The storage area is a second message queue among multiple preset second message queues. The second message queue is used to use a preset processing device as a producer, a server corresponding to the second message queue as a consumer, and cache the terminal event information uploaded by the producer as a message; Wherein, the processing device is used to receive the terminal event information uploaded by each network device and send the terminal event information to the corresponding second message queue according to the terminal identifier in the terminal event information.
6. The method according to any one of claims 1-5, characterized in that, The storing the terminal event information in each sorted information cluster into a database includes: For each sorted information cluster, divide the sorted information cluster with a preset time window length to obtain an information subset corresponding to each time window; For each sorted information cluster, starting from the information subset corresponding to the first time window, merge adjacent and non-empty information subsets to obtain at least one target set, where the target final set includes at least one non-empty information subset; Store each target set corresponding to each information cluster set in a database.
7. The method according to claim 6, wherein The step of starting from the information subset corresponding to the first time window, merging adjacent and non-empty information subsets to obtain at least one target set includes: Take each information subset as a subset to be processed, and in the order of the time windows, sequentially perform the following operations on each subset to be processed to obtain at least one target set: If the subset to be processed is an empty information subset, skip the subset to be processed; If the subset to be processed is a non-empty information subset and meets the first condition, create a new target terminal event set and add the subset to be processed to the target set; If the subset to be processed is a non-empty information subset and does not meet the first condition, add the subset to be processed to the target set where the previous information subset is located; Wherein, meeting the first condition includes: the subset to be processed is the first information subset, or the previous information subset of the subset to be processed is an empty information subset.
8. The method according to claim 6, characterized in that, The type of any communication event is a connection event or a disconnection event. The connection event refers to an event where a terminal connects to a network device, and the disconnection event refers to an event where a terminal disconnects from a network device; The step of storing the terminal event information in each sorted information cluster in a database further includes: For terminal event information of the same type in each information subset, only retain the terminal event information with the latest timestamp among the terminal event information of the same type.
9. A data storage device, characterized in that, Applied to a server cluster, the server cluster includes at least one server, the device is executed by a first server, the first server is any one of the servers in the server cluster, and the device includes: An information set acquisition module, configured to periodically acquire an information set corresponding to the first server, where the information set includes at least one terminal event information, and each terminal event information includes a communication event between a terminal and a network device, a timestamp indicating the occurrence time of the communication event, and a terminal identifier of the corresponding terminal; A classification module, configured to classify the information set according to the terminal identifier to obtain an information cluster corresponding to each terminal identifier, where the information cluster includes at least one terminal event information with the corresponding terminal identifier; A sorting module, configured to sort the terminal event information in each information cluster according to the timestamp in the terminal event information to obtain a sorted information cluster corresponding to each information cluster; A storage module, configured to store the terminal event information in each sorted information cluster in a database.
10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the data storage method according to any one of claims 1-8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data storage method according to any one of claims 1-8.