Monitoring method, apparatus, device, and storage medium

Abstract

The present disclosure provides a monitoring method and device, equipment and storage medium, relating to the technical field of computer, and the method comprises the steps of determining first metadata, differentiating the first metadata from second metadata to obtain a differential result, uploading the differential result to a monitoring center, so that the monitoring center trains a teacher model based on the differential result, and distills a small model after the teacher model is trained, and monitors the first cluster based on the small model, thereby achieving monitoring of the devices in the cluster.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a monitoring method and device, equipment and storage medium. BACKGROUND

[0002] With the development of network technology, the number of network devices gradually increases, and more and more network devices are connected to the network to form a large distributed system, which is usually composed of a large number of devices. How to monitor a large number of devices is a problem to be solved in the field. SUMMARY

[0003] The present disclosure provides a monitoring method, device, equipment and storage medium, which at least partly realizes the monitoring of a large number of devices in a distributed system.

[0004] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0005] According to one aspect of the present disclosure, a monitoring method is provided, applied to a first cluster, comprising:

[0006] determining first metadata;

[0007] differentially processing the first metadata and second metadata to obtain a differential result, the second metadata being determined by a second cluster;

[0008] uploading the differential result to a monitoring center, so that the monitoring center trains a teacher model based on the differential result, and distills a small model after the teacher model is trained;

[0009] receiving the small model;

[0010] monitoring the first cluster based on the small model.

[0011] In one embodiment of the present disclosure, determining the first metadata comprises:

[0012] obtaining state data and basic information data of network devices in the first cluster;

[0013] determining the first metadata according to the state data, the basic information data and a semantic embedding model.

[0014] In one embodiment of the present disclosure, determining the first metadata according to the state data, the basic information data and the semantic embedding model comprises:

[0015] inputting the state data into the semantic embedding model to obtain semantic feature data;

[0016] determining the semantic feature data and the basic information data as the metadata.

[0017] In one embodiment of the present disclosure, the first metadata and the second metadata are differentially processed to obtain a differential result, including:

[0018] The first metadata and the second metadata are normalized so that the normalized first metadata and the normalized second metadata are in the same preset range;

[0019] The first metadata and the second metadata in the same preset range are differentially processed to obtain a differential result.

[0020] In one embodiment of the present disclosure, the first cluster is monitored based on a small model, including:

[0021] The metadata is input into the small model to obtain predicted traffic data;

[0022] In the case where the difference between the predicted traffic data and the actual traffic data in the first cluster is greater than a first preset threshold, an alarm is given.

[0023] In one embodiment of the present disclosure, the method further includes:

[0024] Based on the differential result, the first metadata and the second metadata that exist differences are determined;

[0025] The proportion of the first metadata and the second metadata that exist differences in the first metadata and the second metadata is determined;

[0026] In the case where the proportion exceeds a second preset threshold, an alarm is given.

[0027] According to another aspect of the present disclosure, a monitoring method is provided, applied to a monitoring center, including:

[0028] Receiving a differential result, the differential result being obtained by differentially processing the first metadata and the second metadata, the first metadata being determined by a first cluster, and the second metadata being determined by a second cluster;

[0029] Training a teacher model based on the differential result;

[0030] After the teacher model is trained, distilling a small model based on the trained teacher model;

[0031] Sending the small model to the first cluster so that the small model monitors the first cluster.

[0032] According to still another aspect of the present disclosure, a monitoring device is provided, applied to a first cluster, including:

[0033] A determination module for determining first metadata;

[0034] The difference module is configured to perform a difference process on the first metadata and second metadata to obtain a difference result, and the second metadata is determined by the second cluster;

[0035] The uploading module is configured to upload the difference result to a monitoring center, so that the monitoring center trains the teacher model based on the difference result, and distills the small model after the training of the teacher model is completed;

[0036] The first receiving module is configured to receive the small model;

[0037] The monitoring module is configured to monitor the first cluster based on the small model.

[0038] In an embodiment of the present disclosure, the determining module comprises:

[0039] The obtaining unit is configured to obtain state data and basic information data of the network device in the first cluster;

[0040] The determining unit is configured to determine the first metadata according to the state data, the basic information data, and the semantic embedding model.

[0041] In an embodiment of the present disclosure, the determining unit comprises:

[0042] The first input subunit is configured to input the state data into the semantic embedding model to obtain semantic feature data;

[0043] The determining subunit is configured to determine the semantic feature data and the basic information data as the metadata.

[0044] In an embodiment of the present disclosure, the difference module comprises:

[0045] The normalization unit is configured to normalize the first metadata and the second metadata, so that the normalized first metadata and the normalized second metadata are located in a same preset range;

[0046] The difference unit is configured to perform a difference process on the first metadata and the second metadata in the same preset range to obtain a difference result.

[0047] In an embodiment of the present disclosure, the monitoring module comprises:

[0048] The second input unit is configured to input the metadata into the small model to obtain predicted traffic data;

[0049] The alarm unit is configured to perform an alarm in a case where a difference between the predicted traffic data and actual traffic data in the first cluster is greater than a first preset threshold.

[0050] In an embodiment of the present disclosure, the apparatus further comprises:

[0051] The second determination module is used to determine the first metadata and the second metadata that have differences based on the difference results;

[0052] The third determination module is used to determine the proportion of the first metadata and the second metadata that have differences in the first metadata and the second metadata.

[0053] A second alarm module is used to issue an alarm when the proportion exceeds a second preset threshold. According to another aspect of this disclosure, a monitoring device is provided for use in a monitoring center, comprising:

[0054] The second receiving module is used to receive the differential result, which is obtained by differential processing of the first metadata and the second metadata. The first metadata is determined by the first cluster, and the second metadata is determined by the second cluster.

[0055] The training module is used to train the teacher model based on the difference results;

[0056] The distillation module is used to distill small models based on the trained teacher model after the teacher model has been trained.

[0057] The sending module is used to send the small model to the first cluster so that the small model can monitor the first cluster.

[0058] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described monitoring method by executing the executable instructions.

[0059] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described monitoring method.

[0060] The monitoring method, apparatus, device, and storage medium provided in the embodiments of this disclosure acquire status data and basic information data of network devices in a first cluster, determine first metadata based on the status data, basic information data, and semantic embedding model, perform differential processing on the first metadata and second metadata to obtain a differential result, upload the differential result to the monitoring center so that the monitoring center can train a teacher model based on the differential result, and after the teacher model is trained, distill a small model, and monitor the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0063] Figure 1 This diagram illustrates a monitoring system structure according to an embodiment of the present disclosure;

[0064] Figure 2 This diagram illustrates a monitoring method flowchart according to an embodiment of the present disclosure;

[0065] Figure 3 This diagram illustrates another monitoring method flowchart in an embodiment of the present disclosure;

[0066] Figure 4 This illustration shows a flowchart of yet another monitoring method according to an embodiment of the present disclosure;

[0067] Figure 5 This illustration shows a flowchart of yet another monitoring method in an embodiment of the present disclosure;

[0068] Figure 6 This illustration shows a flowchart of yet another monitoring method in an embodiment of the present disclosure;

[0069] Figure 7 This diagram illustrates a monitoring device according to an embodiment of the present disclosure;

[0070] Figure 8 This diagram illustrates another monitoring device in an embodiment of the present disclosure;

[0071] Figure 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0072] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0073] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0074] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0075] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0076] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0077] Network monitoring in data centers typically involves periodically collecting traffic statistics from nodes in the network topology to detect issues such as traffic congestion, idle traffic, or even device unresponsiveness while providing services. Most network equipment manufacturers, including switches and network interface cards (NICs), provide feedback logs or information display interfaces for their product status. Data centers can determine device status simply by accessing the relevant device information during monitoring. For network traffic, further analysis of the collected data is needed to simulate the current network environment. Centralized monitoring methods require access to all devices within the managed area, centrally processing and analyzing the collected data. This represents a stable system overhead for a stable topology of device clusters. This overhead will continue to increase as the number of devices in the system grows. Furthermore, as the devices managed by data centers become increasingly heterogeneous, the compatibility of centralized monitoring systems will often face ongoing challenges.

[0078] To address the aforementioned technical problems, this disclosure provides a monitoring method, apparatus, device, and storage medium. To provide a detailed description of this disclosure, the migration method is described first.

[0079] It should be noted that, where there is no conflict, the embodiments of this disclosure and the technical features in the embodiments can be combined with each other.

[0080] The specific implementation methods of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0081] Figure 1 A schematic diagram of a monitoring system structure according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the system 10 may include a cluster 101 and a monitoring center 102. The cluster 101 and the monitoring center 102 may be located on different devices. The cluster 101 may be a module on an electronic device with data acquisition capabilities. The monitoring center 102 may be a module on an electronic device with data processing capabilities, such as a computer or server. The cluster 101 and the monitoring center 102 may also be located on the same device, such as the cluster 101 and the monitoring center 102 being an input module and a processing module on a computer or mobile phone.

[0082] A network communication connection is established between cluster 101 and monitoring center 102. This network can be a wired network or a wireless network.

[0083] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0084] The following explains the situation where cluster 101 and monitoring center 102 are located on two different devices.

[0085] Cluster 101 can be located on a terminal device, which can be a variety of electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.

[0086] Optionally, the client for the application installed on different terminal devices can be the same, or the client for the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client can also differ; for example, the application client can be a mobile client, a PC client, etc.

[0087] The monitoring center 102 can be located on a server. This server can provide various services, such as a backend management server that supports user operations via terminal devices. The backend management server can analyze and process received requests and other data, and then feed the results back to the terminal devices.

[0088] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0089] Those skilled in the art will know that Figure 1 The number of clusters 101 and monitoring centers 102 shown is merely illustrative; any number of clusters 101 and monitoring centers 102 can be used as needed. This disclosure does not limit this.

[0090] Figure 2 This diagram illustrates a monitoring method flowchart according to an embodiment of the present disclosure. It should be noted that the monitoring method of this disclosure is applied to a first cluster, such as... Figure 2 As shown, the method may include:

[0091] S210, Determine the first metadata;

[0092] In some embodiments, determining the first metadata includes:

[0093] Obtain the status data and basic information data of the network devices in the first cluster;

[0094] The first metadata is determined based on state data, basic information data, and semantic embedding model.

[0095] In some embodiments, the first cluster may be any one of a plurality of clusters connected to the monitoring center.

[0096] For example, network devices can be network interface cards (NICs), switches, etc.

[0097] In some embodiments, the status data of a network device can be data generated during the operation of the network device, as well as basic parameter data of the network device.

[0098] In some embodiments, basic information data may include relevant data of network devices in the cluster and cluster operation data.

[0099] For example, basic information data can be network interface card (NIC) traffic data, network traffic control algorithms, and switch queue cache data.

[0100] In some embodiments, determining the first metadata based on state data, basic information data, and a semantic embedding model may include:

[0101] Inputting state data into a semantic embedding model yields semantic feature data;

[0102] Semantic feature data and basic information data are defined as metadata.

[0103] In some embodiments, before inputting state data into the semantic embedding model, the method may further include training the intelligent model based on the historical semantic embedding model and historical metadata, and obtaining the semantic embedding model when the training stopping condition is met.

[0104] S220, perform differential processing on the first metadata and the second metadata to obtain the differential result, where the second metadata is determined by the second cluster.

[0105] In some embodiments, the first metadata and the second metadata are highly compatible.

[0106] In some embodiments, the first metadata and the second metadata can be differentially processed using a sequence analysis model.

[0107] In some embodiments, differential processing of the first metadata and the second metadata may include merging duplicate data in the first metadata and the second metadata, and determining the merged duplicate data and the difference data as the differential result.

[0108] For example, the first metadata is 1, 2, 3; the second metadata is 3, 4, 5.

[0109] The difference results can be: 1, 2, 3, 4, 5.

[0110] For example, the first metadata corresponding to the first cluster is 1a_BU and the second metadata corresponding to the second cluster is 2a_BU. The determined differential result can be Delta_a_BU_Norm.

[0111] S230 uploads the difference results to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled.

[0112] In some embodiments, the method may further include the following before uploading the differential results to the monitoring center:

[0113] The monitoring center generates topology information and network communication plans for the clusters. The network communication plans include information such as network connection methods and communication protocols between the clusters.

[0114] After the network communication scheme is determined at the monitoring center, it can be sent to each cluster.

[0115] In some embodiments, after collecting the differential results, the monitoring center first trains the teacher model. This teacher model is a high-performance, high-precision model, obtained through centralized training using a large amount of data collected by the monitoring center. This teacher model is able to learn complex patterns and rules in metadata very well.

[0116] After the teacher model is trained, the monitoring center will also conduct adaptive distillation training on the smaller models. Adaptive distillation refers to selectively distilling the knowledge from the teacher model into the smaller models of each cluster, based on a template specific to the autonomous cluster. This allows the smaller models to fully inherit the capabilities of the teacher model while adapting to the characteristics and needs of their own cluster. This adaptive distillation training can significantly improve the performance of the smaller models, making them more closely aligned with the requirements of the autonomous cluster.

[0117] In some embodiments, the specific implementation process of adaptive distillation training is as follows: First, specific cluster data is prepared. The monitoring center collects and organizes specific data for each cluster. This data includes historical monitoring data, current operating status data, and relevant information under specific conditions. This data will serve as the basic dataset for training the small model. Next, based on the data characteristics of each cluster, an adaptive template is generated. This template contains the unique features and needs of the cluster, allowing the distillation process to adjust the distillation strategy accordingly to ensure that the small model can adapt to the cluster environment to the greatest extent. During the distillation process, the teacher model transfers its knowledge to the small model. Specifically, the teacher model generates corresponding soft labels and feature representations by predicting and analyzing the data of the specific cluster. This information is then used to guide the training of the small model, enabling it not only to learn the knowledge of the teacher model but also to perform well on the data of the specific cluster. Simultaneously, during the distillation process, the distillation strategy is dynamically adjusted based on the feedback and performance of the cluster. For example, the training of the small model can be made more accurate and efficient by optimizing the loss function, adjusting the distillation temperature, or changing the learning rate. After the small model training is completed, the monitoring center will conduct rigorous verification and evaluation of each small model. Through testing on specific cluster data, the performance and adaptability of the small models were ensured to meet expectations. Finally, the monitoring center distributed these trained autonomous cluster adaptive models to various clusters. After deployment, the small models will monitor and analyze cluster data in real time, providing accurate services. Simultaneously, the monitoring center will regularly collect feedback and new data for continuous model updates and optimization to ensure the models always maintain optimal performance.

[0118] S240, receives small models.

[0119] S250 monitors the first cluster based on a small model.

[0120] The monitoring method provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data, and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result, and after the teacher model is trained, distills the small model, and monitors the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0121] Figure 3 A schematic diagram of another monitoring method flow in an embodiment of this disclosure is shown, such as... Figure 3 As shown, the method may include:

[0122] S310, Obtain the status data and basic information data of the network devices in the first cluster;

[0123] S320, determine the first metadata based on the state data, basic information data, and semantic embedding model;

[0124] S330, normalize the first metadata and the second metadata so that the normalized first metadata and the second metadata are within the same preset range.

[0125] In some embodiments, the preset range can be a range of the same type or a range within the same data range.

[0126] For example, the first metadata corresponding to the first cluster is 1a_BU and the second metadata corresponding to the second cluster is 2a_BU. First, 1a_BU and 2a_BU are normalized and scaled to a standard range such as [0,1] or [-1,1]. At this time, the normalized first metadata is 1a_BU_Norm and the normalized second metadata is 2a_BU_Norm.

[0127] S340, perform differential processing on the first metadata and the second metadata within the same preset range to obtain the differential result.

[0128] For example, the first normalized metadata is 1a_BU_Norm and the second normalized metadata is 2a_BU_Norm. By performing difference, the difference result can be Delta_a_BU_Norm.

[0129] S350 uploads the difference results to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled.

[0130] S360, receives small models;

[0131] S370 monitors the first cluster based on a small model.

[0132] The monitoring method provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data, and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result, and after the teacher model is trained, distills the small model, and monitors the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0133] Figure 4 A schematic diagram of another monitoring method flow in an embodiment of this disclosure is shown, such as... Figure 4 As shown, the method may include:

[0134] S410, Obtain the status data and basic information data of the network devices in the first cluster;

[0135] S420, determine the first metadata based on the state data, basic information data, and semantic embedding model;

[0136] S430, perform differential processing on the first metadata and the second metadata to obtain the differential result, where the second metadata is determined by the second cluster;

[0137] S440 uploads the difference results to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled.

[0138] S450, receives small models;

[0139] S460: Input the metadata into the small model to obtain the predicted traffic data;

[0140] S470: If the difference between the predicted traffic data and the actual traffic data in the first cluster is greater than a first preset threshold, an alarm will be issued.

[0141] In some embodiments, the first cluster may collect actual traffic data within the first cluster and then determine the difference between the actual traffic data and the predicted traffic data in the first cluster.

[0142] In some embodiments, the preset threshold can be determined by the user, which will not be elaborated here.

[0143] In some embodiments, an alarm may include an alarm sent to a monitoring center.

[0144] The monitoring method provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data, and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result, and after the teacher model is trained, distills the small model, and monitors the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0145] Figure 5 A schematic diagram of another monitoring method in an embodiment of this disclosure is shown, such as... Figure 5 As shown, the method may include:

[0146] S510: Obtain the status data and basic information data of the network devices in the first cluster;

[0147] S520, determine the first metadata based on state data, basic information data and semantic embedding model;

[0148] S530, perform differential processing on the first metadata and the second metadata to obtain the differential result, where the second metadata is determined by the second cluster;

[0149] S540, Based on the difference results, determine the first metadata and the second metadata that have differences;

[0150] S550, determine the proportion of the first metadata and the second metadata that have differences in the first metadata and the second metadata respectively;

[0151] S560 will issue an alarm if the ratio exceeds the second preset threshold.

[0152] In some embodiments, the second preset threshold may include a threshold determined by the user, which is not specifically limited in this disclosure.

[0153] The monitoring method provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data, and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result, and after the teacher model is trained, distills the small model, and monitors the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0154] Based on the same inventive concept, this disclosure also provides a monitoring method, as shown in the following embodiments. Since the principle by which this method solves the problem is similar to that of the above-described method embodiments, the implementation of this method embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0155] Figure 6 A flowchart of yet another monitoring method according to an embodiment of this disclosure is shown. It should be noted that the monitoring method in this disclosure is applied to a monitoring center.

[0156] like Figure 6 As shown, the method may include:

[0157] S610, receive differential results. The differential results are obtained by differential processing of the first metadata and the second metadata. The first metadata is determined by the first cluster, and the second metadata is determined by the second cluster.

[0158] S620, trains the teacher model based on the difference results;

[0159] S630, after the teacher model is trained, distills the small model based on the trained teacher model;

[0160] S640 sends the small model to the first cluster so that the small model can monitor the first cluster.

[0161] The monitoring method provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data, and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result, and after the teacher model is trained, distills the small model, and monitors the first cluster based on the small model, thereby realizing the monitoring of devices in the cluster.

[0162] Based on the same inventive concept, this disclosure also provides a monitoring device, as shown in the following embodiment. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiment, the implementation of this device embodiment can refer to the implementation of the above-described method embodiment, and repeated details will not be elaborated further.

[0163] Figure 7 A schematic diagram of a monitoring device structure according to an embodiment of this disclosure is shown. It should be noted that the device is applied to a first cluster, and the device 700 includes:

[0164] Module 701 is used to determine the first metadata;

[0165] The differential module 702 is used to perform differential processing on the first metadata and the second metadata to obtain the differential result. The second metadata is determined by the second cluster.

[0166] Upload module 703 is used to upload the difference results to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled.

[0167] The first receiving module 704 is used to receive the small model;

[0168] Monitoring module 705 is used to monitor the first cluster based on a small model.

[0169] The monitoring device provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, and uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result. After the teacher model is trained, a small model is distilled, and the first cluster is monitored based on the small model, thereby realizing the monitoring of devices in the cluster.

[0170] In one embodiment of this disclosure, the determining module includes:

[0171] The acquisition unit is used to acquire status data and basic information data of network devices in the first cluster.

[0172] The determining unit is used to determine the first metadata based on the state data, basic information data, and semantic embedding model.

[0173] In one embodiment of this disclosure, the determining module includes:

[0174] The first input sub-unit is used to input state data into the semantic embedding model to obtain semantic feature data;

[0175] The sub-unit is determined to identify semantic feature data and basic information data as metadata.

[0176] In one embodiment of this disclosure, the differential module includes:

[0177] The normalization unit is used to normalize the first metadata and the second metadata so that the normalized first metadata and the second metadata are within the same preset range;

[0178] The differential unit is used to perform differential processing on the first metadata and the second metadata within the same preset range to obtain the differential result.

[0179] In one embodiment of this disclosure, the monitoring module includes:

[0180] The second input unit is used to input metadata into the small model to obtain predicted traffic data;

[0181] The alarm unit is used to issue an alarm when the difference between the predicted traffic data and the actual traffic data in the first cluster is greater than a first preset threshold.

[0182] In one embodiment of this disclosure, the apparatus further includes:

[0183] The second determination module is used to determine the first metadata and the second metadata that have differences based on the difference results;

[0184] The third determination module is used to determine the proportion of the first metadata and the second metadata that have differences in the first metadata and the second metadata.

[0185] The second alarm module is used to issue an alarm when the proportion exceeds the second preset threshold.

[0186] The monitoring device provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, and uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result. After the teacher model is trained, a small model is distilled, and the first cluster is monitored based on the small model, thereby realizing the monitoring of devices in the cluster.

[0187] Based on the same inventive concept, this disclosure also provides a monitoring device, as shown in the following embodiment. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiment, the implementation of this device embodiment can refer to the implementation of the above-described method embodiment, and repeated details will not be elaborated further.

[0188] Figure 8 A schematic diagram of a monitoring device structure according to an embodiment of this disclosure is shown. It should be noted that this device is applied to a monitoring center, and the device 800 includes:

[0189] The second receiving module 801 is used to receive the differential result, which is obtained by differential processing of the first metadata and the second metadata. The first metadata is determined by the first cluster, and the second metadata is determined by the second cluster.

[0190] Training module 802 is used to train the teacher model based on the difference results;

[0191] Distillation module 803 is used to distill small models based on the trained teacher model after the teacher model has been trained.

[0192] The sending module 804 is used to send the small model to the first cluster so that the small model can monitor the first cluster.

[0193] The monitoring device provided in the embodiments of this disclosure acquires the status data and basic information data of network devices in a first cluster, determines the first metadata based on the status data, basic information data and semantic embedding model, performs differential processing on the first metadata and the second metadata to obtain the differential result, and uploads the differential result to the monitoring center so that the monitoring center can train the teacher model based on the differential result. After the teacher model is trained, a small model is distilled, and the first cluster is monitored based on the small model, thereby realizing the monitoring of devices in the cluster.

[0194] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0195] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0196] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0197] The storage unit stores program code, which can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 910 can perform the following steps of the above method embodiments:

[0198] Obtain the status data and basic information data of the network devices in the first cluster;

[0199] The first metadata is determined based on the state data, basic information data, and semantic embedding model;

[0200] The first metadata and the second metadata are differentially processed to obtain the differential result. The second metadata is determined by the second cluster.

[0201] The difference results are uploaded to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled.

[0202] Receive small models;

[0203] The first cluster is monitored based on a small model.

[0204] Alternatively, receive differential results, which are obtained by differential processing of first metadata and second metadata. The first metadata is determined by the first cluster, and the second metadata is determined by the second cluster.

[0205] The teacher model is trained based on the difference results;

[0206] After the teacher model is trained, distill the small model based on the trained teacher model.

[0207] Send the small model to the first cluster so that the small model can monitor the first cluster.

[0208] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0209] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0210] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0211] Electronic device 900 can also perform image processing with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), with one or more devices that enable user interaction with electronic device 900, and / or with any device that enables electronic device 900 to perform image processing with one or more other computing devices (e.g., router, modem, etc.). This image processing can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also perform image processing with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 performs image processing with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0212] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0213] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0214] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0215] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0216] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0217] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0218] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0219] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0220] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0221] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A monitoring method, characterized in that, Applied to the first cluster, including: Determine the first metadata; The first metadata and the second metadata are differentially processed to obtain the differential result, wherein the second metadata is determined by the second cluster; The first metadata and the second metadata are differentially processed to obtain the differential result, including: The duplicate data in the first metadata and the second metadata are merged, and the merged duplicate data and the difference data are determined as the difference result. The difference results are uploaded to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled. Receive the small model; The first cluster is monitored based on the small model, and the first cluster includes any one of multiple clusters connected to the monitoring center; the determination of the first metadata includes: Obtain the status data and basic information data of the network devices in the first cluster; The state data is input into the semantic embedding model to obtain semantic feature data; The semantic feature data and basic information data are identified as metadata.

2. The monitoring method according to claim 1, characterized in that, The step of performing differential processing on the first metadata and the second metadata to obtain the differential result includes: The first metadata and the second metadata are normalized so that the normalized first metadata and the second metadata are within the same preset range; The first metadata and the second metadata within the same preset range are differentially processed to obtain the differential result.

3. The monitoring method according to claim 1, characterized in that, The monitoring of the first cluster based on the small model includes: The metadata is input into the small model to obtain the predicted traffic data; An alarm is triggered if the difference between the predicted traffic data and the actual traffic data in the first cluster exceeds a first preset threshold.

4. The monitoring method according to claim 1, characterized in that, The method further includes: Based on the difference results, the first metadata and the second metadata that have differences are determined; Determine the proportion of the first metadata and the second metadata that differ in the first metadata and the second metadata respectively; An alarm is triggered if the ratio exceeds a second preset threshold.

5. A monitoring method, characterized in that, Used in monitoring centers, including: Receive differential results, which are obtained by differential processing of first metadata and second metadata, wherein the first metadata is determined by the first cluster and the second metadata is determined by the second cluster; The first metadata and the second metadata undergo differential processing, including: Merge duplicate data in the first metadata and the second metadata; The teacher model is trained based on the difference results; After the teacher model is trained, distill the small model based on the trained teacher model. The small model is sent to the first cluster so that the small model can monitor the first cluster, the first cluster including any one of a plurality of clusters connected to the monitoring center; The first metadata includes: Obtain the status data and basic information data of the network devices in the first cluster; The state data is input into the semantic embedding model to obtain semantic feature data; The semantic feature data and basic information data are identified as metadata.

6. A monitoring device, characterized in that, Applied to the first cluster, including: The determination module is used to determine the first metadata. The differential module is used to perform differential processing on the first metadata and the second metadata to obtain a differential result, wherein the second metadata is determined by the second cluster; The first metadata and the second metadata are differentially processed to obtain the differential result, including: The duplicate data in the first metadata and the second metadata are merged, and the merged duplicate data and the difference data are determined as the difference result. The upload module is used to upload the difference results to the monitoring center so that the monitoring center can train the teacher model based on the difference results. After the teacher model is trained, a small model is distilled out. The first receiving module is used to receive the small model; The monitoring module is used to monitor the first cluster based on the small model. The first cluster includes any one of multiple clusters connected to the monitoring center. The determination module is also used to obtain status data and basic information data of network devices in the first cluster; The state data is input into the semantic embedding model to obtain semantic feature data; The semantic feature data and basic information data are identified as metadata.

7. A monitoring device, characterized in that, Used in monitoring centers, including: The second receiving module is used to receive differential results, which are obtained by differential processing of first metadata and second metadata. The first metadata is determined by the first cluster, and the second metadata is determined by the second cluster. The first metadata and the second metadata undergo differential processing, including: Merge duplicate data in the first metadata and the second metadata; The training module is used to train the teacher model based on the difference results; The distillation module is used to distill small models based on the trained teacher model after the teacher model has been trained. The sending module is used to send the small model to the first cluster so that the small model can monitor the first cluster. The first cluster includes any one of a plurality of clusters connected to the monitoring center. The first metadata includes: Obtain the status data and basic information data of the network devices in the first cluster; The state data is input into the semantic embedding model to obtain semantic feature data; The semantic feature data and basic information data are identified as metadata.

8. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the monitoring method according to any one of claims 1 to 5 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the monitoring method according to any one of claims 1 to 5.

Images