Large-model-based abnormal behavior detection method and system for electric power Internet of Things terminal

Abstract

The invention provides a method and a system for detecting abnormal behaviors of a power Internet of Things terminal based on a large model, which are applied to the technical field of network security. The method comprises the following steps: acquiring service flow data of each Internet of Things terminal of the power Internet of Things in different time periods, and extracting behavior fingerprint features based on data packet analysis of the service flow data; generating an original behavior thermodynamic diagram based on the behavior fingerprint features; carrying out multiple times of dynamic convolution on the original behavior thermodynamic diagram based on adaptive cavity convolution to obtain behavior thermodynamic diagrams with different fine granularities; taking the original behavior thermodynamic diagrams and the behavior thermodynamic diagrams with different fine granularities as a thermodynamic diagram set; for each behavior thermodynamic diagram in the thermodynamic diagram set, determining representative factors based on similarity analysis between the behavior thermodynamic diagram and other behavior thermodynamic diagrams; and sending the thermodynamic diagram set and the representative factor to a cloud server for abnormal behavior detection. According to the invention, the problem of low anomaly detection accuracy and efficiency of the electric power Internet of Things is solved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and particularly to a method and system for detecting abnormal behaviors of power IoT terminals based on large models. Background Art

[0002] In the Power IoT scenario, with the intelligent and networked development of the power system, a large number of business terminal devices are connected to the power grid, and with the construction of the Power IoT, the number of terminals and the access ratio will further increase. Although the application of IoT terminals has brought great convenience to the intelligent advancement of the power system, the increasing number of terminals has also brought huge challenges to the security of the power system.

[0003] When detecting abnormal behaviors of business terminal devices, most of the existing methods judge abnormalities by detecting specified parameters of the business terminal devices. However, due to the complex and changeable actual situation, the detection accuracy and efficiency of this detection method are both low, and the network security of the power system cannot be guaranteed. Summary of the Invention

[0004] In order to overcome the problem of low accuracy and efficiency of abnormal detection in the above-mentioned power IoT, the present invention provides a method and system for detecting abnormal behaviors of power IoT terminals based on large models.

[0005] On the one hand, the present invention provides a method for detecting abnormal behaviors of power IoT terminals based on large models, including:

[0006] Obtain the service traffic data of each IoT terminal in the power IoT at different time periods, and based on the packet parsing of the service traffic data of each time period, extract the behavior fingerprint features of each time period;

[0007] Generate a corresponding original behavior heat map based on the behavior fingerprint features of each time period; perform multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels; use the original behavior heat map and the behavior heat maps with different fine-grained levels as the heat map set corresponding to the time period;

[0008] For each behavior heat map in the heat map set of each time period, determine the representative factor of the behavior heat map based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map;

[0009] Send the heat map sets of each time period and the representative factor of each behavior heat map to the cloud server, so that the cloud server performs abnormal behavior detection through a multimodal large model.

[0010] Optionally, obtaining the service traffic data of each IoT terminal in the power IoT network at different time periods includes:

[0011] Configuring port mirroring on the network traffic aggregation device or network traffic diversion device in the power IoT network, or accessing a traffic monitoring device in the power IoT network to obtain the service traffic data of each IoT terminal at different time periods.

[0012] Optionally, the behavior fingerprint features of each time period include at least one of the following: device identification features, network protocol features, network behavior habit features, client system and environment features, domain name type features, and frequency features.

[0013] Optionally, generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period includes:

[0014] Performing feature mapping on the behavior fingerprint features through a temporal convolutional network to obtain the latent vector representation of the behavior fingerprint features;

[0015] Performing smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix;

[0016] Generating the original behavior heat map corresponding to the time period based on the smoothing matrix;

[0017] Among them, the temporal convolutional network is obtained through self-supervised learning.

[0018] Optionally, after performing smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix, it further includes:

[0019] Performing data distribution estimation on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain the probability density function of the latent vector representation;

[0020] Calculating the dynamic statistical parameters of the smoothing matrix based on the probability density function of the latent vector representation;

[0021] Performing adaptive normalization processing on the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix;

[0022] Generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period includes:

[0023] Generating the original behavior heat map corresponding to the time period based on the behavior matrix.

[0024] Optionally, performing dynamic convolution on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels, including:

[0025] Perform atrous convolution processing on the original behavior heat map respectively based on different dilation rates to obtain a fine-grained behavior heat map corresponding to each dilation rate;

[0026] Among them, the dilation rate is determined based on the local characteristics of the original behavior heat map; the size of the convolution kernel for the atrous convolution processing is determined based on the image size of the original behavior heat map.

[0027] Optionally, the dilation rate is calculated according to the following formula:

[0028]

[0029] Among them, r xy is the dilation rate corresponding to the local area centered at the position (x, y), x and y are the pixel position coordinates of the local area respectively, β is an adjustable parameter, ° is the local area of the original behavior heat map, ° xy is the variance of the local area centered at the position (x, y).

[0030] Optionally, based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map, determine the representative factor of the behavior heat map, including:

[0031] Take the behavior heat map and each behavior heat map in other behavior heat maps except the behavior heat map as a heat map pair;

[0032] Based on the similarity calculation of the two heat maps in each behavior heat map pair, obtain the similarity of each behavior heat map pair;

[0033] For each behavior heat map pair, determine the weight of the heat map pair based on the fine-grainedness of the two heat maps in the heat map pair;

[0034] Based on the weights of each heat map pair, perform weighted fusion processing on the similarities of each heat map pair to obtain the representative factor of the behavior heat map.

[0035] Optionally, the representative factor of the behavior heat map is as follows:

[0036]

[0037] Among them, Q i is the representative factor of the behavior heat map i, v is the number of heat maps in the heat map set for each time period, f i is the behavior heat map i, f θ is the θth heat map among other behavior heat maps except the behavior heat map i, g i is fi The fine-grainedness, g θ is f θ The fine-grainedness; Min represents the minimum value operation, and Max represents the maximum value operation; S MLM (f i , f θ ) is the similarity of the heat map pair (f i , f θ ).

[0038] Optionally, before generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period, it further includes:

[0039] Performing dynamic component analysis on the behavior fingerprint features of each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features;

[0040] Among them, the component analysis bias factor is used to adjust the bias degree of principal component analysis and independent component analysis in the dynamic component analysis.

[0041] On the other hand, the present invention also provides an edge device, including:

[0042] An acquisition and parsing module, configured to acquire the service traffic data of each IoT terminal in the power IoT network at different time periods, and extract the behavior fingerprint features of each time period based on packet parsing of the service traffic data of each time period;

[0043] A heat map generation module, configured to generate a corresponding original behavior heat map based on the behavior fingerprint features of each time period; perform multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grainedness; use the original behavior heat map and the behavior heat maps with different fine-grainedness as the heat map set corresponding to the time period;

[0044] A similarity analysis module, configured to determine the representative factor of each behavior heat map in the heat map set of each time period based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map;

[0045] A sending module, configured to send the heat map sets of each time period and the representative factor of each behavior heat map to the cloud server, so that the cloud server performs abnormal behavior detection through a multi-modal large model.

[0046] Optionally, the acquisition and parsing module is specifically configured to:

[0047] By configuring port mirroring on the network traffic aggregation device or network shunting device of the power Internet of Things, or by accessing a traffic monitoring device in the power Internet of Things, the service traffic data of each IoT terminal in different time periods is obtained.

[0048] Optionally, the behavior fingerprint features of each time period include at least one of the following: device identification features, network protocol features, network behavior habit features, client system and environment features, domain name type features, and frequency features.

[0049] Optionally, the heat map generation module includes an original map generation sub-module, and the original map generation sub-module includes

[0050] a mapping sub-unit, configured to perform feature mapping on the behavior fingerprint features based on a temporal convolutional network to obtain a latent vector representation of the behavior fingerprint features;

[0051] a smoothing sub-unit, configured to perform smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix;

[0052] a map generation sub-unit, configured to generate an original behavior heat map corresponding to a time period based on the smoothing matrix;

[0053] wherein, the temporal convolutional network is obtained through self-supervised learning.

[0054] Optionally, the original map generation sub-module further includes:

[0055] an estimation sub-unit, configured to perform data distribution estimation on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a probability density function of the latent vector representation;

[0056] a dynamic calculation sub-unit, configured to perform statistical parameter calculation on the smoothing matrix based on the probability density function of the latent vector representation to obtain dynamic statistical parameters of the smoothing matrix;

[0057] a normalization sub-unit, configured to perform adaptive normalization processing on the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix;

[0058] The map generation sub-unit is specifically configured to:

[0059] generate an original behavior heat map corresponding to a time period based on the behavior matrix.

[0060] Optionally, the heat map generation module includes:

[0061] a fine-grained map generation sub-module, configured to perform dilated convolution processing on the original behavior heat map based on different dilation rates to obtain a behavior heat map with fine-grainedness corresponding to each dilation rate;

[0062] Among them, the porosity is determined based on the local characteristics of the original behavior heat map; the convolution kernel size of the dilated convolution processing is determined based on the image size of the original behavior heat map.

[0063] Optionally, the porosity is calculated according to the following formula:

[0064]

[0065] Among them, r xy is the porosity corresponding to the local area centered at the position (x, y), x and y are the pixel position coordinates of the local area respectively, β is an adjustable parameter, σ is the local area of the original behavior heat map, and σ xy is the variance of the local area centered at the position (x, y).

[0066] Optionally, the similarity analysis module includes:

[0067] A similarity calculation sub-module, configured to use each behavior heat map in the behavior heat map and other behavior heat maps except the behavior heat map as a heat map pair; based on calculating the similarity of the two heat maps in each behavior heat map pair, obtain the similarity of each behavior heat map pair;

[0068] A weight determination sub-module, configured to, for each behavior heat map pair, determine the weight of the heat map pair based on the fine-grainedness of the two heat maps in the heat map pair;

[0069] A weighted fusion sub-module, configured to perform weighted fusion processing on the similarities of each heat map pair based on the weights of each heat map pair to obtain a representative factor of the behavior heat map.

[0070] Optionally, the representative factor of the behavior heat map is as follows:

[0071]

[0072] Among them, Q i is the representative factor of the behavior heat map i, v is the number of heat maps in the heat map set for each time period, f i is the behavior heat map i, f θ is the θth heat map among other behavior heat maps except the behavior heat map i, g i is for f i 's fine-grainedness, g θ is for f θ 's fine-grainedness; Min represents the operation of taking the minimum value, and Max is the operation of taking the maximum value; S MLM (f i , f θ) is the similarity of the heat map pair (f i , f θ ).

[0073] Optionally, it further includes:

[0074] A component analysis module, configured to perform dynamic component analysis on the behavior fingerprint features of each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features;

[0075] Wherein, the component analysis bias factor is used to adjust the bias degree of principal component analysis and independent component analysis in the dynamic component analysis.

[0076] On the other hand, the present invention also provides a method for detecting abnormal behaviors of power IoT terminals based on a large model, including:

[0077] Receiving a set of heat maps of each time period and a representative factor of each behavior heat map sent by the edge device;

[0078] Based on the representative factor of each behavior heat map, screening a target number of heat maps from the set of heat maps of each time period as candidate heat maps for each time period;

[0079] Using a multi-modal large model to perform visual comparison on the candidate heat maps of each time period, marking abnormal regions, and further determining abnormal behaviors corresponding to the abnormal regions;

[0080] Wherein, the set of heat maps of each time period is obtained based on the service traffic data of each IoT terminal in the power IoT at different time periods, and the representative factor of each behavior heat map is used to indicate the similarity between each behavior heat map and other behavior heat maps in the corresponding set of heat maps except the behavior heat map.

[0081] Optionally, after using the multi-modal large model to perform visual comparison on the candidate heat maps of each time period and determining the abnormal regions, it further includes:

[0082] Based on the ratio of the number of times each position in the candidate heat map is marked as an abnormal region to the target number, determining the abnormal risk level of each position.

[0083] On the other hand, the present invention also provides a cloud server, including:

[0084] A receiving module, configured to receive a set of heat maps of each time period and a representative factor of each behavior heat map sent by the edge device;

[0085] A screening module, configured to screen a target number of heat maps from the set of heat maps of each time period as candidate heat maps for each time period based on the representative factor of each behavior heat map;

[0086] Anomaly detection module, which is used to visually compare the candidate heatmaps of each time period by using a multi-modal large model, mark the abnormal areas, and then determine the abnormal behaviors corresponding to the abnormal areas;

[0087] Among them, the set of heatmaps for each time period is obtained based on the service traffic data of each IoT terminal in the power IoT at different time periods, and the representative factor of each behavior heatmap is used to indicate the similarity between each behavior heatmap and other behavior heatmaps in the corresponding heatmap set except the behavior heatmap.

[0088] Optionally, it further includes:

[0089] Risk level division module, which is used to determine the abnormal risk level of each position based on the ratio of the number of times each position in the candidate heatmap is marked as an abnormal area to the target number.

[0090] On the other hand, the present invention also provides a power IoT terminal abnormal behavior detection system based on a large model, including: the edge device described in any one of the above and the cloud server described in any one of the above.

[0091] Compared with the prior art, the beneficial effects of the present invention are:

[0092] The present invention provides a method for detecting abnormal behaviors of power IoT terminals based on a large model. Starting from service traffic data, a set of heatmaps for each time period and the representative factor of each behavior heatmap are obtained, and abnormal behavior detection is carried out based on the set of heatmaps for each time period and the representative factor of each behavior heatmap. By introducing behavior heatmaps, the data modal information is increased, and by extracting and fusing multi-modal information in service traffic data, the detection accuracy of abnormal behaviors is improved, and the service security of the power IoT is enhanced.

[0093] The present invention generates corresponding original behavior heatmaps based on the behavior fingerprint features of each time period; performs multiple dynamic convolutions on the original behavior heatmaps based on adaptive dilated convolution to obtain behavior heatmaps with different fine-grained levels; uses the original behavior heatmaps and behavior heatmaps with different fine-grained levels as the set of heatmaps for the corresponding time period. Through the multiple dynamic convolution process of adaptive dilated convolution, the convolution process based on dynamic receptive fields is realized. Through behavior heatmaps with different fine-grained levels, the sensitivity of the final abnormal behavior detection to data changes is enhanced, thereby improving the detection accuracy, enabling this method to adapt to the real-time abnormal detection process of complex and changeable actual power IoT scenarios, and realizing automatic abnormal detection and improving the detection efficiency. Brief Description of the Drawings

[0094] Figure 1It is one of the flow schematic diagrams of a method for detecting abnormal behaviors of power IoT terminals based on large models according to the present invention;

[0095] Figure 2 It is the second flow schematic diagram of a method for detecting abnormal behaviors of power IoT terminals based on large models according to the present invention;

[0096] Figure 3 It is the schematic diagram of an exemplary system for detecting abnormal behaviors of power IoT terminals based on large models according to the present invention;

[0097] Figure 4 It is the structural schematic diagram of the electronic device according to the present invention. Specific Embodiments

[0098] The following further elaborates on the specific embodiments of the present invention with reference to the accompanying drawings.

[0099] Embodiment 1

[0100] A method for detecting abnormal behaviors of power IoT terminals based on large models provided by the present invention, as shown in Figure 1 shown, includes:

[0101] Step S110, obtaining the service traffic data of each IoT terminal in the power Internet of Things at different time periods, and based on the packet parsing of the service traffic data of each time period, extracting the behavior fingerprint features of each time period;

[0102] Step S120, generating a corresponding original behavior heat map based on the behavior fingerprint features of each time period; performing multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels; using the original behavior heat map and the behavior heat maps with different fine-grained levels as the heat map set corresponding to the time period;

[0103] Step S130, for each behavior heat map in the heat map set of each time period, determining the representative factor of the behavior heat map based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map;

[0104] Step S140, sending the heat map sets of each time period and the representative factor of each behavior heat map to the cloud server, so that the cloud server performs abnormal behavior detection through a multimodal large model.

[0105] In this exemplary embodiment, the execution entity is an edge device. The IoT terminals in the power IoT can be various power intelligent service terminals, such as smart meters, power sensors, power controllers, etc. Different time periods can be a series of consecutive time periods, and each time period can be a preset duration. Continuous service traffic data can be obtained in real time, and the service traffic data of a specified duration can be used as a group of data, and abnormal behavior detection is performed through adjacent different groups of data. Exemplarily, after obtaining the service traffic data, specific traffic can be filtered and obtained according to conditions such as source IP address, destination IP address, port number, protocol type, etc., so as to reduce the amount of captured data and improve the analysis efficiency. Exemplarily, the service traffic data packet can include protocol type features, port number group features, statistical features, etc. The protocol type features include data link layer protocol, network layer protocol, transport layer protocol, and application layer protocol; the port number group features include source port group and destination port group; the statistical features include data packet size, data packet content, and arrival time interval. The data packet parsing can include link layer parsing, network layer parsing, transport layer parsing, and application layer parsing, and can also include some specific application layer parsing. Among them, the link layer parsing includes the parsing of Ethernet frame packets and the parsing of Point-to-Point Protocol (PPP) frame packets; the network layer parsing includes the parsing of Internet Protocol (IP) data packets, the parsing of Internet Control Message Protocol (ICMP) data packets, and the parsing of Address Resolution Protocol (ARP) data packets; the transport layer parsing includes the parsing of Transmission Control Protocol (TCP) data packets and the parsing of User Datagram Protocol (UDP) data packets; the application layer parsing includes the parsing of Hypertext Transfer Protocol (HTTP) data packets, the parsing of Domain Name System (DNS) data packets, and the parsing of Simple Mail Transfer Protocol (SMTP) data packets, File Transfer Protocol (FTP) data packets, Post Office Protocol Version 3 (POP3), and Internet Message Access Protocol (IMAP) data packets, etc. The behavior fingerprint features of the device are obtained through data packet parsing. The behavior fingerprint features are the key data for recording the system operation status, device working conditions, and various event and alarm information in the power IoT. These data not only contain the identity identifiers of the device (such as MAC address, IP address, etc.), but also cover the operation parameters of the device (such as voltage, current, power, etc.) and environmental information (such as temperature, humidity, etc.). The present invention collects the continuous network traffic data of the IoT terminal device within a certain period of time, performs hierarchical unpacking and parsing on it, obtains the behavior fingerprint features that can be recognized, and the data packet after parsing becomes a series of protocol feature data and user data; by converting the parsed data into a data matrix, an original behavior heat map is generated, and then, multiple fine-grained behavior heat maps are generated through dynamic convolution operations; the multiple fine-grained images and representative factors are submitted to the cloud multi-modal large model for abnormal detection.The multimodal large model conducts visual analysis and comparison on the behavior heat map, identifies the changes in the heat map distribution of the device at different time periods, and at the same time, feedbacks the abnormal identification results and safety suggestions. Through the analysis of these fingerprint data, real-time monitoring, fault diagnosis, and predictive maintenance of the power system can be achieved.

[0106] Exemplarily, the obtaining of the service traffic data of each IoT terminal in the power IoT at different time periods in S110 includes: configuring port mirroring on the network traffic aggregation device or network shunt device in the power IoT, or obtaining the service traffic data of each IoT terminal at different time periods by accessing a traffic monitoring device in the power IoT.

[0107] In this exemplary embodiment, port mirroring can be configured on the network aggregation device, switch (such as a core switch), or network tap in the power IoT to mirror network data packets to the monitoring port to indirectly obtain network traffic data. The traffic monitoring device can also be connected to the power IoT as a network device to directly obtain network traffic data through this traffic monitoring device. In this example, for the existing method of obtaining terminal device fingerprint data through active scanning, such as obtaining terminal device information by sending probe requests (such as Ping, port scanning, etc.) to the device. Although this method can obtain more detailed information, it may be detected by the device and is not suitable for scanning sensitive devices. The problem of this invasive information acquisition method not being applicable to the current power IoT scenario. By designing a passive scanning (such as listening, etc.) method for obtaining device information, there is no need to actively send probe requests to the device. This method will not interfere with the normal operation of the device and will not increase the network load, and can obtain terminal device fingerprint data without affecting the normal operation of the device.

[0108] Exemplarily, the behavior fingerprint features for each time period include at least one of the following: device identification features, network protocol features, network behavior habit features, client system and environment features, domain name type features, and frequency features. Specifically, by parsing protocol type features, port number group features, statistical features, etc. of service traffic data packets into the following fields as behavior fingerprint features: obtaining device identification features through the MAC address; obtaining network protocol features through Ethernet packets, where a value of 0x0800 indicates that the upper layer is the IP protocol, a value of 0x0806 indicates the ARP protocol, and a value of 0x86DD indicates the IPv6 protocol; obtaining the TTL (time to live) value (IPv4) or hop limit (IPv6) through the network protocol type; obtaining the upper layer transport layer protocol type, the type feature of whether it is UDP or TCP; obtaining the program or service type feature of the device through the TCP or UDP port number; obtaining network behavior habit features through TCP flag bits, such as SYN, ACK, FIN, etc.; obtaining relevant system and environment features of the client software through the HTTP request header; obtaining query domain name type and frequency features through DNS.

[0109] In some exemplary embodiments, before generating the corresponding original behavior heat map based on the behavior fingerprint features for each time period in S120, it further includes:

[0110] Performing dynamic component analysis on the behavior fingerprint features for each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features;

[0111] Wherein, the component analysis bias factor is used to adjust the bias degree of principal component analysis and independent component analysis in the dynamic component analysis.

[0112] In this exemplary embodiment, redundant features in the behavior fingerprint features can be removed through dynamic component analysis. For example, duplicate feature dimensions and unimportant feature dimensions can be removed. For example, let the behavior fingerprint feature matrix of each device be S = {S1, S2, …, S k}, S k is the kth behavior feature dimension of the behavior fingerprint feature matrix, and each dimension corresponds to the device feature field after parsing the service traffic data packet. After removing redundant feature dimensions through dynamic component analysis, effective behavior fingerprint features X = {x1, x2, …, x M} are obtained, M ≤ k, and x M is the Mth behavior feature dimension of the effective behavior fingerprint features. The specific dynamic component analysis can be expressed as follows:

[0113]

[0114] Where S is the behavior fingerprint feature matrix, V Mis the eigenvector matrix corresponding to the first M main traffic characteristics, w m is the weight vector of the m-th feature dimension. The superscript T represents the matrix transpose operation. W is the set of weight vectors, G is the information gain function, and α is the component analysis bias factor. When α = 1, principal component analysis is fully used; when α = 0, it is fully biased towards self-component analysis. μ is the mean matrix of each feature, which can be calculated by the following formula:

[0115]

[0116] where, x M,j represents the element in the M-th row and j-th column of the behavior matrix X, and n is the number of elements in each row of the behavior matrix X.

[0117] In some exemplary embodiments, generating the corresponding original behavior heatmap based on the behavior fingerprint features in step S120 includes:

[0118] Performing feature mapping on the behavior fingerprint features based on a temporal convolutional network to obtain a latent vector representation of the behavior fingerprint features;

[0119] Performing smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix;

[0120] Generating the original behavior heatmap corresponding to the time period based on the smoothing matrix.

[0121] In this exemplary embodiment, the temporal convolutional network can be trained through self-supervised learning, and then the trained temporal convolutional network is used to perform feature mapping on the behavior fingerprint features to map the business traffic features to the vector space and obtain a denser feature latent vector representation. Self-supervised learning can improve the feature extraction ability and generalization ability of the model, and at the same time, it can reduce the amount of data processed by edge devices. Specifically, the temporal convolutional network encodes the behavior fingerprint features with the following calculation formula:

[0122]

[0123] where, is the output at the l-th layer at time step t, is the weight of the k'-th element of the convolutional kernel of the l-th layer, d is the dilation factor, K' is the size of the convolutional kernel of the temporal convolutional network, σ' is the non-linear activation function, and b (l) is the bias term of the l-th layer. Then, the corresponding matrix of the latent vector representation output by the temporal convolutional network is smoothed through non-linear transformation Gaussian kernel density estimation (KDE), and finally a smoother and continuous-feature smoothing matrix Ρ is generated:

[0124] P = log(KDE(ReLU(Z))+ε);

[0125] Among them, Z is the potential vector representation of the output of the temporal convolutional network, ReLU is the activation function used for nonlinear transformation to enhance the feature expression ability, KDE is the Gaussian kernel density estimate of nonlinear transformation, and ε is a very small real number to avoid zero values when taking logarithms. This example uses the temporal convolutional network to perform temporal encoding for fingerprint features, and then uses Gaussian kernel density estimation to further calculate on the matrix output by the temporal convolutional network. In this method, the role of KDE is to smooth or transform each time step or feature dimension of the TCN output to generate a new matrix, which can be used for subsequent feature extraction or normalization to improve feature expression ability.

[0126] In some example implementations, after smoothing the latent vector representation of the behavior fingerprint feature based on Gaussian kernel density estimation to obtain a smoothing matrix, the method further includes:

[0127] Based on Gaussian kernel density estimation, data distribution estimation is performed on the latent vector representation of the behavior fingerprint feature to obtain a probability density function of the latent vector representation;

[0128] Calculating statistical parameters of the smoothing matrix based on the probability density function represented by the potential vector to obtain dynamic statistical parameters of the smoothing matrix;

[0129] Adaptively normalizing the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix;

[0130] The generating of the corresponding original behavior heat map based on the behavior fingerprint feature of each time period includes:

[0131] An original behavior heat map of a corresponding time period is generated based on the behavior matrix.

[0132] In this example implementation, Gaussian kernel density estimation is used to model the output distribution of the temporal convolutional network to obtain a probability density function For dynamic calculation of adaptive normalization parameters, the probability density function is calculated as follows:

[0133]

[0134] in, is the probability density function represented by the latent vector, z is the independent variable of the probability density function, K(·) is the Gaussian kernel function, z i′ is the i′th multidimensional vector of the latent vector representation Z output by the temporal convolutional network, N is the number of multidimensional vectors represented by the latent vector output by the temporal convolutional network, and h is the bandwidth coefficient.

[0135] The smoothing matrix P based on non - linear transformation Gaussian kernel density estimation and the modeled probability density function Calculate the dynamic statistical parameters of the smoothing matrix. For example, the dynamic mean and variance, and the calculation formulas are as follows:

[0136]

[0137] where, μ dynamic is the mean of the smoothing matrix, is the variance of the smoothing matrix, and dx represents the differential of the independent variable x. The parameters of adaptive normalization are obtained from the dynamically calculated μ dynamic and and the calculation formula is as follows:

[0138]

[0139] where, KDE_AN(Z) is the calculation result of adaptive normalization, γ and β are parameters for controlling scaling and offset respectively, is the behavior matrix. In this example, adaptive normalization is used to better adapt to the output distribution of the temporal convolutional network and Gaussian kernel density estimation, so as to generate a smoother behavior heatmap. After obtaining the behavior matrix, the original behavior heatmap corresponding to the corresponding time period can be generated based on the behavior matrix.

[0140] In some exemplary embodiments, perform dynamic convolution on the original behavior heatmap based on adaptive dilated convolution in step S120 to obtain behavior heatmaps with different fine - grained levels, including:

[0141] Perform dilated convolution processing on the original behavior heatmap based on different dilation rates respectively to obtain the behavior heatmap with the fine - grained level corresponding to each dilation rate;

[0142] where, the dilation rate is determined based on the local characteristics of the original behavior heatmap; the size of the convolution kernel for the dilated convolution processing is determined based on the image size of the original behavior heatmap.

[0143] In this exemplary embodiment, convert the behavior matrix into the original behavior heatmap, and perform multiple convolution operations on the original behavior heatmap using adaptive dilated convolution to obtain multiple fine - grained behavior heatmaps. First, calculate the size of the convolution kernel based on the size of the original behavior heatmap, and the convolution kernel of the adaptive dilated convolution performs adaptive calculation of dilation and contraction based on temporal changes:

[0144]

[0145] where, α is an adjustable parameter used to control the relationship between the size of the convolution kernel and the size of the input matrix (original behavior heatmap); H and W are the height and width of the original behavior heatmap respectively, is the floor function, which sets the minimum convolution kernel size to 2 to ensure the effectiveness of convolution. After determining the convolution kernel size, by introducing dynamic dilated convolutions, the convolution kernel dynamically expands and contracts to obtain a better dynamic receptive field. The calculation method is as follows:

[0146]

[0147] where Y is the output matrix of the dynamic dilated convolution, and (x, y) is the position of the element in the x-th row and y-th column of the matrix. is the behavior matrix the element in the (x + m′·r xy )-th row and (y + n′·r xy )-th column, K″ is the convolution kernel of the dilated convolution, with a size of c×c; r xy is the dilation rate corresponding to the local area centered at the position (x, y), which is used to adjust the sampling interval; x and y are the pixel position coordinates of the local area, which can correspond to the row and column indices of the matrix; m′ and n′ are the offsets of the convolution kernel, ranging from -(c - 1) / 2 to (c - 1) / 2.

[0148] Exemplarily, the dilation rate is determined based on the local characteristics of the original behavior heatmap, that is, the dilation rate is dynamically adjusted according to the local characteristics of the input matrix. For example, the dilation rate is calculated according to the following formula:

[0149]

[0150] where r xy is the dilation rate corresponding to the local area centered at the position (x, y), β is an adjustable parameter used to control the dilation rate; σ is the local area of the original behavior heatmap, and σ xy is the variance of the local area centered at the position (x, y), that is, the numerical density affecting the small-scale receptive field. The local variance of the input matrix, or the numerical change situation in the local range, is calculated by the following formula:

[0151] σ xy = Var(M x-δ:x+δ,y-δ:y+δ );

[0152] where δ is the window size of the local area, M is the window center coordinate of the local area; Var is the variance function.

[0153] In some exemplary embodiments, the determining the representative factor of the behavior heatmap based on the similarity analysis between the behavior heatmap and other behavior heatmaps in the corresponding heatmap set except the behavior heatmap in step S130 includes:

[0154] Take the behavior heat map and each behavior heat map in other behavior heat maps except the behavior heat map as a heat map pair;

[0155] Based on calculating the similarity between the two heat maps in each behavior heat map pair, obtain the similarity of each behavior heat map pair;

[0156] For each behavior heat map pair, based on the fine-grainedness of the two heat maps in the heat map pair, determine the weight of the heat map pair;

[0157] Based on the weights of each heat map pair, perform weighted fusion processing on the similarities of each heat map pair to obtain the representative factor of the behavior heat map.

[0158] In the present exemplary embodiment, for each behavior heat map in the heat map set of a time period, form a heat map pair by combining this heat map with any other heat map in the set, and calculate the similarity and weight between each behavior heat map pair corresponding to the current heat map, perform weighted summation averaging on the current heat map, and the weighted calculation result is used as the representative factor of the current heat map. The representative factor is used to characterize the high or low representativeness of the current heat map participating in anomaly detection. The higher the representative factor of the heat map, the more it can represent the heat map set participating in the anomaly detection calculation between the heat map sets generated by the large model at multiple different time periods. Each behavior heat map pair can calculate the similarity based on matrix values. For example, the Euclidean distance, cosine similarity, etc. of the two matrices corresponding to the heat map pair can be calculated to determine the similarity between the heat map pairs; other similarity calculation methods of graphs and matrices can also be used, and this example does not limit this.

[0159] Exemplarily, the representative factor calculation of each behavior heat map is as follows:

[0160]

[0161] Among them, Q i is the representative factor of behavior heat map i, v is the number of heat maps in the heat map set of each time period, f i is behavior heat map i, f θ is the θ-th heat map among other behavior heat maps except behavior heat map i, and other behavior heat maps are the heat map set F excluding the map f i after that, g i is the fine-grainedness of f i , g θ is the fine-grainedness of f θ ; Min represents the minimum value operation, Max is the maximum value operation; S MLM (f i , f θ ) is the heat map pair (f i , fθ ) similarity. Through f i and f θ pairwise weighted calculation, finally obtain f i The comprehensive similarity to other heatmaps in the set is called the representative factor. The heatmap sets for each time period are all calculated for the above similarity and representative factor. In this example, similarity discrimination is performed through calculation, providing an enhanced solution for the visual judgment of the cloud multi-modal large model. The heatmap set F = {f1, f2, …, f v} and the calculated representative factor set Q = {Q1, Q2, …, Q v} are uploaded to the cloud multi-modal large model platform through the relay calculation unit for discrimination. f v is the v-th heatmap of the heatmap set F, and Q v is the representative factor of f v . The cloud multi-modal large model judges the similarity of the behavior sets in two time periods, and uses the visual ability to obtain the detection result of abnormal behavior through the local color blocks and the numerical changes of the RGB channels between the heatmaps.

[0162] Embodiment 2

[0163] Based on the same inventive concept, the present invention also provides a method for detecting abnormal behavior of a power Internet of Things terminal based on a large model, including:

[0164] Receiving the heatmap sets for each time period and the representative factors of each behavior heatmap sent by the edge device;

[0165] Based on the representative factors of each behavior heatmap, screen out a target number of heatmaps from the heatmap sets for each time period as the candidate heatmaps for each time period;

[0166] Using the multi-modal large model to perform visual comparison on the candidate heatmaps for each time period, mark the abnormal areas, and further determine the abnormal behavior corresponding to the abnormal areas.

[0167] In the implementation manner of this example, the execution subject is the cloud multi-modal large model. The heatmap sets for each time period are obtained based on the service traffic data of each Internet of Things terminal in the power Internet of Things at different time periods. The representative factor of each behavior heatmap is used to indicate the similarity between each behavior heatmap and other behavior heatmaps except the behavior heatmap in the corresponding heatmap set. The heatmap set and the representative factor of each behavior heatmap can be calculated and obtained in the same ways as in Embodiment 1. The cloud multi-modal large model uses the heatmap sets of different time periods for visual judgment and uses the representative factors for auxiliary screening to detect abnormal behavior. Exemplarily, for two time periods T1 and T2, the corresponding heatmap sets and and the representative factor set With Select ξ heatmaps from each behavioral heatmap set as candidate heatmaps according to the representative factor size. The candidate heatmaps can also be sorted, and ξ depends on the computing power level and the required judgment accuracy. Two heatmap subsets are obtained Is the ξ-th heatmap of the candidate heatmap set , Is the ξ-th heatmap of the candidate heatmap set . After that, And Are visually judged pairwise by the large model to locate the abnormal heatmap areas. Finally, the ξ located abnormal areas are integrated to give an abnormal judgment opinion. Exemplarily, the abnormal area location process is as follows: The multimodal large model boxes the places with large color differences and sets the abnormal area standard threshold:

[0168]

[0169] Among them, Is the j'-th heatmap in the heatmap set of the latest time period T new , Indicates The abnormal situation at the (x, y) position of, with a value of 1 indicating abnormal and a value of 0 indicating normal; ΔR, ΔG, ΔB are the pixel value differences of the R, G, B channels in two time periods, and θ R , θ G , θ B Are the abnormal area standard thresholds of the R, G, B channels respectively. Abnormal marking can be performed through the above formula, and the business location corresponding to the feature dimension to which the marked abnormal position belongs can be traced, and finally the abnormal behavior of the business can be judged.

[0170] In some exemplary embodiments, after using the multimodal large model to visually compare the candidate heatmaps of each time period and determine the abnormal areas, it further includes:

[0171] Determine the abnormal risk level of each position based on the ratio of the number of times each position in the candidate heatmap is marked as an abnormal area to the target quantity.

[0172] In the present exemplary embodiment, for the abnormal area, the risk level of the abnormal position can be divided according to the ratio of the number of times it is marked as abnormal to the target quantity (such as ξ). For example, by counting the number of marking times and setting the judgment return criterion, when the number of times the position (x, y) is marked as 1 is greater than or equal to 3 / 4ξ, it indicates that this position is an extremely high-risk abnormal position, that is, abnormal behavior occurs at this position; when the number of times the position (x, y) is marked as 1 is greater than or equal to 1 / 2ξ and less than 3 / 4ξ, it indicates that this position is a medium-risk abnormal position, that is, abnormal behavior may occur at this position; when the number of times the position (x, y) is marked as 1 is greater than or equal to 1 / 4ξ and less than 1 / 2ξ, it indicates that this position is a low-risk abnormal position, that is, there is a safety risk at this position. By dividing the risk level of the abnormal situation of each position, it helps in the fault diagnosis and predictive maintenance of the power system.

[0173] For example, as Figure 2 shown, an exemplary method for detecting abnormal behavior of a power IoT terminal based on a large model according to the present invention includes: obtaining the fingerprint of the terminal service traffic by passive scanning, and collecting the service behavior data of the power grid terminal in a non-intrusive manner in real time; these data include information such as the communication traffic, interaction mode, timestamp, packet size, and packet frequency of the terminal device. Analyze the collected original network data, that is, parse out the formatted original data according to the protocol rules, and then delete the redundant feature behavior fields through component analysis to complete the data acquisition and preprocessing process. Map the preprocessed data through dense coding to a high-dimensional matrix, smooth the matrix through Gaussian kernel density estimation and adaptive normalization, and convert the smoothed result into a structured behavior matrix to complete the calculation of the behavior matrix. Generate the original behavior heat map based on the behavior matrix, and generate behavior heat maps with different fine-grained levels by using dynamic dilated convolution; calculate the similarity of the behavior heat maps with different fine-grained levels to obtain the representative factors to complete the generation of the heat map. Finally, send the behavior heat maps with different fine-grained levels and the representative factors to the cloud multi-modal large model, and the cloud multi-modal large model conducts comprehensive analysis and detection to give the detection results and policy suggestions. Through multiple dilated convolution and pooling operations, the present invention generates behavior heat maps with different levels and different granularities, and these heat maps present the service traffic patterns of the power grid terminal from different scales, capturing the behavior characteristics of the terminal device at different time periods and in different environments. The fine-grained heat map can accurately reflect the minute fluctuations or abnormal behaviors of the network traffic, while the coarse-grained heat map can show the macroscopic trend of the long-term behavior. Extract the traffic behavior set of the terminal service; the multi-modal large model combines the capabilities of image processing, time series analysis, and natural language processing, and can integrate information from different modalities for comprehensive analysis; ultimately, it realizes the accurate detection of abnormal service traffic.

[0174] In view of the security issues brought by active scanning in obtaining fingerprint data of IoT terminals, as well as the problems such as low accuracy of traditional detection techniques and the need for high-quality labeled data to drive neural network detection techniques. The present invention divides the detection of abnormal behavior of IoT terminal services into two parts. The first is to parse and extract the service traffic data of the terminal device, which is usually also called the terminal device fingerprint. The second is to classify the service traffic data through detection techniques. The present invention detects abnormal behavior of IoT terminal services through passive traffic scanning, traffic abnormal behavior analysis, and self-supervised learning and multi-modal large models. In comparison, passive scanning does not require sending detection requests to devices actively, but obtains information by listening to network traffic. This method will not interfere with the normal operation of the device and will not increase the network load. It has gradually become the mainstream method for obtaining fingerprint data of terminal devices. The abnormal detection method based on self-supervised machine learning of the present invention can automatically learn features from a large amount of fingerprint data and identify potential abnormal patterns without being driven by high-quality labeled data.

[0175] Embodiment 3

[0176] Based on the same inventive concept, the present invention also provides an edge device, including:

[0177] An acquisition and parsing module, configured to acquire the service traffic data of each IoT terminal in the power IoT at different time periods, and extract the behavior fingerprint features of each time period based on the packet parsing of the service traffic data of each time period;

[0178] A heat map generation module, configured to generate a corresponding original behavior heat map based on the behavior fingerprint features of each time period; perform multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels; use the original behavior heat map and the behavior heat maps with different fine-grained levels as the heat map set corresponding to the time period;

[0179] A similarity analysis module, configured to, for each behavior heat map in the heat map set of each time period, determine the representative factor of the behavior heat map based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map;

[0180] A sending module, configured to send the heat map sets of each time period and the representative factor of each behavior heat map to the cloud server, so that the cloud server performs abnormal behavior detection through a multi-modal large model.

[0181] In a possible implementation manner, the acquisition and parsing module is specifically configured to:

[0182] By configuring port mirroring on the network traffic aggregation device or network shunting device of the power Internet of Things, or by accessing a traffic monitoring device in the power Internet of Things, the service traffic data of each IoT terminal in different time periods is obtained.

[0183] In a possible implementation manner, the behavior fingerprint features of each time period include at least one of the following: device identification features, network protocol features, network behavior habit features, client system and environment features, domain name type features, and frequency features.

[0184] In a possible implementation manner, the heat map generation module includes an original map generation sub-module, and the original map generation sub-module includes

[0185] A mapping sub-unit, configured to perform feature mapping on the behavior fingerprint features based on a temporal convolutional network to obtain a latent vector representation of the behavior fingerprint features;

[0186] A smoothing sub-unit, configured to perform smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix;

[0187] A map generation sub-unit, configured to generate an original behavior heat map for the corresponding time period based on the smoothing matrix;

[0188] Wherein, the temporal convolutional network is obtained through self-supervised learning.

[0189] In a possible implementation manner, the original map generation sub-module further includes:

[0190] An estimation sub-unit, configured to perform data distribution estimation on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a probability density function of the latent vector representation;

[0191] A dynamic calculation sub-unit, configured to perform statistical parameter calculation on the smoothing matrix based on the probability density function of the latent vector representation to obtain dynamic statistical parameters of the smoothing matrix;

[0192] A normalization sub-unit, configured to perform adaptive normalization processing on the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix;

[0193] The map generation sub-unit is specifically configured to:

[0194] Generate an original behavior heat map for the corresponding time period based on the behavior matrix.

[0195] In a possible implementation manner, the heat map generation module includes:

[0196] A fine-grained map generation sub-module, which is used to perform dilated convolution processing on the original behavior heat map based on different dilation rates to obtain a fine-grained behavior heat map corresponding to each dilation rate;

[0197] Among them, the dilation rate is determined based on the local characteristics of the original behavior heat map; the size of the convolution kernel for the dilated convolution processing is determined based on the image size of the original behavior heat map.

[0198] In a possible implementation manner, the dilation rate is calculated according to the following formula:

[0199]

[0200] Among them, r xy is the dilation rate corresponding to the local area centered at the position (x, y), x and y are the pixel position coordinates of the local area respectively, β is an adjustable parameter, σ is the local area of the original behavior heat map, and σ xy is the variance of the local area centered at the position (x, y).

[0201] In a possible implementation manner, the similarity analysis module includes:

[0202] A similarity calculation sub-module, which is used to take each behavior heat map in the behavior heat map and other behavior heat maps except the behavior heat map as a heat map pair; based on calculating the similarity between the two heat maps in each behavior heat map pair, obtain the similarity of each behavior heat map pair;

[0203] A weight determination sub-module, which is used for each behavior heat map pair to determine the weight of the heat map pair based on the fine-grainedness of the two heat maps in the heat map pair;

[0204] A weighted fusion sub-module, which is used to perform weighted fusion processing on the similarities of each heat map pair based on the weights of each heat map pair to obtain the representative factor of the behavior heat map.

[0205] In a possible implementation manner, the representative factor of the behavior heat map is as follows:

[0206]

[0207] Among them, Q i is the representative factor of the behavior heat map i, v is the number of heat maps in the heat map set for each time period, f i is the behavior heat map i, f θ is the θ-th heat map among other behavior heat maps except the behavior heat map i, g i is for f i 's fine-grainedness, g θ is for fθ The fine granularity; Min represents the operation of taking the minimum value, and Max is the operation of taking the maximum value; S MLM (f i , f θ ) is the similarity of the heat map pair (f i , f θ ).

[0208] In a possible implementation manner, it further includes:

[0209] A component analysis module, configured to perform dynamic component analysis on the behavior fingerprint features of each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features;

[0210] Wherein, the component analysis bias factor is used to adjust the bias degree of principal component analysis and independent component analysis in the dynamic component analysis.

[0211] Embodiment 4

[0212] Based on the same inventive concept, the present invention further provides a cloud server, including:

[0213] A receiving module, configured to receive the heat map sets of each time period and the representative factor of each behavior heat map sent by the edge device;

[0214] A screening module, configured to screen a target number of heat maps from the heat map sets of each time period as the candidate heat maps of each time period based on the representative factor of each behavior heat map;

[0215] An anomaly detection module, configured to perform visual comparison on the candidate heat maps of each time period by using a multi-modal large model, mark the anomaly areas, and further determine the abnormal behaviors corresponding to the anomaly areas;

[0216] Wherein, the heat map sets of each time period are obtained based on the service traffic data of each IoT terminal in the power IoT at different time periods, and the representative factor of each behavior heat map is used to indicate the similarity between each behavior heat map and other behavior heat maps except the behavior heat map in the corresponding heat map set.

[0217] In a possible implementation manner, it further includes:

[0218] A risk level division module, configured to determine the abnormal risk level of each position based on the ratio of the number of times each position in the candidate heat maps is marked as an anomaly area to the target number.

[0219] Embodiment 5

[0220] Based on the same inventive concept, the present invention further provides a large model-based abnormal behavior detection system for power IoT terminals, including: the edge device according to any one of Embodiment 3 and the cloud server according to any one of Embodiment 4.

[0221] In some embodiments, such as Figure 3As shown in the figure, the edge device includes a fingerprint aggregation module, a data preprocessing module, a matrix construction module, a behavior heatmap generation module, a feature library, a relay computing unit, etc.; the cloud server includes a cloud multi-modal large model platform. The fingerprint aggregation module is mainly responsible for collecting and aggregating the terminal fingerprint data obtained from multiple devices (Device 1, Device 2, N1 devices) through passive scanning. Each device transmits the fingerprint traffic data generated during its operation to the data preprocessing module through passive scanning; this module ensures that the fingerprint data of all devices can be uniformly collected and transmitted for subsequent processing. The data preprocessing module parses out the formatted original behavior data template from the fingerprint data according to the protocol rules, and uses the sliding window mechanism to generate a log sequence for detection; then, it performs formatted parsing and cleaning operations on the received original fingerprint data to ensure the accuracy and consistency of subsequent analysis, and uploads the processed data to the matrix construction module. The matrix construction module is the core part of the entire system, responsible for complex calculations and the generation of behavior matrices. Its workflow includes the following steps: First, it encodes and embeds the received data through a temporal convolutional network, maps the original data to a denser high-dimensional latent feature, then learns the probability density function of the processed data through Gaussian kernel density estimation, and transforms the data so that the distribution can better adapt to heatmap generation, and sends the probability density function and the data to the adaptive normalization unit; the adaptive normalization unit dynamically calculates the normalization parameters for the received data, and finally maps all the values to the range [0,1]; then, the final output is used as the behavior matrix and is uploaded to the behavior heatmap generation module. The behavior heatmap generation module generates RGB three-channel behavior heatmap images through the behavior matrix, and generates multiple behavior heatmap images with different fine-grainedness based on adaptive dilated convolution. These images are uploaded to the feature library for storage. The feature library is a database used to store the feature sets extracted from fingerprint data, including not only the basic features of fingerprint data, but also the high-dimensional behavior heatmaps generated by the behavior heatmap generation module. The relay computing unit is used to extract multiple groups of behavior heatmaps from the feature library for similarity calculation. As an intermediate processing unit, it is responsible for packing and uploading the calculation results and the original image group to the cloud multi-modal large model platform for judgment. The cloud multi-modal large model platform provides services based on a single multi-modal large model or a large model API cluster platform, and can make a comprehensive judgment through the decision results of multiple large models, which has the advantage of being flexible. It combines the calculation results received from the relay computing unit with its own visual capabilities to judge the heatmap, and finally gives suggestions on abnormal behavior of the IoT terminal.

[0222] The present invention combines the advantages of multimodal large models and passive traffic scanning. Through the visual capabilities of large models and dilated convolution, it constructs multiple fine-grained behavior heat maps to achieve efficient and unsupervised anomaly detection, avoiding the implementation difficulties and real-time requirements brought by data-driven approaches. And through the temporal encoding embedding mechanism, it ensures the representativeness and uniqueness of fingerprint data in massive terminal data.

[0223] In some embodiments, as Figure 4 shown, the edge device and the cloud server of the present invention may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected by a bus; the memory can be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, and this data can be called and / or modified when the instructions are executed.

[0224] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of a method for detecting abnormal behaviors of power IoT terminals based on large models in the above embodiments.

[0225] In some embodiments, the steps of the method for detecting abnormal behaviors of a power IoT terminal based on a large model of the present invention can be implemented by a readable storage medium. One or more instructions stored in the storage medium are loaded and executed by a processor, and the steps of the method for detecting abnormal behaviors of a power IoT terminal based on a large model in the above embodiments can be realized. Specifically, it is an electronic device-readable storage medium (Memory). The electronic device-readable storage medium is a memory device in the electronic device, used to store programs and data. It can be understood that the storage medium here can include both the built-in storage medium in the electronic device and, of course, the extended storage medium supported by the electronic device. The storage medium provides a storage space, and the operating system of the terminal is stored in this storage space. Moreover, one or more instructions suitable for being loaded and executed by the processor are stored in this storage space. These instructions can be one or more executable programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory.

[0226] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0227] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0228] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device realizes the functions in the flow Figure 1 one flow or multiple flows and / or blocksFigure 1 The functions specified in one or more boxes.

[0229] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one or more processes and / or boxes Figure 1 One process or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0230] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the scope of its protection. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that after reading the present invention, various changes, modifications or equivalent replacements can still be made to the specific implementation manners of the application. However, these changes, modifications or equivalent replacements are all within the scope of protection of the claims pending for approval of the application.

Claims

1. A method for detecting abnormal behaviors of power IoT terminals based on large models, characterized in that, Including: Obtain the service traffic data of each IoT terminal in the power IoT at different time periods, parse the data packets based on the service traffic data of each time period, and extract the behavior fingerprint features of each time period; Generate the corresponding original behavior heat map based on the behavior fingerprint features of each time period; perform multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels; use the original behavior heat map and the behavior heat maps with different fine-grained levels as the heat map set corresponding to the time period; For each behavior heat map in the heat map set of each time period, determine the representative factor of the behavior heat map based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map; Send the heat map sets of each time period and the representative factors of each behavior heat map to the cloud server so that the cloud server can perform abnormal behavior detection through a multimodal large model.

2. The method according to claim 1, characterized in that, The obtaining the service traffic data of each IoT terminal in the power IoT at different time periods includes: Obtain the service traffic data of each IoT terminal at different time periods by configuring port mirroring on the network traffic aggregation device or network traffic diversion device in the power IoT, or by accessing a traffic monitoring device in the power IoT.

3. The method according to claim 1, wherein The behavior fingerprint features of each time period include at least one of the following: device identification feature, network protocol feature, network behavior habit feature, client system and environment feature, domain name type feature, and frequency feature.

4. The method according to claim 1, wherein The generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period includes: Perform feature mapping on the behavior fingerprint features based on a temporal convolutional network to obtain the latent vector representation of the behavior fingerprint features; Perform smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix; Generate the original behavior heat map corresponding to the time period based on the smoothing matrix; Among them, the temporal convolutional network is obtained through self-supervised learning.

5. The method according to claim 4, characterized in that, After performing smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix, it further includes: Perform data distribution estimation on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain the probability density function of the latent vector representation; Perform statistical parameter calculation on the smoothing matrix based on the probability density function of the latent vector representation to obtain the dynamic statistical parameters of the smoothing matrix; Perform adaptive normalization processing on the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix; The generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period includes: Generate the original behavior heat map corresponding to the time period based on the behavior matrix.

6. The method according to claim 1, wherein Performing dynamic convolution on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grained levels includes: Perform dilated convolution processing on the original behavior heat map based on different dilation rates to obtain behavior heat maps with fine-grained levels corresponding to each dilation rate; Wherein, the void ratio is determined based on the local characteristics of the original behavior heat map; the size of the convolution kernel for the dilated convolution processing is determined based on the image size of the original behavior heat map.

7. The method according to claim 6, wherein The void ratio is calculated according to the following formula: where r xy is the porosity corresponding to the local area centered at the position (x, y), x and y are the pixel position coordinates of the local area respectively, β is an adjustable parameter, σ is the local area of the original behavior heat map, and σ xy is the variance of the local area centered at the position (x, y).

8. The method according to any one of claims 1-7, characterized in that, Based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map, determining the representative factor of the behavior heat map, including: Regarding the behavior heat map and each of the other behavior heat maps in the corresponding heat map set except the behavior heat map as a heat map pair; Based on calculating the similarity of the two heat maps in each behavior heat map pair, obtaining the similarity of each behavior heat map pair; For each behavior heat map pair, determining the weight of the heat map pair based on the fine-grainedness of the two heat maps in the heat map pair; Based on the weights of each heat map pair, performing weighted fusion processing on the similarities of each heat map pair to obtain the representative factor of the behavior heat map.

9. The method according to claim 8, wherein The representative factor of the behavior heat map is as follows: Among them, Q i is the representative factor of behavior heatmap i, v is the number of heatmaps in the heatmap set for each time period, f i is behavior heatmap i, f θ is the θ-th heatmap among other behavior heatmaps except behavior heatmap i, g i is the fine-grainedness of f i , g θ is the fine-grainedness of f θ ; Min represents the operation of taking the minimum value, and Max is the operation of taking the maximum value; S MLM (f i , f θ ) is the similarity of the heatmap pair (f i , f θ ).

10. The method according to claim 1, wherein Before generating the corresponding original behavior heat map based on the behavior fingerprint features of each time period, it further includes: Performing dynamic component analysis on the behavior fingerprint features of each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features; Wherein, the component analysis bias factor is used to adjust the bias degree of principal component analysis and independent component analysis in the dynamic component analysis.

11. An edge device, characterized in that, It includes: An acquisition and parsing module, configured to acquire the service traffic data of each IoT terminal in the power IoT at different time periods, and extract the behavior fingerprint features of each time period based on packet parsing of the service traffic data of each time period; A heat map generation module, configured to generate a corresponding original behavior heat map based on the behavior fingerprint features of each time period; perform multiple dynamic convolutions on the original behavior heat map based on adaptive dilated convolution to obtain behavior heat maps with different fine-grainedness; use the original behavior heat map and the behavior heat maps with different fine-grainedness as the heat map set corresponding to the time period; A similarity analysis module, configured to, for each behavior heat map in the heat map set of each time period, determine the representative factor of the behavior heat map based on the similarity analysis between the behavior heat map and other behavior heat maps in the corresponding heat map set except the behavior heat map; A sending module, configured to send the heat map sets of each time period and the representative factor of each behavior heat map to the cloud server, so that the cloud server performs abnormal behavior detection through a multi-modal large model.

12. The edge device according to claim 11, wherein The acquisition and parsing module is specifically configured to: Obtain the service traffic data of each IoT terminal in different time periods by configuring port mirroring on the network traffic aggregation device or network shunting device in the power IoT, or by accessing a traffic monitoring device in the power IoT.

13. The edge device according to claim 11, wherein The behavior fingerprint features of each time period include at least one of the following: device identification features, network protocol features, network behavior habit features, client system and environment features, domain name type features, and frequency features.

14. The edge device according to claim 11, wherein The heat map generation module includes an original map generation sub-module, and the original map generation sub-module includes A mapping sub-unit, configured to perform feature mapping on the behavior fingerprint features based on a temporal convolutional network to obtain a latent vector representation of the behavior fingerprint features; A smoothing sub-unit, configured to perform smoothing processing on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a smoothing matrix; A graph generation sub-unit, configured to generate an original behavior heatmap for a corresponding time period based on the smoothing matrix; Wherein, the temporal convolutional network is obtained through self-supervised learning.

15. The edge device according to claim 14, wherein The original graph generation sub-module further includes: An estimation sub-unit, configured to perform data distribution estimation on the latent vector representation of the behavior fingerprint features based on Gaussian kernel density estimation to obtain a probability density function of the latent vector representation; A dynamic calculation sub-unit, configured to perform statistical parameter calculation on the smoothing matrix based on the probability density function of the latent vector representation to obtain dynamic statistical parameters of the smoothing matrix; A normalization sub-unit, configured to perform adaptive normalization processing on the smoothing matrix based on the dynamic statistical parameters to obtain a behavior matrix; The graph generation sub-unit is specifically configured to: Generate an original behavior heatmap for a corresponding time period based on the behavior matrix.

16. The edge device according to claim 11, wherein The heatmap generation module includes: A fine-grained graph generation sub-module, configured to perform atrous convolution processing on the original behavior heatmap based on different dilation rates to obtain a behavior heatmap with fine-grainedness corresponding to each dilation rate; Wherein, the dilation rate is determined based on the local characteristics of the original behavior heatmap; the size of the convolution kernel for the atrous convolution processing is determined based on the image size of the original behavior heatmap.

17. The edge device according to claim 16, wherein The dilation rate is calculated according to the following formula: where r xy is the porosity corresponding to the local area centered at the position (x, y), x and y are the pixel position coordinates of the local area respectively, β is an adjustable parameter, σ is the local area of the original behavior heat map, and σ xy is the variance of the local area centered at the position (x, y).

18. The edge device according to any one of claims 11-17, characterized in that, The similarity analysis module includes: A similarity calculation sub-module, configured to use each behavior heatmap in the behavior heatmap and other behavior heatmaps except the behavior heatmap as a heatmap pair; based on calculating the similarity between the two heatmaps in each behavior heatmap pair, obtain the similarity of each behavior heatmap pair; A weight determination sub-module, configured to, for each behavior heatmap pair, determine the weight of the heatmap pair based on the fine-grainedness of the two heatmaps in the heatmap pair; A weighted fusion sub-module, configured to perform weighted fusion processing on the similarities of each heatmap pair based on the weights of each heatmap pair to obtain a representative factor of the behavior heatmap.

19. The edge device according to claim 18, wherein The representative factor of the behavior heatmap is as follows: Among them, Q i is the representative factor of the behavior heat map i, v is the number of heat maps in the heat map set for each time period, f i is the behavior heat map i, f θ is the θ-th heat map among other behavior heat maps except the behavior heat map i, g i is for f i 's fine-grained level, g θ is for f θ 's fine-grained level; Min represents the operation of taking the minimum value, Max is the operation of taking the maximum value; S MLM (f i , f θ ) is the similarity of the heat map pair (f i , f θ ).

20. The edge device according to claim 11, wherein It further includes: A component analysis module, configured to perform dynamic component analysis on the behavior fingerprint features for each time period based on a preset component analysis bias factor to remove redundant features and obtain effective behavior fingerprint features; Wherein, the component analysis bias factor is used to adjust the bias degree between principal component analysis and independent component analysis in the dynamic component analysis.

21. A method for detecting abnormal behaviors of power IoT terminals based on large models, characterized in that, It includes: Receiving the heatmap set for each time period sent by the edge device and the representative factor of each behavior heatmap; Based on the representative factor of each behavior heatmap, screening out a target number of heatmaps from the heatmap set for each time period as candidate heatmaps for each time period; Using a multi-modal large model to perform visual comparison on the candidate heatmaps for each time period, marking abnormal regions, and further determining abnormal behaviors corresponding to the abnormal regions; Among them, the set of heatmaps for each time period is obtained based on the service traffic data of each IoT terminal in the power IoT at different time periods, and the representative factor of each behavior heatmap is used to indicate the similarity between each behavior heatmap and other behavior heatmaps in the corresponding heatmap set except the behavior heatmap.

22. The method according to claim 21, wherein, After visually comparing the candidate heatmaps for each time period using a multi-modal large model to determine the abnormal areas, it further includes: Determining the abnormal risk level of each position based on the ratio of the number of times each position in the candidate heatmap is marked as an abnormal area to the target quantity.

23. A cloud server, characterized in that, It includes: A receiving module for receiving the set of heatmaps for each time period and the representative factor of each behavior heatmap sent by the edge device; A screening module for screening out a target quantity of heatmaps from the set of heatmaps for each time period as the candidate heatmaps for each time period based on the representative factor of each behavior heatmap; An abnormal detection module for visually comparing the candidate heatmaps for each time period using a multi-modal large model, marking the abnormal areas, and further determining the abnormal behaviors corresponding to the abnormal areas; Among them, the set of heatmaps for each time period is obtained based on the service traffic data of each IoT terminal in the power IoT at different time periods; the representative factor of each behavior heatmap is used to indicate the similarity between each behavior heatmap and other behavior heatmaps in the corresponding heatmap set except the behavior heatmap.

24. The cloud server according to claim 23, wherein It further includes: A risk level division module for determining the abnormal risk level of each position based on the ratio of the number of times each position in the candidate heatmap is marked as an abnormal area to the target quantity.

25. An abnormal behavior detection system for power IoT terminals based on large models, characterized in that, It includes: The edge device according to any one of claims 11-20 and the cloud server according to claim 23 or 24.

Images