A hard disk anomaly detection system and method based on the principle of the cerebral neocortex

Abstract

The present invention relates to the field of hard disk anomaly detection technology, specifically a hard disk anomaly detection system and method based on the principle of the cerebral neocortex layer, the system includes a data acquisition layer, a data encoding layer, a neural network layer and a classifier, the data acquisition layer includes multiple data acquisition modules, the data encoding layer includes multiple data encoding modules, the neural network layer includes a network level N, wherein N is greater than or equal to 1, the network level N includes a single or multiple cortical layers, and the cortical layer includes at least a single or multiple cortical column components. Compared with the prior art, the present invention predicts hard disk failures with higher prediction accuracy, has a good inhibitory effect on noise in real-time data, can continuously learn, enhance adaptation to scenarios, requires a small amount of training data, reduces machine performance requirements, and saves hardware costs; and can detect the hard disk and machine status, timely warn, and remind to replace the hard disk in time to reduce economic losses.

Description

Technical Field

[0001] The present invention relates to the technical field of hard disk anomaly detection, and in particular to a hard disk anomaly detection system and method based on the principle of the cerebral neocortex. Background Art

[0002] Hard disk damage is often accompanied by the loss of data in the hard disk, which often causes great economic losses to users. The existing disk anomaly detection technology uses the traditional anomaly detection algorithm. The main idea is to use a collector to collect key indicators of the server disk and the machine, and perform detection based on the key indicators of the disk and the machine. Once a large fluctuation is found, it is positioned as an anomaly. This solution has many false alarms and often generates a large number of junk alarms, which is relatively poor in practicality.

[0003] In addition, traditional anomaly detection algorithms use machine learning (ES, MA, ARIMA, SVM, etc.) algorithms or deep learning (LSTM-AD, LSTM-ED, etc.) algorithms. The machine learning solution has low accuracy and poor robustness, and is only suitable for specific scenarios. The training of deep learning solutions is more complicated, requiring a large amount of data training, and the model is relatively large, requiring the machine to have a large computing power, and the cost of use is high. In addition, since deep learning itself lacks a large amount of training data, it is difficult to train and produce a usable model. Summary of the invention

[0004] The purpose of the present invention is to provide a hard disk anomaly detection method based on the principle of the cerebral neocortex to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A hard disk anomaly detection system based on the principle of the cerebral neocortex layer, the system includes a data acquisition layer, a data encoding layer, a neural network layer and a classifier, the data acquisition layer includes multiple data acquisition modules, the data encoding layer includes multiple data encoding modules, the neural network layer includes a network level N, where N is greater than or equal to 1, the network level N includes a single or multiple cortical layers, and the cortical layer includes at least a single or multiple cortical column components.

[0007] Furthermore, the collected information of the data acquisition module includes real-time index data of the hard disk and related hardware index data, the data encoding module input is connected to the collected information of the data acquisition module, the data encoding module output information includes initial data features for data encoding according to the business characteristics of hard disk anomaly detection, the data encoding module output is connected to the neural network layer, the cortical column component executes data calculation of this level, and performs data feature aggregation and extraction corresponding to this level, and transmits the extracted new data features to the previous network level, which performs data calculation, new data feature extraction and transmission, and recursively until the cortical column component of the top network level sends the extracted final data features to the classifier, the classifier receives the final data features of the top network level and uses them as prediction results to issue a disk damage warning.

[0008] Furthermore, the data of the data encoding layer enters from the Nth network layer, and the number of cortical column components in each layer of the neural network layer decreases step by step from the Nth network layer to the top network layer, and the top network layer has a single cortical column component.

[0009] Furthermore, the cortical column component includes neurons and synaptic components, the neuron component performs data calculation, and the synaptic component performs data exchange transmission between the cortical column components, and the synaptic component includes distal synapses and proximal synapses.

[0010] Furthermore, a hard disk anomaly detection method based on the principle of the cerebral neocortex layer is provided, the method comprising at least the following steps:

[0011] 1. Initialize the anomaly detection system;

[0012] Second, the data collection layer collects real-time index data of hard disks and related hardware index data as system input;

[0013] 3. The data encoding layer encodes the data collected by the data collection layer according to the business characteristics of hard disk anomaly detection to obtain initial data characteristics;

[0014] 4. Initialize each cortical column component of the Nth network layer, input the initial data features obtained in the previous step into the Nth network layer of the neural network layer, where N is greater than or equal to 1, initialize the synaptic components in each cortical column component in the Nth network layer, complete the data transmission in this layer, calculate the initial data features through the neuron components in each cortical column component, and realize the activation and inhibition of neurons, aggregate and extract the data features predicted after processing the cortical column components of this layer and use them as new feature data, recorded as feature data N;

[0015] 5. When N is greater than or equal to 2, each cortical column component of the N-1th network layer is initialized, and the feature data N obtained in the previous step is input to the N-1th network layer of the neural network layer, and the synaptic components in each cortical column component in the N-1th network layer are initialized to complete the data transmission in this layer. The initial data features are calculated through the neuron components in each cortical column component, and the activation and inhibition of neurons are realized. The data features predicted after processing the cortical column components of this layer are aggregated and extracted as new feature data, which is recorded as feature data N-1;

[0016] 6. Recursively execute the previous step to the first network level, that is, the top network level, and aggregate and extract the data features predicted after processing the cortical column components of this layer as the final data features;

[0017] 7. The above final data features are sent to the classifier as the final prediction result for disk damage warning.

[0018] Furthermore, the data encoding layer uses a binary format encoder to encode the data. During encoding: semantically similar data produces encoded output data with overlapping active bits; the same input data produces the same encoded output data; for all input data, the encoded output data has the same dimension; for all input data, the encoded output data has consistent sparsity, and the encoder has at least 20 bits set to 1.

[0019] Furthermore, the activation and inhibition method of the neuron is as follows: first, the initial data of the neuron is a random value, which is compared with the feature calculation of the N-1 layer according to the corresponding bit. When the number of matching features exceeds the specified threshold, the neuron is in an activated state; then, a suitable receptive field and inhibition radius are selected according to the business, and within the inhibition radius, the business and model are tuned to ensure that only a fixed number of cortical columns are in an activated state within the inhibition radius.

[0020] Compared with the prior art, the invention has the following beneficial effects: the hard disk anomaly detection system and detection method based on the principle of the cerebral neocortex layer of the invention simulates the operation mode of the biological neocortex brain area, has higher prediction accuracy, has a good suppression effect on the noise in real-time data, can continuously learn, and continuously enhance the adaptation to the scene, requires a small amount of training data, and the model is mainly binary operation, which reduces the amount of floating-point operations, reduces the machine performance requirements, and saves hardware costs. Through the detection method, the hard disk and machine status are detected, and a disk damage warning is generated in time, which reminds to replace the hard disk in time to reduce the economic losses caused by sudden hard disk damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the hard disk anomaly detection method based on the cerebral neocortex principle of the present invention;

[0022] Figure 2 It is a schematic diagram of the logic architecture of the hard disk anomaly detection system based on the cerebral neocortex principle of the present invention;

[0023] Figure 3 It is a structural illustration of the cortical column assembly in the present invention.

[0024] In the figure: 1. Distal synapse; 2. Neuron; 3. Proximal synapse. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] See also Figure 1-3 , the present invention provides a technical solution:

[0027] See also Figure 2 A hard disk anomaly detection system based on the principle of the cerebral neocortex layer, the system includes a data acquisition layer, a data encoding layer, a neural network layer and a classifier, the data acquisition layer includes multiple data acquisition modules, the data encoding layer includes multiple data encoding modules, the neural network layer includes a network level N, where N is greater than or equal to 1, the network level N includes a single or multiple cortical layers, and the cortical layer at least includes a single or multiple cortical column components.

[0028] Specifically, the collected information of the data acquisition module includes the real-time index data of the hard disk and related hardware index data. The data encoding module input is connected to the collected information of the data acquisition module. The output information of the data encoding module includes the initial data features for data encoding according to the business characteristics of the hard disk anomaly detection. The output of the data encoding module is connected to the neural network layer. The cortical column component performs data calculation at this level, aggregates and extracts the data features corresponding to this level, and transmits the extracted new data features to the previous network level. The previous network level performs data calculation, new data feature extraction and transmission, and recursively until the cortical column component of the top network level sends the extracted final data features to the classifier. The classifier receives the final data features of the top network level and uses them as prediction results to issue a disk damage warning.

[0029] It should be noted that, in this embodiment, since the encoder and its encoding process must capture the semantic features of data that are important to the application, the encoding module must meet the following principles:

[0030] Let 𝒜 be an arbitrary input space, and let 𝑆(𝑛, 𝑘) be a set of length 𝑛 with 𝑘 ON bits. The encoder 𝑓 is just a function 𝑓 ∶ 𝒜 → 𝑆(𝑛, 𝑘). The distance score d𝒜 of the input space 𝒜 is a function d𝒜 ∶ 𝒜 × 𝒜 → ℝ, satisfying the following three conditions:

[0031] 1. ∀solved 𝑦

[0032] 2. ∀solved 𝑦

[0033] 3. |

[0034] Equation 1 requires that the distance value given by the semantic similarity metric is greater than or equal to 0;

[0035] Equation 2 requires the distance metric to be symmetric;

[0036] Equation 3 requires that the distance between identical values ​​be zero.

[0037] When the input space and distance scores are given, the encoder can be evaluated by comparing the distance scores of input data pairs and the degree of overlap of their encodings. Input data pairs with low distance scores should correspond to two sets with high overlap, and vice versa. In addition, if an input data pair has a higher degree of overlap than another input data pair, then the distance score of the former's precoding should be lower than that of the latter. For two sets of the same length, s and t, let O(s,t) be the number of bits in the overlapping part (that is, the number of ON bits in the intersection s&t of the two). Then for encoder f ∶ 𝒜 → S(n,k) and ∀w, x, y, z ∈ 𝒜,

[0038] 4. 𝑂(𝑓(𝑤), 𝑓(𝑥)) ≥ 𝑂(𝑓(𝑦), 𝑓(𝑧)) ⟺ d𝒜(𝑤, 𝑧) ≤ d𝒜(𝑦,𝑧)

[0039] Equation 4 states that code pairs with more overlapping 1-bits mean that they are more semantically similar, and conversely, code pairs with greater semantic similarity have more overlapping 1-bits.

[0040] Furthermore, the encoder should be deterministic, so that the same input data will produce the same output every time, and the encoder's output must always have the same number of bits as its input. Comparisons and other operations between sets are performed bit by bit, so that the bit representing a particular "meaning" is always in the same position.

[0041] The data of the data encoding layer enters from the Nth network layer, and the number of cortical column components in each layer of the neural network layer decreases step by step from the Nth network layer to the top network layer, and the top network layer has a single cortical column component.

[0042] The cortical column component includes a neuron 2 and a synapse component. The neuron component performs data calculation, and the synapse component performs data exchange transmission between the cortical column components. The synapse component includes a distal synapse 1 and a proximal synapse 3.

[0043] It should be specifically explained that the calculation of a single cortical column component is divided into two parts: one is the calculation of receiving the longitudinal transmission value, which mainly reflects the spatial characteristics, and the other is the calculation of the horizontal data transmission between cortical columns, which mainly reflects the temporal characteristics. The input of the spatial feature calculation is the feature vector of the re-encoded output of the encoder and each sensing area in each layer. The output is the characteristic representation value of the activated neuron 2 in the cortical column. Because the data encoding is encoded according to the bit position, the amount of calculation is small, and only the corresponding operation needs to be performed bit by bit. The temporal feature calculation method is the same as the spatial feature calculation method, except that the input comes from other cortical columns instead of the encoder.

[0044] It should be further explained that there will be inhibition between cortical columns. The receptive field and inhibition radius are set by system parameters. The cortical columns within the inhibition radius will compete. The winner is the activated cortical column, and the synaptic related parameters connecting the neuron are updated according to the winning situation.

[0045] For details, please refer to Figure 1 , a hard disk anomaly detection method based on the principle of the cerebral neocortex, the method at least comprises the following steps:

[0046] 1. Initialize the anomaly detection system;

[0047] Second, the data collection layer collects real-time index data of hard disks and related hardware index data as system input;

[0048] 3. The data encoding layer encodes the data collected by the data collection layer according to the business characteristics of hard disk anomaly detection to obtain initial data characteristics;

[0049] 4. Initialize each cortical column component of the Nth network layer, input the initial data features obtained in the previous step into the Nth network layer of the neural network layer, where N is greater than or equal to 1, initialize the synaptic components in each cortical column component in the Nth network layer, complete the data transmission in this layer, calculate the initial data features through the neuron components in each cortical column component, and realize the activation and inhibition of neurons, aggregate and extract the data features predicted after processing the cortical column components of this layer and use them as new feature data, recorded as feature data N;

[0050] 5. When N is greater than or equal to 2, each cortical column component of the N-1th network layer is initialized, and the feature data N obtained in the previous step is input to the N-1th network layer of the neural network layer, and the synaptic components in each cortical column component in the N-1th network layer are initialized to complete the data transmission in this layer. The initial data features are calculated through the neuron components in each cortical column component, and the activation and inhibition of neurons are realized. The data features predicted after processing the cortical column components of this layer are aggregated and extracted as new feature data, which is recorded as feature data N-1;

[0051] 6. Recursively execute the previous step to the first network level, that is, the top network level, and aggregate and extract the data features predicted after processing the cortical column components of this layer as the final data features;

[0052] 7. The above final data features are sent to the classifier as the final prediction result for disk damage warning.

[0053] The data encoding layer uses a binary format encoder to encode the data. During encoding: semantically similar data produces encoded output data with overlapping active bits; the same input data produces the same encoded output data; for all input data, the encoded output data has the same dimension; for all input data, the encoded output data has consistent sparsity, and the encoder has at least 20 bits set to 1.

[0054] Furthermore, the method of activating and inhibiting neurons is as follows: first, the initial data of the neuron is a random value, which is compared with the feature calculation of the N-1 layer according to the corresponding bit. When the number of matching features exceeds the specified threshold, the neuron is in an activated state; then, the appropriate receptive field and inhibition radius are selected according to the business, and within the inhibition radius, the business and model are tuned to ensure that only a fixed number of cortical columns are in an activated state within the inhibition radius to prevent over-activation of the cortical columns.

[0055] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A hard disk anomaly detection system based on the principle of the cerebral neocortex, characterized in that: The system includes a data acquisition layer, a data encoding layer, a neural network layer and a classifier, wherein the data acquisition layer includes a plurality of data acquisition modules, the data encoding layer includes a plurality of data encoding modules, the neural network layer includes a network level N, wherein N is greater than or equal to 1, the network level N includes a single or multiple cortical layers, and the cortical layer includes at least a single or multiple cortical column components; The collected information of the data collection module includes the real-time index data of the hard disk and the related hardware index data. The data encoding module input is connected to the collected information of the data collection module. The output information of the data encoding module includes the initial data features for data encoding according to the business characteristics of the hard disk anomaly detection. The output of the data encoding module is connected to the neural network layer. The cortical column component executes the data calculation of this level, and performs data feature aggregation and extraction corresponding to this level, and transmits the extracted new data features to the upper network level, which performs data calculation, new data feature extraction and transmission, and recursively until the cortical column component of the top network level sends the extracted final data features to the classifier. The classifier receives the final data features of the top network level and uses them as prediction results to issue a disk damage warning.

2. The hard disk anomaly detection system based on the cerebral neocortex principle according to claim 1 is characterized in that: The data of the data encoding layer enters from the Nth network layer, and the number of cortical column components in each layer of the neural network layer decreases step by step from the Nth network layer to the top network layer, and the top network layer has a single cortical column component.

3. The hard disk anomaly detection system based on the cerebral neocortex principle according to claim 1 is characterized in that: The cortical column component includes a neuron (2) and a synaptic component. The neuron component performs data calculation, and the synaptic component performs data exchange transmission between the cortical column components. The synaptic component includes a distal synapse (1) and a proximal synapse (3).

4. A hard disk anomaly detection method based on the principle of the cerebral neocortex, characterized in that: The method comprises at least the following steps:

1. Initialize the anomaly detection system; Second, the data collection layer collects real-time index data of hard disks and related hardware index data as system input; 3. The data encoding layer encodes the data collected by the data collection layer according to the business characteristics of hard disk anomaly detection to obtain initial data characteristics; 4. Initialize each cortical column component of the Nth network layer, input the initial data features obtained in the previous step into the Nth network layer of the neural network layer, where N is greater than or equal to 1, initialize the synaptic components in each cortical column component in the Nth network layer, complete the data transmission in this layer, calculate the initial data features through the neuron components in each cortical column component, and realize the activation and inhibition of neurons, aggregate and extract the data features predicted after processing the cortical column components of this layer and use them as new feature data, recorded as feature data N; 5. When N is greater than or equal to 2, each cortical column component of the N-1th network layer is initialized, and the feature data N obtained in the previous step is input to the N-1th network layer of the neural network layer, and the synaptic components in each cortical column component in the N-1th network layer are initialized to complete the data transmission in this layer. The initial data features are calculated through the neuron components in each cortical column component, and the activation and inhibition of neurons are realized. The data features predicted after processing the cortical column components of this layer are aggregated and extracted as new feature data, which is recorded as feature data N-1; 6. Recursively execute the previous step to the first network level, that is, the top network level, and aggregate and extract the data features predicted after processing the cortical column components of this layer as the final data features; 7. The above final data features are sent to the classifier as the final prediction result for disk damage warning.

5. A hard disk anomaly detection method based on the cerebral neocortex principle according to claim 4, characterized in that: The data encoding layer uses a binary format encoder to encode the data. During encoding: semantically similar data produces encoded output data with overlapping active bits; the same input data produces the same encoded output data; for all input data, the encoded output data has the same dimension; for all input data, the encoded output data has consistent sparsity, and the encoder has at least 20 bits set to 1.

6. A hard disk anomaly detection method based on the cerebral neocortex principle according to claim 4, characterized in that: The activation and inhibition method of the neuron is as follows: first, the initial data of the neuron is a random value, which is compared with the feature calculation of the N-1 layer according to the corresponding bit. When the number of matching features exceeds the specified threshold, the neuron is in an activated state; then, a suitable receptive field and inhibition radius are selected according to the business, and within the inhibition radius, the business and model are tuned to ensure that only a fixed number of cortical columns are in an activated state within the inhibition radius.

Images