Intelligent fusion gateway and method thereof

Abstract

According to the intelligent fusion gateway and the method thereof, in an edge calculation analysis module, a data processing and analysis algorithm based on artificial intelligence is introduced to carry out time sequence dynamic analysis and collaborative association coding on environmental parameter data; in this way, time sequence correlation characteristics among the temperature, the humidity, the smoke concentration, the water accumulation state and the liquid level depth are captured, time sequence representation of the equipment state is obtained, and therefore the equipment state is effectively recognized and predicted. Therefore, the environment condition of the equipment can be effectively identified by using the environment parameter multi-mode data through the intelligent fusion gateway, and powerful support is provided for equipment fault early warning and response.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to an intelligent fusion gateway and method thereof. Background Art

[0002] The intelligent converged gateway is an indispensable component of the modern IoT architecture. It has efficient data collection and transmission capabilities and can act as a bridge between sensors and cloud or local processing systems. Although the intelligent converged gateway has significant advantages in data collection and processing, the traditional intelligent converged gateway still has some defects in the device status detection process. For example, when performing status detection on the device, the traditional gateway usually focuses on the operating parameters of the device itself, such as vibration, current and voltage. Although this solution can provide a certain degree of device health monitoring, it often ignores the environmental conditions of the device. Environmental changes may be a precursor to device failure. For example, high humidity may cause a short circuit inside the device, while water accumulation may cause failure of electrical equipment. These environmental factors help to better understand the environmental conditions of the device and provide strong support for early warning and response of equipment failures. However, the traditional solution does not pay attention to the impact of the environment in which the device is located on the operating status of the device, which will lead to a lag in the response to device failures.

[0003] Therefore, an optimized intelligent converged gateway is desired. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent fusion gateway and method thereof, which, in the edge computing analysis module, introduces artificial intelligence-based data processing and analysis algorithms to perform time-series dynamic analysis and collaborative correlation coding on these environmental parameter data, so as to capture the time-series correlation characteristics between temperature, humidity, smoke concentration, water accumulation state and liquid level depth, so as to obtain the time-series representation of the equipment state, thereby effectively identifying and predicting the equipment state. In this way, the intelligent fusion gateway can use the multimodal data of environmental parameters to effectively identify the environmental conditions of the equipment, providing strong support for equipment failure warning and response.

[0005] According to one aspect of the present application, there is provided an intelligent converged gateway, comprising:

[0006] A data acquisition module is used to collect the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation status and the time series of liquid level depth;

[0007] A data storage module, used for storing the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth;

[0008] an edge computing analysis module, for extracting temperature time series features, smoke concentration time series features, water accumulation state time series features, and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state, and the time series of the liquid level depth, and generating a device state detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features, and the liquid level depth time series features, wherein the device state detection result is used to indicate whether there is an abnormality in the device state;

[0009] A remote communication module is used to transmit the device status detection result.

[0010] According to another aspect of the present application, a detection method of an intelligent converged gateway is provided, which includes:

[0011] Collect the time series of temperature data, humidity data, smoke concentration, water accumulation status and liquid level depth;

[0012] storing the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth;

[0013] Extracting temperature time series features, smoke concentration time series features, water accumulation state time series features and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth, and generating a device state detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features and the liquid level depth time series features, wherein the device state detection result is used to indicate whether the device state is abnormal;

[0014] The device status detection result is transmitted.

[0015] Compared with the prior art, the present application provides an intelligent fusion gateway and method thereof, which, in the edge computing analysis module, introduces artificial intelligence-based data processing and analysis algorithms to perform time-series dynamic analysis and collaborative correlation coding on these environmental parameter data, so as to capture the time-series correlation characteristics between temperature, humidity, smoke concentration, water accumulation state and liquid level depth, so as to obtain the time-series representation of the equipment state, thereby effectively identifying and predicting the equipment state. In this way, the intelligent fusion gateway can use the multimodal data of environmental parameters to effectively identify the environmental conditions of the equipment, providing strong support for equipment failure warning and response. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 is a block diagram of an intelligent converged gateway according to an embodiment of the present application;

[0018] Figure 2 A schematic diagram of data flow of an intelligent converged gateway according to an embodiment of the present application;

[0019] Figure 3 It is a block diagram of an edge computing analysis module in an intelligent fusion gateway according to an embodiment of the present application;

[0020] Figure 4 The present invention is a flow chart of a detection method of an intelligent converged gateway according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0023] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0024] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0026] As an important bridge connecting various sensors with cloud or local processing systems, the intelligent fusion gateway has efficient data collection and transmission capabilities, which can ensure the accuracy and real-time nature of data. It can not only collect multi-dimensional data such as temperature, humidity, smoke concentration, water accumulation status and liquid level depth in real time, but also analyze and process data through edge computing. This capability enables operation and maintenance personnel to quickly obtain equipment status information, make decisions in a timely manner, and reduce potential risks and losses.

[0027] In particular, it is necessary to consider the mutual influence between different environmental parameters. For example, changes in temperature and humidity can affect the operating efficiency of the equipment, an increase in smoke concentration may indicate a fire risk, and there is an implicit correlation and mutual influence between the state of water accumulation and the depth of the water level. However, traditional data analysis and equipment status detection methods usually rely on simple rules or threshold methods to judge the status of the equipment, and lack the ability to conduct coordinated temporal analysis of complex data. This makes them inadequate in identifying complex equipment abnormal patterns, and often results in frequent false alarms when the equipment is operating normally. This limitation may lead to missed detections or misjudgments, affecting the ability to identify, warn and respond to abnormal equipment conditions.

[0028] In the technical solution of the present application, an intelligent fusion gateway is proposed. Figure 1 is a block diagram of an intelligent converged gateway according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the intelligent converged gateway according to an embodiment of the present application. Figure 1 and Figure 2As shown, the intelligent fusion gateway according to the embodiment of the present application includes: a data acquisition module 310, which is used to collect the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation state and the time series of liquid level depth; a data storage module 320, which is used to store the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth; an edge computing analysis module 330, which is used to extract temperature time series features, smoke concentration time series features, water accumulation state time series features and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth, and generate a device status detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features and the liquid level depth time series features, and the device status detection result is used to indicate whether there is an abnormality in the device status; a remote communication module 340, which is used to transmit the device status detection result.

[0029] In particular, the data acquisition module 310 and the data storage module 320 are used to collect the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation state and the time series of liquid level depth; and store the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation state and the time series of liquid level depth. It should be understood that in the technical solution of the present application, by acquiring and analyzing environmental time series data such as temperature, humidity, smoke concentration, water accumulation state and liquid level depth, operation and maintenance personnel can timely understand the environmental conditions of the power grid system and equipment, and provide strong support for equipment failure warning and response. Therefore, first of all, the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation state and the time series of liquid level depth are collected.

[0030] In particular, the edge computing analysis module 330 is used to extract temperature time series features, smoke concentration time series features, water accumulation state time series features, and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state, and the time series of the liquid level depth, and generate a device state detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features, and the liquid level depth time series features, and the device state detection result is used to indicate whether there is an abnormality in the device state. In particular, in a specific example of the present application, as Figure 3As shown, the edge computing analysis module 330 includes: a parameter time series feature extraction unit 331, which is used to extract time series features of the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state, and the time series of the liquid level depth to obtain a temperature time series correlation feature vector, a humidity time series correlation feature vector, a smoke concentration time series correlation feature vector, a water accumulation state time series correlation feature vector, and a liquid level depth time series correlation feature vector; a device parameter time series correlation feature capture unit 332, which is used to extract the temperature time series correlation feature vector, the humidity time series correlation feature vector, the smoke concentration time series correlation feature vector, the water accumulation state time series correlation feature vector, and the liquid level depth time series correlation feature vector. After the vector, the water accumulation state temporal correlation feature vector and the liquid level depth temporal correlation feature vector are arranged into an equipment state multi-parameter correlation matrix, the equipment state multi-parameter correlation matrix is ​​input into an equipment parameter temporal correlation pattern feature extractor based on a void convolutional neural network model to obtain an equipment parameter temporal correlation feature graph; an equipment parameter temporal correlation feature enhancement unit 333 is used to input the equipment parameter temporal correlation feature graph into a cross-channel feature space adaptive enhanced self-attention processing module to obtain an equipment parameter temporal correlation enhanced feature graph; an equipment state detection unit 334 is used to determine the equipment state detection result based on the equipment parameter temporal correlation enhanced feature graph.

[0031] Specifically, the parameter time series feature extraction unit 331 is used to perform time series feature extraction on the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth, respectively, to obtain a temperature time series associated feature vector, a humidity time series associated feature vector, a smoke concentration time series associated feature vector, a water accumulation state time series associated feature vector and a liquid level depth time series associated feature vector. Taking into account that temperature, humidity, smoke concentration, water accumulation state and liquid level depth all have time-series dependencies, that is, these data have time-series dynamic characteristics and correlation relationships in the time dimension, in order to understand and capture the time-series change patterns and trends of these environmental parameters, in the technical solution of the present application, the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth are respectively input into the sequence encoder of the 1D-CNN model for encoding, so as to extract the time-series correlation feature information of temperature, humidity, smoke concentration, water accumulation state and liquid level depth in the time dimension, so as to obtain the temperature time-series correlation feature vector, humidity time-series correlation feature vector, smoke concentration time-series correlation feature vector, water accumulation state time-series correlation feature vector and liquid level depth time-series correlation feature vector.

[0032] It is worth mentioning that the one-dimensional convolutional neural network (1D-CNN) is a convolutional neural network specifically designed to process one-dimensional data (such as time series, text, and audio signals). It is similar to the traditional two-dimensional convolutional neural network (2D-CNN), but the convolution kernel slides on the one-dimensional space of the input data instead of sliding on the two-dimensional space.

[0033] Specifically, the device parameter time series correlation feature capture unit 332 is used to arrange the temperature time series correlation feature vector, the humidity time series correlation feature vector, the smoke concentration time series correlation feature vector, the water accumulation state time series correlation feature vector and the liquid level depth time series correlation feature vector into a device state multi-parameter correlation matrix, and then input the device state multi-parameter correlation matrix into a device parameter time series correlation pattern feature extractor based on a hole convolutional neural network model to obtain a device parameter time series correlation feature graph. Considering that environmental parameter data, such as temperature, humidity, smoke concentration, water accumulation state and liquid level depth, not only have time series dynamic characteristics in the time dimension, but also have time series correlation characteristics and interactive effects between these data. Based on this, in order to effectively capture the temporal relationship between different environmental parameters, so as to more comprehensively analyze the environment in which the equipment is located, help analyze and predict the equipment operation status, and avoid information loss caused by a single data source, in the technical solution of the present application, the temperature time series correlation feature vector, the humidity time series correlation feature vector, the smoke concentration time series correlation feature vector, the water accumulation state time series correlation feature vector and the liquid level depth time series correlation feature vector are further arranged into a multi-parameter correlation matrix of the equipment state, and then the multi-parameter correlation matrix of the equipment state is input into the equipment parameter time series correlation pattern feature extractor based on the hole convolutional neural network model to obtain the equipment parameter time series correlation feature map. It should be understood that the hole convolutional neural network model can handle long-distance dependencies in time series data. By introducing the hole rate, the model can expand the receptive field without increasing the computational complexity. Therefore, through the processing of the device parameter temporal correlation pattern feature extractor based on the void convolutional neural network model, the model can better capture the temporal dependency correlation characteristics of various environmental parameter data under long-distance dependencies in the device state multi-parameter association matrix, so as to characterize the device state and improve the prediction accuracy of the device state.

[0034] Specifically, the device parameter temporal correlation feature strengthening unit 333 is used to input the device parameter temporal correlation feature map into the cross-channel feature space adaptive strengthening self-attention processing module to obtain the device parameter temporal correlation strengthening feature map. It should be understood that the device parameter temporal correlation feature map reflects the characterization characteristics of the device state, which is obtained by extracting the temporal coordination and correlation features of a variety of environmental data. However, since different environmental data have different temporal correlation semantics and importance characteristics, that is, the temporal correlation features between different environmental parameters have different importance and contribution to the subsequent device state detection task. Therefore, in order to better analyze the environment in which the device is located to improve the accuracy of device state abnormality detection, in the technical solution of the present application, the device parameter temporal correlation feature map is further input into the cross-channel feature space adaptive strengthening self-attention processing module to obtain the device parameter temporal correlation strengthening feature map. It should be understood that the cross-channel feature space adaptive strengthening self-attention processing module can effectively capture the contextual relationship between different feature channels in the feature map through the self-attention mechanism. This mechanism enables the model to identify which feature channels are more important in specific situations and scenarios, thereby enhancing the expressive power of these environmental parameter temporal correlation features that are more important for subsequent equipment status detection tasks. While performing feature enhancement, the cross-channel feature space adaptive enhanced self-attention processing module can also retain the structure of the original feature space. This feature ensures that the geometric information of the feature graph is not destroyed, so that subsequent analysis and decision-making are based on the real important semantics between the temporal correlation features of different categories of environmental parameters. The module can also perform feature fusion in units of feature space structure while retaining the feature space structure, which gives the model greater flexibility when processing high-dimensional data. By adaptively adjusting the relationship between channels, the model can optimize the understanding of the temporal correlation features and interactions of different environmental parameters according to actual conditions, helping the model to better understand the operating status of the equipment, especially when multiple environmental factors are jointly affected, so as to better characterize the equipment status and provide a basis for subsequent equipment status detection and abnormal situation prediction.

[0035] In an embodiment of the present application, the temporal correlation feature map between device parameters is input into a cross-channel feature space adaptive enhanced self-attention processing module to obtain a temporal correlation enhanced feature map between device parameters, including: performing layer normalization on the temporal correlation feature map between device parameters to obtain a normalized temporal correlation feature map between device parameters; performing point convolution processing on the normalized temporal correlation feature map between device parameters to obtain a temporal channel context association representation feature map between device parameters; performing convolution encoding on the temporal channel context association representation feature map between device parameters to obtain a temporal space context association representation feature map between device parameters; copying the temporal space context association representation feature map between device parameters to obtain a temporal backup space context association representation feature map between device parameters; performing cross-channel context association representation feature map between device parameters, the temporal space context association representation feature map between device parameters and the temporal backup space context association representation feature map between device parameters. The representation feature map is reshaped to obtain a feature matrix representing the temporal channel context association between device parameters, a feature matrix representing the temporal space context association between device parameters, and a feature matrix representing the temporal backup space context association between device parameters; the temporal cross-channel cross-covariance matrix between device parameters is calculated between the temporal channel context association representation feature matrix and the temporal space context association representation feature matrix between device parameters; the temporal cross-channel cross-covariance matrix between device parameters is activated to obtain a temporal feature global interaction attention matrix between device parameters; the product between the temporal backup space context association representation feature matrix between device parameters and the temporal feature global interaction attention matrix between device parameters is calculated to obtain a temporal attention enhancement feature representation matrix between device parameters; the temporal attention enhancement feature representation matrix between device parameters is reshaped to obtain the temporal association enhancement feature map between device parameters.

[0036] Among them, the process of calculating the device parameter time-series cross-channel cross-covariance matrix between the device parameter time-series channel context association representation feature matrix and the device parameter time-series space context association representation feature matrix includes: calculating the matrix multiplication between the device parameter time-series channel context association representation feature matrix and the device parameter time-series space context association representation feature matrix to obtain the device parameter time-series channel-space context association feature matrix; calculating the positional division between the device parameter time-series channel-space context association feature matrix and the hyperparameter to obtain the device parameter time-series cross-channel cross-covariance matrix.

[0037] More specifically, the process of activating the temporal cross-channel cross-covariance matrix between the device parameters to obtain the global interactive attention matrix of the temporal features between the device parameters includes: using a Softmax function to activate the temporal cross-channel cross-covariance matrix between the device parameters to obtain the global interactive attention matrix of the temporal features between the device parameters.

[0038] In summary, in the above embodiment, the temporal correlation feature map between device parameters is input into the cross-channel feature space adaptive enhanced self-attention processing module to obtain the temporal correlation enhanced feature map between device parameters, including: inputting the temporal correlation feature map between device parameters into the cross-channel feature space adaptive enhanced self-attention processing module to process it with the following self-attention feature enhancement formula to obtain the temporal correlation enhanced feature map between device parameters; wherein the self-attention feature enhancement formula is:

[0039] F ln =Layer Normalization(F i )

[0040] F Q =Conv 1×1 (F ln )

[0041] F K =Conv 3×3 (Conv 1×1 (F ln ))

[0042] F V =Copy(F K )

[0043] M Q =Reshape(F Q )

[0044] M K =Reshape(F K )

[0045] M V =Reshape(F V )

[0046]

[0047]

[0048] Among them, F i is the temporal correlation feature map between the device parameters, Layer Normalization(·) represents the layer normalization operation on the feature map, F lnConv is the time series correlation feature map between normalized device parameters. 1×1 (·) is the point convolution operation, F Q The feature map represents the temporal channel context association between device parameters, Conv 3×3 (·) is a dilated convolution operation with a convolution kernel of 3×3, F K is the temporal spatial context association feature graph between device parameters, Copy(·) is the copy operation, and F V is the temporal backup space context association between device parameters, representing the feature graph, Reshape(·) represents the feature shape reshaping process, M Q M is the feature matrix representing the temporal channel context association between device parameters. K is the feature matrix representing the temporal spatial context association between device parameters, M V The feature matrix represents the temporal backup space context association between device parameters. is matrix multiplication, θ is a hyperparameter, softmax(·) is the softmax activation function, M a is the global interaction attention matrix of temporal features between device parameters, F a A characteristic diagram is provided for strengthening the temporal correlation between the device parameters.

[0049] Specifically, the device state detection unit 334 is used to determine the device state detection result based on the time series correlation enhancement feature diagram between the device parameters. In a specific example of the present application, the time series correlation enhancement feature diagram between the device parameters is input into the classifier-based device state detection module to obtain the device state detection result. That is, the device state representation information after feature enhancement expression is used for classification processing, so as to effectively identify the device state and predict abnormalities. In this way, the intelligent fusion gateway can use the multimodal data of environmental parameters to effectively identify the environmental conditions of the device, providing strong support for equipment failure warning and response.

[0050] In a preferred example, when the temporal correlation feature graph between device parameters expresses the temporal correlation features related to the cross-sample high-order sample domain of the one-dimensional temporal correlation features of temperature data, humidity data, smoke concentration, water accumulation state and liquid level depth, and a cross-channel feature space adaptive enhanced self-attention processing is performed on it, the obtained temporal correlation enhanced feature graph between device parameters will also cause the self-attention weight enhanced distribution regression mode to shift due to the cross-channel difference in the multi-dimensional spatial distribution of features, that is, it will cause the temporal correlation enhanced feature graph between device parameters to be input into the classifier-based device state detection module to obtain the overflow of the device state detection result, affecting its accuracy.

[0051] Based on this, when the applicant of the present application inputs the time series correlation enhancement feature graph between the equipment parameters into the classifier-based equipment state detection module to obtain the equipment state detection result, the applicant optimizes the time series correlation enhancement feature graph between the equipment parameters, and the optimization process includes the steps of:

[0052] Determine the abnormal probability value corresponding to the abnormality of the device state obtained by inputting the time series correlation reinforcement feature graph between the device parameters into the device state detection module based on the classifier;

[0053] Passing the device parameter temporal correlation reinforcement feature map through a probabilistic activation function to obtain a probabilistic device parameter temporal correlation reinforcement feature map;

[0054] Subtracting the probabilistic device parameter time series correlation reinforcement feature graph from the unit feature graph, and using the feature graph resulting from the subtraction as a base to calculate a power function with the abnormal probability value as an exponent;

[0055] Performing a dot multiplication of the power function feature graph and the probabilistic temporal correlation enhancement feature graph between device parameters to obtain an intermediate feature graph of temporal correlation enhancement between device parameters;

[0056] Taking the absolute value of each eigenvalue of the probabilistic device parameter temporal correlation enhancement feature graph as the base, calculating a power function with the inverse of the abnormal probability value as the exponent, and summing all eigenvalues ​​to obtain a device parameter temporal correlation enhancement bias eigenvalue; and

[0057] After multiplying the bias characteristic value of the temporal correlation enhancement between device parameters by a weight serving as a hyperparameter, the value is interpolated with the intermediate characteristic graph of the temporal correlation enhancement between device parameters to obtain an optimized characteristic graph of the temporal correlation enhancement between device parameters.

[0058] Wherein, let the probabilistic device parameter temporal correlation enhancement characteristic diagram be denoted as F, then the optimized device parameter temporal correlation enhancement characteristic diagram F' is expressed as:

[0059]

[0060] Wherein, F is the temporal correlation enhancement feature map between the probabilistic device parameters, F I is a unit feature map with all eigenvalues ​​being one, and f i ∈F, is the characteristic value of the temporal correlation reinforcement feature map between the probabilistic device parameters, α is the weight of the hyperparameter, p is the abnormal probability value, (·) ⊙p It is a power function with the abnormal probability value as the exponent of the characteristic value at each position in the calculation vector, f i represents the i-th eigenvalue of the probabilistic temporal correlation reinforcement feature map between device parameters, ⊙ represents point multiplication by position, ⊕ represents addition by position, represents the difference by position, and F' is the time series correlation enhancement characteristic diagram between the optimized device parameters.

[0061] That is, for the time series correlation enhanced feature graph between the device parameters, based on the convergence limit of the probability distribution of the power series in the form of Cauchy-Hadamard, the feature set of the time series correlation enhanced feature graph between the device parameters is represented by a power series based on the convergence probability, and a set convergence bounded bias based on each eigenvalue of the time series correlation enhanced feature graph between the device parameters is added as a convergence radius representation, so as to avoid the invalid overflow of the classification result caused by the deviation of the distribution mode of the time series correlation enhanced feature graph between the device parameters due to the gradient propagation of the model parameters based on the convergence probability in the regression process of the device state detection module based on the classifier under the unified probability convergence limit standard, thereby improving the accuracy of the device state detection result obtained by the device state detection module based on the classifier. In this way, the intelligent fusion gateway can be used to effectively identify the environmental conditions of the device by using the multimodal time series characteristics of the environmental parameters, so as to more accurately predict the abnormal conditions of the device state, and provide strong support for the early warning and response of equipment failures.

[0062] It is worth mentioning that in other specific examples of the present application, temperature time series features, smoke concentration time series features, water accumulation state time series features and liquid level depth time series features can also be extracted from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth in other ways, and a device state detection result is generated based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features and the liquid level depth time series features, and the device state detection result is used to indicate whether there is an abnormality in the device state, for example: normalizing the time series of temperature data, humidity data, smoke concentration data, water accumulation state data and liquid level depth data; extracting features from the time series using a time series model such as 1D-CNN or LSTM; fusing the extracted temperature time series features, humidity time series features, smoke concentration time series features, water accumulation state time series features and liquid level depth time series features through an attention mechanism; classifying the device state based on the fused time series features using a machine learning algorithm (such as a support vector machine, a decision tree or a neural network); the algorithm outputs the device state detection result, indicating whether there is an abnormality in the device state.

[0063] In particular, the remote communication module 340 is used to transmit the device status detection result. In one example, in response to the device status detection result being abnormal, an alarm is triggered, thereby providing strong support for device failure warning and response.

[0064] As described above, the intelligent fusion gateway 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a detection algorithm of the intelligent fusion gateway. In a possible implementation, the intelligent fusion gateway 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the intelligent fusion gateway 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the intelligent fusion gateway 300 can also be one of the many hardware modules of the wireless terminal.

[0065] Alternatively, in another example, the intelligent converged gateway 300 and the wireless terminal may also be separate devices, and the intelligent converged gateway 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0066] Furthermore, a detection method for an intelligent fusion gateway is also provided.

[0067] Figure 4 FIG. 1 is a flow chart of a detection method of an intelligent converged gateway according to an embodiment of the present application. Figure 4 As shown, the detection method of the intelligent fusion gateway according to the embodiment of the present application includes: S1, collecting the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation state and the time series of liquid level depth; S2, storing the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth; S3, extracting the temperature time series characteristics, the smoke concentration time series characteristics, the water accumulation state time series characteristics and the liquid level depth time series characteristics from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth, and generating a device status detection result based on the temperature time series characteristics, the smoke concentration time series characteristics, the water accumulation state time series characteristics and the liquid level depth time series characteristics, and the device status detection result is used to indicate whether there is an abnormality in the device status; S4, transmitting the device status detection result.

[0068] In summary, the detection method of the intelligent fusion gateway according to the embodiment of the present application is explained. In the edge computing analysis module, the data processing and analysis algorithm based on artificial intelligence is introduced to perform time-series dynamic analysis and collaborative correlation coding on these environmental parameter data, so as to capture the time-series correlation characteristics between temperature, humidity, smoke concentration, water accumulation state and liquid level depth, so as to obtain the time-series representation of the equipment state, thereby effectively identifying and predicting the equipment state. In this way, the intelligent fusion gateway can use the multimodal data of environmental parameters to effectively identify the environmental conditions of the equipment, providing strong support for equipment failure warning and response.

[0069] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An intelligent fusion gateway, characterized in that: include: A data acquisition module is used to collect the time series of temperature data, the time series of humidity data, the time series of smoke concentration, the time series of water accumulation status and the time series of liquid level depth; A data storage module, used for storing the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth; an edge computing analysis module, for extracting temperature time series features, smoke concentration time series features, water accumulation state time series features, and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state, and the time series of the liquid level depth, and generating a device state detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features, and the liquid level depth time series features, wherein the device state detection result is used to indicate whether there is an abnormality in the device state; A remote communication module is used to transmit the device status detection result.

2. The intelligent converged gateway according to claim 1, characterized in that: The edge computing analysis module includes: A parameter time series feature extraction unit is used to extract time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state, and the time series of the liquid level depth to obtain a temperature time series correlation feature vector, a humidity time series correlation feature vector, a smoke concentration time series correlation feature vector, a water accumulation state time series correlation feature vector, and a liquid level depth time series correlation feature vector; a device parameter time series correlation feature capture unit, configured to arrange the temperature time series correlation feature vector, the humidity time series correlation feature vector, the smoke concentration time series correlation feature vector, the water accumulation state time series correlation feature vector, and the liquid level depth time series correlation feature vector into a device state multi-parameter correlation matrix, and then input the device state multi-parameter correlation matrix into a device parameter time series correlation pattern feature extractor based on a hole convolutional neural network model to obtain a device parameter time series correlation feature graph; A device parameter temporal correlation feature enhancement unit, used for inputting the device parameter temporal correlation feature map into a cross-channel feature space adaptive enhancement self-attention processing module to obtain a device parameter temporal correlation enhancement feature map; The device state detection unit is used to determine the device state detection result based on the time series correlation enhancement feature diagram between the device parameters.

3. The intelligent converged gateway according to claim 2, characterized in that: The parameter time series feature extraction unit is used to: input the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth into the sequence encoder of the 1D-CNN model respectively to obtain the temperature time series associated feature vector, the humidity time series associated feature vector, the smoke concentration time series associated feature vector, the water accumulation state time series associated feature vector and the liquid level depth time series associated feature vector.

4. The intelligent converged gateway according to claim 3, characterized in that: The device parameter temporal correlation feature strengthening unit includes: A normalization subunit, used for performing layer normalization on the timing correlation feature graph between device parameters to obtain a normalized timing correlation feature graph between device parameters; A channel context association feature extraction subunit, used for performing point convolution processing on the normalized device parameter temporal association feature map to obtain a device parameter temporal channel context association representation feature map; A spatial context association feature extraction subunit, used for performing convolution encoding on the temporal channel context association representation feature map between device parameters to obtain a temporal spatial context association representation feature map between device parameters; A spatial context association representation feature backup subunit, used for copying the temporal spatial context association representation feature graph between device parameters to obtain a temporal backup spatial context association representation feature graph between device parameters; A feature shape reshaping subunit is used to perform feature shape reshaping on the feature graph of the temporal channel context association representation between device parameters, the feature graph of the temporal space context association representation between device parameters, and the feature graph of the temporal backup space context association representation between device parameters to obtain a feature matrix of the temporal channel context association representation between device parameters, a feature matrix of the temporal space context association representation between device parameters, and a feature matrix of the temporal backup space context association representation between device parameters; A time series cross-channel cross-covariance calculation subunit, used to calculate a time series cross-channel cross-covariance matrix between device parameters between the time series channel context association representation feature matrix between the device parameters and the time series space context association representation feature matrix between the device parameters; An activation subunit, used for activating the temporal cross-channel cross-covariance matrix between the device parameters to obtain a temporal feature global interaction attention matrix between the device parameters; A temporal attention enhancement representation subunit, used to calculate the product between the temporal backup space context association representation feature matrix between the device parameters and the temporal feature global interaction attention matrix between the device parameters to obtain the temporal attention enhancement feature representation matrix between the device parameters; The device parameter temporal association reinforcement subunit is used to reshape the feature representation matrix of the device parameter temporal attention reinforcement to obtain the device parameter temporal association reinforcement feature map.

5. The intelligent converged gateway according to claim 4, characterized in that: The time series cross-channel cross covariance calculation subunit is used for: Calculating the matrix multiplication between the temporal channel context association representation feature matrix between the device parameters and the temporal spatial context association representation feature matrix between the device parameters to obtain the temporal channel-spatial context association feature matrix between the device parameters; The position-wise division between the temporal channel-spatial contextual association feature matrix between the device parameters and the hyperparameters is calculated to obtain the temporal cross-channel cross-covariance matrix between the device parameters.

6. The intelligent converged gateway according to claim 5, characterized in that: The activation subunit is used to: use a Softmax function to activate the temporal cross-channel cross-covariance matrix between the device parameters to obtain a global interactive attention matrix of the temporal features between the device parameters.

7. The intelligent converged gateway according to claim 6, characterized in that: The device state detection unit is used to: input the time series correlation reinforcement feature diagram between the device parameters into the classifier-based device state detection module to obtain the device state detection result.

8. A detection method for an intelligent fusion gateway, characterized in that: include: Collect the time series of temperature data, humidity data, smoke concentration, water accumulation status and liquid level depth; storing the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth; Extracting temperature time series features, smoke concentration time series features, water accumulation state time series features and liquid level depth time series features from the time series of the temperature data, the time series of the humidity data, the time series of the smoke concentration, the time series of the water accumulation state and the time series of the liquid level depth, and generating a device state detection result based on the temperature time series features, the smoke concentration time series features, the water accumulation state time series features and the liquid level depth time series features, wherein the device state detection result is used to indicate whether the device state is abnormal; The device status detection result is transmitted.

Images