Fault determination method and device based on multi-sensor fusion hypergraph neural network

CN116678455BActive Publication Date: 2026-08-18TSINGHUA UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310652817.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-02
Publication Date
2026-08-18
Estimated Expiration
2043-06-02

AI Technical Summary

Technical Problem

[0005]本申请提供一种基于多传感器融合超图神经网络的故障确定方法和设备,用以解决因获取故障样本的机会较少导致基于少量样本所预测出的故障类型不准确问题

Benefits of technology

[0029] The fault determination method and apparatus based on multi-sensor fusion hypergraph neural network provided in this application construct a single-sensor fusion hypergraph and its correlation matrix based on each sensor data of an electromechanical system, and construct a multi-sensor fusion hypergraph and its correlation matrix based on multiple sensor data. This enriches the input information used to determine faults in the electromechanical system. Then, the single-sensor fusion hypergraph and its correlation matrix and the multi-sensor fusion hypergraph and its correlation matrix are analyzed and processed to determine the fault type of the electromechanical system. This improves the accuracy of fault determination based on a small amount of sensor data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116678455B_ABST
    Figure CN116678455B_ABST
Patent Text Reader

Abstract

The application provides a fault determination method and device based on a multi-sensor fusion hypergraph neural network. The method comprises: acquiring a set of to-be-processed data of an electromechanical system; performing hypergraph calculation processing on the set of to-be-processed data to obtain a first correlation matrix of a first hypergraph and a second correlation matrix of a second hypergraph; and determining a fault type of the electromechanical system according to the set of to-be-processed data, the first correlation matrix and the second correlation matrix. The method of the application improves the accuracy of fault determination based on a small amount of sensor data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of electromechanical systems technology, and in particular to a fault determination method and device based on a multi-sensor fusion hypergraph neural network. Background Technology

[0002] For complex electromechanical systems such as rotating machinery and nuclear power plant electromechanical systems, if a critical component is damaged, it could cause the equipment to malfunction or even lead to an accident. Therefore, it is necessary to monitor the equipment in real time and diagnose faults to promptly identify equipment failure risks.

[0003] In existing technologies, artificial intelligence technology is used to construct a fault diagnosis model based on equipment fault samples and manual diagnostic results. This fault diagnosis model is then used to diagnose newly emerging equipment faults, thereby completing the fault diagnosis of the electromechanical system.

[0004] However, in the above methods, the number of failures in the electromechanical system is relatively small, especially the opportunity to obtain failure samples from large equipment, which leads to inaccurate prediction of the failure type based on a small number of samples. Summary of the Invention

[0005] This application provides a fault determination method and device based on a multi-sensor fusion hypergraph neural network to solve the problem of inaccurate fault type prediction based on a small number of samples due to the limited opportunities to obtain fault samples.

[0006] In a first aspect, this application provides a fault determination method based on a multi-sensor fusion hypergraph neural network, comprising:

[0007] Acquire a set of data to be processed from an electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes sensor data collected by the sensor at each moment;

[0008] Hypergraph computation is performed on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed.

[0009] The fault type of the electromechanical system is determined based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

[0010] Secondly, this application provides a model training method based on a multi-sensor fusion hypergraph neural network, including:

[0011] Obtain a training data set for the electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes sensor data collected by the sensor at each moment;

[0012] Hypergraph computation is performed on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained.

[0013] Based on the dataset to be trained, the first correlation matrix, and the second correlation matrix, the initial model is trained to obtain a preset fault diagnosis model; wherein, the initial model is a model obtained based on a hypergraph neural network model;

[0014] The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the data set to be processed in the fault determination method based on multi-sensor fusion hypergraph neural network.

[0015] Thirdly, this application provides a fault determination device based on a multi-sensor fusion hypergraph neural network, comprising:

[0016] An acquisition unit is used to acquire a set of data to be processed from an electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes sensor data collected by the sensor at each moment;

[0017] A processing unit is configured to perform hypergraph computation on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed.

[0018] The determining unit is used to determine the fault type of the electromechanical system based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

[0019] Fourthly, this application provides a model training device based on a multi-sensor fusion hypergraph neural network, comprising:

[0020] An acquisition unit is used to acquire a training data set of an electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes sensor data collected by the sensor at each moment;

[0021] A processing unit is configured to perform hypergraph computation on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained.

[0022] The training unit is used to train the initial model based on the dataset to be trained, the first correlation matrix, and the second correlation matrix to obtain a preset fault diagnosis model; wherein the initial model is a model obtained based on a hypergraph neural network model;

[0023] The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the data set to be processed by the fault determination device based on multi-sensor fusion hypergraph neural network.

[0024] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0025] The memory stores computer-executed instructions;

[0026] The processor executes computer execution instructions stored in the memory to implement the methods described in the first and second aspects.

[0027] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the first and second aspects.

[0028] In a seventh aspect, this application provides a computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods described in the first and second aspects.

[0029] The fault determination method and apparatus based on multi-sensor fusion hypergraph neural network provided in this application construct a single-sensor fusion hypergraph and its correlation matrix based on each sensor data of an electromechanical system, and construct a multi-sensor fusion hypergraph and its correlation matrix based on multiple sensor data. This enriches the input information used to determine faults in the electromechanical system. Then, the single-sensor fusion hypergraph and its correlation matrix and the multi-sensor fusion hypergraph and its correlation matrix are analyzed and processed to determine the fault type of the electromechanical system. This improves the accuracy of fault determination based on a small amount of sensor data. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0031] Figure 1 A flowchart illustrating a fault determination method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0032] Figure 2 A flowchart illustrating another fault determination method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0033] Figure 3 A schematic diagram of a single-sensor hypergraph construction process based on an electromechanical system is provided for an embodiment of this application;

[0034] Figure 4 A schematic diagram of a multi-sensor hypergraph construction process based on an electromechanical system is provided for an embodiment of this application;

[0035] Figure 5 A schematic diagram of a hypergraph neural network structure based on an electromechanical system is provided in this application embodiment;

[0036] Figure 6 A flowchart illustrating a model training method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0037] Figure 7 A schematic flowchart illustrating another model training method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0038] Figure 8 A schematic diagram of a fault determination device based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0039] Figure 9A schematic diagram of another fault determination device based on a multi-sensor fusion hypergraph neural network provided in an embodiment of this application;

[0040] Figure 10 A schematic diagram of the structure of a model training device based on a multi-sensor fusion hypergraph neural network provided in an embodiment of this application;

[0041] Figure 11 A schematic diagram of another model training device based on a multi-sensor fusion hypergraph neural network provided in this application embodiment;

[0042] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0043] Figure 13 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0044] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0046] First, let me explain the terms used in this application:

[0047] Hypergraph: refers to a subset system of a finite set. It is the most general structure in discrete mathematics and can be connected to any number of vertices via hyperedges. We can regard any natural join in a relational database as a hypergraph, where attributes are vertices and the relation graph, i.e., the set of attributes of a relation, is the edge.

[0048] Hypergraph Neural Networks: A general framework for modeling multimodal data and complex data correlations, used for representation learning using hypergraph structures; capable of forming complex high-order data correlations through its hypergraph structure, and able to efficiently use hyper-edge convolution operations.

[0049] For complex electromechanical systems such as rotating machinery and nuclear power plant electromechanical systems, if a critical component is damaged, it could cause the equipment to malfunction or even lead to an accident. Therefore, it is necessary to monitor the equipment in real time and diagnose faults to promptly identify equipment failure risks.

[0050] In one example, artificial intelligence technology is used to build a fault diagnosis model based on equipment fault samples and human diagnostic results. This fault diagnosis model is then used to diagnose newly emerging equipment faults, thereby completing the fault diagnosis of the electromechanical system.

[0051] However, in the above methods, the number of failures in the electromechanical system is relatively small, especially the opportunity to obtain failure samples from large equipment, which leads to inaccurate prediction of the failure type based on a small number of samples.

[0052] The fault determination method and device based on multi-sensor fusion hypergraph neural network provided in this application aim to solve the above-mentioned technical problems in the prior art.

[0053] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0054] It should be noted that the fault determination method and device based on multi-sensor fusion hypergraph neural network of this application can be used in the field of electromechanical system technology, and can also be used in any field other than the field of electromechanical system technology. The application field of the fault determination method and device based on multi-sensor fusion hypergraph neural network of this application is not limited.

[0055] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0056] Figure 1 A flowchart illustrating a fault determination method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 1 As shown, the method includes:

[0057] S101. Obtain the set of data to be processed from the electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes the sensor data collected by the sensor at each moment.

[0058] For example, the execution subject of this embodiment can be an electronic device. First, for a certain electromechanical system, the system is equipped with multiple sensors to collect the operating data of the electromechanical system at the current moment. The electronic device obtains the real-time operating data collected from all the sensors at the electromechanical system. This data can be the operating data collected from multiple similar sensors or multiple different types of sensors. The electronic device then segments the acquired data into several fragments and organizes them into a data set, i.e., the data set to be processed. The data set to be processed includes a data subset corresponding to each sensor in the electromechanical system. Each data subset includes the sensor data collected by the corresponding sensor at each moment, i.e., the operating data of the electromechanical system.

[0059] S102. Perform hypergraph computation on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed.

[0060] For example, after acquiring real-time operational data from all sensors, a hypergraph and its association matrix can be used to characterize various complex relationships. For the sensor data collected by each sensor, the relationships between the sensor data collected by each sensor are analyzed, and a hypergraph is constructed based on these relationships to obtain a first hypergraph corresponding to each sensor. A first association matrix is ​​then obtained by transforming this first hypergraph to characterize the relationships between the sensor data collected by each sensor. Similarly, for all sensor data collected from all sensors, the relationships between all sensor data collected from all sensors are analyzed, and a second hypergraph is constructed based on these relationships to obtain a second hypergraph corresponding to each sensor. A second association matrix is ​​then obtained by transforming this second hypergraph to characterize the relationships between the sensor data. Thus, by fusing a large amount of information from a small amount of data, the effectiveness of fault diagnosis is significantly improved.

[0061] S103. Determine the fault type of the electromechanical system based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

[0062] For example, after obtaining the first correlation matrix of the first hypergraph and the second correlation matrix of the second hypergraph, the sensor data in the dataset to be processed, the first correlation matrix of the first hypergraph, and the second correlation matrix of the second hypergraph are fused and analyzed to obtain the current fault type of the electromechanical system. For example, the electromechanical system is currently fault-free; or the current fault type of the electromechanical system is a type 1 fault or a type 2 fault.

[0063] This embodiment provides a fault determination method based on a multi-sensor fusion hypergraph neural network. For multiple sensor data from an electromechanical system, a single-sensor fusion hypergraph and its correlation matrix are constructed based on each sensor data point, and a multi-sensor fusion hypergraph and its correlation matrix are constructed based on multiple sensor data points. This enriches the input information for diagnosing electromechanical system faults. The single-sensor fusion hypergraph and its correlation matrix, as well as the multi-sensor fusion hypergraph and its correlation matrix, are then analyzed and processed to determine the fault type of the electromechanical system. This improves the accuracy of fault determination based on a limited amount of sensor data.

[0064] Figure 2 A flowchart illustrating another fault determination method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 2 As shown, the method includes:

[0065] S201. Obtain the set of data to be processed from the electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes the sensor data collected by the sensor at each moment.

[0066] For example, this step can be referred to step 101, and will not be repeated here.

[0067] S202. Perform Fourier transform on the sensor data in the dataset to be processed to obtain the processed dataset.

[0068] For example, after acquiring the sensor data collected by each sensor, for each sensor, when the sensor data is time-series data, it can be directly processed to obtain the corresponding data of each sensor data, such as mean, variance, maximum value, and minimum value, and thus obtain the corresponding processed data set to be processed; optionally, based on the Fast Fourier Transform (FFT) algorithm, Fourier transform processing can be performed on the sensor data of each sensor to transform the time-series data into frequency domain data; furthermore, feature processing can be performed on the frequency domain data of the sensor to obtain the feature data corresponding to the sensor data, such as amplitude and phase, and thus obtain the corresponding processed data set to be processed.

[0069] S203. For each subset of data from each sensor in the dataset to be processed, determine the first distance information between each pair of sensor data in the subset of data from that sensor; wherein the first distance information represents the data difference between a pair of sensor data.

[0070] For example, for the sensor data in the data subset collected by each sensor, the sensor data at each time moment corresponds to a specific value. The difference between the values ​​of a pair of corresponding sensor data at any two time moments can be calculated, and this difference can then be determined as the distance information between each pair of sensor data in the data subset of that sensor, i.e., the first distance information. Optionally, for the sensor data in the data subset collected by each sensor, after Fourier transform processing, each pair of data in the resulting processed data set is processed to obtain the difference between the values ​​corresponding to each pair of Fourier-transformed data. This difference can then be determined as the distance information between each pair of sensor data in the data subset of that sensor, i.e., the first distance information.

[0071] S204. For each data subset of the sensor in the dataset to be processed, based on the first distance information corresponding to the data subset of the sensor, the data subset of the sensor is processed according to the K-nearest neighbor method to obtain the third correlation matrix of the third hypergraph of the sensor; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between the sensor data at each time.

[0072] For example, based on the calculated first distance information corresponding to the data subset of the sensor, for the sensor data at each time moment, using the K nearest neighbor method, K first distance information with the smallest corresponding values ​​are selected from the first distance information related to the sensor data. Then, based on these K first distance information, K sensor data at all times corresponding to these K first distance information are selected from the sensor data at all times, and the sensor data is associated with these K sensor data. For each sensor, based on the association relationship between the various sensor data, a third hypergraph corresponding to that sensor is constructed. Each third hypergraph has corresponding feature information in that sensor, namely first feature information, which characterizes the association relationship between the sensor data in the data subset corresponding to that sensor. Further, the third hypergraph is transformed to obtain a third association matrix of the third hypergraph, which characterizes the association relationship between the sensor data at each time moment in the data subset corresponding to that sensor.

[0073] S205. The third correlation matrices of the third hypergraphs of each sensor are concatenated to obtain the first correlation matrix of the first hypergraph; wherein the first correlation matrix includes the first feature information of each sensor.

[0074] For example, after obtaining the third association matrix of the third hypergraph corresponding to each sensor, the third association matrices of the third hypergraphs of each sensor are spliced ​​to obtain the first association matrix of the first hypergraph corresponding to each sensor; wherein, the first association matrix includes the first feature information of each sensor to characterize the association relationship between the sensor data in each sensor.

[0075] In one example, Figure 3 A schematic diagram of a single-sensor hypergraph construction process based on an electromechanical system is provided for an embodiment of this application, as shown below. Figure 3 As shown, three sensors, Sensor1, Sensor2, and Sensor3, are installed in the electromechanical system. Sensor data is collected at five time points: x1, x2, x3, x4, and x5. The data collected by Sensor1 at these five time points is then analyzed. 11 x 21 x 31 x 41 and x 51 , sensor data x 11 With data from 2 sensors x 31 and x 51 Perform correlation separately, and associate the sensor data x 21 With data from 2 sensors x 31 and x 41 Perform correlations separately; for the sensor data x at five time points in the dataset collected by the second sensor, Sensor2. 12 x 22 x 32 x 42 and x 52 , sensor data x 12 With data from 2 sensors x 32 and x 42 Perform correlation separately, and associate the sensor data x 22 With data from 2 sensors x 32 and x 52 Perform correlations separately; for the sensor data x at five time points in the dataset collected by the third sensor, Sensor3. 13 x 23 x 33 x 43 and x 53 , sensor data x13 With data from 2 sensors x 43 and x 53 Perform correlation separately, and associate the sensor data x 23 With data from 2 sensors x 33 and x 53 Each sensor is associated separately. Based on the association relationships between the data from each sensor, a single-sensor hypergraph (the third hypergraph) and its corresponding association matrices H1, H2, and H3 (the third association matrix) are constructed. These three hypergraphs are then concatenated along the time dimension to obtain the single-sensor fusion hypergraph s-Hypergraph (the first hypergraph) corresponding to each sensor's data. The association matrices of the three hypergraphs are then concatenated along the time dimension to obtain the association matrix H corresponding to the single-sensor fusion hypergraph s-Hypergraph. sf , namely the first correlation matrix; wherein, each single sensor fusion hypergraph has the corresponding feature information of that sensor, including E1 and E2 corresponding to sensor Sensor1, E3 and E4 corresponding to sensor Sensor2, and E5 and E6 corresponding to sensor Sensor3, so as to characterize the correlation relationship between sensor data in the data subset corresponding to each sensor at each time.

[0076] S206. Based on the data set to be processed, determine the sensor data belonging to the same time in the data set to be processed, which are the hypergraph nodes at that time; wherein, the hypergraph nodes include the sensor data belonging to the same time under each sensor in the electromechanical system.

[0077] For example, based on the sensor data sets collected by each sensor (i.e., the sensor data in the dataset to be processed), for each sensor data acquisition time, the sensor data collected at that time are analyzed and processed to determine the hypergraph node at that time. Optionally, based on the sensor data in the subset of data collected by each sensor, after Fourier transform processing, for each pair of data in the processed dataset to be processed, for each acquisition time, the sensor data collected at that time are analyzed and processed to determine the hypergraph node at that time. For each hypergraph node, the hypergraph node includes the sensor data belonging to the same time from each sensor in the electromechanical system.

[0078] For example, in an electromechanical system, three sensors are installed: a first pressure sensor, a second pressure sensor, and a temperature sensor. These three sensors collect sensor data, including pressure data and temperature data, at two consecutive time points. This data is then organized into a dataset to be processed according to the time of collection. At each time point, the three sensors collect the first pressure data, the second pressure data, and the temperature data, respectively. For time t1, the data belonging to that time point, including the first pressure data t11, the second pressure data t12, and the temperature data t13, are fused and analyzed to determine a hypergraph node t1(t11, t12, t13). This node integrates all sensor data t11, t12, and t13 collected at the corresponding time t1.

[0079] S207. For each pair of hypergraph nodes, determine the second distance information between each pair of hypergraph nodes based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair; wherein, the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension.

[0080] For example, based on the sensors in the electromechanical system, a multi-dimensional sensor space is pre-established. The number of dimensions is determined by the number of sensors in the electromechanical system, with each dimension representing one sensor. Each obtained hypergraph node contains all sensor data collected at a corresponding time, or contains processed feature data corresponding to all sensor data collected at a corresponding time. For each obtained hypergraph node, its corresponding position can be determined in the multi-dimensional sensor space. For each pair of hypergraph nodes, based on the sensor data of one hypergraph node and the sensor data of the other hypergraph node in the pair, the distance between the two corresponding positions of the pair of hypergraph nodes in the sensor space can be calculated, which is the second distance information between each pair of hypergraph nodes.

[0081] In one example, the second distance information is

[0082]

[0083] in, , , Let be the hypergraph node in the preset space at time i. Let be the hypergraph node in the preset space at time j. Let S be the sensor data collected by the S-th sensor at time i. Let S be the sensor data collected by the S-th sensor at time j. Let be the total number of sensors, i be an integer greater than or equal to 1, j be an integer greater than or equal to 1, and S be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

[0084] For example, for three sensors in an electromechanical system, a sensor space R can be defined. S If each dimension represents a sensor, then in the sensor space R 3 In this context, each sensor data point represents a "coordinate value" along a specific dimension; that is, at each given moment, each sensor data point is a vector, not a numerical value. The multiple sensor data points at five different moments in the dataset can be represented as hypergraph nodes corresponding to the three sensor data points at that moment, each corresponding to a point x in the sensor space R. S In the diagram, the distance d between hypergraph nodes at any two time points in the sensor space is calculated using the following formula:

[0085]

[0086] in, Let be the hypergraph node in the multidimensional space corresponding to time i. Let be the hypergraph node in the multidimensional space corresponding to time j. In sensor space The coordinate value in the S-th dimension, To the point in the sensor space In the coordinate value of the S-th sensor dimension, i is an integer greater than or equal to 1 and less than or equal to 5, j is an integer greater than or equal to 1 and less than or equal to 3, and S is an integer greater than or equal to 1. If the value is 3, then the calculated distance is the distance between the hypergraph nodes. and The second distance information between them.

[0087] S208. Based on each second distance information, the dataset to be processed is processed using the K-nearest neighbor method to obtain the second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph node; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

[0088] For example, based on the distance information between hypergraph nodes calculated in the sensor space, for each hypergraph node in the dataset to be processed at each time step, the K nearest neighbors method is used to select the K smallest distance information from the distance information related to the hypergraph node. Then, based on these K distance information, sensor data corresponding to the K distance information at all times are selected from the hypergraph nodes at all times, and the hypergraph node is associated with these K hypergraph nodes. Based on the association relationships between the hypergraph nodes at each time step, a multi-sensor fusion hypergraph corresponding to the dataset to be processed is constructed, i.e., a second hypergraph. This hypergraph has corresponding feature information for the hypergraph nodes, i.e., second feature information, which characterizes the association relationships between the sensor data in the dataset to be processed corresponding to each sensor. Further, for the multi-sensor fusion hypergraph corresponding to the dataset to be processed, the hypergraph is transformed to obtain the association matrix of the hypergraph, i.e., the second association matrix, to characterize the association relationships between the sensor data in the dataset to be processed corresponding to each sensor.

[0089] In one example, Figure 4 A schematic diagram of a multi-sensor hypergraph construction process based on an electromechanical system is provided for an embodiment of this application, such as... Figure 4 As shown, three sensors, sensor 1, sensor 2, and sensor 3, are installed in the electromechanical system. Sensor data is collected at each of the five time points. The data x collected by sensor 1 at the five time points is then analyzed. 11 x 21 x 31 x 41 and x 51 The sensor data x at five time points in the data set collected by the second sensor 2. 12 x 22 x 32 x 42 and x 52 The sensor data x at five time points in the data set collected by the third sensor 3. 13 x 23 x 33 x 43 and x 53 For each moment, in the preset sensor space, there are three dimensions. These five moments correspond to hypergraph nodes x1, x2, x3, x4, and x5, respectively. For example, based on the hypergraph node x1 corresponding to the first moment, the coordinates in each dimension of this space are (x... 11 x 12 x 13This allows us to obtain the corresponding hypergraph nodes and their coordinate values ​​at other times. Then, based on the coordinate values ​​of the hypergraph nodes at each time point, we calculate the second distance information in the sensor space using a formula. Using the K-nearest neighbor method, we associate hypergraph node x1 with two hypergraph nodes x3 and x4, and hypergraph node x2 with two hypergraph nodes x3 and x5. Based on these associations, we construct the multi-sensor hypergraph m-Hypergraph corresponding to this sensor, i.e., the second hypergraph, and the associated matrix H corresponding to the second hypergraph. mf The third correlation matrix is ​​the second hypergraph, which has second feature information corresponding to the hypergraph nodes, including F1 and F2, to characterize the correlation between sensor data in the dataset to be processed for each sensor.

[0090] S209. Input the data set to be processed and the first correlation matrix into the first hypergraph neural network branch of the preset fault diagnosis model to obtain the first feature vector; wherein, the first feature vector represents the fusion feature of the data set to be processed and the first correlation matrix; the preset fault diagnosis model is a model obtained based on the hypergraph neural network model.

[0091] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network model, i.e., a fault diagnosis model, can be pre-set. This model can include a hypergraph neural network branch. Sensor data collected by each sensor, along with the correlation matrix of the obtained single-sensor hypergraph, can be input into this pre-set fault diagnosis model. Based on the hypergraph neural network branch of this model, computational processing is performed to obtain the feature vector of the data set to be processed, i.e., the first feature vector, which characterizes the fusion characteristics between the data set to be processed and the correlation matrix corresponding to the single-sensor hypergraph. Optionally, the first hypergraph neural network branch of this model can be used to process the sensor data collected by each sensor after Fourier transform processing to obtain the feature vector of the Fourier-processed data set to be processed, i.e., the first feature vector, which characterizes the fusion characteristics between the Fourier-processed data set to be processed and the correlation matrix corresponding to the single-sensor hypergraph of the processed data set.

[0092] In one example, step S209 includes the following steps:

[0093] The first step of step S209 is to input the data set to be processed and the first correlation matrix into the first hypergraph neural network branch of the preset fault diagnosis model, and to perform convolution processing on the data set to be processed and the first correlation matrix to obtain the first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the data set to be processed and the first correlation matrix.

[0094] The second step of step S209 is to perform convolution processing on the first intermediate feature and the first correlation matrix to obtain the first feature vector.

[0095] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network model, i.e., a fault diagnosis model, can be pre-set. This model can include a hypergraph neural network branch. Sensor data collected by each sensor and the correlation matrix corresponding to the single-sensor hypergraph are input into the pre-set fault diagnosis model. Based on the first graph convolutional layer of the hypergraph neural network branch, convolution calculation is performed to obtain the intermediate features of the data set to be processed, i.e., the first intermediate features, representing the fusion features between the data set to be processed and the first correlation matrix corresponding to the data set. Then, based on the second graph convolutional layer of the first hypergraph neural network branch, convolution processing is further performed on the intermediate features and the correlation matrix corresponding to the single-sensor hypergraph to obtain the feature vector of the data set to be processed, i.e., the first feature vector, representing the fusion features between the data set to be processed and the correlation matrix corresponding to the single-sensor hypergraph.

[0096] In one example, Figure 5 A schematic diagram of a hypergraph neural network structure based on an electromechanical system is provided for an embodiment of this application, such as... Figure 5 As shown, the preset fault diagnosis model includes two hypergraph neural network branches, s-HGNN and m-HGNN, each consisting of two HConv graph convolutional layers, with Leaky ReLU activation function selected for both. The preset fault diagnosis model also includes a multilayer perceptron (MLP) composed of fully connected layers (ReLU activation function) and a softmax function. Based on the preset computational framework, the multi-sensor fusion hypergraph neural network (MsfHGNN) integrates the hypergraph s-Hypergraph and its correlation matrix H... sf The dataset X to be processed, the hypergraph m-Hypergraph, and its correlation matrix H mf Input into the model, targeting the hypergraph s-Hypergraph and the correlation matrix H sf And the dataset X to be processed, based on the first graph convolutional layer, the correlation matrix H... sf And perform convolution calculation on the dataset X to be processed to obtain the intermediate feature HConv(X, H) sf Based on the second graph convolutional layer, the feature vector HConv(X, H) is further processed. sf ) and the correlation matrix H sf Perform convolution to obtain the first feature vector Z. sf=( HConv(HConv(X,H sf ), H sf The feature vector represents the relationship between the data set X to be processed and the first correlation matrix H corresponding to the data set X. sf The fusion characteristics between them.

[0097] S210. Input the data set to be processed and the second correlation matrix into the second hypergraph neural network branch of the preset fault diagnosis model to obtain the second feature vector; wherein, the second feature vector represents the fusion feature of the data set to be processed and the second correlation matrix.

[0098] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network model, i.e., a fault diagnosis model, can be pre-set. Sensor data collected by each sensor and the correlation matrix of the multi-sensor fusion hypergraph are input into this pre-set fault diagnosis model. Based on the second hypergraph neural network branch of this model, calculations are performed to obtain the second feature vector of the sensor data collected by each sensor, i.e., the second feature vector, which characterizes the fusion characteristics between the sensor data collected by each sensor and the correlation matrix of the multi-sensor fusion hypergraph. Optionally, the second hypergraph neural network branch of this model can be used to process the sensor data collected by each sensor after Fourier transform processing to obtain the second feature vector of the Fourier-processed dataset, i.e., the second feature vector, which characterizes the fusion characteristics between the Fourier-processed dataset and the correlation matrix of the multi-sensor fusion hypergraph.

[0099] In one example, step S210 includes the following steps:

[0100] The first step of step S210 is to input the data set to be processed and the second correlation matrix into the second hypergraph neural network branch of the preset fault diagnosis model, and to perform convolution processing on the data set to be processed and the second correlation matrix to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the data set to be processed and the second correlation matrix.

[0101] The second step of step S210 involves convolving the second intermediate feature and the second correlation matrix to obtain the second feature vector.

[0102] For example, in order to determine faults in an electromechanical system, a model based on a hypergraph neural network model can be pre-set, which is the fault diagnosis model. This pre-set fault diagnosis model includes two hypergraph neural network branches, s-HGNN and m-HGNN, each including two HConv graph convolutional layers, with LeakyReLU selected as the activation function. The pre-set fault diagnosis model also includes a multilayer perceptron (MLP) composed of fully connected layers (activation function ReLU) and a softmax function. The sensor data collected by each sensor and the obtained second correlation matrix are input into the preset fault diagnosis model. Based on the first graph convolutional layer in the second hypergraph neural network branch of the model, convolution calculation is performed to obtain the second intermediate feature of the data set to be processed, which represents the fusion feature between the data set to be processed and the corresponding second correlation matrix. Then, based on the second graph convolutional layer in the second hypergraph neural network branch of the model, the intermediate feature and the corresponding second correlation matrix are further convolved to obtain the second feature vector of the data set to be processed, which represents the fusion feature between the data set to be processed and the corresponding second correlation matrix.

[0103] In one example, based on a pre-defined computational framework, a multi-sensor fusion hypergraph neural network (MsfHGNN), a second hypergraph neural network branch, m-HGNN, is set up to target the hypergraph m-Hypergraph and the correlation matrix H. mf And the dataset X to be processed, based on the first graph convolutional layer, the correlation matrix H... mf And perform convolution calculation on the dataset X to be processed to obtain the intermediate feature HConv(X, H) mf Based on the second graph convolutional layer, the feature vector HConv(X, H) is further processed. mf ) and the correlation matrix H sf Perform convolution to obtain the first feature vector Z. mf =( HConv(HConv(X,H mf ), H mf The feature vector represents the data set X to be processed and the second correlation matrix H corresponding to the data set X. mf The fusion characteristics between them.

[0104] S211. Connect the first feature vector and the second feature vector to obtain the feature vector to be processed.

[0105] For example, after convolution processing by a preset fault diagnosis model, the first and second feature processing vectors are concatenated to obtain the feature vector to be processed corresponding to the set of data to be processed collected by each sensor.

[0106] For example, regarding the obtained first feature vector Z... sf Second feature processing vector Z mf Perform a concat operation on the two to obtain Z = Concat(Z). sf Z mf ), which is the feature vector to be processed corresponding to the set of data to be processed collected by each sensor.

[0107] S212. Based on the preset fault diagnosis model, identify the feature vector to be processed to obtain the fault type of the electromechanical system.

[0108] For example, after the two-layer HConv convolutional layer processing and concatenation processing of the two hypergraph neural network branches in the preset fault diagnosis model, the feature vector to be processed is obtained. Based on the multilayer perceptron in the preset fault diagnosis model, the feature vector to be processed is calculated and identified to obtain the fault type of the electromechanical system. For example, the electromechanical system is currently fault-free; or the fault type of the electromechanical system is currently a type 1 fault or a type 2 fault.

[0109] In one example, the preset fault diagnosis model includes two hypergraph neural network branches, s-HGNN and m-HGNN, each consisting of two HConv convolutional layers, with LeakyReLU activation function selected for both. This preset fault diagnosis model also includes a multilayer perceptron (MLP) composed of fully connected layers (ReLU activation function) and a softmax function. After processing and concatenating the two HConv convolutional layers from the two hypergraph neural network branches in the preset fault diagnosis model to obtain the feature vector to be processed, the MLP in the preset fault diagnosis model calculates, processes, and identifies this feature vector, outputting label information Label Y = MLP(Z). This label information Label Y represents the current fault type of the electromechanical system; for example, the electromechanical system currently has no fault; or the current fault type of the electromechanical system is either type 1 or type 2.

[0110] S213. Generate alarm information according to the fault type of the electromechanical system and issue alarm information; wherein, the alarm information is one or more of the following: voice information, text information, image information, video information, and light information.

[0111] For example, after determining the fault type of the current electromechanical system, an alarm message corresponding to each fault type is generated according to the fault type of the electromechanical system, and the alarm message is sent out. This can prompt the current staff to control whether the electromechanical system continues to operate. The alarm message can be one or more of the following: voice message, text message, image message, video message, and light message.

[0112] In one example, after determining the fault type of the current electromechanical system, if the fault type is classified as Class 1, an alarm message corresponding to that fault type is generated and issued in the form of an illuminated light to prompt the current staff to check and control the electromechanical system to stop operation.

[0113] In this embodiment, based on the above embodiments, firstly, for multiple sensor data of the electromechanical system, a single-sensor fusion hypergraph and its correlation matrix are constructed according to each sensor data. Then, based on the multi-dimensional sensor hypergraph, a multi-sensor fusion hypergraph and its correlation matrix are constructed. This enriches the input information for diagnosing electromechanical system faults. Next, the single-sensor fusion hypergraph and its correlation matrix, and the multi-sensor fusion hypergraph and its correlation matrix are analyzed and processed to determine the fault type of the electromechanical system. This improves the accuracy of fault determination based on a small amount of sensor data. Furthermore, after determining the fault type, an alarm is issued according to the fault type, prompting the current staff to control whether the electromechanical system continues to operate, thus avoiding various production losses caused by the fault.

[0114] Figure 6 A flowchart illustrating a model training method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 6 As shown, the method includes:

[0115] S301. Obtain the training data set of the electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes the sensor data collected by the sensor at each moment.

[0116] For example, the execution subject of this embodiment can be an electronic device. First, for a certain electromechanical system, the system is equipped with multiple sensors to collect the operating data of the electromechanical system at each moment in a historical time period. The electronic device obtains the real-time operating data collected from all sensors in the electromechanical system. This data can be the operating data collected from multiple similar sensors or multiple different types of sensors. The acquired data is then organized into a dataset, namely the training dataset. The training dataset includes a subset of data corresponding to each sensor in the electromechanical system. Each subset of data includes the sensor data collected by the corresponding sensor at each moment, that is, the operating data of the electromechanical system in the historical time period.

[0117] S302. Perform hypergraph computation on the dataset to be trained to obtain a first correlation matrix of the first hypergraph and a second correlation matrix of the second hypergraph; wherein, the first hypergraph represents the correlation between sensor data in the data subset, and the first correlation matrix includes the correlation between sensor data in each data subset; the second hypergraph represents the correlation between sensor data in the dataset to be trained, and the second correlation matrix includes the correlation between sensor data in the dataset to be trained.

[0118] For example, after acquiring the operational data collected from all sensors over a historical time period, a hypergraph and its association matrix can be used to characterize various complex relationships. For the training data collected by each sensor, the relationships between these training data are analyzed, and a hypergraph is constructed based on these relationships to obtain a first hypergraph for each sensor. This first hypergraph is then transformed to obtain a first association matrix, which characterizes the relationships between the training data collected by each sensor. Similarly, for all sensor data collected from all sensors, the relationships between these data are analyzed, and a second hypergraph is constructed based on these relationships to obtain a second hypergraph. This second hypergraph is then transformed to obtain a second association matrix, which characterizes the relationships between the training data collected by each sensor. Thus, by fusing a large amount of information from a small amount of data, the effectiveness of fault diagnosis is significantly improved.

[0119] S303. Based on the training dataset, the first correlation matrix, and the second correlation matrix, the initial model is trained to obtain a preset fault diagnosis model; wherein, the initial model is a model obtained based on a hypergraph neural network model; wherein, the preset fault diagnosis model is used to identify the fault type of the electromechanical system from the processing dataset of the fault determination method based on multi-sensor fusion hypergraph neural network.

[0120] For example, after obtaining the first correlation matrix of the first hypergraph and the second correlation matrix of the second hypergraph corresponding to the dataset to be trained, the sensor data in the dataset to be trained, the first correlation matrix of the first hypergraph and the second correlation matrix of the second hypergraph are input into a preset initial model. The initial model is a model obtained based on a hypergraph neural network model. The initial model is trained to obtain a preset fault diagnosis model. This model is used to identify the fault type of the electromechanical system in the dataset to be processed based on the fault determination method of multi-sensor fusion hypergraph neural network.

[0121] This embodiment provides a model training method based on a multi-sensor fusion hypergraph neural network. For multiple sensor data points within a historical time period of an electromechanical system, a single-sensor fusion hypergraph and its correlation matrix are constructed based on each sensor data point to be trained. Similarly, a multi-sensor fusion hypergraph and its correlation matrix are constructed based on multiple sensor data points to be trained. This enriches the input information for diagnosing electromechanical system faults and improves the accuracy of fault determination based on a limited amount of sensor data to be trained. The initial model is then sub-trained using the single-sensor fusion hypergraph and its correlation matrix, as well as the multi-sensor fusion hypergraph and its correlation matrix, to obtain a preset fault diagnosis model. Furthermore, using this hypergraph to train the model simplifies the training process, improves algorithm efficiency, and allows the determination of the fault type of the electromechanical system based on the model.

[0122] Figure 7 A flowchart illustrating another model training method based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 7 As shown, the method includes:

[0123] S401. Obtain the training data set of the electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes the sensor data collected by the sensor at each moment.

[0124] For example, this step can be referred to step 301, and will not be repeated here.

[0125] S402. Perform Fourier transform on the sensor data in the dataset to be trained to obtain the processed dataset to be trained.

[0126] For example, after acquiring the sensor data to be trained collected by each sensor, for each sensor, when the sensor data in that sensor is time-series data, it can be directly processed to obtain the data corresponding to each sensor data in that sensor, such as mean, variance, maximum value, and minimum value, and thus obtain the corresponding processed training data set; optionally, based on the Fast Fourier Transform (FFT) algorithm, Fourier transform processing can be performed on each sensor data in that sensor to transform the time-series data into frequency domain data; furthermore, feature processing can be performed on the frequency domain data of the sensor to obtain the feature data corresponding to the sensor data of that sensor, such as amplitude and phase, and thus obtain the corresponding processed training data set.

[0127] S403. For each subset of data from each sensor in the dataset to be trained, determine the first distance information between each pair of sensor data in the subset of data from that sensor; wherein the first distance information represents the data difference between a pair of sensor data.

[0128] For example, for the sensor data in each data subset of the dataset to be trained, the sensor data at each time step corresponds to a specific value. The difference between the values ​​of a pair of sensor data at each two time steps can be calculated, and then the difference can be determined as the first distance information between each pair of sensor data in the dataset of that sensor. Alternatively, for the sensor data in the dataset collected by each sensor, after Fourier transform processing, each pair of data in the processed dataset to be trained can be processed to obtain the difference between the values ​​of each pair of Fourier transformed data, and then the difference can be determined as the first distance information between each pair of sensor data in the dataset of that sensor.

[0129] S404. For each data subset of the sensor in the dataset to be trained, based on the first distance information corresponding to the data subset of the sensor, the data subset of the sensor is processed according to the K-nearest neighbor method to obtain the third correlation matrix of the third hypergraph of the sensor; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between the sensor data at each time.

[0130] For example, based on the calculated first distance information corresponding to each subset of the sensor's data, for the sensor data to be trained at each time step, using the K nearest neighbor method, K first distance information with the smallest corresponding values ​​are selected from the first distance information related to the sensor data. Then, based on these K first distance information, K sensor data corresponding to these K first distance information are selected from the sensor data to be trained at all time steps, and the sensor data is associated with these K sensor data. For each sensor, based on the association relationships between the various sensor data, a hypergraph corresponding to the sensor is constructed, i.e., a third hypergraph. Each third hypergraph has corresponding feature information in the sensor, i.e., first feature information, which characterizes the association relationships between the sensor data in the subset of the data corresponding to the sensor. Further, the hypergraph corresponding to the sensor is transformed to obtain the association matrix of the hypergraph, i.e., the third association matrix, to characterize the association relationships between the sensor data to be trained at each time step in the subset of the data corresponding to the sensor.

[0131] S405. The third correlation matrices of the third hypergraphs of each sensor are concatenated to obtain the first correlation matrix of the first hypergraph; wherein the first correlation matrix includes the first feature information of each sensor.

[0132] For example, after obtaining the third association matrix of the third hypergraph corresponding to each sensor, the third association matrices of the third hypergraphs of each sensor are concatenated to obtain the first association matrix of the first hypergraph corresponding to each sensor; wherein, the first association matrix includes the first feature information of each sensor to characterize the association relationship between the sensor data to be trained in each sensor.

[0133] S406. Based on the training data set, determine the sensor data belonging to the same time in the training data set, which are the hypergraph nodes at that time; wherein, the hypergraph nodes include the sensor data belonging to the same time under each sensor in the electromechanical system.

[0134] For example, based on the sensor data set collected by each sensor, i.e., the sensor data in the training dataset; or, based on the sensor data in the subset of data collected by each sensor, after Fourier transform processing, for each pair of data in the processed training dataset, for each acquisition time, the sensor data belonging to that acquisition time are analyzed and processed to determine the hypergraph node at that time.

[0135] S407. For each pair of hypergraph nodes, determine the second distance information between each pair of hypergraph nodes based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair; wherein, the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension.

[0136] For example, based on the sensors in the electromechanical system, a multi-dimensional sensor space is pre-established. The number of dimensions is determined by the number of sensors in the electromechanical system, and each dimension represents a sensor. For each obtained hypergraph node, it contains all the sensor data to be trained collected at the corresponding time. The position of the hypergraph node can be determined in the multi-dimensional sensor space. For each pair of hypergraph nodes, based on the sensor data to be trained for one hypergraph node in each pair and the sensor data to be trained for the other hypergraph node in each pair, the distance between the two positions corresponding to the pair of hypergraph nodes in the sensor space can be calculated, which is the second distance information between each pair of hypergraph nodes.

[0137] In one example, the second distance information is

[0138]

[0139] in, , , Let be the hypergraph node in the preset space at time k. Let be the hypergraph node in the preset space at time q. Let T be the sensor data collected by the T-th sensor at time k. Let be the sensor data collected by the T-th sensor at time q. Let be the total number of sensors, k be an integer greater than or equal to 1, q be an integer greater than or equal to 1, and T be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

[0140] For example, for three sensors in an electromechanical system, a sensor space P can be defined. T If each dimension represents a sensor, then in the sensor space P 3 In this context, each sensor data point represents a "coordinate value" along a specific dimension; that is, at each given moment, each sensor data point is a vector, not a numerical value. The multiple sensor data points at five different moments in the dataset to be processed can be represented as hypergraph nodes corresponding to the three sensor data points at that moment, corresponding to a point n in the sensor space R. TIn the diagram, the distance D between hypergraph nodes at any two time points in the sensor space is defined and calculated using the following formula:

[0141]

[0142] in, Let k be the hypergraph node in the multidimensional space corresponding to time k. Let be the hypergraph node in the multidimensional space corresponding to time q. In sensor space The coordinate value in the T-th dimension, In sensor space In the coordinate value of the T-th dimension, k is an integer greater than or equal to 1 and less than or equal to 5, q is an integer greater than or equal to 1 and less than or equal to 3, and T is an integer greater than or equal to 1. If the value is 3, then the calculated distance is the distance between the hypergraph nodes. and The second distance information between them.

[0143] S408. Based on each second distance information, the training data set is processed using the K-nearest neighbor method to obtain the second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph node; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

[0144] For example, based on the second distance information calculated in the sensor space, for each hypergraph node in the training dataset at each time step, using the K-nearest neighbor method, K second distance information with the smallest corresponding values ​​are selected from the second distance information related to the hypergraph node. Then, based on these K second distance information, sensor data corresponding to the K first and second distance information at K time steps are selected from the hypergraph nodes at all time steps, and the hypergraph node is associated with these K hypergraph nodes. Based on the association relationships between the hypergraph nodes at each time step, a hypergraph corresponding to the training dataset is constructed, namely a second hypergraph. The third hypergraph has corresponding feature information of the hypergraph nodes, namely second feature information, to characterize at least one second feature information of the sensor. This feature information characterizes the association relationships between the sensor data in the training dataset corresponding to each sensor. Further, the second hypergraph is transformed to obtain a second association matrix of the second hypergraph, to characterize the association relationships between the sensor data in the training dataset corresponding to each sensor.

[0145] S409. Input the training dataset and the first correlation matrix into the first hypergraph neural network branch of the initial model to obtain the first feature vector; wherein, the first feature vector represents the fusion feature of the training dataset and the first correlation matrix.

[0146] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network model can be pre-set, which is the initial model. A first hypergraph neural network branch can be set in this model. Sensor data collected by each sensor and the obtained first correlation matrix are input into the pre-set initial model. Based on the first hypergraph neural network branch of the model, calculation processing is performed to obtain the first feature vector of the training dataset, which represents the fusion feature between the training dataset and the first correlation matrix corresponding to the training dataset. Alternatively, the first hypergraph neural network branch of the model can be used to process the sensor data collected by each sensor after Fourier transform processing to obtain the first feature vector of the Fourier-processed training dataset, which represents the fusion feature between the Fourier-processed training dataset and the first correlation matrix corresponding to the processed training dataset.

[0147] In one example, step S409 includes the following steps:

[0148] The first step of step S409 is to input the training dataset and the first correlation matrix into the first hypergraph neural network branch of the initial model, and to perform convolution processing on the training dataset and the first correlation matrix to obtain the first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the training dataset and the first correlation matrix.

[0149] The second step of step S409 is to perform convolution processing on the first intermediate feature and the first correlation matrix to obtain the first feature vector.

[0150] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network can be pre-set, which is the initial model. In this model, a first hypergraph neural network branch can be set. Sensor data collected by each sensor and the obtained first correlation matrix are input into the preset initial model. Based on the first graph convolutional layer in the first hypergraph neural network branch of the model, convolution calculation is performed to obtain the first intermediate feature of the dataset to be trained, which represents the fusion feature between the dataset to be trained and the corresponding first correlation matrix. Then, based on the second graph convolutional layer in the first hypergraph neural network branch of the model, the intermediate feature and the corresponding first correlation matrix are further convolved to obtain the first feature vector of the dataset to be trained, which represents the fusion feature between the dataset to be trained and the corresponding first correlation matrix.

[0151] S410. Input the training dataset and the second correlation matrix into the second hypergraph neural network branch of the initial model to obtain the second feature vector; wherein, the second feature vector represents the fusion feature of the training dataset and the second correlation matrix.

[0152] For example, to determine faults in an electromechanical system, a model based on a hypergraph neural network model can be pre-set as an initial model. Sensor data collected by each sensor and the resulting second correlation matrix are input into this pre-set initial model. Based on the second hypergraph neural network branch of this model, calculations are performed to obtain the second feature vector of the dataset to be trained, which represents the fusion feature between the dataset to be trained and the corresponding second correlation matrix. Alternatively, the second hypergraph neural network branch of this model can be used to process the sensor data collected by each sensor after Fourier transform processing to obtain the second feature vector of the Fourier-processed dataset to be trained, which represents the fusion feature between the Fourier-processed dataset to be trained and the corresponding second correlation matrix.

[0153] In one example, step S410 includes the following steps:

[0154] The first step of step S410 is to input the training dataset and the second correlation matrix into the second hypergraph neural network branch of the initial model, and to perform convolution processing on the training dataset and the second correlation matrix to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the training dataset and the second correlation matrix.

[0155] The second step of step S410 involves convolving the second intermediate feature and the second correlation matrix to obtain the second feature vector.

[0156] For example, in order to determine faults in an electromechanical system, a model based on a hypergraph neural network model can be pre-set, which is the initial model. Sensor data collected by each sensor and the obtained second correlation matrix are input into the pre-set initial model. Based on the first graph convolutional layer in the second hypergraph neural network branch of the model, convolution calculation is performed to obtain the second intermediate feature of the dataset to be trained, which represents the fusion feature between the dataset to be trained and the corresponding second correlation matrix. Then, based on the second graph convolutional layer in the second hypergraph neural network branch of the model, the intermediate feature and the corresponding second correlation matrix are further convolved to obtain the second feature vector of the dataset to be trained, which represents the fusion feature between the dataset to be trained and the corresponding second correlation matrix.

[0157] S411. Connect the first feature vector and the second feature vector to obtain the feature vector to be processed.

[0158] For example, after convolution processing of the preset initial model, the obtained first feature processing vector and second feature processing vector are concatenated to obtain the feature vector to be processed corresponding to the training data set collected by each sensor.

[0159] For example, regarding the obtained first feature vector z sf Second feature processing vector z mf Perform a concat operation on the two to obtain z = Concat(z) sf , z mf ), which is the feature vector to be processed corresponding to the training data set collected by each sensor.

[0160] S412. Based on the initial model, identify the feature vector to be processed to obtain the identified fault type of the electromechanical system.

[0161] For example, after the two convolutional layers of the two hypergraph neural network branches in the initial model are processed and concatenated to obtain the feature vector to be processed, the feature vector to be trained is calculated and identified based on the multilayer perceptron in the preset initial model to obtain the fault type of the electromechanical system. For example, the electromechanical system is currently fault-free; or the current fault type of the electromechanical system is a type 1 fault or a type 2 fault.

[0162] S413. Based on the actual fault type and the identified fault type, adjust the parameters of the initial model to obtain the preset fault diagnosis model.

[0163] In one example, the dataset to be trained corresponds to actual fault types.

[0164] For example, after determining the fault type of the electromechanical system through the model, the actual fault type corresponding to the training data set is compared with the fault type determined by the initial model. Furthermore, the parameters of the initial model are optimized through the loss function of the initial model until the value of the loss function is minimized, indicating that the fault type output by the model is consistent with the actual fault type. This is the trained initial model, which is the preset fault diagnosis model.

[0165] In this embodiment, based on the above embodiments, on the one hand, for multiple sensor data within a historical time period of the electromechanical system, a single-sensor fusion hypergraph and its correlation matrix are constructed according to each sensor data to be trained, and a multi-sensor fusion hypergraph and its correlation matrix are constructed according to multiple sensor data to be trained; thereby enriching the input information for diagnosing electromechanical system faults and improving the accuracy of fault determination based on a small amount of sensor data to be trained; on the other hand, the single-sensor fusion hypergraph and its correlation matrix and the multi-sensor fusion hypergraph and its correlation matrix are used to sub-train the initial model to obtain a preset fault diagnosis model; then, using the hypergraph to calculate and train the model can simplify the training process, improve the efficiency of the algorithm, and use the model to determine the fault type of the electromechanical system.

[0166] Figure 8 This application provides a schematic diagram of the structure of a fault determination device based on a multi-sensor fusion hypergraph neural network, as shown in the embodiments of this application. Figure 8 As shown, the device 500 includes:

[0167] The acquisition unit 501 is used to acquire the set of data to be processed of the electromechanical system; wherein the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes the sensor data collected by the sensor at each moment.

[0168] The processing unit 502 is used to perform hypergraph calculation processing on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed.

[0169] The determination unit 503 is used to determine the fault type of the electromechanical system based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

[0170] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0171] Figure 9 A schematic diagram of another fault determination device based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 9 As shown, the device 600 includes:

[0172] The acquisition unit 601 is used to acquire the set of data to be processed of the electromechanical system; wherein the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes the sensor data collected by the sensor at each moment.

[0173] The processing unit 602 is used to perform hypergraph calculation processing on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed.

[0174] The determining unit 603 is used to determine the fault type of the electromechanical system based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

[0175] In one example, unit 603 is defined as including:

[0176] The first generation module 6031 is used to input the data set to be processed and the first correlation matrix into the first hypergraph neural network branch of the preset fault diagnosis model to obtain the first feature vector; wherein, the first feature vector represents the fusion feature of the data set to be processed and the first correlation matrix; the preset fault diagnosis model is a model obtained based on the hypergraph neural network model.

[0177] The second generation module 6032 is used to input the data set to be processed and the second correlation matrix into the second hypergraph neural network branch of the preset fault diagnosis model to obtain the second feature vector; wherein, the second feature vector represents the fusion feature of the data set to be processed and the second correlation matrix.

[0178] The connection module 6033 is used to connect the first feature vector and the second feature vector to obtain the feature vector to be processed.

[0179] The identification module 6034 is used to identify the feature vector to be processed based on a preset fault diagnosis model to obtain the fault type of the electromechanical system.

[0180] In one example, the first generation module 6031 includes:

[0181] The first processing submodule is used to input the data set to be processed and the first correlation matrix into the first hypergraph neural network branch of the preset fault diagnosis model, and to perform convolution processing on the data set to be processed and the first correlation matrix to obtain the first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the data set to be processed and the first correlation matrix.

[0182] The first generation submodule is used to perform convolution processing on the first intermediate feature and the first correlation matrix to obtain the first feature vector.

[0183] In one example, the second generation module 6032 includes:

[0184] The second processing submodule is used to input the data set to be processed and the second correlation matrix into the second hypergraph neural network branch of the preset fault diagnosis model, and to perform convolution processing on the data set to be processed and the second correlation matrix to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the data set to be processed and the second correlation matrix.

[0185] The second generation submodule is used to perform convolution processing on the second intermediate feature and the second correlation matrix to obtain the second feature vector.

[0186] In one example, processing unit 602 includes:

[0187] The first determining module 6021 is used to determine, for each subset of data from a sensor in the dataset to be processed, a first distance information between each pair of sensor data in that subset; wherein the first distance information represents the data difference between a pair of sensor data.

[0188] The first processing module 6022 is used to process the data subset of each sensor in the dataset to be processed based on the first distance information corresponding to the data subset of the sensor and the K-nearest neighbor method to obtain the third correlation matrix of the third hypergraph of the sensor; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between the sensor data at each time.

[0189] The first stitching module 6023 is used to stitch together the third correlation matrix of the third hypergraph of each sensor to obtain the first correlation matrix of the first hypergraph; wherein, the first correlation matrix includes the first feature information of each sensor.

[0190] In one example, processing unit 602 further includes:

[0191] The second determining module 6024 is used to determine, based on the data set to be processed, the sensor data belonging to the same time in the data set to be processed, as the super graph node at that time; wherein, the super graph node includes the sensor data belonging to the same time under each sensor in the electromechanical system.

[0192] The second processing module 6025 is used to determine, for each pair of hypergraph nodes, a second distance information between each pair of hypergraph nodes based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair of hypergraph nodes; wherein, the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension.

[0193] The second splicing module 6026 is used to process the dataset to be processed based on the K-nearest neighbor method according to each second distance information to obtain the second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph node; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

[0194] In one example, the second distance information is

[0195]

[0196] in, , , Let be the hypergraph node in the preset space at time i. Let be the hypergraph node in the preset space at time j. Let S be the sensor data collected by the S-th sensor at time i. Let S be the sensor data collected by the S-th sensor at time j. Let be the total number of sensors, i be an integer greater than or equal to 1, j be an integer greater than or equal to 1, and S be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

[0197] In one example, before the processing unit 602 performs hypergraph computation on the dataset to be processed to obtain the first association matrix of the first hypergraph and the second association matrix of the second hypergraph, the apparatus 600 further includes:

[0198] The transformation unit 604 is used to perform Fourier transform processing on the sensor data in the dataset to be processed, so as to obtain the processed dataset.

[0199] In one example, device 600 also includes:

[0200] The alarm unit 605 is used to generate alarm information and issue alarm information according to the fault type of the electromechanical system; wherein the alarm information is one or more of the following: voice information, text information, image information, video information, and light information.

[0201] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0202] Figure 10 This application provides a schematic diagram of the structure of a model training device based on a multi-sensor fusion hypergraph neural network, as shown in the embodiments of this application. Figure 10 As shown, the device 700 includes:

[0203] The acquisition unit 701 is used to acquire the training data set of the electromechanical system; wherein the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes the sensor data collected by the sensor at each moment.

[0204] The processing unit 702 is used to perform hypergraph computation processing on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained.

[0205] Training unit 703 is used to train the initial model based on the dataset to be trained, the first correlation matrix, and the second correlation matrix to obtain a preset fault diagnosis model; wherein, the initial model is a model obtained based on the hypergraph neural network model.

[0206] The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the set of data to be processed in the fault determination device based on multi-sensor fusion hypergraph neural network.

[0207] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0208] Figure 11 A schematic diagram of another model training device based on a multi-sensor fusion hypergraph neural network provided in this application embodiment is shown below. Figure 11 As shown, the device 800 includes:

[0209] The acquisition unit 801 is used to acquire the training data set of the electromechanical system; wherein the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes the sensor data collected by the sensor at each moment.

[0210] The processing unit 802 is used to perform hypergraph computation processing on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained.

[0211] Training unit 803 is used to train the initial model based on the dataset to be trained, the first correlation matrix, and the second correlation matrix to obtain a preset fault diagnosis model; wherein, the initial model is a model obtained based on the hypergraph neural network model.

[0212] The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the set of data to be processed in the fault determination device based on multi-sensor fusion hypergraph neural network.

[0213] In one example, the dataset to be trained corresponds to actual fault types; training unit 803 includes:

[0214] The first generation module 8031 ​​is used to input the training dataset and the first correlation matrix into the first hypergraph neural network branch of the initial model to obtain the first feature vector; wherein the first feature vector represents the fusion feature of the training dataset and the first correlation matrix.

[0215] The second generation module 8032 is used to input the training dataset and the second correlation matrix into the second hypergraph neural network branch of the initial model to obtain the second feature vector; wherein the second feature vector represents the fusion feature of the training dataset and the second correlation matrix.

[0216] The connection module 8033 is used to connect the first feature vector and the second feature vector to obtain the feature vector to be processed.

[0217] The identification module 8034 is used to identify the feature vector to be processed based on the initial model, and to obtain the identified fault type of the electromechanical system.

[0218] The adjustment module 8035 is used to adjust the parameters of the initial model according to the actual fault type and the identified fault type to obtain the preset fault diagnosis model.

[0219] In one example, the first generation module 8031 ​​includes:

[0220] The first processing submodule is used to input the training dataset and the first correlation matrix into the first hypergraph neural network branch of the initial model, and to perform convolution processing on the training dataset and the first correlation matrix to obtain the first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the training dataset and the first correlation matrix.

[0221] The first generation submodule is used to perform convolution processing on the first intermediate feature and the first correlation matrix to obtain the first feature vector.

[0222] In one example, the second generation module 8032 includes:

[0223] The second processing submodule is used to input the training dataset and the second correlation matrix into the second hypergraph neural network branch of the initial model, and to perform convolution processing on the training dataset and the second correlation matrix to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the training dataset and the second correlation matrix.

[0224] The second generation submodule is used to perform convolution processing on the second intermediate feature and the second correlation matrix to obtain the second feature vector.

[0225] In one example, processing unit 802 includes:

[0226] The first determining module 8021 is used to determine, for each subset of data from a sensor in the dataset to be trained, a first distance information between each pair of sensor data in that subset; wherein the first distance information represents the data difference between a pair of sensor data.

[0227] The first processing module 8022 is used to process the data subset of each sensor in the dataset to be trained based on the first distance information corresponding to the data subset of the sensor and the K-nearest neighbor method to obtain the third correlation matrix of the third hypergraph of the sensor; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between the sensor data at each time.

[0228] The first stitching module 8023 is used to stitch together the third correlation matrix of the third hypergraph of each sensor to obtain the first correlation matrix of the first hypergraph; wherein the first correlation matrix includes the first feature information of each sensor.

[0229] In one example, processing unit 802 includes:

[0230] The second determining module 8024 is used to determine, based on the training data set, the sensor data belonging to the same time in the training data set, as the hypergraph node at that time; wherein, the hypergraph node includes the sensor data belonging to the same time under each sensor in the electromechanical system.

[0231] The second processing module 8025 is used to determine the second distance information between each pair of hypergraph nodes based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair; wherein the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension.

[0232] The second splicing module 8026 is used to process the training data set based on the K-nearest neighbor method according to each second distance information to obtain the second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph node; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

[0233] In one example, the second distance information is

[0234]

[0235] in, , , Let be the hypergraph node in the preset space at time k. Let be the hypergraph node in the preset space at time q. Let T be the sensor data collected by the T-th sensor at time k. Let be the sensor data collected by the T-th sensor at time q. Let be the total number of sensors, k be an integer greater than or equal to 1, q be an integer greater than or equal to 1, and T be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

[0236] In one example, before the processing unit 802 performs hypergraph computation on the dataset to be trained to obtain the first association matrix of the first hypergraph and the second association matrix of the second hypergraph, the apparatus 800 further includes:

[0237] The transformation unit 804 is used to perform Fourier transform processing on the sensor data in the training dataset to obtain the processed training dataset.

[0238] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0239] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 12 As shown, the electronic device 900 includes: a memory 901 and a processor 902; the memory 901 is a memory for storing executable instructions of the processor 902.

[0240] The processor 902 is configured to perform the method provided in the above embodiments.

[0241] The terminal device also includes a receiver 903 and a transmitter 904. The receiver 903 is used to receive instructions and data sent by other devices, and the transmitter 904 is used to send instructions and data to external devices.

[0242] Figure 13 This is a block diagram illustrating an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0243] The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0244] Processing component 1002 typically controls the overall operation of device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0245] Memory 1004 is configured to store various types of data to support the operation of device 1000. Examples of such data include instructions for any application or method operating on device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0246] Power supply component 1006 provides power to various components of device 1000. Power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1000.

[0247] Multimedia component 1008 includes a screen that provides an output interface between device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0248] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0249] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0250] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of device 1000. For example, sensor assembly 1014 may detect the on / off state of device 1000, the relative positioning of components such as the display and keypad of device 1000, changes in the position of device 1000 or a component of device 1000, the presence or absence of user contact with device 1000, the orientation or acceleration / deceleration of device 1000, and temperature changes of device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0251] Communication component 1016 is configured to facilitate wired or wireless communication between device 1000 and other devices. Device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0252] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0253] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of the device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0254] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described method.

[0255] This application also provides a computer program product, which includes: a computer program stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the solution provided in any of the above embodiments.

[0256] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0257] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A fault determination method based on a multi-sensor fusion hypergraph neural network, characterized in that, The method includes: Acquire a set of data to be processed from an electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes sensor data collected by the sensor at each moment; Hypergraph computation is performed on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed. The fault type of the electromechanical system is determined based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

2. The method of claim 1, wherein, Based on the dataset to be processed, the first correlation matrix, and the second correlation matrix, the fault type of the electromechanical system is determined, including: The data set to be processed and the first correlation matrix are input into the first hypergraph neural network branch of the preset fault diagnosis model to obtain a first feature vector; wherein, the first feature vector represents the fusion feature of the data set to be processed and the first correlation matrix; the preset fault diagnosis model is a model obtained based on the hypergraph neural network model; The data set to be processed and the second correlation matrix are input into the second hypergraph neural network branch of the preset fault diagnosis model to obtain the second feature vector; wherein, the second feature vector represents the fusion feature of the data set to be processed and the second correlation matrix; The first feature vector and the second feature vector are concatenated to obtain the feature vector to be processed; Based on the preset fault diagnosis model, the feature vector to be processed is identified to obtain the fault type of the electromechanical system.

3. The method according to claim 2, characterized in that, The dataset to be processed and the first correlation matrix are input into the first hypergraph neural network branch of the preset fault diagnosis model to obtain the first feature vector, including: The data set to be processed and the first correlation matrix are input into the first hypergraph neural network branch of the preset fault diagnosis model, and the data set to be processed and the first correlation matrix are convolved to obtain the first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the data set to be processed and the first correlation matrix. The first intermediate feature and the first correlation matrix are convolved to obtain the first feature vector.

4. The method according to claim 2, characterized in that, The dataset to be processed and the second correlation matrix are input into the second hypergraph neural network branch of the preset fault diagnosis model to obtain the second feature vector, including: The data set to be processed and the second correlation matrix are input into the second hypergraph neural network branch of the preset fault diagnosis model, and the data set to be processed and the second correlation matrix are convolved to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the data set to be processed and the second correlation matrix; The second intermediate feature and the second correlation matrix are convolved to obtain the second feature vector.

5. The method according to claim 1, characterized in that, Perform hypergraph computation on the dataset to be processed to obtain the first association matrix of the first hypergraph, including: For each subset of data from each sensor in the dataset to be processed, a first distance information is determined between each pair of sensor data in that subset; wherein, the first distance information represents the data difference between a pair of sensor data. For each data subset of the sensor in the dataset to be processed, based on the first distance information corresponding to the data subset of the sensor, the data subset of the sensor is processed according to the K-nearest neighbor method to obtain the third correlation matrix of the third hypergraph of the sensor; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between the sensor data at each time point; The third correlation matrices of the third hypergraphs of each sensor are concatenated to obtain the first correlation matrix of the first hypergraph; wherein the first correlation matrix includes the first feature information of each sensor.

6. The method according to claim 1, characterized in that, The dataset to be processed is subjected to hypergraph computation to obtain the second association matrix of the second hypergraph, including: Based on the set of data to be processed, sensor data belonging to the same time in the set of data to be processed are identified as hypergraph nodes at that time; wherein, the hypergraph nodes include sensor data belonging to the same time under each sensor in the electromechanical system; For each pair of hypergraph nodes, a second distance information between each pair of hypergraph nodes is determined based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair; wherein, the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension. Based on the second distance information, the dataset to be processed is processed using the K-nearest neighbor method to obtain a second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph nodes; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

7. The method according to claim 6, characterized in that, The second distance information is in, , , Let be the hypergraph node in the preset space at time i. Let be the hypergraph node in the preset space at time j. Let S be the sensor data collected by the S-th sensor at time i. Let S be the sensor data collected by the S-th sensor at time j. Let be the total number of sensors, i be an integer greater than or equal to 1, j be an integer greater than or equal to 1, and S be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

8. The method according to any one of claims 1-7, characterized in that, Before performing hypergraph computation on the dataset to be processed to obtain the first association matrix of the first hypergraph and the second association matrix of the second hypergraph, the process further includes: The sensor data in the dataset to be processed is subjected to Fourier transform to obtain the processed dataset.

9. The method according to any one of claims 1-7, characterized in that, The method further includes: An alarm message is generated and issued according to the fault type of the electromechanical system; wherein the alarm message is one or more of the following: voice message, text message, image message, video message, and light message.

10. A model training method based on a multi-sensor fusion hypergraph neural network, characterized in that, The method includes: Obtain a training data set for the electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes sensor data collected by the sensor at each moment; Hypergraph computation is performed on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained. Based on the dataset to be trained, the first correlation matrix, and the second correlation matrix, the initial model is trained to obtain a preset fault diagnosis model; wherein, the initial model is a model obtained based on a hypergraph neural network model; The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the data set to be processed in the fault determination method according to any one of claims 1-9.

11. The method according to claim 10, characterized in that, The dataset to be trained corresponds to actual fault types; based on the dataset to be trained, the first correlation matrix, and the second correlation matrix, the initial model is trained to obtain a preset fault diagnosis model, including: The training dataset and the first correlation matrix are input into the first hypergraph neural network branch of the initial model to obtain a first feature vector; wherein, the first feature vector represents the fusion feature of the training dataset and the first correlation matrix; The training dataset and the second correlation matrix are input into the second hypergraph neural network branch of the initial model to obtain a second feature vector; wherein the second feature vector represents the fusion feature of the training dataset and the second correlation matrix; The first feature vector and the second feature vector are concatenated to obtain the feature vector to be processed; Based on the initial model, the feature vector to be processed is identified to obtain the identified fault type of the electromechanical system; Based on the actual fault type and the identified fault type, the parameters of the initial model are adjusted to obtain the preset fault diagnosis model.

12. The method according to claim 11, characterized in that, The training dataset and the first correlation matrix are input into the first hypergraph neural network branch of the initial model to obtain the first feature vector, including: The training dataset and the first correlation matrix are input into the first hypergraph neural network branch of the initial model, and convolution processing is performed on the training dataset and the first correlation matrix to obtain a first intermediate feature; wherein, the first intermediate feature represents the fusion feature of the training dataset and the first correlation matrix; The first intermediate feature and the first correlation matrix are convolved to obtain the first feature vector.

13. The method according to claim 11, characterized in that, The training dataset and the second correlation matrix are input into the second hypergraph neural network branch of the initial model to obtain the second feature vector, including: The training dataset and the second correlation matrix are input into the second hypergraph neural network branch of the initial model, and convolution processing is performed on the training dataset and the second correlation matrix to obtain the second intermediate feature; wherein, the second intermediate feature represents the fusion feature of the training dataset and the second correlation matrix; The second intermediate feature and the second correlation matrix are convolved to obtain the second feature vector.

14. The method according to claim 10, characterized in that, The dataset to be trained is subjected to hypergraph computation to obtain the first correlation matrix of the first hypergraph, including: For each subset of data from each sensor in the dataset to be trained, a first distance information is determined between each pair of sensor data in that subset; wherein the first distance information represents the data difference between a pair of sensor data. For each sensor's data subset in the dataset to be trained, based on the first distance information corresponding to that sensor's data subset, the data subset of that sensor is processed using the K-nearest neighbor method to obtain the third correlation matrix of the sensor's third hypergraph; wherein, the third hypergraph represents at least one first feature information of the sensor; the first feature information represents the correlation relationship between sensor data at each time step; The third correlation matrices of the third hypergraphs of each sensor are concatenated to obtain the first correlation matrix of the first hypergraph; wherein the first correlation matrix includes the first feature information of each sensor.

15. The method according to claim 10, characterized in that, The dataset to be trained is subjected to hypergraph computation to obtain the second correlation matrix of the first hypergraph, including: Based on the set of data to be trained, sensor data belonging to the same time in the set of data to be trained are identified as hypergraph nodes at that time; wherein, the hypergraph nodes include sensor data belonging to the same time under each sensor in the electromechanical system; For each pair of hypergraph nodes, a second distance information between each pair of hypergraph nodes is determined based on the sensor data of one hypergraph node in each pair and the sensor data of the other hypergraph node in each pair; wherein, the second distance information represents the distance between a pair of hypergraph nodes in a preset space; the preset space is a multi-dimensional space established with the number of sensors in the electromechanical system as the dimension. Based on the second distance information, the training dataset is processed using the K-nearest neighbor method to obtain a second association matrix of the second hypergraph; wherein, the second hypergraph represents at least one second feature information of the hypergraph nodes; the second feature information represents the association relationship between the hypergraph nodes; the second association matrix includes each second feature information of each hypergraph node.

16. The method according to claim 15, characterized in that, The second distance information is in, , , Let be the hypergraph node in the preset space at time k. Let be the hypergraph node in the preset space at time q. Let T be the sensor data collected by the T-th sensor at time k. Let be the sensor data collected by the T-th sensor at time q. Let be the total number of sensors, k be an integer greater than or equal to 1, q be an integer greater than or equal to 1, and T be an integer greater than or equal to 1. It is an integer greater than or equal to 1.

17. The method according to any one of claims 10-16, characterized in that, Before performing hypergraph computation on the dataset to be trained to obtain the first correlation matrix of the first hypergraph and the second correlation matrix of the second hypergraph, the process further includes: The sensor data in the dataset to be trained is subjected to Fourier transform processing to obtain the processed dataset to be trained.

18. A fault determination device based on a multi-sensor fusion hypergraph neural network, characterized in that, The device includes: An acquisition unit is used to acquire a set of data to be processed from an electromechanical system; wherein, the set of data to be processed includes a subset of data from each sensor in the electromechanical system, and the subset of data includes sensor data collected by the sensor at each moment; A processing unit is configured to perform hypergraph computation on the dataset to be processed to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be processed, and the second association matrix includes the association relationship between sensor data in the dataset to be processed. The determining unit is used to determine the fault type of the electromechanical system based on the data set to be processed, the first correlation matrix, and the second correlation matrix.

19. A model training device based on a multi-sensor fusion hypergraph neural network, characterized in that, The device includes: An acquisition unit is used to acquire a training data set of an electromechanical system; wherein, the training data set includes a data subset of each sensor in the electromechanical system, and the data subset includes sensor data collected by the sensor at each moment; A processing unit is configured to perform hypergraph computation on the dataset to be trained to obtain a first association matrix of the first hypergraph and a second association matrix of the second hypergraph; wherein, the first hypergraph represents the association relationship between sensor data in the data subset, and the first association matrix includes the association relationship between sensor data in each data subset; the second hypergraph represents the association relationship between sensor data in the dataset to be trained, and the second association matrix includes the association relationship between sensor data in the dataset to be trained. The training unit is used to train the initial model based on the dataset to be trained, the first correlation matrix, and the second correlation matrix to obtain a preset fault diagnosis model; wherein the initial model is a model obtained based on a hypergraph neural network model; The preset fault diagnosis model is used to identify the fault type of the electromechanical system from the data set to be processed in the fault determination device of claim 18.

20. An electronic device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-17.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-17.

22. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-17.

Citation Information

Patent Citations

  • Hypergraph neural network classification method and device

    CN113792768A

  • Network fault prediction method and device based on self-supervised hypergraph

    CN115587612A