Assembly line fault identification method, system, equipment and medium

Abstract

The invention provides an assembly line fault identification method and system, and belongs to the technical field of assembly line fault identification, and the method comprises the steps: obtaining the multi-mode monitoring data of an assembly line; for the obtained multi-modal monitoring data, fusing the multi-modal monitoring data through a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to different dimensions of data in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; based on the fusion data, a pre-trained space-time adaptive multi-modal analysis network model is utilized to obtain a fault identification result; wherein the space-time adaptive multi-modal analysis network model specifically executes the following processing processes: based on the fusion data, respectively extracting features of the fusion data on different time scales and distribution and relation of the fusion data on a spatial dimension, and obtaining time features and spatial features; and performing dynamic fusion on the time features and the space features to obtain fusion features.

Description

Technical Field

[0001] The present invention belongs to the technical field of pipeline fault identification, and in particular relates to a pipeline fault identification method, system, equipment and medium. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of industrial automation technology, the application of automated verification lines in production is becoming more and more extensive. However, various faults often occur in the long-term operation of the lines, which not only affect production efficiency, but also increase the work pressure of operation and maintenance personnel.

[0004] Traditional monitoring and maintenance methods often rely on manual judgment, which is inefficient and prone to errors. Therefore, in order to improve the operation and maintenance efficiency and intelligence level of the assembly line, reduce labor costs, and achieve more efficient production management, a new technical solution is urgently needed to achieve multi-parameter perception, automatic fault identification, and unmanned operation and maintenance of the assembly line.

[0005] The inventors have found that existing fault monitoring methods usually have the following technical problems: the existing technology is unable to comprehensively process multimodal data, resulting in limited monitoring accuracy and efficiency; fault identification and prediction are inaccurate, relying on simple threshold judgments or basic data analysis, making it difficult to effectively prevent or take maintenance measures in advance; lack of adaptive and dynamic optimization capabilities, unable to adjust strategies according to environmental changes, affecting the timeliness and accuracy of responses; high dependence on operation and maintenance, over-reliance on personnel experience and judgment, increasing labor costs, and reducing operation and maintenance efficiency and accuracy. Summary of the invention

[0006] Embodiments of the present invention provide a pipeline fault identification method, system, device and medium to solve the problem that the prior art cannot comprehensively process multimodal data, resulting in limited monitoring accuracy and efficiency; the problem that the prior art relies on simple threshold judgment or basic data analysis, making it difficult to effectively prevent or take maintenance measures in advance; and the problem that the prior art lacks adaptive and dynamic optimization capabilities and cannot adjust strategies according to environmental changes, affecting the timeliness and accuracy of the response.

[0007] According to a first aspect of an embodiment of the present invention, a pipeline fault identification method is provided, comprising: Obtain multimodal monitoring data of the pipeline; For the obtained multimodal monitoring data, the multimodal monitoring data is fused by a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the data of different dimensions in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; Based on the fused data, a pre-trained spatiotemporal adaptive multimodal analysis network model is used to obtain a fault identification result; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing procedures: based on the fused data, its features at different time scales and its distribution and relationship in the spatial dimension are extracted respectively to obtain time features and space features; by dynamically fusing the time features and space features, a fused feature is obtained; based on the comparison result of the fused feature with the preset benchmark data, the fault is identified.

[0008] Furthermore, for the data fusion strategy, the current data fusion strategy is evaluated based on the conditional information entropy, and the data fusion strategy is dynamically optimized based on the evaluation result.

[0009] Furthermore, the extraction of the time feature is specifically as follows: based on the fused data, based on the results of Gaussian modes and trigonometric functions, the features of the fused data at different time scales are obtained.

[0010] Furthermore, the extraction of the spatial features is specifically as follows: based on the fused data, a spatial feature reconstruction network is constructed through a logistic function, the spatial features of the fused data are reconstructed, and the distribution and relationship of the fused data in the spatial dimension are obtained.

[0011] Furthermore, the dynamic weighted fusion is specifically: generating adaptive dynamic weights based on spatial features and temporal features, and fusing temporal features and spatial features based on the obtained dynamic weights.

[0012] Furthermore, the preset data fusion strategy specifically includes: converting the obtained original multimodal monitoring data into a new dimensional space through nonlinear mapping to obtain a new data feature set; performing feature extraction on the data feature set, performing tensor decomposition based on the extracted features, and obtaining features of different dimensions; performing weighted fusion based on the obtained features of different dimensions to obtain fused data.

[0013] Furthermore, the obtained original multimodal monitoring data is converted into a new dimensional space through nonlinear mapping, which is specifically expressed as:

[0014] in, Represents the mapped data. represents the i-th data point in the original data, is the amplitude coefficient of the kth Gaussian function, is the center position of the kth Gaussian function, is the standard deviation of the kth Gaussian function, yes The coefficient of It is The exponent of the polynomial, and are the constant offset and the linear coefficient.

[0015] Furthermore, the fused data is specifically expressed as follows:

[0016] in, represents the fused data set, represents the data vector of the i-th dimension in the r-th decomposition pattern, is the weight of the rth decomposition mode, represents the fusion weight.

[0017] Furthermore, the data fusion weights corresponding to the data of different dimensions in the data fusion strategy are dynamically adjusted based on the distance between the sensor data. The data fusion weights are specifically expressed as follows:

[0018] in, is the dynamic weight between the pth and qth data points at time t, , , is the adjustment parameter of the weight formula, represents the distance between the pth and qth data points, represents the fused value of the p-th and q-th data points at time t.

[0019] Furthermore, the multimodal monitoring data includes light intensity, sound frequency, temperature, current and voltage changes, and equipment movement.

[0020] According to a second aspect of an embodiment of the present invention, a pipeline fault identification system is provided, comprising: A data acquisition unit, which is used to acquire multimodal monitoring data of the pipeline; A data fusion unit, which is used to fuse the obtained multimodal monitoring data through a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the different dimensional data in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; A fault identification unit is used to obtain a fault identification result based on the fused data using a pre-trained spatiotemporal adaptive multimodal analysis network model; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing procedures: based on the fused data, extracting its features at different time scales and its distribution and relationship in the spatial dimension to obtain time features and space features; obtaining fused features by dynamically fusing the time features and space features; and realizing fault identification based on the comparison results of the fused features with preset benchmark data.

[0021] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored and running on the memory, wherein the processor implements the pipeline fault identification method when executing the program.

[0022] According to a fourth aspect of an embodiment of the present invention, there is provided a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the pipeline fault identification method when executed by a processor.

[0023] One or more of the above technical solutions have the following beneficial effects: (1) The present invention provides a method and system for identifying pipeline faults. The scheme collects data in real time through a multimodal sensor network and uses data fusion technology to accurately monitor the operating status of the pipeline from multiple dimensions, which not only improves the accuracy of data collection, but also improves the analysis efficiency through intelligent processing. By analyzing the fused data set, it is possible to effectively identify and predict abnormal patterns and potential faults on the pipeline, making it possible to take maintenance measures in advance, thereby preventing the occurrence of serious faults. At the same time, the data fusion strategy and feature fusion in the scheme described in this embodiment both adopt an adaptive dynamic fusion method, which can effectively respond to environmental changes in the pipeline.

[0024] (2) The present invention provides a dynamic data fusion strategy that can effectively cope with the dynamic changes in the pipeline operating environment. By adjusting the weight of data fusion through dynamic weight adjustment, it can achieve self-adaptation to changes in the pipeline environment. At the same time, in order to identify and reduce the uncertainty in the data fusion process, especially when processing complex and changeable pipeline data, the conditional information entropy formula is used to guide the data fusion strategy, and the calculation of information entropy is used to evaluate the uncertainty of the data. By minimizing the information entropy, the information loss in the fusion process can be reduced and the data fusion strategy can be optimized.

[0025] (3) The present invention uses a combination of Gaussian modes and trigonometric functions to capture the characteristics of the fused data set at different time scales, and reveals subtle changes in the data in the time dimension by adjusting the Gaussian mode parameters (mean, standard deviation) and trigonometric function parameters (period). By fusing the data features of the time and space dimensions together and using a dynamic weight system to conduct in-depth analysis of the data, the potential patterns and information in the multimodal data are revealed, effectively ensuring the accuracy of subsequent fault identification.

[0026] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0028] Figure 1 A flow chart of a pipeline fault identification method described in an embodiment of the present invention; Figure 2 This is a structural diagram of a pipeline fault identification system described in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0030] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0031] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0032] like Figure 1 As shown, an embodiment of the present invention provides a pipeline fault identification method, which specifically includes the following processing procedures: Step 1: Obtain multimodal monitoring data of the assembly line, wherein the multimodal monitoring data includes light intensity, sound frequency, temperature, current and voltage changes, and equipment movement; In the specific implementation, in order to obtain real-time data, the following process is performed: First, user demand analysis is performed to determine the key parameters for pipeline monitoring. According to the needs, corresponding sensors are selected and arranged, such as optical, acoustic, thermodynamic and electrical parameter sensors, to form a multimodal sensor network.

[0033] For automated verification lines, data mainly comes from multimodal sensor networks. Multimodal sensors are not limited to traditional optical, acoustic, and thermodynamic sensors, but also include electrical parameter signal sensors and audio and video equipment that are unique to the line. These devices are placed at key locations on the line to capture key indicators such as mechanical motion and electrical performance.

[0034] The multimodal sensor network collects raw data in real time, such as light intensity, sound frequency, temperature, current and voltage changes, equipment movement, etc., and transmits the raw data to the central processing system in real time.

[0035] Step 2: For the obtained multimodal monitoring data, the multimodal monitoring data is fused through a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the different dimensional data in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; Step 3: Based on the fused data, a pre-trained spatiotemporal adaptive multimodal analysis network model is used to obtain a fault identification result; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing: based on the fused data, the features at different time scales and their distribution and relationship in the spatial dimension are extracted to obtain time features and space features; fusion features are obtained by dynamically fusing the time features and space features; and fault identification is achieved based on the comparison results between the fusion features and the preset benchmark data.

[0036] In this way, the above scheme of this embodiment collects data in real time through a multimodal sensor network, and uses data fusion technology to accurately monitor the operating status of the assembly line from multiple dimensions, which not only improves the accuracy of data collection, but also improves the analysis efficiency through intelligent processing; by analyzing the fused data set, it is possible to effectively identify and predict abnormal patterns and potential faults on the assembly line, making it possible to take maintenance measures in advance, thereby preventing the occurrence of serious faults; at the same time, the data fusion strategy and feature fusion in the scheme described in this embodiment both adopt an adaptive dynamic fusion method, which can effectively respond to environmental changes in the assembly line.

[0037] In one or more embodiments, the fusion of multimodal monitoring data is performed by using a preset data fusion strategy, and specifically the following data fusion strategy is adopted: The obtained original multimodal monitoring data is converted into a new dimensional space through nonlinear mapping to obtain a new data feature set; Extracting features from the data feature set, and performing tensor decomposition based on the extracted features to obtain features of different dimensions; Perform weighted fusion based on the features of different dimensions to obtain fused data; Specifically, multi-modal data collection is performed in real time based on the multi-modal sensor network deployed on the assembly line to achieve multi-dimensional perception, and multi-dimensional data fusion processing is performed through the tensor decomposition method. At the same time, by adjusting the weight of data fusion and calculating the information entropy value, the uncertainty under the current data fusion strategy is evaluated and the data fusion strategy is optimized; Specifically: In order to increase the distinction between data and provide more abundant and distinct features for subsequent data fusion, the original data is converted into a new dimensional space through a nonlinear mapping function. The mapping formula is:

[0038] in, Represents the mapped data. represents the i-th data point in the original data, is the amplitude coefficient of the kth Gaussian function, which is used to adjust the contribution of the Gaussian term to the mapping result. is the center position of the kth Gaussian function, which determines the position of the Gaussian function within the data range. is the standard deviation of the kth Gaussian function, affecting the width of the function, yes The coefficient of adjustment The impact on the mapping results, It is The exponent of the polynomial is used to determine the shape of the polynomial function. and are the constant offset and the linear coefficient.

[0039] Raw data Generate a new data feature set through nonlinear mapping transformation , increasing the data's differentiation and expressiveness. Perform feature extraction, which uses existing techniques, such as dimensionality reduction techniques (such as principal component analysis PCA) or feature selection methods, to extract the most useful information. The extracted features are used as multidimensional data vectors required for tensor decomposition , according to different dimensions (such as time, space, sensor type), the feature data is organized into corresponding vector forms. Multi-dimensional data fusion processing is performed through tensor decomposition method, and the tensor representation method is:

[0040] in, Represents the fused data set, which is multi-dimensional and contains data from different time points, different spatial locations, and different sensor types. represents the fusion weight, represents the data vector of the i-th dimension in the r-th decomposition pattern, is the weight of the rth decomposition mode. In tensor decomposition, accumulation is used to integrate data from different sources. For example, data collected by different sensors in the pipeline (such as optical, acoustic, and thermodynamic sensors) will be accumulated. The information from each independent data source is aggregated into a collection, with each source contributing a part of the information. Accumulation is used for deep integration of data, correlating data of different dimensions (such as time series, spatial location, and sensor type) with each other. It not only retains the independent information of each data source, but also reveals the correlation between them.

[0041] Furthermore, in the specific implementation, considering the dynamic changes of the pipeline operation environment, the weight of data fusion is adjusted through dynamic weight adjustment; the specific processing process is as follows: Dynamically adjust the importance of data according to the changes in the actual monitoring line operation environment to enhance the adaptability and accuracy of data fusion, such as factors that may affect sensor data, such as equipment wear and temperature fluctuations. Dynamic weight adjustment is to adjust the weight of data fusion according to the real-time distance between sensors. The weight formula is:

[0042] in, is the dynamic weight between the pth and qth data points at time t, , , is the adjustment parameter of the weight formula, represents the distance between the pth and qth data points, represents the fused value of the p-th and q-th data points at time t.

[0043] The following should be noted: and There are two different parameters, specifically: It is a time-related weight coefficient used to adjust the contribution of data fusion at different time points. This is a global time weight, which is used to adjust the data weights at different time points in the entire tensor decomposition process, reflecting the impact of time dynamic changes on data fusion results.

[0044] The dynamic weight between the pth and qth data points at time t. This weight is used to dynamically adjust the data fusion according to the actual monitoring environment of the sensor data. Specifically, it takes into account the distance between the data points and their fusion value at the current time point.

[0045] For example: There is an automated assembly line system that is equipped with multiple sensors (optical, acoustic, thermodynamic, and electrical parameter sensors) to monitor the operation status of the assembly line. The data from these sensors needs to be fused to obtain a comprehensive monitoring result. , , ) sensor data is collected, assuming that there are different sensor data points at each moment , data points It can be temperature, light intensity, current, voltage, etc. The data points of different sensors are initially fused to generate a mapped feature data set .

[0046] In order to reflect the importance of data at different times in the fusion results, the time-related weight coefficient is introduced. For example, the data at a certain moment has a greater impact on the monitoring results, so you can set different value to reflect this effect.

[0047] Assumptions: =0.5, =1.0, =0.8 The tensor decomposition formula reflects the importance weighting of data at different time points. For example, time The data has the greatest impact on the fusion results.

[0048] Dynamic Weight Used to weight data points and reflect the real-time correlation between data points. The following data points were collected: the distance between data point p (temperature) and data point q (voltage) = 0.1, data points p and q at time The fusion value =2.5.

[0049] According to the weight calculation formula:

[0050] It can be calculated The specific value of this weight reflects the data points p and q in time If the distance between two data points is close and the fusion value is large, then are also larger, indicating that they contribute more to the fusion results.

[0051] In summary, : Used for weighting of the time dimension, reflecting the importance of data at different times in the fusion result. For example, at some critical moments (such as when the device starts or stops), the importance of data may be higher.

[0052] :Used for dynamic weighting between data points to reflect the real-time correlation between data points. By dynamically adjusting the weights between different data points, multimodal sensor data can be more accurately fused. It should be noted here that in the assembly line monitoring system, sensors are usually fixed at specific locations to ensure continuous and stable monitoring of specific areas or equipment.

[0053] Although the sensor position is fixed, the state of the monitoring environment and the monitored object may change dynamically. For example, factors such as equipment wear and temperature fluctuations can affect the accuracy and relevance of sensor data. Therefore, dynamic weight adjustment adjusts the weight of each sensor data in the fusion process in real time according to these changes.

[0054] In this context, the distance between sensors refers to the relative distance in the data space, not the physical location. For example, if a device produces abnormal vibrations during operation, the frequency data captured by the acoustic sensor and the image data captured by the optical sensor may show a strong correlation. At this time, the "distance" of these data in the data space becomes closer, and the fusion weight should be increased accordingly.

[0055] Among them, the dynamic weight formula represents the distance in data space, Represents data fusion value, weight Dynamically adjust over time to adapt to changes in the monitoring environment. This adjustment mechanism can improve the accuracy and adaptability of data fusion and ensure that high-quality monitoring data can be obtained in different environments and states.

[0056] Furthermore, in order to identify and reduce the uncertainty in the data fusion process, especially when dealing with complex and variable pipeline data, the conditional information entropy formula is used to guide the data fusion strategy. The calculation of information entropy is used to evaluate the uncertainty of the data. By minimizing the information entropy, the information loss in the fusion process can be reduced and the data fusion strategy can be optimized. The information entropy formula is:

[0057] in, Represents the given fusion data In the case of dynamic weight The conditional information entropy of Represents dynamic weight and fused data The joint probability distribution of Represents fused data The probability distribution of .

[0058] The uncertainty of the current data fusion strategy is evaluated by calculating the information entropy value. If the information entropy value is greater than the preset threshold, it means that the uncertainty of the prediction weight is large when the fusion data is known. It is necessary to adjust the parameters in the fusion strategy according to the information entropy result, such as adjusting the weight of tensor decomposition. or parameters in the dynamic weight formula , , , to change the influence of different data dimensions in fusion. The following is the weight decomposition for the tensor and the parameters in the dynamic weight formula , , Adjustment method and corresponding formula.

[0059] If the information entropy value is greater than the preset threshold, the weight of tensor decomposition can be adjusted by the following formula:

[0060] in, is the adjusted weight, is the weight before adjustment, is the learning rate parameter, which controls the magnitude of the adjustment. is the current information entropy value. If the information entropy is high, it means that the uncertainty is large and the weight will be reduced accordingly, thereby reducing the influence of this decomposition mode on the fusion result.

[0061] The specific adjustment method of the dynamic weight formula is as follows: parameter Adjustments:

[0062] in, and Before and after adjustment Parameters. Add The value will increase the distance The influence in weight calculation reduces the weight of distant data points and is suitable for strengthening the weight of local data when the information entropy is high.

[0063] parameter Adjustments:

[0064] in, and Before and after adjustment Parameter. Reduce The value will decrease The influence in the weight calculation balances the influence of the fusion value and is suitable for reducing the weight of a specific fusion value when the information entropy is high.

[0065] parameter Adjustments:

[0066] in, and Before and after adjustment Parameter. Increasing the value of γ will reduce the weight value as a whole, thereby reducing the weight difference between data points, which is suitable for smoothing weight distribution when the information entropy is high.

[0067] According to the adjusted parameters, the dynamic weights and fusion data are recalculated until the information entropy value is lower than the preset threshold or the maximum number of iterations is reached. This process is iterated continuously, and the fusion strategy is adjusted each time according to the result of the information entropy to reduce uncertainty.

[0068] In this way, the above design provides a dynamic data fusion strategy that can effectively cope with the dynamic changes in the pipeline operation environment. By adjusting the weight of data fusion through dynamic weight adjustment, it can achieve self-adaptation to changes in the pipeline environment. At the same time, in order to identify and reduce the uncertainty in the data fusion process, especially when dealing with complex and changeable pipeline data, the conditional information entropy formula is used to guide the data fusion strategy, and the calculation of information entropy is used to evaluate the uncertainty of the data. By minimizing information entropy, the information loss in the fusion process can be reduced and the data fusion strategy can be optimized.

[0069] In one or more implementations, step 3 specifically performs the following processing: The scheme described in this embodiment constructs a fault recognition model based on a spatiotemporal adaptive multimodal analysis network by comprehensively applying multi-scale time feature decomposition, spatial feature reconstruction network, dynamic weight system and comprehensive fusion analysis, which is used to reveal the potential patterns and information of fused data in time and space dimensions and obtain the final recognition results. Specifically: For the fused data X obtained from the multimodal sensor network, a spatiotemporal adaptive multimodal analysis network model is designed by comprehensively applying the multi-scale time feature decomposition combining Gaussian mode and trigonometric function, the spatial feature reconstruction network constructed by logistic function, the dynamic weight system and its comprehensive fusion analysis to reveal the potential patterns and information of the data in the time and space dimensions. This network model is suitable for processing multimodal data in complex and dynamically changing industrial environments, especially for assembly line monitoring, and can effectively improve the accuracy and efficiency of fault identification.

[0070] In the specific implementation, in order to extract the multi-scale features of the fused dataset X from the time dimension, a combination of Gaussian mode and trigonometric function is used to capture the features of the fused dataset at different time scales. By adjusting the Gaussian mode parameters (mean, standard deviation) and trigonometric function parameters (period), the subtle changes in the data in the time dimension are revealed. The data features of the time and space dimensions are fused together, and the dynamic weight system is used to conduct in-depth analysis of the data to reveal the potential patterns and information in the multimodal data. The specific formula is:

[0071] in, Represents the fusion data set The time features extracted from and denote the mean and variance of the i-th Gaussian mode, respectively. is the period of the ith trigonometric function, used to capture the time variation of different frequencies, Represents the fused dataset The i-th element in .

[0072] By constructing a spatial feature reconstruction network through logistic functions, the spatial features of the data are reconstructed, and the distribution and relationship of the data in the spatial dimension are understood, and the change pattern of the data in different spatial regions is identified. The specific formula is:

[0073] in, Represents the fusion data set The spatial features extracted from represents the weight parameter of the jth spatial reconstruction unit, represents the bias parameter of the jth spatial reconstruction unit.

[0074] Combine the extracted time and space features to generate highly adaptable dynamic weights to optimize the final fusion of data. Dynamic weights can be dynamically adjusted according to the combination of time and space features, making the final data fusion more accurate and targeted. The specific formula is:

[0075] in, represents the dynamic weight generated based on the temporal feature T and the spatial feature S, is the normalization constant, is the kth dynamic weight, is the weight parameter of the kth dynamic weight generation unit, is the bias parameter of the kth dynamic weight generation unit, is element-wise addition.

[0076] Combining time and space features, applying dynamic weights for final data fusion and intelligent analysis, and providing comprehensive data analysis by integrating time and space features. The specific formula is:

[0077] in, The final result of fusion and analysis is a composite data structure that contains temporal and spatial features. Represents element-wise multiplication.

[0078] Continuously monitor the pipeline operation and calculate in real time , the real-time calculation Compare with normal operating baseline data. If some parameters in the system deviate from the normal range, the system will mark this as a potential abnormal pattern. Using historical data, it can identify whether these deviations are consistent with past failure patterns. The deviation degree and pattern of the equipment can be used to predict possible fault types, such as equipment overheating, component wear, circuit failure, etc. Once a potential fault is identified, an early warning is immediately issued to the operation and maintenance team.

[0079] In the specific implementation, As shown, the temporal feature T, spatial feature S and dynamic weight According to the formula fusion The analysis part is to Compare with normal operating baseline data to flag potential abnormal patterns and predict possible failure types.

[0080] In this way, the above strategy captures the characteristics of the fused data set at different time scales by using a combination of Gaussian modes and trigonometric functions, and reveals subtle changes in the data in the time dimension by adjusting the Gaussian mode parameters (mean, standard deviation) and trigonometric function parameters (period); by fusing the data features of the time and space dimensions and using a dynamic weight system to conduct in-depth analysis of the data, the potential patterns and information in the multimodal data are revealed, effectively ensuring the accuracy of subsequent fault identification.

[0081] In one or more embodiments, Figure 2 As shown, corresponding to the above-mentioned pipeline fault identification method, a pipeline fault identification system is provided, including: A data acquisition unit, which is used to acquire multimodal monitoring data of the pipeline; A data fusion unit, which is used to fuse the obtained multimodal monitoring data through a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the different dimensional data in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; A fault identification unit is used to obtain a fault identification result based on the fused data using a pre-trained spatiotemporal adaptive multimodal analysis network model; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing procedures: based on the fused data, extracting its features at different time scales and its distribution and relationship in the spatial dimension to obtain time features and space features; obtaining fused features by dynamically fusing the time features and space features; and realizing fault identification based on the comparison results of the fused features with preset benchmark data.

[0082] It can be understood that the system described in this embodiment corresponds one-to-one to the method in the above embodiment, and its technical details have been described in detail in the first embodiment, so they will not be repeated here.

[0083] In more embodiments: An electronic device is also provided, comprising a processor, wherein the processor is used to implement the steps of the above-mentioned method for predicting regional photovoltaic power generation when executing a computer program stored in a memory.

[0084] A non-transitory readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for predicting regional photovoltaic power generation as described above are implemented.

Claims

1. A pipeline fault identification method, It is characterized in that include: Obtain multimodal monitoring data of the pipeline; For the obtained multimodal monitoring data, the multimodal monitoring data is fused by a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the data of different dimensions in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; Based on the fused data, a pre-trained spatiotemporal adaptive multimodal analysis network model is used to obtain a fault identification result; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing procedures: based on the fused data, its features at different time scales and its distribution and relationship in the spatial dimension are extracted respectively to obtain time features and space features; by dynamically fusing the time features and space features, a fused feature is obtained; based on the comparison result of the fused feature with the preset benchmark data, the fault is identified.

2. A pipeline fault identification method as claimed in claim 1, It is characterized in that For the data fusion strategy, the current data fusion strategy is evaluated based on conditional information entropy, and the data fusion strategy is dynamically optimized based on the evaluation result.

3. A pipeline fault identification method as claimed in claim 1, It is characterized in that The extraction of the time feature is specifically as follows: based on the fused data, based on the results of Gaussian modes and trigonometric functions, the features of the fused data at different time scales are obtained.

4. A pipeline fault identification method as claimed in claim 1, It is characterized in that The extraction of the spatial features is specifically as follows: based on the fused data, a spatial feature reconstruction network is constructed through a logistic function, the spatial features of the fused data are reconstructed, and the distribution and relationship of the fused data in the spatial dimension are obtained.

5. A pipeline fault identification method as claimed in claim 1, It is characterized in that The dynamic weighted fusion is specifically: generating adaptive dynamic weights based on spatial features and temporal features, and fusing temporal features and spatial features based on the obtained dynamic weights.

6. A pipeline fault identification method as claimed in claim 1, It is characterized in that The preset data fusion strategy specifically includes: converting the obtained original multimodal monitoring data into a new dimensional space through nonlinear mapping to obtain a new data feature set; performing feature extraction on the data feature set, performing tensor decomposition based on the extracted features to obtain features of different dimensions; performing weighted fusion based on the obtained features of different dimensions to obtain fused data.

7. A pipeline fault identification method as claimed in claim 6, It is characterized in that The obtained original multimodal monitoring data is converted into a new dimensional space through nonlinear mapping, which is specifically expressed as: in, Represents the mapped data. represents the i-th data point in the original data, is the amplitude coefficient of the kth Gaussian function, is the center position of the kth Gaussian function, is the standard deviation of the kth Gaussian function, yes The coefficient of It is The exponent of the polynomial, and are the constant offset and the linear coefficient.

8. A pipeline fault identification method as claimed in claim 1, It is characterized in that The fusion data is specifically expressed as follows: in, represents the fused data set, represents the data vector of the i-th dimension in the r-th decomposition pattern, is the weight of the rth decomposition mode, represents the fusion weight.

9. A pipeline fault identification method as claimed in claim 1, It is characterized in that The data fusion weights corresponding to the data of different dimensions in the data fusion strategy are dynamically adjusted based on the distance between the sensor data. The data fusion weights are specifically expressed as follows: in, is the dynamic weight between the pth and qth data points at time t, , , is the adjustment parameter of the weight formula, represents the distance between the pth and qth data points, represents the fused value of the p-th and q-th data points at time t.

10. A pipeline fault identification method as claimed in claim 1, It is characterized in that The multimodal monitoring data includes light intensity, sound frequency, temperature, current and voltage changes, and device movement.

11. A pipeline fault identification system, It is characterized in that include: A data acquisition unit, which is used to acquire multimodal monitoring data of the pipeline; A data fusion unit, which is used to fuse the obtained multimodal monitoring data through a preset data fusion strategy to obtain fused data; wherein the data fusion weights corresponding to the different dimensional data in the data fusion strategy are dynamically adjusted based on the distance between the sensor data; A fault identification unit is used to obtain a fault identification result based on the fused data using a pre-trained spatiotemporal adaptive multimodal analysis network model; wherein the spatiotemporal adaptive multimodal analysis network model specifically performs the following processing procedures: based on the fused data, extracting its features at different time scales and its distribution and relationship in the spatial dimension to obtain time features and space features; obtaining fused features by dynamically fusing the time features and space features; and realizing fault identification based on the comparison results of the fused features with preset benchmark data.

12. An electronic device comprising a memory, a processor and a computer program stored and executed on the memory, It is characterized in that When the processor executes the program, a pipeline fault identification method as described in any one of claims 1-10 is implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, a pipeline fault identification method as described in any one of claims 1 to 10 is implemented.

Images