Industrial equipment fault instant monitoring method and system based on neural network

Abstract

The invention discloses an industrial equipment fault real-time monitoring method and system based on a neural network, and the method comprises the steps: carrying out the segmentation and classification of a monitoring video frame of target industrial equipment, obtaining recognition frames, selecting doubt recognition image blocks which are strongly correlated with other recognition frames, and reference recognition image blocks which are weakly correlated with other recognition frames, and carrying out the recognition of the doubt recognition image blocks. And performing brightness and texture feature analysis and processing on the doubtful recognition image block to enlarge the features of the doubtful recognition image block which do not belong to the equipment device, enhancing the reference image block to enhance the features of the reference image block which belong to the equipment device, then performing fault recognition based on the doubtful fusion feature map and the reference enhancement feature map, and outputting a fault monitoring result. In the scheme, the doubtful image block corresponding to the obstacle is weakened, and meanwhile, the related characteristics of the reference image block belonging to the equipment device are enhanced, so that the influence of obstacle shielding and the like on the fault monitoring result can be effectively avoided, and the fault monitoring accuracy of the industrial equipment is improved. The system also has the above technical effects.

Description

Technical Field

[0001] This application relates to the field of equipment monitoring technology, and in particular to a method and system for real-time monitoring of industrial equipment faults based on neural networks. Background Technology

[0002] With the increasing complexity and automation of industrial equipment in sectors such as machinery and transportation, condition monitoring of industrial equipment is crucial. Given the limitations of human resources and the subjective biases inherent in manual monitoring, intelligent monitoring of industrial equipment is becoming a trend. For example, CN118426442A – An IoT-based Intelligent Fault Diagnosis System for Industrial Equipment – ​​prioritizes equipment on the production line by combining historical fault data with equipment operating parameters, thereby achieving fault diagnosis. However, current intelligent monitoring of industrial equipment may be affected by factors such as video capture quality and obstructions, leading to inaccurate fault detection and hindering the normal progress of industrial processes. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method and system for real-time monitoring of industrial equipment faults based on neural networks, so as to solve the problem of inaccurate detection of industrial equipment faults. The specific solution is as follows:

[0004] To achieve the above objectives, this application provides a method for real-time fault monitoring of industrial equipment based on neural networks, comprising:

[0005] Acquire surveillance video of the target industrial equipment, and perform the following steps on at least a portion of the target video frames using a pre-trained neural network model:

[0006] Image segmentation and classification are used to determine the bounding boxes and corresponding feature information of objects contained in the area where the target industrial equipment is located from the target video frame;

[0007] A correlation analysis is performed on the feature information between each pair of recognition boxes; based on the correlation analysis results, suspicious recognition image blocks and reference recognition image blocks are determined; wherein, the correlation between the recognition box corresponding to the suspicious recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are the recognition boxes in the recognition boxes of objects contained in the area where the target industrial equipment is located, excluding the recognition boxes corresponding to the suspicious recognition image block or the reference recognition image block;

[0008] Brightness and texture information are extracted from the suspicious image block. Different feature extraction processes are performed on the brightness and texture information, and the extracted features are fused to obtain the suspicious fusion feature map corresponding to the suspicious image block. Among them, the feature extraction process corresponding to the texture information is used to focus on texture space feature information.

[0009] The reference recognition image block is enhanced to obtain the corresponding reference enhanced feature map;

[0010] Based on the doubt fusion feature map and the reference enhanced feature map, the fault operation status of the equipment and components included in the target industrial equipment is identified, and the fault monitoring results are output.

[0011] For example, the feature information includes location feature information and object type feature information;

[0012] The correlation analysis of feature information between pairwise recognition boxes includes:

[0013] For any two target bounding boxes:

[0014] The positional feature information between the target recognition boxes is analyzed to obtain the positional association relationship;

[0015] The object type feature information between the target recognition boxes is analyzed for correlation to obtain the object type correlation relationship;

[0016] The location association and the object type association are fused to obtain the association analysis results between the target recognition boxes.

[0017] For example, determining the suspected identification image block and the reference identification image block based on the correlation analysis results includes:

[0018] For any given bounding box:

[0019] The correlation analysis results between the two boxes are weighted and summed to obtain the total correlation analysis result.

[0020] The overall correlation analysis results are compared with the correlation threshold;

[0021] If the correlation threshold is lower than the threshold, the image block corresponding to the recognition box is identified as a suspicious recognition image block.

[0022] If the correlation threshold is higher than or equal to the correlation threshold, then the image block corresponding to the recognition box is determined as the reference recognition image block.

[0023] For example, the step of performing different feature extraction processes on the brightness information and the texture information, and then fusing the extracted features to obtain the doubt fusion feature map corresponding to the doubt identification image patch, includes:

[0024] The brightness information is subjected to a first convolution process and a second convolution process to obtain a first brightness feature and a second brightness feature; the first convolution process and the second convolution process are the same convolution process;

[0025] The texture information is subjected to third convolution processing and spatial attention processing respectively to obtain texture features and texture spatial attention features;

[0026] The first brightness feature and the texture feature are fused to obtain a first fusion result;

[0027] The second brightness feature and the texture space attention feature are fused, and the fusion result is enhanced to obtain the second fusion result;

[0028] Cross-fusion is performed based on the first fusion result and the second fusion result, and the images are stitched together based on the brightness channel and the texture channel respectively to obtain the doubt fusion feature map corresponding to the doubt identification image block.

[0029] For example, spatial attention processing is performed on the texture information to obtain texture spatial attention features, including:

[0030] The texture information is subjected to a fourth convolution process to obtain the first texture sub-feature;

[0031] Perform a real-number fast Fourier transform on the first texture sub-feature to obtain the first frequency domain feature;

[0032] The first frequency domain feature is subjected to a fifth convolution process to obtain the second frequency domain feature;

[0033] Perform a real-number inverse fast Fourier transform on the second frequency domain feature to obtain the second texture sub-feature;

[0034] The first texture sub-feature and the second texture sub-feature are fused, and the fusion result is convolved to obtain texture space attention features.

[0035] For example, the enhancement processing of the reference recognition image block to obtain the corresponding reference enhanced feature map includes:

[0036] Obtain the correlation map between the devices and components of the target industrial equipment; wherein, the correlation map is used to characterize the degree of mutual influence when each device or component fails, and each device or component has a corresponding influence value on other devices or components; wherein, the other devices or components are the devices or components included in the target industrial equipment other than the current device or component;

[0037] Based on the association map, core reference recognition image blocks with influence values ​​higher than or equal to a set value and non-core reference recognition image blocks with influence values ​​lower than the set value are identified in the reference recognition image blocks.

[0038] The core reference recognition image block is enhanced according to a first enhancement factor, and the non-core reference recognition image block is enhanced according to a second enhancement factor to obtain a corresponding reference enhanced feature map; wherein, the enhancement intensity of the first enhancement factor is greater than the enhancement intensity of the second enhancement factor.

[0039] For example, the step of identifying the fault operating status of the equipment components included in the target industrial equipment based on the doubt fusion feature map and the reference enhanced feature map, and outputting fault monitoring results, includes:

[0040] Based on the doubtful fusion feature map and the reference enhanced feature map, the fault operating state of the equipment device is identified, and the target faulty device is determined.

[0041] Based on the correlation map, related devices with a strong influence correlation with the target faulty device are identified;

[0042] The fault monitoring results are output based on the target faulty device and the associated devices.

[0043] For example, after acquiring the monitoring video of the target industrial equipment, the process further includes:

[0044] The timing sequence corresponding to each video frame of the monitoring video is determined, and the high-incidence video segment corresponding to the high-incidence period of the fault and the low-incidence video segment corresponding to the low-incidence period of the fault are determined based on the timing sequence corresponding to each video frame.

[0045] The high-frequency video segments are sampled at a first frame rate, and the low-frequency video segments are sampled at a second frame rate; wherein the first frame rate is higher than the second frame rate.

[0046] The extracted video frames are arranged in chronological order to obtain the target video frames.

[0047] In another aspect, this application also provides a real-time monitoring system for industrial equipment faults based on neural networks, the system including interconnected camera equipment and electronic equipment;

[0048] The camera device is used to acquire surveillance video of the target industrial equipment and send the surveillance video to the electronic device;

[0049] The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the method described above.

[0050] Furthermore, this application also provides a neural network-based real-time fault monitoring device for industrial equipment, comprising:

[0051] The video acquisition module is used to acquire monitoring videos of the target industrial equipment and performs the following steps on at least a portion of the target video frames using a pre-trained neural network model:

[0052] The device identification module is used to determine the identification box and corresponding feature information of the objects contained in the area where the target industrial equipment is located from the target video frame through image segmentation and classification;

[0053] The correlation analysis module is used to perform correlation analysis on the feature information between pairs of recognition boxes; based on the correlation analysis results, it determines suspicious recognition image blocks and reference recognition image blocks; wherein, the correlation between the recognition box corresponding to the suspicious recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are the recognition boxes in the recognition boxes of objects contained in the area where the target industrial equipment is located, excluding the recognition boxes corresponding to the suspicious recognition image block or the reference recognition image block;

[0054] The feature processing module is used to extract brightness information and texture information from the suspicious image block, perform different feature extraction processes on the brightness information and the texture information, and perform feature fusion on the extracted features to obtain the suspicious fusion feature map corresponding to the suspicious image block; wherein, the feature extraction processing corresponding to the texture information is used to focus on texture space feature information;

[0055] The feature enhancement module is used to enhance the reference recognition image block to obtain the corresponding reference enhanced feature map;

[0056] The fault prediction module is used to identify the fault operation status of the equipment components included in the target industrial equipment based on the doubt fusion feature map and the reference enhanced feature map, and output the fault monitoring results.

[0057] The method for real-time fault monitoring of industrial equipment based on neural networks provided in this application includes: acquiring monitoring video of a target industrial equipment; performing the following steps on at least a portion of the target video frames of the monitoring video using a pre-trained neural network model: determining the bounding boxes and corresponding feature information of objects contained in the area where the target industrial equipment is located from the target video frames through image segmentation and classification; performing correlation analysis on the feature information between pairs of bounding boxes; determining suspicious identification image blocks and reference identification image blocks based on the correlation analysis results; wherein, the correlation between the bounding box corresponding to the suspicious identification image block and other bounding boxes is lower than a set condition, and the reference identification image block... The correlation between the recognition box corresponding to the block and other recognition boxes is higher than or equal to a set condition; brightness and texture information are extracted from the suspected recognition image block, and different feature extraction processes are performed on the brightness and texture information, and the extracted features are fused to obtain the suspected recognition image block corresponding to the suspected fusion feature map; among them, the feature extraction process corresponding to the texture information is used to focus on the texture space feature information; the reference recognition image block is enhanced to obtain the corresponding reference enhanced feature map; based on the suspected fusion feature map and the reference enhanced feature map, the fault operation status of the equipment and components contained in the target industrial equipment is identified, and the fault monitoring results are output.

[0058] In the above method, the monitoring video frames of the target industrial equipment are segmented and classified to obtain recognition boxes. Suspicious image blocks strongly correlated with other recognition boxes and reference image blocks weakly correlated with other recognition boxes are selected. Brightness and texture feature analysis and processing are performed on the suspicious image blocks to amplify their features that do not belong to the equipment components. The reference image blocks are enhanced to strengthen their features that belong to the equipment components. Then, fault identification is performed on the suspicious fusion feature map and the reference enhanced feature map obtained based on the aforementioned processing, and the fault monitoring results are output. Obstacles such as pedestrians and dust often do not directly cause industrial equipment failures, but they can affect the accuracy of fault identification by the neural network. For example, obstacles may be mistakenly identified as abnormal states of equipment components, leading to the output of warning information. This scheme weakens the suspicious image blocks corresponding to obstacles such as pedestrians and dust, while enhancing the relevant features of the reference image blocks belonging to the equipment components. Fault prediction is performed based on the corresponding suspicious fusion feature map and reference enhanced feature map. This not only effectively avoids the influence of video shooting quality on fault monitoring results but also effectively avoids the influence of obstacle occlusion on fault monitoring results, thus improving the accuracy of fault monitoring for industrial equipment.

[0059] Correspondingly, the neural network-based real-time fault monitoring system and device provided in this application also have the above-mentioned technical effects. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0061] Figure 1 A flowchart illustrating a method for real-time fault monitoring of industrial equipment based on neural networks, provided in this application embodiment;

[0062] Figure 2 A schematic diagram of a neural network-based real-time fault monitoring system for industrial equipment provided in this application embodiment;

[0063] Figure 3 A structural diagram of an electronic device provided in an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of a neural network-based real-time fault monitoring device for industrial equipment, provided as an embodiment of this application. Detailed Implementation

[0065] Please see Figure 1 , Figure 1 This document presents a flowchart of a method for real-time fault monitoring of industrial equipment based on neural networks, as provided in an embodiment of this application. This method can be applied to electronic devices such as personal computers and smartphones. Figure 1 As shown, this neural network-based real-time fault monitoring method for industrial equipment may include the following steps:

[0066] S101. Acquire monitoring video of the target industrial equipment, and perform the following steps on at least a portion of the target video frames of the monitoring video using a pre-trained neural network model.

[0067] The industrial equipment can be electrical equipment, transportation equipment, etc. For example, the target industrial equipment can be at least one piece of industrial equipment; further, these industrial equipment can be multiple pieces of the same type of industrial equipment or multiple pieces of different types of industrial equipment.

[0068] The camera device can continuously monitor the target industrial equipment to capture relevant video footage. The camera device can continuously record video of the target industrial equipment, acquiring several hours or days of monitoring video.

[0069] Furthermore, the camera can be pointed towards the location of the equipment or component most likely to malfunction in the target industrial equipment. Additionally, there can be multiple camera devices. In the case of multiple camera devices, fault identification can be performed on the monitoring video acquired by each device individually. As long as a fault is identified in the monitoring video corresponding to any one camera, a fault warning is output. Alternatively, the fault identification results from each camera device can be integrated, and a fault warning can be output based on the integrated result.

[0070] The target video frame can be all frames in the surveillance video or a subset of frames. If it's a subset of frames, it can be any segment of the surveillance video containing all frames, or it can be a subset of frames obtained by extracting frames from the surveillance video.

[0071] The neural network model can be any type of network model capable of feature extraction and feature-based classification, such as Convolutional Neural Networks (CNN), Fully Connected Neural Networks (FCN), Generative Adversarial Networks (GAN), Recurrent Neural Networks (RNN), and Long Short-Term Memory Networks (LSTM). This application does not limit the type of neural network model. When training the neural network model, images of various objects that may appear in the area where the target industrial equipment is located can be used as training samples. For example, images of equipment components, dust, pedestrians, shelves, and smoke can be used as training samples. In this way, the trained neural network model can identify various objects that may appear in the area where the target industrial equipment is located, and then select the bounding box.

[0072] S102. By image segmentation and classification, the bounding boxes and corresponding feature information of the objects contained in the area where the target industrial equipment is located are determined from the target video frame.

[0073] For example, feature extraction, segmentation, and classification of target video frames can be performed using common image segmentation and classification methods to determine the bounding boxes of various objects.

[0074] For example, the target video frame can be cropped first to remove the image portion outside the area where the target industrial equipment is located. That is, the neural network model only processes the image portion of the area where the target industrial equipment is located, thereby reducing the computational load of the neural network model. The image portion of the area where the target industrial equipment is located can be the image area corresponding to a certain number of pixels outward from the largest rectangular bounding box of the target industrial equipment.

[0075] The objects identified can be equipment components within industrial equipment, or obstacles such as dust and pedestrians. Equipment components refer to various parts included in the industrial equipment, and the types of equipment components can vary depending on the type of industrial equipment. Equipment types can include transformers, radiators, conveyor belts, etc. For example, equipment components are the objects of fault analysis, and it is necessary to determine whether the target industrial equipment has malfunctioned based on the corresponding bounding boxes of the equipment components. While pedestrians, dust, and other obstacles often do not directly cause malfunctions in industrial equipment, they can affect the accuracy of fault identification by the neural network model. Therefore, special processing is required for the bounding boxes corresponding to obstacles to avoid affecting fault identification and causing false alarms.

[0076] The feature information corresponding to the recognition box can include the location information of the recognition box, the object type, etc. It is understandable that each device on industrial equipment has its own reasonable and relatively fixed location, while obstacles and other objects have significant positional deviations from these devices; furthermore, the types of devices and obstacles differ considerably.

[0077] S103. Perform correlation analysis on the feature information between pairs of recognition boxes; determine the suspected recognition image block and the reference recognition image block based on the correlation analysis results; wherein, the correlation between the recognition box corresponding to the suspected recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are the recognition boxes other than the recognition boxes corresponding to the suspected recognition image block or the reference recognition image block among the recognition boxes of objects contained in the area where the target industrial equipment is located.

[0078] Correlation analysis is the process of determining the relationship between objects corresponding to each pair of bounding boxes. Specifically, it can determine whether the positional relationship between the two objects matches the positional relationship between devices and components on industrial equipment. For example, if one object is located at the center of the target industrial equipment while the other is located outside, the relationship is weak; if both are located at the center, the relationship is strong. It can also determine the similarity in type. For example, if one is a transformer and the other is a human arm, the relationship is weak; if one is a transformer and the other is a radiator, the relationship is strong. Through correlation analysis, the devices and components within the area containing the target industrial equipment can be accurately selected, resulting in suspicious and reference image blocks for subsequent differentiation processing.

[0079] The correlation setting conditions can be preset correlation thresholds, such as 60%, 70%, etc.

[0080] For example, the feature information includes location feature information and object type feature information; the correlation analysis of the feature information between pairs of recognition boxes includes: for any two target recognition boxes: performing correlation analysis on the location feature information between the target recognition boxes to obtain a location correlation relationship; performing correlation analysis on the object type feature information between the target recognition boxes to obtain an object type correlation relationship; and fusing the location correlation relationship and the object type correlation relationship to obtain the correlation analysis result between the target recognition boxes.

[0081] By combining location and type characteristics for correlation analysis, obstacles that differ significantly from equipment or devices in location or type can be identified, which facilitates mitigating the impact of obstacles and improving the accuracy of fault identification.

[0082] For example, after integrating the correlation analysis results between each bounding box and other identified bounding boxes, reference bounding boxes strongly correlated with other bounding boxes and questionable bounding boxes weakly correlated with other bounding boxes can be identified. The image blocks circled by the bounding boxes are used as output to obtain questionable image blocks and reference image blocks. Among them, questionable image blocks are more likely to be obstacles, which may lead to poor fault identification results; while reference image blocks are more likely to be equipment components, which need to be analyzed to determine whether a fault has occurred, and thus determine whether the target industrial equipment has malfunctioned.

[0083] For example, determining the suspected identification image block and the reference identification image block based on the correlation analysis results includes: for any identification box: weighted summing of the correlation analysis results with other identification boxes to obtain a total correlation analysis result; comparing the total correlation analysis result with a correlation threshold; if it is lower than the correlation threshold, the image block corresponding to the identification box is determined as a suspected identification image block; if it is higher than or equal to the correlation threshold, the image block corresponding to the identification box is determined as a reference identification image block. The correlation threshold can be a pre-set correlation value, such as 60%, 70%, etc.

[0084] In the above scheme, for any recognition box, the correlation analysis between it and other recognition boxes is integrated to obtain the total correlation analysis result corresponding to the recognition box. The total correlation analysis result can better characterize the relatively special object in the video frame, and then pedestrians, dust and other obstacles with weak correlation with other recognition boxes can be extracted as suspicious recognition image blocks.

[0085] As mentioned earlier, the suspected image block and the reference image block differ significantly. Therefore, different processing is required for these two types of image blocks to improve the accuracy of fault identification.

[0086] S104. Extract brightness information and texture information from the suspected image block, perform different feature extraction processes on the brightness information and the texture information, and perform feature fusion on the extracted features to obtain the suspected image block corresponding to the suspected image block; wherein, the feature extraction process corresponding to the texture information is used to focus on texture space feature information.

[0087] Specifically, luminance and texture information can be extracted from the suspicious image block according to the luminance and texture channels. Considering that obstacles differ significantly from devices in both luminance and texture, feature extraction is performed on the suspicious image block based on information from these two channels. When extracting texture features, attention is paid to its spatial feature information to improve the accuracy of texture feature extraction.

[0088] In one embodiment, the step of performing different feature extraction processes on the brightness information and the texture information, and then fusing the extracted features to obtain a suspicious fusion feature map corresponding to the suspicious image block, includes: performing a first convolution process and a second convolution process on the brightness information to obtain a first brightness feature and a second brightness feature; the first convolution process and the second convolution process are the same convolution process; performing a third convolution process and spatial attention processing on the texture information to obtain a texture feature and a texture spatial attention feature; fusing the first brightness feature and the texture feature to obtain a first fusion result; fusing the second brightness feature and the texture spatial attention feature, and enhancing the fusion result to obtain a second fusion result; performing cross-fusion based on the first fusion result and the second fusion result, and concatenating them based on the brightness channel and the texture channel respectively to obtain the suspicious fusion feature map corresponding to the suspicious image block. The suspicious fusion feature map can be an image that can characterize relevant features, such as a mask image corresponding to the suspicious image block.

[0089] Optionally, the first and second convolution processes are the same, and their corresponding kernel sizes can be the same. Further, both the first and second convolution processes can be implemented using at least one convolution block. Convolution is a commonly used computational technique in signal processing and image processing, capable of achieving image processing objectives such as feature extraction, noise reduction, and compression. In this embodiment, the kernel size can be 3*3, etc.

[0090] In the above embodiments, features corresponding to the suspected image blocks are extracted through convolution and spatial attention cross-processing, as well as feature integration, to obtain a suspected fusion feature map. This suspected fusion feature map fully integrates brightness features and texture space features, effectively highlighting these features. In other words, it emphasizes the differentiated features between obstacles and equipment, facilitating the distinction between obstacles and genuine equipment features and improving the accuracy of fault identification.

[0091] In one embodiment, spatial attention processing is performed on the texture information to obtain texture spatial attention features, including: performing a fourth convolution processing on the texture information to obtain a first texture sub-feature; performing a real-number fast Fourier transform on the first texture sub-feature to obtain a first frequency domain feature; performing a fifth convolution processing on the first frequency domain feature to obtain a second frequency domain feature; performing a real-number inverse fast Fourier transform on the second frequency domain feature to obtain a second texture sub-feature; fusing the first texture sub-feature and the second texture sub-feature, and performing convolution processing on the fusion processing result to obtain texture spatial attention features.

[0092] Among them, combining spatial and frequency domain convolution processing to highlight the spatial features of texture can effectively highlight the spatially distinct features of obstacles and devices.

[0093] The above processing enhances the brightness and texture features of the suspected image block. Since obstacles and devices are significantly different in brightness and texture, this processing further strengthens this distinction when the suspected image block is an obstacle. However, if the suspected image block is a device, its brightness and texture differences from other devices are not significant, so even with this enhancement, the impact on subsequent fault identification is minimal. Therefore, this processing not only strengthens the difference between obstacles and devices in the suspected image block, facilitating accurate model identification of the corresponding image block as an obstacle, but also prevents the model from misidentifying a suspected image block as a device as an obstacle.

[0094] S105. Enhance the reference recognition image block to obtain the corresponding reference enhanced feature map.

[0095] Enhancement processing of the reference recognition image can strengthen the relevant features of the device components, thereby further distinguishing it from the suspected recognition image block. The reference fusion feature map can be an image that represents relevant features, such as a mask image corresponding to the reference recognition image block.

[0096] For example, the enhancement processing of the reference recognition image block to obtain the corresponding reference enhancement feature map includes: acquiring an association map between the devices of the target industrial equipment; wherein the association map is used to characterize the degree of mutual influence when each device fails, and each device has an influence value on other devices; wherein the other devices are devices other than the current device among the devices included in the target industrial equipment; determining, based on the association map, core reference recognition image blocks with influence values ​​higher than or equal to a set value and non-core reference recognition image blocks with influence values ​​lower than the set value in the reference recognition image block; enhancing the core reference recognition image block according to a first enhancement factor, and enhancing the non-core reference recognition image block according to a second enhancement factor to obtain the corresponding reference enhancement feature map; wherein the enhancement intensity of the first enhancement factor is greater than the enhancement intensity of the second enhancement factor.

[0097] In the above scheme, core and non-core reference image blocks are determined by combining the correlation map. Different enhancement processes are applied to these two types of image blocks. Using a stronger enhancement factor effectively strengthens the core reference image block. In subsequent fault identification, the model can more accurately identify whether there are anomalies in the core reference image block, thus improving the accuracy of fault identification. Simultaneously, since smaller enhancement factors often consume less computational resources during enhancement processing, it is possible to balance recognition accuracy with computer resource efficiency, ensuring computational speed.

[0098] S106. Based on the doubt fusion feature map and the reference enhanced feature map, identify the fault operation status of the equipment components included in the target industrial equipment, and output the fault monitoring results.

[0099] Among them, the fault state can be that the equipment or components are in a state different from the normal state. For example, the equipment or components may fall and cause the position to shift, the equipment or components may be stuck and cause the operating speed to be lower than the normal speed, or the equipment or components may overheat and cause the color to turn red.

[0100] As mentioned earlier, the doubtful fusion feature map is a feature map obtained by enhancing the brightness and texture of image blocks that may be obstacles, while the reference enhanced feature map is a feature map obtained by enhancing image blocks that may be devices. Based on this, the model is unlikely to misidentify the doubtful fusion feature map corresponding to obstacles as devices. At the same time, since the impact on the brightness and texture of devices is not significant, devices in the doubtful fusion feature map will not be missed. Moreover, since the relevant features in the doubtful fusion feature map and the reference enhanced feature map are enhanced, the model can accurately identify the features of devices, thereby accurately achieving fault identification.

[0101] For example, the doubt fusion feature map and the reference enhancement feature map can be fused, and then the fused feature map can be input into the fault prediction unit of the neural network model for fault prediction.

[0102] For example, the step of identifying the fault operating status of the equipment components included in the target industrial equipment based on the doubt fusion feature map and the reference enhanced feature map, and outputting fault monitoring results, includes: identifying the fault operating status of the equipment components based on the doubt fusion feature map and the reference enhanced feature map, and determining the identified target faulty device; determining the associated devices that have a strong influence association with the target faulty device based on the association map; and outputting fault monitoring results based on the target faulty device and the associated devices.

[0103] Specifically, determining the associated devices with a strong influence relationship with the target faulty device based on the correlation map can be achieved by identifying devices whose influence value with the target faulty device is greater than a set threshold, and using these devices as associated devices. This influence value can specifically refer to the influence of the target faulty device on other devices, that is, the probability that other devices will also fail when the target faulty device fails.

[0104] Since the associated device is not a directly faulty device, the fault monitoring results corresponding to the target faulty device and the associated device can be different. The fault monitoring result corresponding to the target faulty device can be a warning message, while the fault monitoring result corresponding to the associated device can be a reminder message, used to remind the user to pay attention rather than directly serving as a warning of the fault.

[0105] The above scheme not only outputs results based on the identified target faulty device, but also combines the results with its associated devices, which can effectively ensure the accuracy of fault warning.

[0106] For example, to improve the efficiency of fault prediction, the suspected fusion feature map can be type-identified first. If it is determined that the type is not a device, it is deleted, and the fault operation status of the device is only identified when the type is a device. Based on the above processing, the suspected fusion feature map achieves differentiated amplification of the distinguishing features between devices and obstacles. Therefore, it can accurately determine the type of the corresponding object, avoid fault prediction for obstacles, and improve the accuracy of fault prediction.

[0107] For example, there are often many target video frames, which can be analyzed frame by frame. An alarm can be triggered when a fault is determined to exist in a single frame, or an alarm can be triggered when a fault is found in several consecutive frames.

[0108] The neural network-based real-time fault monitoring method for industrial equipment provided in the above embodiments segment and classify the monitoring video frames of the target industrial equipment to obtain recognition boxes. It selects suspicious image blocks strongly correlated with other recognition boxes and weakly correlated reference image blocks. The suspicious image blocks undergo brightness and texture feature analysis and processing to amplify their features that do not belong to the equipment components. The reference image blocks are enhanced to strengthen their features that belong to the equipment components. Then, fault identification is performed on the suspicious image fusion feature map and the reference enhanced feature map obtained from the aforementioned processing, and the fault monitoring result is output. This scheme weakens suspicious image blocks corresponding to obstacles such as pedestrians and dust, while enhancing the relevant features of reference image blocks belonging to the equipment components. Fault prediction is performed based on the corresponding suspicious image fusion feature map and reference enhanced feature map. This not only effectively avoids the influence of video shooting quality on fault monitoring results but also effectively avoids the influence of obstacle occlusion on fault monitoring results, improving the accuracy of fault monitoring for industrial equipment.

[0109] In one embodiment, after acquiring the monitoring video of the target industrial equipment, the method further includes: determining the time sequence corresponding to each video frame of the monitoring video; determining high-incidence video segments corresponding to high-incidence fault periods and low-incidence video segments corresponding to low-incidence fault periods based on the time sequence corresponding to each video frame; extracting frames from the high-incidence video segments according to a first frame extraction rate and extracting frames from the low-incidence video segments according to a second frame extraction rate; wherein the first frame extraction rate is higher than the second frame extraction rate; and arranging the extracted video frames in time sequence to obtain the target video frame.

[0110] Among them, the high-incidence period of failure can be the period when the target industrial equipment is running at high speed, and the low-incidence period of failure can be the period when the target industrial equipment is running at low speed.

[0111] For example, high-incidence and low-incidence periods of faults can be determined based on historical fault alarm information of the target industrial equipment.

[0112] In the above embodiments, frames are extracted from the surveillance video to reduce the amount of data. At the same time, different frame extraction rates are used for videos in high-incidence and low-incidence periods to achieve a balance between efficiency and accuracy.

[0113] The following describes a neural network-based real-time fault monitoring system for industrial equipment provided in an embodiment of this application. The neural network-based real-time fault monitoring system described below and the neural network-based real-time fault monitoring method and corresponding technical effects described above can be referred to each other.

[0114] like Figure 2As shown, the monitoring system 200 includes a camera device 201 and an electronic device 202 connected to each other; the camera device 201 is used to acquire monitoring video of the target industrial equipment and send the monitoring video to the electronic device 202; the electronic device 202 includes a processor and a memory; wherein, the memory is used to store a computer program, which is loaded and executed by the processor to implement the methods described above.

[0115] Among them, electronic device 202 can be a terminal. The specific structure of the aforementioned electronic device can be as follows: Figure 3 As shown. Figure 3 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be considered as any limitation on the scope of this application. Please see [link to diagram]. Figure 3 The terminal includes a processor 31 and a memory 32.

[0116] The processor 31 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 31 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 31 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 31 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 31 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0117] The memory 32 may include one or more computer-readable storage media, which may be non-transitory. The memory 32 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 32 is used to store at least the following computer program 322, which, after being loaded and executed by the processor 31, is capable of implementing the relevant steps in the neural network-based real-time fault monitoring method for industrial equipment executed on the terminal side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 32 may also include an operating system 321 and data 323, and the storage method may be temporary or permanent storage. The operating system 321 may include Windows, Unix, Linux, etc. The data 323 may include, but is not limited to, application update information.

[0118] In some embodiments, the terminal may further include a power supply 33, a communication interface 34, an input / output interface 35, a display screen (not shown), a sensor (not shown), and a communication bus 36.

[0119] Those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than illustrated.

[0120] The following describes a neural network-based real-time fault monitoring device for industrial equipment provided in an embodiment of this application. The neural network-based real-time fault monitoring device described below and the neural network-based real-time fault monitoring method and corresponding technical effects described above can be referred to each other.

[0121] Please see Figure 4 , Figure 4 A schematic diagram of a neural network-based real-time fault monitoring device 400 for industrial equipment, provided in an embodiment of this application, includes:

[0122] The video acquisition module 401 is used to acquire monitoring video of the target industrial equipment and perform the following steps on at least a portion of the target video frames of the monitoring video using a pre-trained neural network model:

[0123] The device identification module 402 is used to determine the identification box and corresponding feature information of the objects contained in the area where the target industrial equipment is located from the target video frame through image segmentation and classification;

[0124] The correlation analysis module 403 is used to perform correlation analysis on the feature information between pairs of recognition boxes; based on the correlation analysis results, it determines suspicious recognition image blocks and reference recognition image blocks; wherein, the correlation between the recognition box corresponding to the suspicious recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are recognition boxes other than the recognition boxes corresponding to the suspicious recognition image block or the reference recognition image block among the recognition boxes of objects contained in the area where the target industrial equipment is located;

[0125] Feature processing module 404 is used to extract brightness information and texture information from the suspicious identification image block, perform different feature extraction processes on the brightness information and the texture information, and perform feature fusion on the extracted features to obtain a suspicious fusion feature map corresponding to the suspicious identification image block; wherein, the feature extraction processing corresponding to the texture information is used to focus on texture space feature information;

[0126] Feature enhancement module 405 is used to enhance the reference recognition image block to obtain a corresponding reference enhanced feature map;

[0127] The fault prediction module 406 is used to identify the fault operation status of the equipment components included in the target industrial equipment based on the doubt fusion feature map and the reference enhanced feature map, and output the fault monitoring results.

[0128] The neural network-based real-time fault monitoring device for industrial equipment provided in the above embodiments segments and classifies the monitoring video frames of the target industrial equipment to obtain recognition boxes. It selects suspicious image blocks strongly correlated with other recognition boxes and weakly correlated reference image blocks. The suspicious image blocks undergo brightness and texture feature analysis and processing to amplify their features that do not belong to equipment components. The reference image blocks are enhanced to strengthen their features that belong to equipment components. Then, fault identification is performed on the suspicious image fusion feature map and the enhanced reference feature map obtained from the aforementioned processing, and the fault monitoring results are output. This solution weakens suspicious image blocks corresponding to obstacles such as pedestrians and dust, while enhancing the relevant features of reference image blocks belonging to equipment components. This effectively avoids the impact of video shooting quality and obstacle occlusion on the fault monitoring results, improving the accuracy of fault monitoring for industrial equipment.

[0129] In one embodiment, the feature information includes location feature information and object type feature information;

[0130] The correlation analysis module includes:

[0131] For any two target bounding boxes:

[0132] The location association analysis submodule is used to perform association analysis on the location feature information between the target recognition boxes to obtain the location association relationship;

[0133] The type association analysis submodule is used to perform association analysis on the object type feature information between the target recognition boxes to obtain the object type association relationship;

[0134] The association relationship fusion submodule is used to fuse the location association relationship and the object type association relationship to obtain the association analysis results between the target recognition boxes.

[0135] In one embodiment, the association analysis module includes:

[0136] For any given bounding box:

[0137] The relation weighted summation submodule is used to perform a weighted summation of the correlation analysis results with other recognition boxes to obtain the total correlation analysis result;

[0138] The relationship comparison submodule is used to compare the total correlation analysis results with the correlation threshold;

[0139] The doubtful block determination submodule is used to determine the image block corresponding to the recognition box as a doubtful recognition image block if it is lower than the correlation threshold.

[0140] The reference block determination submodule is used to determine the image block corresponding to the recognition box as the reference recognition image block if it is higher than or equal to the correlation threshold.

[0141] In one embodiment, the feature processing module includes:

[0142] A brightness extraction submodule is used to perform a first convolution process and a second convolution process on the brightness information to obtain a first brightness feature and a second brightness feature; the first convolution process and the second convolution process are the same convolution process;

[0143] The texture extraction submodule is used to perform third convolution processing and spatial attention processing on the texture information to obtain texture features and texture spatial attention features;

[0144] The first fusion submodule is used to fuse the first brightness feature and the texture feature to obtain a first fusion result;

[0145] The second fusion submodule is used to fuse the second brightness feature and the texture space attention feature, and to enhance the fusion result to obtain the second fusion result;

[0146] The fusion and stitching submodule is used to perform cross-fusion based on the first fusion result and the second fusion result, and to stitch them based on the brightness channel and the texture channel respectively, to obtain the doubt fusion feature map corresponding to the doubt identification image block.

[0147] In one embodiment, the texture extraction submodule includes:

[0148] The first convolutional unit is used to perform a fourth convolutional process on the texture information to obtain the first texture sub-feature;

[0149] The transformation unit is used to perform a real-number fast Fourier transform on the first texture sub-feature to obtain the first frequency domain feature;

[0150] The second convolutional unit is used to perform a fifth convolution on the first frequency domain features to obtain the second frequency domain features;

[0151] The inverse transform unit is used to perform a real-number fast Fourier inverse transform on the second frequency domain feature to obtain the second texture sub-feature;

[0152] The fusion unit is used to fuse the first texture sub-feature and the second texture sub-feature, and to perform convolution processing on the fusion result to obtain texture space attention features.

[0153] In one embodiment, the feature enhancement module includes:

[0154] The graph acquisition submodule is used to acquire the correlation graph between the devices and components of the target industrial equipment; wherein, the correlation graph is used to characterize the degree of mutual influence when each device or component fails, and each device or component has a corresponding influence value on other devices or components; wherein, the other devices or components are the devices or components included in the target industrial equipment other than the current device or component;

[0155] The image patch classification submodule is used to determine, based on the association map, core reference recognition image patches with an influence value higher than or equal to a set value and non-core reference recognition image patches with an influence value lower than the set value in the reference recognition image patches;

[0156] The difference enhancement submodule is used to enhance the core reference recognition image block according to a first enhancement factor and enhance the non-core reference recognition image block according to a second enhancement factor to obtain the corresponding reference enhancement feature map; wherein, the enhancement intensity of the first enhancement factor is greater than the enhancement intensity of the second enhancement factor.

[0157] In one embodiment, the fault prediction module includes:

[0158] The fault identification submodule is used to identify the fault operating status of the equipment based on the doubt fusion feature map and the reference enhanced feature map, and to determine the identified target faulty device.

[0159] The associated device determination submodule is used to determine, based on the association map, associated devices that have a strong influence association with the target faulty device;

[0160] The result output submodule is used to output fault monitoring results based on the target faulty device and the associated device.

[0161] In one embodiment, the device further includes:

[0162] The video segment determination module is used to determine the timing of each video frame of the monitoring video, and to determine the high-incidence video segment corresponding to the high-incidence period of the fault and the low-incidence video segment corresponding to the low-incidence period of the fault based on the timing of each video frame.

[0163] The frame extraction module is used to extract frames from the high-frequency video segments according to a first frame extraction rate and to extract frames from the low-frequency video segments according to a second frame extraction rate; wherein the first frame extraction rate is higher than the second frame extraction rate.

[0164] The video frame determination module is used to arrange the extracted video frames in time sequence to obtain the target video frame.

[0165] The following describes a storage medium provided in an embodiment of this application. The storage medium described below can be referred to in conjunction with the embodiments of the neural network-based real-time monitoring method, system and device for industrial equipment faults described above, as well as their corresponding technical effects.

[0166] This application also discloses a storage medium storing computer-executable instructions. When these instructions are loaded and executed by a processor, they implement the neural network-based real-time fault monitoring method for industrial equipment disclosed in any of the foregoing embodiments. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0167] It should be noted that the above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0169] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for real-time fault monitoring of industrial equipment based on neural networks, characterized in that, include: Acquire surveillance video of the target industrial equipment, and perform the following steps on at least a portion of the target video frames using a pre-trained neural network model: Image segmentation and classification are used to determine the bounding boxes and corresponding feature information of objects contained in the area where the target industrial equipment is located from the target video frame; A correlation analysis is performed on the feature information between each pair of recognition boxes; based on the correlation analysis results, suspicious recognition image blocks and reference recognition image blocks are determined; wherein, the correlation between the recognition box corresponding to the suspicious recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are the recognition boxes in the recognition boxes of objects contained in the area where the target industrial equipment is located, excluding the recognition boxes corresponding to the suspicious recognition image block or the reference recognition image block; Brightness and texture information are extracted from the suspicious image block. Different feature extraction processes are performed on the brightness and texture information, and the extracted features are fused to obtain the suspicious fusion feature map corresponding to the suspicious image block. Among them, the feature extraction process corresponding to the texture information is used to focus on texture space feature information. The reference recognition image block is enhanced to obtain the corresponding reference enhanced feature map; Based on the doubt fusion feature map and the reference enhanced feature map, the fault operation status of the equipment and components included in the target industrial equipment is identified, and the fault monitoring results are output.

2. The method according to claim 1, characterized in that, The feature information includes location feature information and object type feature information; The correlation analysis of feature information between pairwise recognition boxes includes: For any two target bounding boxes: The positional feature information between the target recognition boxes is analyzed to obtain the positional association relationship; The object type feature information between the target recognition boxes is analyzed for correlation to obtain the object type correlation relationship; The location association and the object type association are fused to obtain the association analysis results between the target recognition boxes.

3. The method according to claim 2, characterized in that, The determination of the suspected image block and the reference image block based on the correlation analysis results includes: For any given bounding box: The correlation analysis results between the two boxes are weighted and summed to obtain the total correlation analysis result. The overall correlation analysis results are compared with the correlation threshold; If the correlation threshold is lower than the threshold, the image block corresponding to the recognition box is identified as a suspicious recognition image block. If the correlation threshold is higher than or equal to the correlation threshold, then the image block corresponding to the recognition box is determined as the reference recognition image block.

4. The method according to claim 1, characterized in that, The step of performing different feature extraction processes on the brightness information and the texture information, and then fusing the extracted features to obtain the doubt fusion feature map corresponding to the doubt identification image block includes: The brightness information is subjected to a first convolution process and a second convolution process to obtain a first brightness feature and a second brightness feature; the first convolution process and the second convolution process are the same convolution process; The texture information is subjected to third convolution processing and spatial attention processing respectively to obtain texture features and texture spatial attention features; The first brightness feature and the texture feature are fused to obtain a first fusion result; The second brightness feature and the texture space attention feature are fused, and the fusion result is enhanced to obtain the second fusion result; Cross-fusion is performed based on the first fusion result and the second fusion result, and the images are stitched together based on the brightness channel and the texture channel respectively to obtain the doubt fusion feature map corresponding to the doubt identification image block.

5. The method according to claim 4, characterized in that, Spatial attention processing is performed on the texture information to obtain texture spatial attention features, including: The texture information is subjected to a fourth convolution process to obtain the first texture sub-feature; Perform a real-number fast Fourier transform on the first texture sub-feature to obtain the first frequency domain feature; The first frequency domain feature is subjected to a fifth convolution process to obtain the second frequency domain feature; Perform a real-number inverse fast Fourier transform on the second frequency domain feature to obtain the second texture sub-feature; The first texture sub-feature and the second texture sub-feature are fused, and the fusion result is convolved to obtain texture space attention features.

6. The method according to claim 1, characterized in that, The enhancement process of the reference recognition image block to obtain the corresponding reference enhanced feature map includes: Obtain the correlation map between the devices and components of the target industrial equipment; wherein, the correlation map is used to characterize the degree of mutual influence when each device or component fails, and each device or component has a corresponding influence value on other devices or components; wherein, the other devices or components are the devices or components included in the target industrial equipment other than the current device or component; Based on the association map, core reference recognition image blocks with influence values ​​higher than or equal to a set value and non-core reference recognition image blocks with influence values ​​lower than the set value are identified in the reference recognition image blocks. The core reference recognition image block is enhanced according to a first enhancement factor, and the non-core reference recognition image block is enhanced according to a second enhancement factor to obtain a corresponding reference enhanced feature map; wherein, the enhancement intensity of the first enhancement factor is greater than the enhancement intensity of the second enhancement factor.

7. The method according to claim 6, characterized in that, The process of identifying the fault operating status of the equipment components included in the target industrial equipment based on the doubtful fusion feature map and the reference enhanced feature map, and outputting fault monitoring results, includes: Based on the doubtful fusion feature map and the reference enhanced feature map, the fault operating state of the equipment device is identified, and the target faulty device is determined. Based on the correlation map, related devices with a strong influence correlation with the target faulty device are identified; The fault monitoring results are output based on the target faulty device and the associated devices.

8. The method according to claim 1, characterized in that, After acquiring the monitoring video of the target industrial equipment, the process also includes: The timing sequence corresponding to each video frame of the monitoring video is determined, and the high-incidence video segment corresponding to the high-incidence period of the fault and the low-incidence video segment corresponding to the low-incidence period of the fault are determined based on the timing sequence corresponding to each video frame. The high-frequency video segments are sampled at a first frame rate, and the low-frequency video segments are sampled at a second frame rate; wherein the first frame rate is higher than the second frame rate. The extracted video frames are arranged in chronological order to obtain the target video frames.

9. A real-time fault monitoring system for industrial equipment based on neural networks, characterized in that, The system includes interconnected camera equipment and electronic equipment; The camera device is used to acquire surveillance video of the target industrial equipment and send the surveillance video to the electronic device; The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 8.

10. A real-time fault monitoring device for industrial equipment based on neural networks, characterized in that, include: The video acquisition module is used to acquire monitoring videos of the target industrial equipment and performs the following steps on at least a portion of the target video frames using a pre-trained neural network model: The device identification module is used to determine the identification box and corresponding feature information of the objects contained in the area where the target industrial equipment is located from the target video frame through image segmentation and classification; The correlation analysis module is used to perform correlation analysis on the feature information between pairs of recognition boxes; based on the correlation analysis results, it determines suspicious recognition image blocks and reference recognition image blocks; wherein, the correlation between the recognition box corresponding to the suspicious recognition image block and other recognition boxes is lower than a set condition, and the correlation between the recognition box corresponding to the reference recognition image block and other recognition boxes is higher than or equal to the set condition; wherein, the other recognition boxes are the recognition boxes in the recognition boxes of objects contained in the area where the target industrial equipment is located, excluding the recognition boxes corresponding to the suspicious recognition image block or the reference recognition image block; The feature processing module is used to extract brightness information and texture information from the suspicious image block, perform different feature extraction processes on the brightness information and the texture information, and perform feature fusion on the extracted features to obtain the suspicious fusion feature map corresponding to the suspicious image block; wherein, the feature extraction processing corresponding to the texture information is used to focus on texture space feature information; The feature enhancement module is used to enhance the reference recognition image block to obtain the corresponding reference enhanced feature map; The fault prediction module is used to identify the fault operation status of the equipment components included in the target industrial equipment based on the doubt fusion feature map and the reference enhanced feature map, and output the fault monitoring results.

Images