Method, system and equipment for identifying defects of coal-fired equipment based on large model

Abstract

A method, system and device for identifying coal-fired equipment defects based on a large model relate to the technical field of equipment defect identification, and the method comprises the following steps: training labeled target domain data to generate a primary coal-fired equipment defect identification model; migrating the general features and the invariant domain features extracted from the source domain data to the first-level model through transfer learning to generate a second-level model; inputting the sensor data and preset random noise into the generative adversarial network embedded with the thermodynamic residual project to generate standard counterfeit defect data; training the standard counterfeit defect data to generate a third-level model; generating a cross-modal fusion feature based on a preset strategy and adding the cross-modal fusion feature into the third-stage model to generate a fourth-stage model; and inputting a to-be-identified coal-fired equipment picture and working condition parameters into the four-stage model to obtain an identification result and confidence. Through multi-stage model training and cross-modal fusion feature extraction, the problems of low recognition precision and poor generalization ability in related technologies are solved, and the accuracy and reliability of coal-fired equipment defect recognition are improved.

Description

Technical Field

[0001] The present application relates to the technical field of equipment defect identification, and specifically to a method, system, equipment and storage medium for identifying defects in coal-fired equipment based on a large model. Background Art

[0002] Coal-fired power generation plays a crucial role in my country's energy mix, and the safe and stable operation of coal-fired equipment has a significant impact on the reliability of the power system. Coal-fired equipment is susceptible to various defects, such as cracks, wear, and corrosion, under long-term high-temperature, high-pressure, and high-load operation. If these defects are not discovered and addressed promptly, they can lead to equipment failure or even major safety accidents. Therefore, the accurate identification of coal-fired equipment defects is of great practical significance.

[0003] Currently, coal-fired equipment defect identification primarily relies on manual inspections and traditional image processing techniques, but these methods suffer from low efficiency and accuracy. With the development of artificial intelligence (AI), deep learning-based defect identification methods are increasingly being applied to coal-fired equipment defect detection. For example, CN119205743A discloses a defect identification method that integrates a discriminant model with a generative model. This method generates training data using a pre-set generative model and a set of production parameters. The discriminant model is then trained based on the generated data set to improve the accuracy and efficiency of defect identification.

[0004] In practical applications, sample data on coal-fired equipment defects is often insufficient, especially for new or rare defects, and it is difficult to obtain sufficient labeled data for model training. To solve the problem of insufficient data, transfer learning technology has been widely used in the field of defect recognition. CN114972299B proposed a defect detection method based on deep transfer learning. By designing a pre-trained model and utilizing the discriminant joint distribution adaptive similarity feature, it effectively reduces the differences between features and enhances the generalization ability of the model. Similarly, CN118155066A introduced a defect recognition method based on transfer learning and improved YOLOv5. This method uses a defect image dataset as the source domain and another defect image dataset as the target domain, and obtains the target defect recognition model through transfer learning.

[0005] For complex industrial environments such as coal-fired equipment, single-modal data often cannot fully reflect the equipment status. CN115115919B proposed a method for identifying thermal defects in power grid equipment. By performing cross-modal feature information interaction on visible light and infrared light image features, synthetic modal shared features are obtained, and image features of different modalities are enhanced and fused, thereby improving the accuracy of thermal defect identification. CN114881940A, targeting the problem of identifying head defects of high-temperature alloy bolts after hot upsetting, proposed an identification method that can handle differences in image brightness. By establishing a residual neural network model and conducting two training sessions, it solved the problem of unstable image quality in industrial environments.

[0006] However, existing technologies still have the following problems in identifying defects in coal-fired equipment:

[0007] 1. The operating environment of coal-fired equipment is complex. Harsh working conditions such as high dust, high temperature, and strong vibration lead to high sensor noise and easy failure of single-modal data. Existing multimodal fusion methods lack effective alignment and fusion of spatiotemporal information, making it difficult to fully utilize the complementary advantages of multi-source heterogeneous data.

[0008] 2. It is difficult to obtain real defect samples, especially new or rare coal-fired equipment defects, which lack labeled data. Existing data augmentation and generation methods often ignore physical constraints, and the generated samples may not conform to physical laws such as thermodynamics, reducing the practicality of the model.

[0009] 3. There are significant domain differences between different equipment types (such as coal-fired and photovoltaic equipment). Existing transfer learning methods lack generalization capabilities when dealing with cross-domain knowledge transfer, and it is difficult to effectively use source domain knowledge to guide defect identification tasks in the target domain.

[0010] Therefore, there is an urgent need for a coal-fired equipment defect identification method that can comprehensively utilize multimodal data, data generation methods that conform to physical laws, effective transfer learning strategies, and knowledge graphs to improve the accuracy, robustness, and interpretability of identification. Summary of the Invention

[0011] The present application provides a method, device / system, equipment and computer-readable storage medium for identifying defects in coal-fired equipment based on a large model, which can solve xx technical problems existing in the prior art.

[0012] In a first aspect, an embodiment of the present application provides a method for identifying defects in coal-fired equipment based on a large model, the method comprising:

[0013] Training the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data;

[0014] By transfer learning, the common features extracted from the source domain data and the invariant domain features obtained by adversarial training between the source domain data and the target domain data are transferred to the first-level coal-fired equipment defect recognition model to generate a second-level coal-fired equipment defect recognition model;

[0015] Inputting a coal-fired equipment defect dataset and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws;

[0016] Based on the secondary coal-fired equipment defect recognition model, the standard forged defect data is trained to generate a tertiary coal-fired equipment defect recognition model;

[0017] generating a cross-modal fusion feature based on the sensor data and a preset strategy, and adding the cross-modal fusion feature to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model;

[0018] The image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment are input into the four-level coal-fired equipment defect recognition model to obtain the recognition results and confidence levels.

[0019] In conjunction with the first aspect, in one embodiment, inputting the coal-fired equipment defect dataset and preset random noise into a generative adversarial network (GAN) to generate standard simulated defect data includes:

[0020] Embedding a thermodynamic residual term in a loss function of a GAN generator, wherein the thermodynamic residual term includes a plurality of preset thermodynamic equations;

[0021] The coal-fired equipment defect dataset and preset random noise are input into the generator of the Generative Adversarial Network (GAN) to generate basic counterfeit defect data;

[0022] The preset real defect data or the basic simulated defect data is input into the discriminator of the Generative Adversarial Network (GAN) to generate standard simulated defect data.

[0023] In conjunction with the first aspect, in one embodiment, embedding a thermodynamic residual term in the loss function of a GAN generator includes:

[0024] Physical quantities related to thermodynamic residual items, including crack orientation angle, thermal stress, and thermal diffusion residual, are embedded in the loss function of the GAN generator.

[0025] In conjunction with the first aspect, in one embodiment, generating a cross-modal fusion feature based on the sensor data and a preset strategy includes:

[0026] Preprocessing sensor data, wherein the sensor data includes: thermal imaging data, acoustic signals, and vibration data;

[0027] Extracting single modal features from each of the sensor data, wherein the single modal features include thermal imaging modal features, acoustic modal features, and vibration modal features;

[0028] Performing spatiotemporal alignment processing on the plurality of unimodal features to obtain spatiotemporal aligned unimodal features;

[0029] fusing the timestamps and spatial coordinates of the spatiotemporally aligned unimodal features through spatiotemporal encoding to generate spatiotemporal fused features;

[0030] A cross-modal fusion feature is generated based on the attention mechanism and the spatiotemporal fusion feature.

[0031] In combination with the first aspect, in one embodiment, generating a cross-modal fusion feature based on the attention mechanism and the spatiotemporal fusion feature includes:

[0032] Generate multi-head single-modal self-fusion features based on the spatiotemporal fusion features and the multi-head self-attention mechanism;

[0033] Cross-modal fusion features are generated based on the multi-head single-modal self-fusion features and the cross-modal attention mechanism (Cross-modal Attention).

[0034] In conjunction with the first aspect, in one embodiment, performing spatiotemporal alignment processing on the plurality of unimodal features to obtain spatiotemporal aligned unimodal features includes:

[0035] performing linear interpolation on the vibration modal feature according to the timestamp of the thermal imaging modal feature to generate a vibration modal feature that is synchronized with the thermal imaging data;

[0036] intercepting the acoustic modal feature according to the imaging frame time window of the thermal imaging modal feature to generate an acoustic modal feature synchronized with the thermal imaging data;

[0037] Establishing a mapping relationship between a pixel coordinate system and a physical coordinate system based on parameters of the thermal imaging device, and inputting the thermal imaging modal features into the physical coordinate system;

[0038] Acquiring the azimuth angle of the coal-fired equipment defect by forming a microphone beam array, and inputting the acoustic modal characteristics into the physical coordinate system;

[0039] By associating the vibration measurement points with the physical coordinates, the vibration modal characteristics are input into the physical coordinate system.

[0040] In conjunction with the first aspect, in one embodiment, the step of inputting the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect recognition model to obtain the recognition result and confidence level further includes:

[0041] Constructing a knowledge graph, which includes: defining structural data, material properties, historical maintenance records, and known defect descriptions of the coal-fired equipment as inputs; defining equipment components, material properties, defect types, and failure modes of the coal-fired equipment as entities; and defining associations between defects caused by materials, symptoms caused by known defects, and maintenance plans as relationships;

[0042] Extracting semantic information features of known defects from the knowledge graph through a graph neural network (GNN);

[0043] Training a linear projection matrix based on the semantic information features of the known defects and the cross-modal fusion features of the known defects, and then inputting the cross-modal fusion features of the unknown defects into the linear projection matrix to obtain the semantic information features of the unknown defects;

[0044] Calculating the cosine similarity between each unknown defect and each known defect based on the semantic information features of the known defects and the semantic information features of the unknown defects;

[0045] According to the cosine similarity, the probability distribution of the category to which the unknown defect belongs is obtained to obtain a recognition result.

[0046] In a second aspect, an embodiment of the present application provides a system for identifying defects in coal-fired equipment based on a large model, the system comprising:

[0047] a first generation module, configured to train the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data;

[0048] The second generation module is used to transfer the common features extracted from the source domain data and the invariant domain features obtained by adversarial training between the source domain data and the target domain data to the first-level coal-fired equipment defect recognition model through transfer learning, so as to generate a second-level coal-fired equipment defect recognition model;

[0049] a third generation module, configured to input the sensor data and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in a loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws;

[0050] a fourth generating module, configured to train the standard forged defect data based on the secondary coal-fired equipment defect recognition model to generate a tertiary coal-fired equipment defect recognition model;

[0051] a fifth generating module, configured to generate a cross-modal fusion feature based on a preset strategy, and add the fusion feature to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model;

[0052] The recognition module is used to input the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect recognition model to obtain the recognition result and confidence level.

[0053] In a third aspect, an embodiment of the present application provides a device for identifying defects in coal-fired equipment based on a large model, wherein the device for identifying defects in coal-fired equipment based on a large model comprises a processor, a memory, and a program for identifying defects in coal-fired equipment based on a large model stored in the memory and executable by the processor, wherein when the program for identifying defects in coal-fired equipment based on a large model is executed by the processor, the steps of the method for identifying defects in coal-fired equipment based on a large model as described above are implemented.

[0054] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which is stored a program for identifying defects in coal-fired equipment based on a large model. When the program for identifying defects in coal-fired equipment based on a large model is executed by a processor, the steps of the method for identifying defects in coal-fired equipment based on a large model as described above are implemented.

[0055] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0056] A first-level coal-fired equipment defect recognition model is generated by training on labeled target domain data, wherein the target domain data is acquired coal-fired equipment data; common features extracted from source domain data and invariant domain features obtained by adversarial training of source domain data and target domain data are transferred to the first-level coal-fired equipment defect recognition model through transfer learning to generate a second-level coal-fired equipment defect recognition model; sensor data and preset random noise are input into a generative adversarial network (GAN) to generate standard simulated defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN). To ensure that the standard forged defect data conforms to the laws of physics; on the basis of the second-level coal-fired equipment defect recognition model, the standard forged defect data is trained to generate a third-level coal-fired equipment defect recognition model; based on the sensor data and preset strategies, cross-modal fusion features are generated, and the cross-modal fusion features are added to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model; the coal-fired equipment image to be identified and the coal-fired equipment operating parameters are input into the fourth-level coal-fired equipment defect recognition model to obtain recognition results and confidence levels, thereby solving the technical problems of low recognition accuracy and poor generalization ability in related technologies.

[0057] The beneficial effects of the present invention are:

[0058] 1. Through the integration of thermal imaging, acoustic data and vibration data through multimodal fusion technology, the system can detect the presence of dust in extreme working conditions (such as dust concentration of 500μg / m 3 , high temperature, and strong vibration environment) can still maintain high detection accuracy through multi-modal redundancy verification, significantly improving the robustness and reliability of industrial equipment fault detection in complex environments.

[0059] 2. The generative model based on physical constraints can produce defect samples that conform to the physical laws of coal-fired equipment. By embedding thermodynamic residual terms in the GAN loss function, it ensures that the generated crack direction is consistent with the temperature gradient, effectively solving the small sample problem.

[0060] 3. Cross-domain transfer learning technology enables the model to migrate from source domain knowledge (such as photovoltaic equipment) to the coal-fired equipment field. By aligning feature distribution through ResNet-50 pre-training and gradient reversal layers, domain shift is reduced and model generalization ability is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of an embodiment of a method for identifying defects in coal-fired equipment based on a large model of the present application;

[0062] Figure 2 For this application Figure 1 Detailed flow chart of step S30;

[0063] Figure 3 This is a schematic diagram of the architecture of an embodiment of a system for identifying coal-fired equipment defects based on a large model according to the present application;

[0064] Figure 4 This is a schematic diagram of the hardware structure of the equipment for identifying defects in coal-fired equipment based on a large model involved in the embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] First, some technical terms in this application are explained to facilitate those skilled in the art to understand this application.

[0067] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0068] In a first aspect, an embodiment of the present application provides a method for identifying defects in coal-fired equipment based on a large model.

[0069] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the method for identifying coal-fired equipment defects based on a large model in this application. Figure 1 As shown in FIG, the method for identifying defects in coal-fired equipment based on a large model includes:

[0070] Step S10: training the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data;

[0071] For example, the target domain data includes thermal images, acoustic and vibration data of coal-fired equipment, and corresponding defect annotation information. This data is collected on-site at coal-fired power plants and annotated by professional technicians. The annotations include information such as defect type (such as cracks, corrosion, and wear), defect location, and defect severity.

[0072] The training process adopts a deep learning framework and uses convolutional neural network (CNN) as the basic model architecture.

[0073] Step S20: Through transfer learning, the common features extracted from the source domain data and the invariant domain features obtained by adversarial training of the source domain data and the target domain data are transferred to the first-level coal-fired equipment defect recognition model to generate a second-level coal-fired equipment defect recognition model;

[0074] For example, the source data comes from a trained and publicly available dataset of industrial equipment defects, containing a large number of different types of industrial equipment defect images and sensor data. Through transfer learning, the knowledge in the source domain data can be used to improve defect recognition performance in the target domain (i.e., coal-fired equipment).

[0075] The transfer learning process consists of two main parts: universal feature extraction and domain-invariant feature learning. Universal feature extraction uses a pre-trained ResNet-50 network, pre-trained on the ImageNet dataset, capable of extracting universal features from the source domain data. Domain-invariant feature learning uses a Domain Adversarial Neural Network (DANN). Through adversarial training, the features learned by the model are invariant between the source and target domains.

[0076] The adversarial relationship between the feature extractor and the domain classifier is realized through the gradient reversal layer, so that the features generated by the feature extractor can deceive the domain classifier, thereby learning the domain-invariant feature representation.

[0077] Step S30: Inputting the coal-fired equipment defect dataset and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws;

[0078] For example, the process of constructing a coal-fired equipment dataset includes: collecting images of real coal-fired equipment defects (such as cracks, corrosion, and dust accumulation), and annotating the defect type and location. Recording equipment operating parameters: Synchronously collecting sensor data such as temperature, pressure, and vibration, and constructing a correspondence between time series and images. First, image preprocessing is performed, adjusting the image size to a uniform resolution, normalizing it to the range of [-1,1], and applying enhancement operations such as random cropping, rotation, and noise addition to improve generalization. Then, the equipment parameters are standardized. Among them, numerical parameters such as temperature and pressure are normalized to prevent dimensional differences from affecting model training.

[0079] After constructing the coal-fired equipment dataset, if there are insufficient real defect samples, image editing tools are used to simulate defect areas on normal equipment images to form a pseudo coal-fired equipment defect dataset.

[0080] The preset random noise obeys the standard normal distribution z~N(0,1), and its dimension matches the characteristic dimension of sensor data, which can improve the diversity of generated data.

[0081] The GAN loss function embeds thermodynamic residuals, including multiple preset thermodynamic equations, such as the heat conduction equation and the thermal stress equation. These equations describe the physical laws of coal-fired equipment during operation, ensuring that the generated defect data conforms to actual physical phenomena.

[0082] The physical quantities in the thermodynamic residual project include crack direction angle, thermal stress, and thermal diffusion residual. The crack direction angle represents the angle of the crack relative to the equipment surface, with a value range of 0-180 degrees; thermal stress represents the stress caused by uneven heating of the equipment material, with the unit of MPa; thermal diffusion residual represents the difference between actual thermal diffusion and theoretical thermal diffusion, with the unit of m 2 / s.

[0083] During GAN training, the coal-fired equipment dataset and preset random noise are first input into the GAN generator to generate basic simulated defect data. The preset real defect data or basic simulated defect data is then input into the GAN discriminator for multimodal discrimination. The discriminator outputs the discrimination results, and the generator continuously adjusts its parameters based on the discrimination results, ultimately generating standard simulated defect data.

[0084] Among them, multimodal discrimination includes: the discriminator simultaneously receives the coal-fired equipment defect image and the corresponding equipment parameters, judges the authenticity of the data through feature splicing, and enhances the consistency of the generated sample with the physical state. The specific implementation process includes:

[0085] 1. Feature extraction of coal-fired equipment defect images: Input coal-fired equipment defect images and gradually downsample them through the convolutional layer (Conv2d) to obtain high-dimensional feature maps.

[0086] 2. Coal-fired equipment parameter encoding: Equipment parameters such as temperature and pressure are mapped into 64-dimensional vectors through a fully connected layer, expanding to the same spatial dimension as the image features.

[0087] 3. Feature merging: The coal-fired equipment defect image features (512 dimensions) and the equipment parameter vector (64 dimensions) are spliced ​​along the channel dimension to form a joint feature.

[0088] 4. Classification and discrimination: The joint features are input into the fully connected layer, and the authenticity probability is output through the Sigmoid function, indicating the confidence that the image is real data.

[0089] Step S40: Based on the secondary coal-fired equipment defect recognition model, the standard forged defect data is trained to generate a tertiary coal-fired equipment defect recognition model;

[0090] As an example, the standard simulated defect data generated in the previous step is combined with the original real defect data to form an expanded training data set. Based on the secondary coal-fired equipment defect recognition model, the expanded training data set is used for further training to generate a tertiary coal-fired equipment defect recognition model.

[0091] Step S50: generating a cross-modal fusion feature based on the sensor data and the preset strategy, and adding the cross-modal fusion feature to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model;

[0092] For example, sensor data is first preprocessed. The sensor data includes thermal imaging data, acoustic data, and vibration data. Thermal imaging data preprocessing includes image normalization, noise removal, and dehazing enhancement; acoustic data preprocessing includes filtering, time-frequency transformation, noise reduction, and feature extraction; and vibration data preprocessing includes detrending, filtering, and time-frequency analysis.

[0093] Then, we extract unimodal features from each sensor data. Thermal imaging modal features are spatial features extracted from thermal imaging data using ResNet-18. Acoustic modal features are frequency domain features extracted from acoustic wave data using a three-layer convolution and pooling 1D-CNN. Vibration modal features are temporal dependency features captured from vibration data using an LSTM network.

[0094] Step S60: Inputting the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect identification model to obtain the identification result and confidence level.

[0095] For example, images of the coal-fired equipment to be identified are captured using a thermal imaging camera with a resolution of 640×480 pixels and a sampling frequency of 30Hz. Coal-fired equipment operating parameters, including operating temperature, pressure, and load, are collected in real time by the equipment monitoring system.

[0096] The preprocessed coal-fired equipment images and operating parameters are fed into the four-stage coal-fired equipment defect recognition model. The model then outputs the defect type, location, and corresponding confidence level. The confidence level indicates the model's confidence in the recognition result, ranging from 0 to 1, with higher values ​​indicating greater confidence.

[0097] In this embodiment, a first-level coal-fired equipment defect recognition model is generated by training the labeled target domain data, wherein the target domain data is the acquired coal-fired equipment data;

[0098] Through transfer learning, the common features extracted from the source domain data and the invariant domain features obtained by adversarial training of the source domain data and the target domain data are transferred to the first-level coal-fired equipment defect recognition model to generate a second-level coal-fired equipment defect recognition model; the coal-fired equipment defect data set and the preset random noise are input into the generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to the laws of physics; in the second-level coal-fired equipment defect recognition model On the basis of this, the standard forged defect data is trained to generate a three-level coal-fired equipment defect recognition model; based on the sensor data and the preset strategy, a cross-modal fusion feature is generated, and the cross-modal fusion feature is added to the three-level coal-fired equipment defect recognition model to generate a four-level coal-fired equipment defect recognition model; the coal-fired equipment image to be identified and the coal-fired equipment operating parameters are input into the four-level coal-fired equipment defect recognition model to obtain the recognition result and confidence, which solves the problem of insufficient generalization ability in cross-domain knowledge transfer in related technologies and difficulty in effectively using source domain knowledge to guide the defect recognition task in the target domain.

[0099] Furthermore, in one embodiment, referring to Figure 2 , Figure 2 For this application Figure 1 Detailed flow chart of step S30 in FIG. Figure 2 As shown, the inputting of sensor data and preset random noise into a GAN to generate standard counterfeit defect data includes:

[0100] Step S31: embedding a thermodynamic residual term into the loss function of a generative adversarial network (GAN) generator, wherein the thermodynamic residual term includes a plurality of preset thermodynamic equations;

[0101] For example, the residual items in thermodynamics include the heat conduction equation, the thermal stress equation, and the thermal expansion equation. The heat conduction equation describes the heat transfer law in the material, the thermal stress equation describes the stress distribution caused by temperature changes, and the thermal expansion equation describes the dimensional changes of the material caused by temperature changes.

[0102] These thermodynamic equations are transformed into constraints in the loss function to ensure that the generated defect data conforms to physical laws. The weight coefficient of the thermodynamic residual term is set to 0.5 and dynamically adjusted during training to balance the authenticity and physical plausibility of the generated data.

[0103] Step S32: inputting the coal-fired equipment defect data set and preset random noise into the generator of the Generative Adversarial Network (GAN) to generate basic simulated defect data;

[0104] For demonstration purposes, a coal-fired equipment defect dataset was preprocessed and concatenated with preset random noise before serving as input to the generator. The generator uses a U-Net architecture, consisting of an encoder and a decoder. The encoder maps the input data into a latent space, while the decoder converts the latent representation into the underlying simulated defect data.

[0105] Step S33: inputting the preset real defect data or the basic simulated defect data into the discriminator of the Generative Adversarial Network (GAN) to generate standard simulated defect data.

[0106] For example, the discriminator receives preset real defect data or basic simulated defect data as input and outputs a discrimination result. The discriminator uses the PatchGAN structure to divide the input image into multiple small blocks for discrimination, improving the precision of the discrimination.

[0107] The generator continuously optimizes parameters according to the feedback from the discriminator, and the basic simulated defect data finally generated is post-processed (such as filtering, normalization, etc.) to form standard simulated defect data.

[0108] After integrating device parameters with image features, the discriminator can simultaneously verify the visual plausibility and physical consistency of the generated samples. For example, corrosion defects generated under high-pressure conditions must match the corresponding stress distribution pattern; otherwise, they are judged as false samples.

[0109] Specifically, embedding a thermodynamic residual item in the loss function of a generative adversarial network (GAN) generator includes: embedding physical quantities related to the thermodynamic residual item in the loss function of the generative adversarial network (GAN) generator, wherein the physical quantities include crack direction angle, thermal stress, and thermal diffusion residual.

[0110] Demonstratively, physical quantities related to thermodynamic residual items are embedded in the loss function of the GAN generator, including crack direction angle, thermal stress, and thermal diffusion residual.

[0111] The loss function of the embedded physical quantity is L total =L 对抗 +λ1||θ 生成 -θ 理论 || 2 +λ2R 热扩散 +λ3||σ 生成 -σ 理论 || 2 , where L total is the total loss, L 对抗 is the WGAN-GP loss, θ 生成 is the crack direction angle in the generated data, θ 理论 is the crack direction angle predicted by the theoretical model, R 热扩散 is the thermal diffusion residual, σ生成 To generate the thermal stress in the data, σ 理论 is the thermal stress predicted by the theoretical model, and λ1, λ2, and λ3 are weight coefficients.

[0112] This embodiment provides a physical constraint-based generative model that can generate defect samples that conform to the physical laws of coal-fired equipment. By embedding the thermodynamic residual equation in the GAN loss function, it can ensure that the generated crack direction is consistent with the temperature gradient, effectively solving the small sample problem.

[0113] Furthermore, in one embodiment, referring to Figure 3 , Figure 3 For this application Figure 1 Detailed flow chart of step S50 in FIG. Figure 3 As shown, generating cross-modal fusion features based on the sensor data and the preset strategy includes:

[0114] Step S51: pre-processing the sensor data, wherein the sensor data includes: thermal imaging data, acoustic wave data, and vibration data;

[0115] Exemplarily, thermal imaging data preprocessing includes: inputting an original thermal imaging image blurred by dust interference, using a dark channel prior dehazing algorithm, estimating a dust scattering model, and restoring a clear thermal imaging image.

[0116] The acoustic wave data preprocessing includes: taking the original acoustic wave data collected by the microphone array as input, first performing a 5-layer decomposition using a 4dB wavelet basis, applying a soft threshold to the high-frequency coefficients to remove noise, and retaining the defect-related impact acoustic wave data.

[0117] The vibration data preprocessing includes: taking the accelerator timing sequence signal as input, segmenting the data according to 1s window, retaining the 5-500Hz frequency band through bandpass filtering, and finally normalizing it to make its mean equal to 0 and variance equal to 1.

[0118] Step S52: extracting single modal features from each of the sensor data, wherein the single modal features include thermal imaging modal features, acoustic modal features, and vibration modal features;

[0119] For demonstration, thermal imaging modal features are extracted using a ResNet-18 network, with an output feature dimension of 512. The network parameters are obtained through pre-training and fine-tuned on coal-fired equipment data.

[0120] The acoustic modal features are extracted from the frequency domain features of the sound wave data through three-layer convolution and pooling 1D-CNN, and the output feature dimension is 256.

[0121] The vibration modal feature is the time-dependent feature captured by the LSTM network on the vibration data, and the output feature dimension is 128.

[0122] Step S53: performing spatiotemporal alignment processing on the plurality of unimodal features to obtain spatiotemporal aligned unimodal features;

[0123] For example, the vibration modal features are linearly interpolated according to the timestamps of the thermal imaging modal features to generate vibration modal features that are time-synchronized with the thermal imaging data. The interpolation method uses cubic spline interpolation to ensure smooth data transition.

[0124] For example, the sampling frequency of thermal imaging data is 30fps, and the sampling frequency of vibration data is 1000Hz, which has a significant frequency difference. The timestamp of thermal imaging data: t1 = 0.033s, t2 = 0.066s, ....t 30 = 1.0s, timestamp of vibration data: T1 = 0.001s, T2 = 0.002s, ..., T 1000 = 1.0s, taking the timestamp t1 = 0.033s of the thermal imaging data as an example, find the T closest to t1 in the vibration data. 33 =0.033s,T 34 = 0.034s, vibration value interpolation: in is the vibration value at time t1 after vibration data synchronization, Vibration data at T 33 The vibration value at the moment, Vibration data at T 34 The vibration value at time T 33 =The vibration value at 0.033s is 5.2mm / s, so the vibration value at time t1 after synchronization is 5.2mm / s.

[0125] The acoustic modal features are intercepted according to the imaging frame time window of the thermal imaging modal features to generate acoustic modal features that are time-synchronized with the thermal imaging data.

[0126] For example, the time window size

[0127] Intercept the waveform of the corresponding time period in the acoustic signal:

[0128] First frame (0s-0.033s): Extract the spectral characteristics (such as peak frequency and energy) of the acoustic signal in this interval.

[0129] Second frame (0.033s-0.066s): Repeat the operation to ensure that each acoustic frame strictly corresponds to the thermal imaging time.

[0130] A mapping relationship between the pixel coordinate system and the physical coordinate system is established based on the parameters of the thermal imaging device, and the thermal imaging modal features are input into the physical coordinate system. The mapping relationship is determined by the intrinsic and extrinsic parameters of the thermal imaging device. The intrinsic parameters include focal length, principal point coordinates, etc., and the extrinsic parameters include camera position and attitude.

[0131] For example, calibration parameters:

[0132] Internal parameters: focal length fx = 1000, principal point (u0, v0) = (640, 512)

[0133] External parameters: Equipment installation height Z = 2.0m, no rotation (R = IR = I).

[0134] Conversion steps:

[0135] Thermal imaging pixel coordinates (u, v) = (700, 500)

[0136] Convert to the thermal imaging device coordinate system:

[0137]

[0138] Physical coordinates (thermal imaging equipment reference): (X, Y, Z) = (0.12, -0.024, 2.0) m

[0139] By forming a microphone beam array, the azimuth of the coal-fired equipment defect is determined and the acoustic modal characteristics are input into the physical coordinate system. The microphone array consists of four omnidirectional microphones arranged in a circle with a diameter of 0.5 meters. The azimuth of the sound source is calculated using a delay and sum beamforming algorithm.

[0140] For example, acoustic data orientation calibration

[0141] Microphone array: 4 microphones in a linear array, the distance between each microphone is d = 0.1m, and the time difference between the sound wave reaching microphones 1 and 2 is Δt = 0.0002s

[0142] Defect azimuth calculation: Where φ is the defect azimuth and c is the speed of sound.

[0143] Defect location: Combined with the physical coordinates (0.12, -0.024, 2.0), confirm that the direction of the sound source is consistent with the high temperature area.

[0144] By associating vibration measurement points with physical coordinates, the vibration modal characteristics are input into the physical coordinate system. The physical position of the vibration sensor is determined through pre-calibration, and the corresponding relationship between the vibration measurement points and the three-dimensional space coordinates is established.

[0145] Step S54: fusing the timestamps and spatial coordinates of the spatiotemporally aligned unimodal features through spatiotemporal encoding to generate spatiotemporal fusion features;

[0146] For example, the spatiotemporal encoding uses a spatiotemporal positional encoding method to encode time and space information into a high-dimensional vector. The spatiotemporal fusion features include thermal imaging spatiotemporal fusion features, vibration spatiotemporal fusion features, and acoustic spatiotemporal fusion features.

[0147] In the specific implementation, for example, the input thermal imaging modal features F after time-space alignment A , acoustic modal characteristics F b , timestamp code E t , spatial coordinate encoding E s , fusion method: F′ A =F A +E t +E s ,F′ B =F B +E t , where F′ A is the spatiotemporal fusion feature of thermal imaging, F′ B It is the vibration space-time fusion feature.

[0148] Step S55: Generate cross-modal fusion features based on the attention mechanism and the spatiotemporal fusion features.

[0149] For demonstration purposes, we generate multi-head single-modality self-fusion features based on spatiotemporal fusion features and a multi-head self-attention mechanism. In the multi-head self-attention mechanism, the number of heads is set to 8, and the dimension of each head is 64. Self-attention is used to calculate feature correlations within each modality.

[0150] Cross-modal fusion features are generated based on multi-head single-modal self-fusion features and the cross-modal attention mechanism. The cross-modal attention mechanism calculates the feature correlation between different modalities and realizes the effective fusion of multimodal information.

[0151] Specifically, the generation of cross-modal fusion features based on the attention mechanism and the spatiotemporal fusion features includes: generating multi-head unimodal self-fusion features based on the spatiotemporal fusion features and a multi-head self-attention mechanism (Multi-head Self-Attention); generating cross-modal fusion features based on the multi-head unimodal self-fusion features and a cross-modal attention mechanism (Cross-modalAttention).

[0152] Demonstratively, multi-head single-modal self-fusion features are generated based on spatiotemporal fusion features and multi-head self-attention mechanism;

[0153] The multi-head self-attention mechanism maps the input features into three spaces: Query, Key, and Value. By calculating the similarity between the query and the key, the attention weights are obtained, and then the values are weighted and summed to generate the attention output.

[0154] Based on the multi-head single-modal self-fused features and the cross-modal attention mechanism (Cross-modal Attention), cross-modal fused features are generated.

[0155] The cross-modal attention mechanism calculates the feature correlations between different modalities. In the specific implementation, the thermal imaging modality features are used as the query, and the acoustic modality features and vibration modality features are used as the key and value respectively. Through the attention mechanism, the attention weights of the thermal imaging modality to other modalities are calculated to generate the cross-modal fused features.

[0156] The cross-modal attention calculation formula is the same as that of the multi-head self-attention, but the query, key, and value come from different modalities. After the calculation, the attention outputs of each modality are concatenated and fused through a fully connected layer to generate the final cross-modal fused features.

[0157] This example provides a method that enables the system to maintain high detection accuracy through multi-modal redundancy verification under extreme working conditions (such as a dust concentration of 500 μg / m 3 , high temperature, and strong vibration environment), significantly improving the robustness and reliability of industrial equipment fault detection in complex environments.

[0158] Further, in one embodiment, the spatio-temporal alignment processing of the multiple single-modal features to obtain the spatio-temporally aligned single-modal features includes:

[0159] Step S531: Linearly interpolate the vibration modality features according to the timestamps of the thermal imaging modality features to generate vibration modality features synchronized with the thermal imaging data in time;

[0160] The sampling frequency of the vibration data is 10 kHz, while the sampling frequency of the thermal imaging data is 30 Hz, and there is a difference in time resolution between the two. Through the linear interpolation method, the time resolution of the vibration modality features is adjusted to be the same as that of the thermal imaging modality features.

[0161] In the specific implementation, first determine the timestamp sequence {t_i} of the thermal imaging data, and then for each timestamp t_i, find the two closest time points t_a and t_b (satisfying t_a < t_i < t_b) in the vibration data time series, and calculate the vibration feature at the t_i moment through linear interpolation: f(t_i) = f(t_a) + (f(t_b) - f(t_a)) * (t_i - t_a) / (t_b - t_a).

[0162] Step S532: intercepting the acoustic modal feature according to the imaging frame time window of the thermal imaging modal feature to generate an acoustic modal feature synchronized with the thermal imaging data;

[0163] For example, the acoustic data is sampled at 44.1kHz and needs to be captured according to the time window of the thermal imaging data. For each thermal imaging frame, the corresponding time window [t_start, t_end] is determined. The data within this time window is then captured from the acoustic data, and features are extracted to generate acoustic modal features that are time-synchronized with the thermal imaging data.

[0164] The time window size is set to 100 ms with a step size of 50 ms (corresponding to a frame interval of approximately 33.3 ms when the thermal imaging frame rate is 30 Hz) to ensure the temporal alignment of the acoustic features with the thermal imaging features.

[0165] Step S533: establishing a mapping relationship between a pixel coordinate system and a physical coordinate system based on the parameters of the thermal imaging device, and inputting the thermal imaging modal features into the physical coordinate system;

[0166] Exemplarily, the parameters of the thermal imaging device include focal length f=18 mm, pixel size p=17 μm, principal point coordinates (cx, cy)=(320, 240), distortion coefficients k1=-0.15, k2=0.1.

[0167] Based on these parameters, a mapping relationship is established from pixel coordinates (u, v) to physical coordinates (X, Y, Z):

[0168] X=Z*(u-cx) / f

[0169] Y=Z*(v-cy) / f

[0170] Where Z is the depth information, which is obtained through an additional depth sensor or structured light technology.

[0171] Step S534: obtaining the azimuth angle of the coal-fired equipment defect by forming a microphone beam array, and inputting the acoustic modal characteristics into the physical coordinate system;

[0172] For example, the microphone beam array consists of 8 omnidirectional microphones arranged in a circle with a diameter of 0.5 meters. The sampling frequency of the microphones is 44.1kHz and the quantization accuracy is 24 bits.

[0173] The delay-and-sum beamforming algorithm calculates the intensity of sound sources in different directions. Specifically, for the azimuth angle σ of the coal-fired equipment defect, the delay of the signal received by each microphone is calculated. These delayed signals are then summed to determine the beam output for that direction. By scanning different azimuth angles, the direction with the maximum beam output is found, which is the azimuth angle of the sound source.

[0174] The azimuth angle θ is scanned from 0 to 360 degrees with a step size of 1 degree. The calculated azimuth angle corresponds to the angle in the physical coordinate system, and the acoustic modal features are mapped to the corresponding positions in the physical coordinate system.

[0175] Step S535: inputting the vibration modal characteristics into the physical coordinate system by associating the vibration measurement points with the physical coordinates.

[0176] After establishing the correspondence between vibration measurement points and 3D spatial coordinates, the vibration modal characteristics are associated with the physical coordinates and input into a unified physical coordinate system. This allows the characteristics of the three modalities—thermal imaging, acoustics, and vibration—to be mapped into the same physical coordinate system, achieving spatial alignment.

[0177] Through this example, the technical problem of failing to align the sensor data of coal-fired equipment in time and space in related technologies is solved, which leads to the inability to further and better complete the cross-modal fusion of the features of each sensor data. This example provides a method for aligning the sensor data of coal-fired equipment in time and space.

[0178] Furthermore, in one embodiment, the generating of the four-stage coal-fired equipment defect recognition model further includes:

[0179] Step S601: Constructing a knowledge graph, which includes: defining the structural data, material properties, historical maintenance records, and known defect descriptions of the coal-fired equipment as inputs; defining the equipment components, material properties, defect types, and failure modes of the coal-fired equipment as entities; and defining the associations between defects caused by materials, symptoms caused by known defects, and maintenance plans as relationships;

[0180] For example, the knowledge graph construction process includes four steps: data collection, entity recognition, relationship extraction and graph storage.

[0181] During the data collection phase, data is collected from various sources, including equipment manuals, maintenance manuals, and historical records. Structural data includes the equipment's components and connection relationships; material properties include material type, strength, and heat resistance; historical maintenance records include maintenance time, maintenance content, and repair results; and descriptions of known defects include defect type, location, and severity.

[0182] During the entity recognition phase, named entity recognition technology is used to extract entities from the text. Entity types include equipment components (such as boilers, turbines, and pipelines), material properties (such as strength, hardness, and heat resistance), defect types (such as cracks, corrosion, and wear), and failure modes (such as overheating, overvoltage, and vibration). Entity recognition uses a BERT-based sequence labeling model, achieving an F1 score of 0.92.

[0183] During the relationship extraction phase, relationships between entities are identified using rules and machine learning methods. Relationship types include material-induced defects (e.g., a certain material is prone to a certain defect under specific conditions), symptoms caused by known defects (e.g., cracks leading to leaks), and repair solution associations (e.g., the repair method corresponding to a certain defect). This relationship extraction utilizes remote supervision, achieving an F1 score of 0.85.

[0184] During the graph storage phase, a graph database (Neo4j) was used to store the constructed knowledge graph. The knowledge graph contains approximately 10,000 entities and 30,000 relationships, covering the main components of coal-fired equipment and common defect types.

[0185] Step S62: extracting semantic information features of known defects from the knowledge graph through a graph neural network (GNN);

[0186] For example, the graph neural network adopts the graph convolutional network (GCN) structure, which contains three layers of graph convolution layers, and the output dimensions of each layer are 128, 64, and 32 respectively. Through the message passing mechanism, GNN can capture the structural and semantic information in the knowledge graph.

[0187] In the implementation, for each defect entity in the knowledge graph, the semantic information features of the defect are generated by aggregating the information of its neighboring nodes using RGCN (Relational Graph Convolutional Network). The aggregation process uses an attention mechanism, weighting the relationship type and the importance of the neighboring nodes.

[0188] GNN training uses supervised learning, using the defect classification task as the training objective. The training parameters are set as follows: learning rate of 0.001, batch size of 64, number of training epochs of 100, and Adam optimizer.

[0189] Step S63: training a linear projection matrix based on the semantic information features of the known defect and the cross-modal fusion features of the known defect, and then inputting the cross-modal fusion features of the unknown defect into the linear projection matrix to obtain the semantic information features of the unknown defect;

[0190] For example, a linear projection matrix is ​​used to map the cross-modal fusion feature space to the semantic information feature space. The training process uses the least squares method, with the goal of minimizing the Euclidean distance between the projected cross-modal fusion features of a known defect and its semantic information features.

[0191] Assume that the cross-modal fusion feature of the known defect is X, the semantic information feature is Y, and the linear projection matrix is ​​W. The optimization objective is: min||XW-Y||_F^2, where ||·||_F represents the Frobenius norm.

[0192] Solving the optimization problem yields the linear projection matrix W, and then the cross-modal fusion features of the unknown defect are input into the matrix to obtain the semantic information features of the unknown defect.

[0193] Step S64: Calculating the cosine similarity between each unknown defect and each known defect based on the semantic information features of the known defects and the semantic information features of the unknown defects;

[0194] Exemplarily, the cosine similarity calculation formula is: cos(θ)=(A·B) / (||A||·||B||), where A and B represent two feature vectors, · represents the vector dot product, and ||·|| represents the L2 norm of the vector.

[0195] For each unknown defect, the cosine similarity between it and all known defects is calculated to form a similarity matrix. The higher the similarity value, the more similar the two defects are in the semantic space.

[0196] Step S65: According to the cosine similarity, the probability distribution of the category to which the unknown defect belongs is obtained to obtain a recognition result.

[0197] Exemplarily, the cosine similarity between the unknown defect and each known defect is converted into a probability value through a softmax function.

[0198] For each defect category, the probability values ​​of all known defects belonging to that category are summed to obtain the probability that the unknown defect belongs to that category. The identification result is the defect category with the highest probability.

[0199] Through this embodiment, the technical problem of difficulty in identifying unknown coal-fired equipment defects in related technologies is solved. This embodiment provides a dynamic weighted similarity measurement method by fusing knowledge graphs with multimodal features, which can identify unknown defect types and improve the system's detection capabilities for new or rare defects.

[0200] In a second aspect, an embodiment of the present application also provides a system for identifying defects in coal-fired equipment based on a large model.

[0201] In one embodiment, referring to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of a device for identifying defects in coal-fired equipment based on a large model. Figure 3 As shown, the device for identifying defects in coal-fired equipment based on a large model includes:

[0202] The first generation module 01 is used to train the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data;

[0203] The second generation module 02 is used to transfer the common features extracted from the source domain data and the invariant domain features obtained by adversarial training between the source domain data and the target domain data to the first-level coal-fired equipment defect recognition model through transfer learning, so as to generate a second-level coal-fired equipment defect recognition model;

[0204] A third generation module 03 is configured to input the sensor data and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws;

[0205] The fourth generating module 04 is configured to train the standard forged defect data based on the second-level coal-fired equipment defect recognition model to generate a third-level coal-fired equipment defect recognition model;

[0206] a fifth generating module 05, configured to generate a cross-modal fusion feature based on a preset strategy, and add the fusion feature to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model;

[0207] The identification module 06 is used to input the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect identification model to obtain the identification result and confidence level.

[0208] Among them, the functional implementation of each module in the above-mentioned device for identifying coal-fired equipment defects based on a large model corresponds to the various steps in the above-mentioned method embodiment for identifying coal-fired equipment defects based on a large model, and their functions and implementation processes will not be repeated here one by one.

[0209] In a third aspect, an embodiment of the present application provides a device for identifying defects in coal-fired equipment based on a large model. The device for identifying defects in coal-fired equipment based on a large model can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0210] Reference Figure 4 , Figure 4 Schematic diagram of the hardware structure of the device for identifying coal-fired equipment defects based on a large model in the embodiment of the present application. In the embodiment of the present application, the device for identifying coal-fired equipment defects based on a large model may include a processor, a memory, a communication interface, and a communication bus.

[0211] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0212] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces are used to interconnect components within the large-scale model-based coal-fired equipment defect identification device, as well as interfaces used to interconnect the large-scale model-based coal-fired equipment defect identification device with other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can include displays, keyboards, etc.

[0213] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0214] The processor may be a general-purpose processor that can call a program for identifying coal-fired equipment defects based on a large model stored in a memory and execute the method for identifying coal-fired equipment defects based on a large model provided in the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the program for identifying coal-fired equipment defects based on a large model is called can be referred to in the various embodiments of the method for identifying coal-fired equipment defects based on a large model of the present application, and will not be further described here.

[0215] Those skilled in the art will understand that Figure 4 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0216] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0217] The computer-readable storage medium of the present application stores a program for identifying defects in coal-fired equipment based on a large model, wherein when the program for identifying defects in coal-fired equipment based on a large model is executed by a processor, the steps of the method for identifying defects in coal-fired equipment based on a large model as described above are implemented.

[0218] Among them, the method implemented when the program for identifying coal-fired equipment defects based on a large model is executed can refer to the various embodiments of the method for identifying coal-fired equipment defects based on a large model in this application, and will not be repeated here.

[0219] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0220] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0221] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0222] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0223] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0224] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of the present application.

[0225] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for identifying defects in coal-fired equipment based on a large model, characterized in that: The method for identifying coal-fired equipment defects based on a large model includes: Training the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data; By transfer learning, the common features extracted from the source domain data and the invariant domain features obtained by adversarial training between the source domain data and the target domain data are transferred to the first-level coal-fired equipment defect recognition model to generate a second-level coal-fired equipment defect recognition model; Inputting a coal-fired equipment defect dataset and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in the loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws; Based on the secondary coal-fired equipment defect recognition model, the standard forged defect data is trained to generate a tertiary coal-fired equipment defect recognition model; generating a cross-modal fusion feature based on the sensor data and a preset strategy, and adding the cross-modal fusion feature to the three-level coal-fired equipment defect recognition model to generate a four-level coal-fired equipment defect recognition model; The image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment are input into the four-level coal-fired equipment defect recognition model to obtain the recognition results and confidence levels.

2. The method for identifying coal-fired equipment defects based on a large model according to claim 1, characterized in that: The step of inputting the coal-fired equipment defect dataset and preset random noise into a generative adversarial network (GAN) to generate standard simulated defect data includes: Embedding a thermodynamic residual term in a loss function of a GAN generator, wherein the thermodynamic residual term includes a plurality of preset thermodynamic equations; The coal-fired equipment defect dataset and preset random noise are input into the generator of the Generative Adversarial Network (GAN) to generate basic counterfeit defect data; The preset real defect data or the basic simulated defect data is input into the discriminator of the Generative Adversarial Network (GAN) to generate standard simulated defect data.

3. The method for identifying coal-fired equipment defects based on a large model according to claim 2, characterized in that: The project of embedding thermodynamic residual in the loss function of the GAN generator includes: Physical quantities related to thermodynamic residual items, including crack orientation angle, thermal stress, and thermal diffusion residual, are embedded in the loss function of the GAN generator.

4. The method for identifying coal-fired equipment defects based on a large model according to claim 1, characterized in that: Generating cross-modal fusion features based on the sensor data and the preset strategy includes: Preprocessing sensor data, wherein the sensor data includes: thermal imaging data, acoustic signals, and vibration data; Extracting single modal features from each of the sensor data, wherein the single modal features include thermal imaging modal features, acoustic modal features, and vibration modal features; Performing spatiotemporal alignment processing on the plurality of unimodal features to obtain spatiotemporal aligned unimodal features; fusing the timestamps and spatial coordinates of the spatiotemporally aligned unimodal features through spatiotemporal encoding to generate spatiotemporal fused features; A cross-modal fusion feature is generated based on the attention mechanism and the spatiotemporal fusion feature.

5. The method for identifying coal-fired equipment defects based on a large model according to claim 4, characterized in that: The generating of cross-modal fusion features based on the attention mechanism and the spatiotemporal fusion features includes: Generate multi-head single-modal self-fusion features based on the spatiotemporal fusion features and the multi-head self-attention mechanism; Cross-modal fusion features are generated based on the multi-head single-modal self-fusion features and the cross-modal attention mechanism (Cross-modal Attention).

6. The method for identifying coal-fired equipment defects based on a large model according to claim 4, characterized in that: The performing spatiotemporal alignment processing on the plurality of unimodal features to obtain spatiotemporal aligned unimodal features includes: performing linear interpolation on the vibration modal feature according to the timestamp of the thermal imaging modal feature to generate a vibration modal feature that is synchronized with the thermal imaging data; intercepting the acoustic modal feature according to the imaging frame time window of the thermal imaging modal feature to generate an acoustic modal feature synchronized with the thermal imaging data; Establishing a mapping relationship between a pixel coordinate system and a physical coordinate system based on parameters of the thermal imaging device, and inputting the thermal imaging modal features into the physical coordinate system; Acquiring the azimuth angle of the coal-fired equipment defect by forming a microphone beam array, and inputting the acoustic modal characteristics into the physical coordinate system; By associating the vibration measurement points with the physical coordinates, the vibration modal characteristics are input into the physical coordinate system.

7. The method for identifying coal-fired equipment defects based on a large model according to claim 1, characterized in that: The process of inputting the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect recognition model to obtain the recognition result and confidence level also includes: Constructing a knowledge graph, which includes: defining structural data, material properties, historical maintenance records, and known defect descriptions of the coal-fired equipment as inputs; defining equipment components, material properties, defect types, and failure modes of the coal-fired equipment as entities; and defining associations between defects caused by materials, symptoms caused by known defects, and maintenance plans as relationships; Extracting semantic information features of known defects from the knowledge graph through a graph neural network (GNN); Training a linear projection matrix based on the semantic information features of the known defects and the cross-modal fusion features of the known defects, and then inputting the cross-modal fusion features of the unknown defects into the linear projection matrix to obtain the semantic information features of the unknown defects; Calculating the cosine similarity between each unknown defect and each known defect based on the semantic information features of the known defects and the semantic information features of the unknown defects; According to the cosine similarity, the probability distribution of the category to which the unknown defect belongs is obtained to obtain a recognition result.

8. A system for identifying defects in coal-fired equipment based on a large model, characterized in that: The system for identifying coal-fired equipment defects based on a large model includes: a first generation module, configured to train the labeled target domain data to generate a first-level coal-fired equipment defect recognition model, wherein the target domain data is the acquired coal-fired equipment data; The second generation module is used to transfer the common features extracted from the source domain data and the invariant domain features obtained by adversarial training between the source domain data and the target domain data to the first-level coal-fired equipment defect recognition model through transfer learning, so as to generate a second-level coal-fired equipment defect recognition model; a third generation module, configured to input the sensor data and preset random noise into a generative adversarial network (GAN) to generate standard counterfeit defect data, wherein a thermodynamic residual term is embedded in a loss function of the generative adversarial network (GAN) to ensure that the standard counterfeit defect data conforms to physical laws; a fourth generating module, configured to train the standard forged defect data based on the secondary coal-fired equipment defect recognition model to generate a tertiary coal-fired equipment defect recognition model; a fifth generating module, configured to generate a cross-modal fusion feature based on a preset strategy, and add the fusion feature to the third-level coal-fired equipment defect recognition model to generate a fourth-level coal-fired equipment defect recognition model; The recognition module is used to input the image of the coal-fired equipment to be identified and the operating parameters of the coal-fired equipment into the four-level coal-fired equipment defect recognition model to obtain the recognition result and confidence level.

9. A device for identifying defects in coal-fired equipment based on a large model, characterized in that: The device for identifying defects in coal-fired equipment based on a large model includes a processor, a memory, and a program for identifying defects in coal-fired equipment based on a large model stored in the memory and executable by the processor. When the program for identifying defects in coal-fired equipment based on a large model is executed by the processor, the steps of the method for identifying defects in coal-fired equipment based on a large model as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for identifying defects in coal-fired equipment based on a large model, wherein when the program for identifying defects in coal-fired equipment based on a large model is executed by a processor, the steps of the method for identifying defects in coal-fired equipment based on a large model as described in any one of claims 1 to 7 are implemented.

Images