Substation equipment defect detection method and system based on saliency, terminal device and computer readable storage medium

Abstract

The application relates to a substation equipment defect detection method and system based on saliency, terminal equipment and a computer readable storage medium, which comprises the following steps: S1, data set preprocessing, S2, neural network model construction, S3, global feature extraction of a defect main body, S4, adaptive multi-scale feature fusion and S5, multi-scale feature fusion. The application mainly adopts a multi-scale deep learning method to construct a neural network model, extracts effective features in a deep feature guiding shallow feature mode, models layer by layer from coarse to fine, adopts multi-scale fusion, improves the robustness of substation equipment defect detection, crosswise and mutually fuses deep features and shallow features, selectively integrates multi-level context information, obtains more abundant and stable and effective feature representation, and thus improves the effect of substation equipment defect detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, system, terminal equipment, and computer-readable storage medium for detecting defects in substation equipment based on saliency. Background Technology

[0002] In substation safety management, defect detection of power transmission and transformation equipment plays a central role. Considering geographical differences, the challenges of complex climatic environments, and the inevitability of equipment aging, these factors combine to affect the operation of power transmission and transformation equipment, potentially triggering a series of unpredictable faults that significantly threaten the stability and reliability of substations. Traditionally, defect detection relies primarily on manual visual inspection. This method heavily depends on the operator's experience and judgment, resulting in low efficiency, a high risk of missed detections, and high labor costs. Therefore, both academia and industry are committed to exploring more efficient and accurate automated detection methods. In recent years, computer vision-based defect detection technology has attracted considerable attention due to its significant advantages. Among them, salient object detection, as an important algorithmic branch, has demonstrated superior performance. Salient detection methods automatically identify and enhance defect regions in images through algorithms, extracting feature information closely related to defects, and realizing an end-to-end automated processing flow from image input to defect recognition output. This method not only improves detection efficiency but also significantly reduces errors caused by human factors, enhancing the accuracy and reliability of detection results. It has broad application prospects and research value in the field of defect detection for power transmission and transformation equipment, providing strong technical support for the safety management of substations.

[0003] Traditional salient object detection methods follow a hierarchical principle, extracting low-level, mid-level, and high-level features. Low-level features mainly involve contrast characteristics, such as color and edges, which are obtained through numerical calculations and provide basic visual information for salient object detection, but lack direct entity association. Mid-level features further incorporate attribute information such as the object's shape and position, including object contours and shape features, as well as contextual or global features. These features help the model understand the layout and relationships of objects in the scene. High-level features are closely related to the semantic information of objects. Based on prior knowledge or high-level abstract representations learned by deep learning models, they can guide the salient object detection model to locate salient objects from top to bottom, significantly improving detection accuracy and robustness. This low-to-high-level feature extraction strategy provides comprehensive and in-depth visual information for salient object detection, contributing to more accurate salient object detection. However, traditional salient object detection methods mainly rely on manually designed feature extraction and fusion, heavily depending on prior knowledge and many constraints. They often fail in highly complex backgrounds and suffer from drawbacks such as long processing time, cumbersome operation steps, and poor robustness. Fortunately, deep learning-based methods can effectively solve these problems.

[0004] With the performance breakthroughs brought by deep learning in computer vision, many saliency detection methods based on deep convolutional neural networks have been proposed. Relying on the powerful feature extraction capabilities of neural networks, deep learning-based methods eliminate the need for handcrafted features and reduce reliance on prior bias. The application scenarios for saliency detection are often in the real physical world, where salient targets in relevant datasets exhibit very large scale variations. Therefore, many methods are designed to extract multi-scale features. Some researchers have integrated features from three branches—ordinary convolution, dilated convolution, and pooling layers—at different scales, enabling the model to more accurately capture image details and global context. Subsequent work has not only focused on multi-scale feature extraction but also investigated the fusion of multi-scale features. Existing methods primarily rely on detecting salient targets in natural images. While existing saliency detection methods achieve good results, they still have some shortcomings when used to detect defects in power equipment. False positives or false negatives frequently occur, manifesting as background being detected as a defect or vice versa. This is caused by the significant differences between natural and defective images. By analyzing the detection results of existing methods and the substation equipment defect dataset, we identified three main problems. First, the dataset has a cluttered background. Background noise affects the quality of saliency maps. The scale, shape, category, lighting, and external interference of salient objects in images differ from those in natural images, making it difficult to accurately separate the defect subject from the background during training. Therefore, direct use may be inappropriate. Second, the defects are numerous and scattered, making it difficult for previous methods to detect all defects completely. Third, when detecting foreign object defects (such as bird nests and plastic bags) or simple defects (such as broken manhole covers), overly complex defect boundaries often lead to blurred edges in the predicted image or even prediction errors. Nevertheless, we can still utilize the ideas and principles of salient object detection methods in natural images to study salient defect features in equipment defects. For example, we can use mainstream architectures of saliency methods, such as encoder-decoder networks and side-fusion networks, to build models and deploy structures. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, terminal equipment, and computer-readable storage medium for detecting defects in substation equipment based on saliency. This invention aims to solve the problems in the prior art where the dataset background is cluttered, the types of defects are numerous and the defect distribution is scattered, and when detecting foreign object defects or simple defects, the defect boundaries are too complex, often leading to unclear edges of the prediction map or even prediction errors.

[0006] A first aspect of this invention provides a method for detecting defects in substation equipment based on saliency, comprising the steps of:

[0007] S1. Dataset Preprocessing: Construct a defect dataset of substation equipment containing various defects for training and optimization of neural network models, and perform preprocessing operations on the dataset.

[0008] S2. Neural Network Model Construction: A multi-scale deep learning method is used to construct a neural network model. The defect image of the substation equipment is input into the backbone network Swin Transformer model of the neural network model to extract feature maps of different scales. The model is built layer by layer from coarse to fine to realize comprehensive feature extraction and processing of the defect image of the substation equipment.

[0009] S3. Global feature extraction of the defect subject: The feature maps of different levels extracted from the backbone network are processed to filter the complex background in the substation, screen and enhance useful defect features, and at the same time, global features are extracted to provide effective guidance for the subsequent defect feature processing and feature fusion stages.

[0010] S4. Adaptive Multi-Scale Feature Fusion: Used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects.

[0011] S5. Multi-scale feature fusion: Used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map;

[0012] Preferably, in the neural network model construction step, the backbone network first performs hierarchical processing on the defect image of the substation equipment, extracts five layers of features in sequence, cascades the features of specific layers and performs specific operations to obtain four specific channel features, and finally obtains four features with 32 channels, which are labeled as f1, f2, f3 and f4 respectively.

[0013] Preferably, in the global feature extraction step of the defect subject, features f2, f3, and f4 are sent to the cross-level feature fusion module, and the extracted global features are then used to further optimize the four multi-scale features.

[0014] Preferably, in the adaptive multi-scale feature fusion step, feature map information at different scales is screened, interfering defect edge features are filtered out, and the defect subject information is enhanced and fused under the guidance of high-level features.

[0015] A second aspect of the present invention provides a substation equipment defect detection system based on saliency, comprising: constructing a substation equipment defect dataset containing multiple defects and performing preprocessing operations on the dataset; and a defect feature processing module for training and optimizing a neural network.

[0016] The neural network model building module uses a multi-scale deep learning approach to construct a neural network model. The defect images of substation equipment are input into the backbone network model of the neural network model, and feature maps of different scales are extracted. The model is built layer by layer from coarse to fine to achieve comprehensive feature extraction and processing of defect images of substation equipment.

[0017] The global feature extraction module for the defect subject processes the feature maps of different levels extracted from the backbone network to filter the complex background within the substation, select and enhance useful defect features, and extract global features to provide effective guidance for subsequent defect feature processing and feature fusion stages.

[0018] The adaptive multi-scale feature fusion module is used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects.

[0019] The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0020] Preferably, the dataset building module constructs a defective dataset for training and optimization, the network framework building module constructs a neural network using multi-scale deep learning and extracts feature maps, and the feature fusion module fuses feature maps of different scales.

[0021] Preferably, the feature generation module generates equipment defect information, the training module trains the model, and the testing module tests the model.

[0022] Preferably, the data management module manages the dataset, the model optimization module optimizes the neural network model, and the results display module displays the detection results.

[0023] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0024] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0025] Based on the above technical solution, compared with the prior art, this invention mainly adopts a multi-scale deep learning method to construct a neural network model. Deep features guide shallow features in extracting effective features, and modeling is performed layer by layer from coarse to fine to achieve comprehensive feature extraction of the image. Multi-scale fusion is employed to improve the robustness of substation equipment defect detection. Deep and shallow features are cross-fused, selectively integrating multi-level contextual information to obtain richer and more stable and effective feature representations, thereby improving the effectiveness of substation equipment defect detection. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Attached Figure Description

[0028] Figure 1 The diagram shows a flowchart of the abnormal sound recognition method in this invention.

[0029] Figure 2 The diagram shown is a structural block diagram of the abnormal sound recognition system in this invention.

[0030] Figure 3 The diagram shown is a schematic diagram of a terminal device according to an embodiment of the present invention;

[0031] Figure 4 A schematic diagram of the network structure framework provided by the present invention;

[0032] Figure 5 A schematic diagram of the cross-level feature fusion module provided by the present invention;

[0033] Figure 6 This is a schematic diagram of the feature optimization module provided by the present invention;

[0034] Figure 7 This is a schematic diagram of the adaptive multi-level feature fusion module provided by the present invention;

[0035] Figure 8 This invention provides an image of a defect in a substation device.

[0036] Figure 9 This is a schematic diagram illustrating the effect of defect detection in substation equipment provided by the present invention;

[0037] Figure 10 This invention provides a label image corresponding to a defect image of substation equipment;

[0038] Figure 11 Another image of a substation equipment defect provided by the present invention;

[0039] Figure 12 This is a schematic diagram illustrating the effect of defect detection in substation equipment provided by the present invention;

[0040] Figure 13 This is a label image corresponding to another substation equipment defect image provided by the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The implementation of this invention will be described in detail below with reference to specific embodiments.

[0042] This invention provides a method, system, terminal equipment, and computer-readable storage medium for detecting defects in substation equipment based on saliency. The method includes the following steps:

[0043] S1. Dataset Preprocessing: Construct a defect dataset of substation equipment containing various defects for training and optimization of neural network models, and perform preprocessing operations on the dataset.

[0044] S2. Neural Network Model Construction: A multi-scale deep learning method is used to construct a neural network model. The defect images of substation equipment are input into the backbone network Swin Transformer model of the neural network model to extract feature maps of different scales. The model is built layer by layer from coarse to fine to achieve comprehensive feature extraction and processing of substation equipment defect images.

[0045] S3. Global feature extraction of the defect subject: The feature maps of different levels extracted from the backbone network are processed to filter the complex background in the substation, screen and enhance useful defect features, and at the same time, global features are extracted to provide effective guidance for the subsequent defect feature processing and feature fusion stages.

[0046] S4. Adaptive Multi-Scale Feature Fusion: Used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects.

[0047] S5. Multi-scale feature fusion: This is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0048] In the neural network model construction steps, the backbone network first performs hierarchical processing on the defect image of the substation equipment, extracts five layers of features in sequence, concatenates specific layer features and performs specific operations to obtain four specific channel features, and finally obtains four features with 32 channels, which are labeled as f1, f2, f3 and f4 respectively.

[0049] In the global feature extraction step of the defect subject, features f2, f3, and f4 are sent to the cross-level feature fusion module. The extracted global features are then used to further optimize the four multi-scale features.

[0050] In the adaptive multi-scale feature fusion step, feature map information at different scales is screened, interfering defect edge features are filtered out, and the defect main information is enhanced and fused under the guidance of high-level features.

[0051] The above content includes the following effects: First, by constructing a defect dataset of substation equipment containing various defects and performing preprocessing, a rich and accurate data foundation is provided for subsequent detection, which improves the training effect and generalization ability of the model.

[0052] In the construction of the neural network model, the backbone network is used to perform hierarchical processing on the defect images of substation equipment and extract feature maps of different scales. This can comprehensively capture various details and overall structure of equipment defects, providing strong feature support for accurate detection. In particular, the four specific channel features obtained provide more targeted information for subsequent processing.

[0053] The global feature extraction step of the defect subject can effectively filter complex backgrounds, select and enhance useful defect features, and at the same time, the extracted global features provide clear guidance for subsequent processing and fusion, which greatly improves the accuracy and reliability of detection.

[0054] The adaptive multi-scale feature fusion step filters out interference by selecting feature map information at different scales to obtain accurate defect edge features, and enhances the main information of defect features under the guidance of high-level features, thus achieving effective fusion from both edge and main aspects, making the detection results more accurate and stable.

[0055] The multi-scale feature fusion step combines defect feature maps of different scales, achieving effective multi-scale feature aggregation. The resulting final feature output map can more comprehensively reflect the equipment defect situation, providing a strong basis for equipment maintenance and repair.

[0056] In summary, the saliency-based substation equipment defect detection method of the present invention is characterized by high efficiency, accuracy, and reliability, and can effectively improve the level of substation equipment defect detection, playing an important role in ensuring the safe operation of substations.

[0057] A saliency-based substation equipment defect detection system includes:

[0058] The dataset preprocessing module constructs a substation equipment defect dataset containing various defects for training and optimization of neural network models, and performs preprocessing operations on the dataset.

[0059] The neural network model building module uses a multi-scale deep learning method to build a neural network model. The defect images of substation equipment are input into the backbone network Swin Transformer model of the neural network model to extract feature maps of different scales. The model is built layer by layer from coarse to fine to achieve comprehensive feature extraction and processing of defect images of substation equipment.

[0060] The global feature extraction module for the defect subject processes the feature maps of different levels extracted from the backbone network to filter the complex background within the substation, select and enhance useful defect features, and extract global features to provide effective guidance for subsequent defect feature processing and feature fusion stages.

[0061] The adaptive multi-scale feature fusion module is used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects.

[0062] The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0063] The dataset building module constructs a defective dataset for training and optimization. The network framework building module builds a neural network using multi-scale deep learning and extracts feature maps. The feature fusion module fuses feature maps from different scales.

[0064] The feature generation module generates equipment defect information, the training module trains the model, and the testing module tests the model.

[0065] The data management module manages the dataset, the model optimization module optimizes the neural network model, and the results display module displays the detection results.

[0066] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0067] A computer-readable storage medium storing a computer program.

[0068] In a specific implementation, this utility model includes the following inner cylinder:

[0069] like Figure 1 As shown, it includes:

[0070] Step 1: Collect image data of substation equipment defects, construct a dataset containing substation equipment defects, and annotate the defects at the pixel level for use in training and optimization of neural network models, including training and test sets.

[0071] Step 2, Neural Network Model Construction: A multi-scale deep learning approach is used to construct the neural network model, building it layer by layer from coarse to fine to achieve comprehensive feature extraction and processing of substation defect characteristics. The backbone network for feature extraction uses the Swin Transformer to extract five layers of features. The fourth and fifth layer features are cascaded to obtain four multi-scale features. These four features are then processed using dilated spatial convolution pooling pyramid and channel pooling operations to obtain four features with 32 channels each: f1, f2, f3, and f4. The operation can be represented as:

[0072] S i =A(S) i (1)

[0073] f i =CP(S) i (2)

[0074]

[0075] Where A(*) is the dilated spatial convolutional pooling pyramid, S(i) are the features extracted from each layer by the Swin Transformer, X is the original feature map containing n channels, and j and k are integers. X is divided into m groups, and max pooling is applied to each group of features, then the m feature maps are concatenated together. In this specification, m is set to 32.

[0076] Step 3, Global Feature Extraction (CFM Module): This step processes the feature maps extracted by the Swin Transformer at different levels to extract global features, guiding subsequent feature processing and fusion stages. The input features are f2, f3, and f4. Since f1 contains a large number of low-level features, f2 contains fewer low-level but cluttered background features. The cross-level feature fusion module consists of two parts: spatial attention processing and channel attention processing. First, the features undergo spatial information processing. Specifically, the features are processed through two convolutional modules and a sigmoid activation function to obtain spatial attention weights, which are then multiplied with the original features. The multiplied feature is added to f3 to obtain a new feature, which is then subjected to channel attention operations. Specifically, the features are processed through global max pooling and global average pooling, the two resulting features are concatenated, and then convolutional operations are performed to obtain channel attention weights. These weights are then multiplied with the original channel weights and finally added to f4 to obtain the final output feature F. cfm The convolution operation consists of two 3x3 convolutions, with batch normalization added after each convolution to force the data input to each network layer to follow a uniform data distribution (normal distribution), and the ReLU function for non-linear transformation. The specific operations are as follows:

[0077] f3=f3+DP(f2×Sig(Conv(Conv(f2)))) (4)

[0078] F cfm =f4+f3×Sig(Conv(Cat(GAP(f3),GMP(f3)))) (5)

[0079] Where Conv(*) represents two 1*1 convolutional layers, GAP(*) represents global average pooling, GMP(*) represents global max pooling, Sig(*) represents the sigmoid activation function, and + and × represent element-wise addition and multiplication, respectively.

[0080] Step 4, Multi-Scale Feature Processing (FOM Module): This step filters information from feature maps at different scales, removes interference, and enhances effective information to reduce interference in subsequent feature fusion processes. The module takes two input features: the current layer's feature and the output feature of the previous feature optimization module. The highest-level feature optimization module takes the global feature output from the cross-layer feature fusion module as its input. First, two convolution operations are performed on the two input features, and the residuals are summed. Then, the two features are multiplied, and finally, the resulting feature is summed with the two input features to obtain the output feature of the feature optimization module. The convolution operation consists of two 3x3 convolutions, and batch normalization is added after each convolution to force the data input to each network layer to a uniform data distribution (normal distribution), and the ReLU function is used for non-linear transformation. The specific operations are as follows:

[0081] f i =f i +Conv(Conv(f i (6)

[0082] f i+1 =f i+1 +Conv(Conv(f i+1 (7)

[0083] F fom =f i +f i+1 +(f i+1 ×f i (8)

[0084] Conv(*) includes 3x3 convolution, batch normalization, and normalization operations. + and × represent element-wise addition and multiplication, respectively.

[0085] Step 5, Multi-Scale Feature Fusion (FFM Module): This module fuses feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map. The input to the multi-level feature fusion module consists of two parts: the output of the current layer's feature optimization module and the output of the previous layer's multi-level feature fusion module. Specifically, the input to the highest-level multi-level feature fusion module is the output of the current layer's feature optimization module and the output of the cross-layer feature fusion module. The specific process is as follows: First, the output features of the previous layer's multi-level feature fusion module are downsampled to the same size as the output features of the current layer's feature optimization module. These two features are then concatenated and subjected to a 1x1 convolution to halve the number of channels before being fed into the multi-level feature fusion module. Features are then sequentially passed through dilated convolutions with different dilation rates to gradually extract features at different scales. Each level of dilated convolution captures contextual information of varying sizes, thereby enhancing the richness and diversity of features. The output feature map of each level of dilated convolution is added to the output feature map of the previous level's dilated convolution to achieve multi-scale feature fusion. This hierarchical fusion can integrate feature information at different scales at a higher level, enhancing the expressive power of the features and ultimately yielding output features that contain rich scale information and context. The entire process can be represented as:

[0086]

[0087] Here, + represents element-wise addition, and Conv(d=r) includes convolution, batch normalization, and batch standardization operations with a dilation rate of r.

[0088] Step 6, Training: Train the model using the training set.

[0089] Step 7: Testing: Test the improved model using the test set.

[0090] This scheme designs a saliency-based defect detection algorithm for substation equipment from two perspectives: feature extraction and feature fusion. In feature extraction, high-level features are first processed to extract high-level semantic information. This high-level semantic information is then used to guide shallow feature extraction, filtering out interfering information and extracting effective features. In feature fusion, a method is proposed to cross-fuse deep and shallow features to maximize the utilization of effective information from each layer. This network can selectively integrate multi-level contextual information, ensuring that the obtained features are stable and effective. These features contain both high-level and low-level information, greatly improving the robustness of the network's extraction results.

[0091] Figure 3 The diagram shown illustrates a terminal device according to an embodiment of the present invention. Figure 3As shown, the terminal device in this embodiment includes: a processor, a memory, and a computer program, such as a software development program, stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the various software development method embodiments described above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0092] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device. For example, the computer program can be divided into an acquisition module, an execution module, and a generation module (a module in a virtual device), with the specific functions of each module as follows:

[0093] The acquisition module is used for software development requirements; the execution module is used to determine the target components of the target page to be generated based on the software development requirements and the template components on the preset page basic framework template; the target components include at least one of the following: an adjustment component based on the template component, a new component created based on the template component, and a component to be replaced based on the template component; the generation module is used to assemble the template components and the target components into the target page in the page basic framework template based on the component name and position information of the target components.

[0094] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 3 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0095] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0096] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the terminal device. It can also be used to temporarily store data that has been output or will be output.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0099] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0100] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0104] This invention relates to an abnormal sound recognition method, system, terminal device, and computer-readable storage medium. Through sound signal analysis and processing, it proposes a feature extraction algorithm that fuses two sound signal processing methods, improving comprehensiveness and the differentiation of individual feature differences. Based on a typical single input, the feature fusion algorithm processes important inputs from two directions, enhancing the network's generalization ability and accuracy. Furthermore, by using a two-dimensional convolutional neural network to further extract features from the sound signal, and using an "acoustic image" as the input signal for the two-dimensional convolutional neural network, information processing is fast and efficient.

[0105] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for detecting defects in substation equipment based on saliency, characterized in that, Including the following steps: S1. Dataset Preprocessing: Construct a defect dataset of substation equipment containing various defects for training and optimization of neural network models, and perform preprocessing operations on the dataset. S2. Neural Network Model Construction: A multi-scale deep learning method is used to construct a neural network model. The defect image of the substation equipment is input into the backbone network Swin Transformer model of the neural network model to extract feature maps of different scales. The model is built layer by layer from coarse to fine to realize comprehensive feature extraction and processing of the defect image of the substation equipment. S3. Global feature extraction of the defect subject: The feature maps of different levels extracted from the backbone network are processed to filter the complex background in the substation, screen and enhance useful defect features, and at the same time, global features are extracted to provide effective guidance for the subsequent defect feature processing and feature fusion stages. S4. Adaptive Multi-Scale Feature Fusion: Used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects. S5. Multi-scale feature fusion: Used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map; In the neural network model construction step, the backbone network first performs hierarchical processing on the defect image of the substation equipment, extracts five layers of features in sequence, concatenates specific layer features and performs specific operations to obtain four specific channel features, and finally obtains four features with 32 channels, which are labeled as f1, f2, f3 and f4 respectively. In the global feature extraction step of the defect subject, features f2, f3, and f4 are sent to the cross-level feature fusion module. The extracted global features are then used to further optimize the four multi-scale features. In the adaptive multi-scale feature fusion step, feature map information at different scales is screened, interfering defect edge features are filtered out, and the defect main information is enhanced and fused under the guidance of high-level features.

2. A substation equipment defect detection system based on saliency, characterized in that, include: The dataset preprocessing module constructs a substation equipment defect dataset containing various defects for training and optimization of neural network models, and performs preprocessing operations on the dataset. The neural network model building module uses a multi-scale deep learning method to build a neural network model. The defect image of the substation equipment is input into the backbone network Swin Transformer model of the neural network model, and feature maps of different scales are extracted. The model is built layer by layer from coarse to fine to realize comprehensive feature extraction and processing of the defect image of the substation equipment. The global feature extraction module for the defect subject processes the feature maps of different levels extracted from the backbone network to filter the complex background within the substation, select and enhance useful defect features, and extract global features to provide effective guidance for subsequent defect feature processing and feature fusion stages. The adaptive multi-scale feature fusion module is used to filter information from feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of high-level features, effectively fusing features from both edge and main aspects. The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map. The neural network model construction module is specifically used to first process the defect image of the substation equipment in a hierarchical manner through the backbone network, extract five layers of features in sequence, cascade the features of specific layers and obtain four specific channel features through specific operations, and finally obtain four features with 32 channels, which are labeled as f1, f2, f3 and f4 respectively. The global feature extraction module of the defect subject is specifically used to send the three features f2, f3, and f4 into the cross-level feature fusion module. The extracted global features are then used to further optimize the four multi-scale features. The adaptive multi-scale feature fusion module is specifically used to filter feature map information at different scales, filter out interfering defect edge features, and enhance and fuse the defect subject information under the guidance of high-level features.

3. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in claim 1.

Images