Substation equipment defect detection method and system based on saliency, terminal equipment and computer readable storage medium

Abstract

The invention relates to a substation equipment defect detection method and system based on saliency, terminal equipment and a computer readable storage medium. The method comprises the steps of S1, data set preprocessing, S2, trunk convolutional network model construction, S3, global feature extraction of a defect subject, S4, adaptive multi-scale feature fusion and S5, multi-scale feature fusion. According to the method, a neural network model is constructed by mainly adopting a multi-scale deep learning method, effective features are extracted in a mode that shallow features are guided by deep features, modeling is performed layer by layer from coarse to fine so as to realize omnibearing feature extraction of images, multi-scale fusion is adopted, the robustness of substation equipment defect detection is improved, and the defect detection efficiency is improved. According to the method, the deep features and the shallow features are crossed and fused, multi-level context information is selectively integrated, richer, more stable and more effective feature representation is obtained, and therefore the effect of substation equipment defect detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, system, terminal equipment and computer-readable storage medium for detecting substation equipment defects based on saliency. Background Art

[0002] In the safety management of substations, defect detection of power transmission and transformation equipment occupies a core position. Considering the differences in geographical locations, the challenges of complex climate environments, and the inevitability of natural aging of equipment, these factors work together in the operation process of power transmission and transformation equipment, which may cause a series of unpredictable failures, thus posing a significant threat to the stability and reliability of substations. Traditionally, defect detection mainly relies on manual visual inspection, which is highly dependent on the operator's experience and judgment. It is not only inefficient, but also has a high risk of missed detection, and is accompanied by high labor costs. Therefore, both academia and industry are committed to exploring more efficient and accurate automated detection methods. In recent years, defect detection technology based on computer vision has attracted much attention due to its significant advantages. Among them, salient object detection, as an important algorithm branch, has shown excellent performance. The saliency detection method automatically identifies and enhances the defective areas in the image through algorithms, extracts feature information closely related to the defects, and realizes an end-to-end automated processing flow from image input to defect recognition output. This method not only improves the detection efficiency, but also significantly reduces the errors caused by human factors, improves the accuracy and reliability of the detection results. It has broad application prospects and research value in the field of defect detection of power transmission and transformation equipment, and provides strong technical support for the safety management of substations.

[0003] Traditional salient object detection methods follow the principle of hierarchy and extract low-level, middle-level and high-level features. Low-level features mainly involve contrast characteristics, such as color and edge, which are obtained through numerical calculations and provide basic visual information for salient object detection, but lack direct entity relevance. Middle-level features further introduce attribute information such as shape and position of objects, such as object contour and shape features, as well as contextual or global features, which help the model understand the layout and relationship of objects in the scene. High-level features are closely related to the semantic information of objects. They are based on prior knowledge or high-level abstract representations learned by deep learning models, and can guide the salient object detection model to locate salient objects from top to bottom, significantly improving the accuracy and robustness of detection. This feature extraction strategy from low to high levels provides comprehensive and in-depth visual information for salient detection, which helps to achieve more accurate salient object detection. However, traditional salient object detection methods mainly rely on manually designed feature extraction and fusion, which is heavily dependent on prior knowledge and many constraints, and often fail in highly complex backgrounds. They also have the disadvantages of being time-consuming, cumbersome operation steps and poor robustness. Fortunately, deep learning based methods can effectively solve these problems.

[0004] With the performance breakthrough brought by deep learning in the field of computer vision, many saliency detection methods based on deep convolutional neural networks have been proposed one after another. Relying on the powerful feature extraction ability of neural networks, deep learning-based methods eliminate the need for manual features and reduce the reliance on prior biases. The application scenarios of saliency detection are often in the real physical world. From the image features of relevant data sets, it can be found that salient targets have very large scale changes. Therefore, many methods design networks for the extraction of multi-scale features. Some people integrate the features of different scales obtained by three branches of ordinary convolution, dilated convolution and pooling layer, so that the model can more accurately capture the details and global context of the image. Subsequent work not only focuses on the extraction of multi-scale features, but also studies the fusion of multi-scale features. Existing methods are mainly based on the detection of salient targets in natural images. Although the existing saliency detection methods can achieve good results, these methods still have some shortcomings when used to detect defects in substation equipment. Missed detection or false detection often occurs, which is reflected in the detection of background as defects, or the detection of defects as background. This is caused by the huge difference between natural images and defect images. By analyzing the detection results of existing methods and the defect dataset of substation equipment, the following three problems are found. First, the background of the dataset is cluttered. Background noise will affect the quality of the salient map. The scale, shape, category, lighting, external interference, etc. of the salient targets in the image are different from those in natural images. This makes it difficult to accurately separate the defect body from the background during training. Therefore, direct use may be inappropriate. Second, there are many types of defects and the defects are scattered. It is difficult for previous methods to detect all defects completely. Third, when detecting foreign body defects (such as bird nests and plastic bags) or simple defects (such as broken manhole covers), the defect boundaries are too complex, which often leads to blurred edges of the prediction map or even prediction errors. Despite this, we can still use the ideas and principles of salient target detection methods in natural images to study the salient defect features in equipment defects. For example, we can use the mainstream architecture of saliency methods, such as encoder-decoder networks and side fusion networks to build models and deploy structures. Summary of the invention

[0005] The purpose of the present invention is to provide a substation equipment defect detection method, system, terminal equipment and computer-readable storage medium based on significance, aiming to solve the problems in the prior art that the background of the data set is messy, the types of defects are many and the defect distribution is scattered, and when detecting foreign body defects or simple defects, the defect boundaries are too complex, which often leads to blurred edges of the prediction graph or even wrong predictions.

[0006] A first aspect of an embodiment of the present invention provides a method for detecting substation equipment defects based on significance, comprising the steps of:

[0007] S1. Dataset preprocessing: Construct a substation equipment defect dataset containing multiple defects for neural network training and optimization, and perform preprocessing operations on the dataset;

[0008] S2. Construction of backbone convolutional network model: A multi-scale deep learning method is used to construct a neural network model. The defect image of the substation equipment is input into the Swin Transformer model, and feature maps of different scales are extracted. The model is then built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment.

[0009] S3. Global feature extraction of defect entities: Process the feature maps of different levels extracted from the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features to provide effective guidance for the subsequent defect feature processing and feature fusion stages;

[0010] S4, Adaptive multi-scale feature fusion: used to filter the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects;

[0011] S5, multi-scale feature fusion: used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0012] Preferably, in the step of constructing the backbone convolutional network model, the backbone network first performs hierarchical processing on the defect image of the substation equipment, extracts five layers of features in sequence, cascades the features of the specific layers and obtains four specific channel features through specific operations, and finally obtains features of 32 for each of the four channels, which are marked as f1, f2, f3 and f4 respectively.

[0013] Preferably, in the step of extracting the global features of the defect body, the three features f2, f3 and f4 are sent to a cross-level feature fusion module, and the extracted global features are then used to perform further feature optimization processing on the four multi-scale features.

[0014] Preferably, in the adaptive multi-scale feature fusion step, feature map information of different scales is screened, interfering defect edge features are filtered, and defect main body information is enhanced and fused under the guidance of high-level features.

[0015] A second aspect of an embodiment of the present invention provides a substation equipment defect detection system based on significance, comprising: constructing a substation equipment defect data set containing multiple defects, and performing a preprocessing operation on the data set, a defect feature processing module, and a neural network training and optimization;

[0016] The backbone convolutional network model building module uses a multi-scale deep learning method to build a neural network model. The defect image of the substation equipment is input into the backbone network model, and feature maps of different scales are extracted. The model is built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment.

[0017] The global feature extraction module of the defect subject processes the feature maps of different levels extracted by the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features at the same time, which effectively guides the subsequent defect feature processing and feature fusion stages;

[0018] Adaptive multi-scale feature fusion module, used to screen the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects;

[0019] The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0020] Preferably, the data set construction module constructs a defective data set for training optimization, the network framework construction module constructs a neural network with multi-scale deep learning and extracts feature maps, and the feature fusion module fuses feature maps of different scales.

[0021] Preferably, the feature generation module generates equipment defect information, the training module trains the model, and the testing module tests the model.

[0022] Preferably, the data management module manages the data set, the model optimization module optimizes the neural network model, and the result display module displays the detection results.

[0023] A third aspect of an embodiment of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the computer program.

[0024] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0025] Based on the above technical solution, compared with the prior art, the present invention mainly adopts a multi-scale deep learning method to construct a neural network model, and extracts effective features in a way that deep features guide shallow features, and models layer by layer from coarse to fine to achieve all-round feature extraction of the image. Multi-scale fusion is used to improve the robustness of substation equipment defect detection. Deep features and shallow features are cross-fused with each other, and multi-level context information is selectively integrated to obtain a richer, more stable and effective feature representation, thereby improving the effect of substation equipment defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 Shown is a flowchart of the abnormal sound recognition method of the present invention;

[0029] Figure 2 Shown is a schematic diagram of the structure of the abnormal sound recognition system of the present invention;

[0030] Figure 3 A schematic diagram showing a terminal device according to an embodiment of the present invention;

[0031] Figure 4 A schematic diagram of the network structure framework provided by the present invention;

[0032] Figure 5 A schematic diagram of a cross-level feature fusion module provided by the present invention;

[0033] Figure 6 A schematic diagram of a feature optimization module provided by the present invention;

[0034] Figure 7 A schematic diagram of an adaptive multi-level feature fusion module provided by the present invention;

[0035] Figure 8 A defect image of a substation equipment provided by the present invention;

[0036] Fig. 9 A schematic diagram of a substation equipment defect detection effect provided by the present invention;

[0037] Fig.10 A label image corresponding to a defect image of a substation equipment provided by the present invention;

[0038] Fig.11 Another substation equipment defect image provided by the present invention;

[0039] Fig.12 A schematic diagram of another substation equipment defect detection effect provided by the present invention;

[0040] Fig.13 A label image corresponding to another substation equipment defect image provided by the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The implementation of the present invention is described in detail below in conjunction with the specific embodiments.

[0042] The present invention provides a method, system, terminal device and computer-readable storage medium for detecting defects in substation equipment based on saliency, including a method for detecting defects in substation equipment based on saliency, comprising the steps of:

[0043] S1. Dataset preprocessing: Construct a substation equipment defect dataset containing multiple defects for neural network training and optimization, and perform preprocessing operations on the dataset;

[0044] S2. Construction of backbone convolutional network model: A multi-scale deep learning method is used to construct a neural network model. The defect image of the substation equipment is input into the Swin Transformer model, and feature maps of different scales are extracted. The model is then built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment.

[0045] S3. Global feature extraction of defect entities: Process the feature maps of different levels extracted from the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features to provide effective guidance for the subsequent defect feature processing and feature fusion stages;

[0046] S4, Adaptive multi-scale feature fusion: used to filter the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects;

[0047] S5, multi-scale feature fusion: used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0048] In the step of constructing the backbone convolutional network model, the backbone network first performs hierarchical processing on the defective image of the substation equipment, extracts five layers of features in sequence, cascades the features of the specific layers and obtains four specific channel features through specific operations, and finally obtains features of 32 for all four channels, which are marked as f1, f2, f3 and f4 respectively.

[0049] In the global feature extraction step of the defect body, the three features f2, f3, and f4 are sent to the cross-level feature fusion module, and the extracted global features are then used to perform further feature optimization processing on the four multi-scale features.

[0050] In the adaptive multi-scale feature fusion step, feature map information of different scales is screened, interfering defect edge features are filtered, and defect main body information is enhanced and fused under the guidance of high-level features.

[0051] The above content includes the following effects: first, by constructing a substation defect data set containing multiple defects and performing preprocessing, a rich and accurate data foundation is provided for subsequent detection, which improves the training effect and generalization ability of the model;

[0052] In the construction of the backbone convolutional network model, the backbone network is used to perform hierarchical processing on the defective images of substation equipment and extract feature maps of different scales, which can comprehensively capture various details and overall structures of equipment defects and provide strong feature support for accurate detection. In particular, the four specific channel features obtained provide more targeted information for subsequent processing.

[0053] The global feature extraction step of the defect body can effectively filter the complex background, screen and enhance the useful defect features. At the same time, the extracted global features provide a clear guidance direction for subsequent processing and fusion, greatly improving the accuracy and reliability of detection;

[0054] The adaptive multi-scale feature fusion step obtains accurate defect edge features by screening feature map information of different scales and filtering interference, and enhances the main defect feature information under the guidance of advanced features, achieving effective fusion from both edge and main aspects, making the detection results more accurate and stable;

[0055] The multi-scale feature fusion step combines defect feature maps of different scales to achieve effective multi-scale feature aggregation. The generated final feature output map can more comprehensively reflect the equipment defects and provide a strong basis for equipment maintenance and overhaul.

[0056] In summary, the substation equipment defect detection method based on significance of the present invention is efficient, accurate and reliable, which can effectively improve the level of substation equipment defect detection and play an important role in ensuring the safe operation of substations.

[0057] A substation equipment defect detection system based on saliency, comprising:

[0058] The data set preprocessing module constructs a defect data set of substation equipment containing various defects for training and optimization of neural networks, and performs preprocessing operations on the data set;

[0059] The backbone convolutional network model building module uses a multi-scale deep learning method to build a neural network model. The defect image of the substation equipment is input into the Swin Transformer model, and feature maps of different scales are extracted. The model is built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment.

[0060] The global feature extraction module of the defect subject processes the feature maps of different levels extracted by the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features at the same time, which effectively guides the subsequent defect feature processing and feature fusion stages;

[0061] Adaptive multi-scale feature fusion module, used to screen the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects;

[0062] The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

[0063] The dataset construction module constructs a defective dataset for training optimization. The network framework construction module constructs a neural network with multi-scale deep learning and extracts feature maps. The feature fusion module fuses feature maps of different scales.

[0064] The feature generation module generates equipment defect information, the training module trains the model, and the testing module tests the model.

[0065] The data management module manages the data set, the model optimization module optimizes the neural network model, and the result display module displays the detection results.

[0066] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor.

[0067] A computer-readable storage medium stores a computer program.

[0068] The present invention, when implemented, comprises the following inner cylinder:

[0069] like Figure 1As shown, including:

[0070] Step 1: Collect defect image data of substation equipment, build a data set containing substation equipment defects and annotate the defects at the pixel level for training and optimization of neural networks, including training sets and test sets.

[0071] Step 2, neural network model construction: Use multi-scale deep learning methods to construct a neural network model, and model it layer by layer from coarse to fine, to achieve all-round feature extraction and processing of substation defect features. The backbone network for feature extraction uses Swin Transformer to extract five layers of features, and cascades the fourth and fifth layer features to obtain four multi-scale features. First, use the void space convolution pooling pyramid and channel pooling operations to process these four features to obtain the features of f1, f2, f3, and f4, all of which have 32 channels. The operation can be expressed as:

[0072] S i =A(S i )(1)

[0073] f i ＝CP ( S i )(2)

[0074]

[0075] Where A(*) is the atrous spatial convolutional pooling pyramid, S(i) is the features of each layer extracted by Swin Transformer, X is the original feature map containing n channels, j and k are integers. X is divided into m groups, and the maximum pooling is applied to each group of features and then the m feature maps are concatenated together. In this description, m is set to 32.

[0076] Step 3, global feature extraction (CFM module): Process the feature maps of different levels extracted by Swin Transformer to extract global features, and guide the subsequent feature processing and feature fusion stages. The input features are f2, f3, and f4. Since f1 contains a large number of low-level features, f2 contains low-level features but fewer messy background features. The cross-level feature fusion module consists of two parts, namely the spatial attention processing part and the channel attention processing part. First, the features pass through the spatial information processing part. The specific operation is that the features pass through two convolution modules and the sigmoid activation function to obtain the spatial attention weights and then multiply them with the original features. The multiplied features are added to f3 to obtain new features, and then the channel attention operation is performed on them. Specifically, the features are subjected to global maximum pooling and global average pooling respectively, the two features obtained are connected, and then a convolution operation is performed to obtain the channel attention weights, which are then multiplied with the original channels, and finally added to f4 to obtain the final output feature F cfm The convolution operation consists of two 3*3 convolutions, and batch normalization is added after each convolution to force the data input to each network layer to be under a unified data distribution (normal distribution), and the ReLU function is used for nonlinear transformation. The specific operations are as follows:

[0077] f3=f3+DP(f2×Sig(Conv(Conv(f2)))) (4)

[0078] F cfm =f4+f3×Sig(Conv(Cat(GAP(f3),GMP(f3)))) (5)

[0079] Among them, Conv(*) represents two 1*1 convolutional layers, GAP(*) represents global average pooling, GMP(*) represents global maximum pooling, Sig(*) represents Sigmoid activation function, + and × represent element-by-element addition and multiplication, respectively.

[0080] Step 4, multi-scale feature processing (FOM module): Filter the information of feature maps of different scales, filter interference information, and enhance effective information to reduce interference in the subsequent feature fusion process. The module has two input features, namely the features of the current level and the output features of the previous feature optimization module. The input of the highest-level feature optimization module is the global feature output by the cross-level feature fusion module. First, two convolution operations are performed on the two input features and the residuals are added. Then the two features are multiplied, and finally the obtained features are added to the two input features to obtain the output features of the feature optimization module. The convolution operation consists of two 3*3 convolutions, and batch normalization is added after each convolution so that the data input to each network layer is forced to be under a unified data distribution (normal distribution), and the ReLU function is used for nonlinear transformation. The specific operations are as follows:

[0081] f i =f i +Conv(Conv(f i )) (6)

[0082] f i+1 =f i+1 +Conv(Conv(f i+1 )) (7)

[0083] F fom =f i +f i+1 +(f i+1 ×f i ) (8)

[0084] Among them, Conv(*) includes 3*3 convolution, batch standardization, and normalization operations. + and × represent element-by-element addition and multiplication, respectively.

[0085] Step 5, multi-scale feature fusion (FFM module): Fuse feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map. The input of the multi-level feature fusion module has two parts, namely the output of the feature optimization module of the current layer and the output of the multi-level feature fusion module of the previous layer. In particular, the input of the highest-level multi-level feature fusion module is the output of the feature optimization module of the current layer and the output of the cross-level feature fusion module. The specific process is: first, the output feature of the multi-level feature fusion module of the previous layer is downsampled to the same size as the output feature size of the feature optimization module of the current layer, and the two features are connected and 1*1 convolved to reduce the number of channels by half, and then sent to the multi-level feature fusion module. The features are sequentially extracted through dilated convolutions with different dilation rates. The dilated convolutions of each level can capture contextual information of different sizes, thereby improving the richness and diversity of features. The output feature map of each dilated convolution is added to the output feature map of the dilated convolution of the previous level to achieve multi-scale feature fusion. This step-by-step fusion can integrate feature information of different scales at a higher level, improve the expressiveness of features, and ultimately obtain output features containing rich scale information and context. The whole process can be expressed as:

[0086] f i =Conv(f fomi +f ffmi+1 ) (9)

[0087] F r (f i )=Conv d＝r (Conv d＝1 (f i )) (10)

[0088] f i + = F r (f i ),r=3,6,12,18,24 (11)

[0089] F ffmi =f i (12)

[0090] Among them, + represents element-level addition operation, Conv(d=r) includes convolution with a dilation rate of r, batch standardization, and batch normalization operations.

[0091] Step 6: Training: Use the training set to train the model.

[0092] Step 7: Test: Use the test set to test the improved model.

[0093] This scheme designs a defect detection algorithm for substation equipment based on saliency from the perspectives of feature extraction and feature fusion. In terms of feature extraction, high-level features are first processed to extract high-level semantic information, and high-level semantic information is used to guide shallow features to filter interference information and extract effective features. In terms of feature fusion, deep features and shallow features are cross-fused to maximize the use of effective information of each layer of features. The network can selectively integrate multi-level context information to ensure that the features it obtains are stable and effective. These features contain both high-level information and low-level information, which greatly improves the robustness of the network extraction results.

[0094] Figure 3 This is a schematic diagram of a terminal device according to an embodiment of the present invention. Figure 3 As shown, the terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a software development program. When the processor executes the computer program, the steps in the above-mentioned software development method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0095] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the terminal device. For example, the computer program may be divided into an acquisition module, an execution module, and a generation module (module in a virtual device), and the specific functions of each module are as follows:

[0096] The acquisition module is used for software development requirements; the execution module is used to determine the target components of the target page to be generated based on the software development requirements and the template components on the preset page basic framework template; the target components include at least one of the following: adjustment components modified based on template components, new components created based on template components, and components to be replaced determined based on template components; the generation module is used to assemble the template components and the target components in the page basic framework template to form the target page based on the component name and location information of the target components.

[0097] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 3It is only an example of a terminal device and does not constitute a limitation of the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access equipment, buses, etc.

[0098] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0099] The memory may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. Furthermore, the memory may include both an internal storage unit and an external storage device of the terminal device. The memory is used to store the computer program and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0100] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0101] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0103] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0106] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. . Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0107] The abnormal sound recognition method, system, terminal device and computer-readable storage medium of the present invention analyze and process sound signals, and propose a feature extraction algorithm that integrates two sound signal processing, thereby improving the distinction between comprehensiveness and individual feature differences. On the basis of a general single input, a feature fusion algorithm is used to process important inputs in two directions, thereby improving the generalization ability and accuracy of the network. By using a two-dimensional convolutional neural network to further extract features from the sound signal, and using the "sound map" as the input signal of the two-dimensional convolutional neural network, information can be processed quickly and efficiently.

[0108] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for detecting substation equipment defects based on significance, characterized in that: Includes steps: S1. Dataset preprocessing: Construct a substation equipment defect dataset containing multiple defects for neural network training and optimization, and perform preprocessing operations on the dataset; S2. Construction of backbone convolutional network model: A multi-scale deep learning method is used to construct a neural network model. The defect image of the substation equipment is input into the Swin Transformer model, and feature maps of different scales are extracted. The model is then built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment. S3. Global feature extraction of defect entities: Process the feature maps of different levels extracted from the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features to provide effective guidance for the subsequent defect feature processing and feature fusion stages; S4, Adaptive multi-scale feature fusion: used to filter the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects; S5, multi-scale feature fusion: used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

2. The substation equipment defect detection method based on significance according to claim 1 is characterized in that: In the step of constructing the backbone convolutional network model, the backbone network first performs hierarchical processing on the defective image of the substation equipment, extracts five layers of features in sequence, cascades the features of the specific layers and obtains four specific channel features through specific operations, and finally obtains features of 32 for all four channels, which are marked as f1, f2, f3 and f4 respectively.

3. The substation equipment defect detection method based on significance according to claim 1 is characterized in that: In the global feature extraction step of the defect body, the three features f2, f3, and f4 are sent to the cross-level feature fusion module, and the extracted global features are then used to perform further feature optimization processing on the four multi-scale features.

4. The substation equipment defect detection method based on significance according to claim 1 is characterized in that: In the adaptive multi-scale feature fusion step, feature map information of different scales is screened, interfering defect edge features are filtered, and defect main body information is enhanced and fused under the guidance of high-level features.

5. A substation equipment defect detection system based on significance, characterized in that: include: The data set preprocessing module constructs a defect data set of substation equipment containing various defects for training and optimization of neural networks, and performs preprocessing operations on the data set; The backbone convolutional network model building module uses a multi-scale deep learning method to build a neural network model. The defect image of the substation equipment is input into the Swin Transformer model, and feature maps of different scales are extracted. The model is built layer by layer from coarse to fine, which is used to realize the full range of feature extraction and processing of defect images of substation equipment. The global feature extraction module of the defect subject processes the feature maps of different levels extracted by the backbone network to filter the complex background in the substation, screen and enhance useful defect features, and extract global features at the same time, which effectively guides the subsequent defect feature processing and feature fusion stages; Adaptive multi-scale feature fusion module, used to screen the information of feature maps of different scales, filter out interference information to obtain accurate defect edge features, and enhance the main information of defect features under the guidance of advanced features, and effectively fuse features from both edge and main aspects; The multi-scale feature fusion module is used to fuse defect feature maps of different scales to achieve effective multi-scale feature aggregation and generate the final feature output map.

6. The significant substation equipment defect detection system according to claim 5 is characterized in that: include: The dataset construction module constructs a defective dataset for training optimization. The network framework construction module constructs a neural network with multi-scale deep learning and extracts feature maps. The feature fusion module fuses feature maps of different scales.

7. The significant substation equipment defect detection system according to claim 5 is characterized in that: include: The feature generation module generates equipment defect information, the training module trains the model, and the testing module tests the model.

8. The significant substation equipment defect detection system according to claim 5 is characterized in that: include: The data management module manages the data set, the model optimization module optimizes the neural network model, and the result display module displays the detection results.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images