GIS switch foreign matter detection method based on multispectral image fusion and related equipment

Abstract

The invention discloses a GIS switch foreign matter detection method based on multispectral image fusion and related equipment, and belongs to the technical field of electrical equipment foreign matter detection, and the method comprises the steps: obtaining a multispectral image of a to-be-detected GIS switch; and inputting the multispectral image into a pre-trained multispectral fusion model to obtain a multispectral feature map, and inputting the multispectral feature map into a pre-trained multispectral image foreign matter detection model to obtain a foreign matter detection result of the to-be-detected GIS switch. The development and automatic identification of the tiny foreign matter material under different light wavelengths are realized by researching the development technology of the tiny foreign matter material under different light wavelengths and combining multispectral fusion and a multispectral image foreign matter detection algorithm, and the accuracy and robustness of detection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of foreign body detection of electrical equipment, and in particular relates to a GIS switch foreign body detection method based on multi-spectral image fusion and related equipment. Background Art At present, the detection technology of tiny foreign objects in GIS (gas insulated switchgear) switches mainly relies on traditional image processing methods and simple visual sensors. These methods usually perform detection under a single light source and fixed wavelength, which is difficult to deal with the detection of tiny foreign objects inside GIS switches under different materials and low light conditions. Existing visual sensors also have certain limitations in resolution and sensitivity, resulting in insufficient accuracy in detecting tiny foreign objects.

[0002] Although multispectral imaging technology has made some progress in other fields, its application in GIS switches is still immature. Existing multispectral imaging systems are mainly used for the detection of large-scale objects. GIS switch micro-foreign body detection technology has deficiencies in light source selection, sensor performance and image processing algorithms. It is difficult to achieve high-accuracy detection in complex environments, and the detection effect of micro-foreign bodies is limited. At the same time, the recognition rate and accuracy of existing image processing algorithms still need to be improved when facing complex backgrounds and low-light conditions. Summary of the invention

[0003] The purpose of the present invention is to provide a GIS switch foreign body detection method and related equipment based on multispectral image fusion, aiming to solve the shortcomings of the prior art in the detection of tiny foreign bodies in GIS switches. By studying the development technology of tiny foreign body materials under different light wavelengths, combined with multispectral fusion and multispectral image foreign body detection algorithms, the development and automatic identification of tiny foreign body materials under different light wavelengths can be achieved, thereby improving the accuracy and robustness of detection.

[0004] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a GIS switch foreign body detection method based on multispectral image fusion, comprising: Acquire a multispectral image of the GIS switch to be inspected; The multispectral image is input into a pre-trained multispectral fusion model to obtain a multispectral feature map, and the multispectral feature map is input into a pre-trained multispectral image foreign body detection model to obtain a foreign body detection result of the GIS switch to be detected.

[0005] Furthermore, the step of acquiring a multispectral image of the GIS switch to be detected includes: Collect images of the inside of the GIS switch at different wavelengths, light source types and intensities to obtain multi-band images, and record the development effects of foreign objects of different materials at different light wavelengths; The multispectral image inside the GIS switch is collected under the combination of light source type and intensity that provides the best development quality for the image sensor.

[0006] Furthermore, the light source type and intensity combination with the best development effect quality is determined by the following steps: Experimentally obtain image contrast under different light source types and intensities; Choose the light source type and intensity combination that gives the highest contrast.

[0007] Further, the image sensor is selected by the following steps: Use different image sensors to capture images; Calculate the clarity of the image; Choose an image sensor that maximizes clarity.

[0008] Furthermore, the pre-trained multi-spectral fusion model is constructed by the following steps: Collect images of GIS switches in multiple bands; For each band of images Perform denoising and correction; Align the images at different wavelengths to obtain the registered image ; for Band image, i=1,2,3,…n; After registration Band images; For each registered band image Perform convolution processing to extract feature maps : The feature map Use splicing technology to fuse and get the fused feature map : The fused feature map Further convolution processing is performed to obtain the final multispectral feature map .

[0009] Furthermore, the pre-trained multispectral image foreign body detection model includes a target position prediction module and a target category prediction module; The target position module expression is as follows:

[0010] in, It is a regression function used to predict the bounding box parameters of the target, and bbox represents the predicted target bounding box; The prediction target category module expression is as follows:

[0011] in, It is a classification function used to predict the category label of the target; cls represents the predicted target category.

[0012] Furthermore, the pre-trained multispectral fusion model and the pre-trained multispectral image foreign body detection model are trained by the following steps: The multispectral image fusion model and the category prediction module are jointly trained using the gradient descent method of deep learning; the loss used in training is the sum of the cross entropy loss and the regression loss: The samples used for training include multispectral images collected from the inside of GIS using image sensors, and the multispectral images include images with foreign objects and normal images; the labels of the images include the marking of the positions of foreign objects in the images with foreign objects and the marking of the types of foreign objects.

[0013] In a second aspect, the present invention provides a GIS switch foreign body detection system based on multispectral image fusion, comprising: An acquisition module is used to obtain a multispectral image of the GIS switch to be detected; The detection module inputs the multispectral image into a pre-trained multispectral fusion model to obtain a multispectral feature map, and inputs the multispectral feature map into a pre-trained multispectral image foreign body detection model to obtain a foreign body detection result of the GIS switch to be detected.

[0014] In a third aspect, the present invention provides an electronic device, comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the GIS switch foreign object detection method based on multi-spectral image fusion as described in any one of the first aspects of the present invention.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the GIS switch foreign body detection method based on multispectral image fusion as described in any one of the first aspects of the present invention.

[0016] Compared with the prior art, the present invention has at least the following beneficial technical effects: The present invention analyzes the development effect of tiny foreign bodies inside the GIS switch. First, a multispectral camera is used to collect image data at different wavelengths, and the appropriate light source type and intensity are selected according to the development results, and a high-resolution image sensor is selected to ensure the detection performance under low-light conditions. Then, using the image information at different light wavelengths, a multispectral fusion model is established through the registration and feature fusion technology of multispectral images to improve the development effect of tiny foreign body materials. Finally, a multispectral image foreign body detection algorithm based on deep learning is used to realize the automatic recognition and classification of tiny foreign bodies inside the GIS cavity, and a multispectral image recognition model for foreign bodies is established, thereby improving the accuracy and robustness of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a GIS switch foreign body detection method based on multispectral image fusion provided in Example 1 of the present invention; Figure 2 It is a schematic diagram of the multi-spectral feature fusion process; Figure 3 A structural block diagram of a GIS switch foreign body detection system based on multispectral image fusion provided by an embodiment of the present invention; Figure 4 A block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] The present invention is further described in detail below in conjunction with the accompanying drawings: This application proposes a technical solution based on multispectral fusion and multispectral image foreign body detection for the detection of tiny foreign bodies inside GIS switches. The present invention solves these technical problems by studying the development technology of tiny foreign body materials under different light wavelengths, combining multispectral fusion models and multispectral image foreign body detection algorithms.

[0021] Example 1 Reference Figure 1 This embodiment provides a GIS switch foreign body detection method based on multispectral image fusion, comprising the following steps: Step 1: Multispectral data acquisition: A multispectral camera is used to collect image data inside the GIS switch at different wavelengths, light source types and intensities to obtain multi-band images, and to record the development effects of foreign objects of different materials at different light wavelengths. In this embodiment, the contrast index is used to characterize the development effect.

[0022] Step 2: Select the light source type and intensity, and select the visual sensor Analyze the development effect, select the best light source type and intensity combination, and select a high-resolution (more than 2 million) image sensor to ensure image quality under low-light conditions, that is, when natural light or indoor light shines into the opening on the GIS switch housing, foreign objects with a diameter of 1mm can be clearly seen. The quality assessment process is as follows: (1) Selection of light source type and intensity The image quality index under different light source types and intensities is obtained through experiments. The present invention uses the contrast index. The contrast index is defined as follows:

[0023] in, is the average brightness of the area where the foreign matter is located, is the average brightness of the background area. Choose the light source type and intensity combination that gives the highest contrast.

[0024] (2) Image sensor selection A high-resolution multispectral camera is used as the image sensor, and its performance in low-light conditions is evaluated experimentally. The sharpness of the image can be evaluated by calculating the sharpness of the image. The image sharpness can be expressed by the mean of the gradient amplitude:

[0025] in, is the gradient of the i-th pixel, and N is the total number of pixels. A multispectral camera with the highest sharpness is selected as the image sensor.

[0026] Step 3: Multispectral fusion model construction Reference Figure 2 First, the collected multi-band images are preprocessed, including denoising, correction and other operations. Perform denoising and correction. Use the registration algorithm to align images at different wavelengths to obtain the registered image . for Band image, i=1,2,3,…n; After registration Band images; Next, for each registered band image Perform convolution processing to extract feature maps :

[0027] Then, the feature maps of different modalities Fusion is performed to obtain the fused feature map :

[0028] It represents feature splicing technology.

[0029] Finally, the convolutional network is used to fused the feature maps Further convolution processing is performed to obtain the final multispectral feature map :

[0030] in, It is the multi-spectral fusion convolutional network layer, multi-spectral fusion feature map Processed by the same convolutional network layer with shared parameters, the output This is the final multispectral feature map.

[0031] This step results in a feature map extraction model architecture, which is responsible for fusing multispectral images into multispectral feature maps.

[0032] Step 4: Multispectral image foreign body detection model construction and training Based on the multispectral feature map obtained by the multispectral fusion model in Step 3, a multispectral micro-foreign matter detection algorithm is designed. Using deep learning methods, the detection model is trained to achieve automatic detection and identification of micro-foreign matters, and an image recognition model for foreign matters inside the GIS cavity is established to improve the accuracy and robustness of detection.

[0033] The specific steps are as follows: First, build a multispectral target detection model. The specific method is: use the target detection head to detect the multispectral feature map Processing, predicting the target location and category. Target location prediction:

[0034] in, It is a regression function used to predict the bounding box parameters of the target, and bbox represents the predicted target bounding box.

[0035] Category prediction:

[0036] in, is a classification function used to predict the target category label. cls represents the predicted target category.

[0037] Finally, the multispectral image fusion model module of step 3 and the target position prediction module and category prediction module of this step (step 4) are jointly trained using the gradient descent method of deep learning. The loss L used in training is the sum of the cross entropy loss and the regression loss:

[0038] in, is the classification loss, is the bounding box regression loss.

[0039] The training samples include multispectral images collected from the GIS using a multispectral image sensor under the image sensor and light source type and intensity determined in step 2. The multispectral images include images with foreign objects and normal (no foreign objects) images. The labels include the annotation of the foreign object location bbox in the foreign object image and the annotation of the foreign object type cls.

[0040] Step 5: Multispectral image foreign body detection Using the image sensor determined in Step 2, under the light source type and intensity determined in Step 2, collect the internal image data of the GIS switch to be inspected, input the data into the trained multispectral fusion model and the multispectral image foreign body detection model, and obtain the inspection result of the GIS switch to be inspected. The foreign body detection result of the GIS switch to be inspected includes whether there is a foreign body, and if there is a foreign body, it also includes the location and type of the foreign body.

[0041] Through the above steps, the multi-spectral fusion model and high-definition visual recognition algorithm proposed in the present invention can effectively improve the detection accuracy and robustness of tiny foreign objects inside GIS switches.

[0042] Example 2 See also Figure 3In this embodiment, a GIS switch foreign body detection system based on multispectral image fusion is provided, including: An acquisition module is used to obtain a multispectral image of the GIS switch to be detected; The detection module inputs the multispectral image into a pre-trained multispectral fusion model to obtain a multispectral feature map, and inputs the multispectral feature map into a pre-trained multispectral image foreign body detection model to obtain a foreign body detection result of the GIS switch to be detected.

[0043] All relevant contents of each step involved in the aforementioned embodiment of the GIS switch foreign object detection method based on multi-spectral image fusion can be referred to the functional description of the functional modules corresponding to the GIS switch foreign object detection system based on multi-spectral image fusion in the embodiment of the present invention, and will not be repeated here.

[0044] Example 3 Reference Figure 4 , This embodiment provides an electronic device, which includes a processor and a memory, and the processor and the memory are connected through a bus; the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the GIS switch foreign body detection method based on multispectral image fusion. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0045] Example 4 This embodiment provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the GIS switch foreign body detection method based on multispectral image fusion in the above embodiment.

[0046] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0047] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0048] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0049] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0050] Example 5 This embodiment provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program product, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0051] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0052] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A GIS switch foreign body detection method based on multispectral image fusion, characterized in that: include: Acquire a multispectral image of the GIS switch to be inspected; The multispectral image is input into a pre-trained multispectral fusion model to obtain a multispectral feature map, and the multispectral feature map is input into a pre-trained multispectral image foreign body detection model to obtain a foreign body detection result of the GIS switch to be detected.

2. The GIS switch foreign body detection method based on multispectral image fusion according to claim 1 is characterized in that: The method of acquiring a multispectral image of a GIS switch to be detected comprises: Collect images of the inside of the GIS switch at different wavelengths, light source types and intensities to obtain multi-band images, and record the development effects of foreign objects of different materials at different light wavelengths; The multispectral image inside the GIS switch is collected under the combination of light source type and intensity that provides the best development quality for the image sensor.

3. The GIS switch foreign body detection method based on multispectral image fusion according to claim 2 is characterized in that: The light source type and intensity combination with the best development quality is determined by the following steps: Experimentally obtain image contrast under different light source types and intensities; Choose the light source type and intensity combination that gives the highest contrast.

4. The GIS switch foreign body detection method based on multispectral image fusion according to claim 2 is characterized in that: The image sensor is selected by the following steps: Use different image sensors to capture images; Calculate the clarity of the image; Choose an image sensor that maximizes clarity.

5. The GIS switch foreign body detection method based on multispectral image fusion according to claim 1 is characterized in that: The pre-trained multispectral fusion model is constructed by the following steps: Collect images of GIS switches in multiple bands; For each band of images Perform denoising and correction; Align the images at different wavelengths to obtain the registered image ; for Band image, i=1,2,3,…n; After registration Band images; For each registered band image Perform convolution processing to extract feature maps : The feature map Use splicing technology to fuse and get the fused feature map : The fused feature map Further convolution processing is performed to obtain the final multispectral feature map .

6. The GIS switch foreign body detection method based on multispectral image fusion according to claim 1 is characterized in that: The pre-trained multispectral image foreign body detection model includes a target position prediction module and a target category prediction module; The target position module expression is as follows: in, It is a regression function used to predict the bounding box parameters of the target, and bbox represents the predicted target bounding box; The prediction target category module expression is as follows: in, It is a classification function used to predict the category label of the target; cls represents the predicted target category.

7. The GIS switch foreign body detection method based on multispectral image fusion according to claim 1 is characterized in that: The pre-trained multispectral fusion model and the pre-trained multispectral image foreign body detection model are trained by the following steps: The multispectral image fusion model and the category prediction module are jointly trained using the gradient descent method of deep learning; the loss used in training is the sum of the cross entropy loss and the regression loss: The samples used for training include multispectral images collected from the inside of GIS using image sensors, and the multispectral images include images with foreign objects and normal images; the labels of the images include the marking of the positions of foreign objects in the images with foreign objects and the marking of the types of foreign objects.

8. GIS switch foreign body detection system based on multispectral image fusion, characterized in that: include: An acquisition module is used to obtain a multispectral image of the GIS switch to be detected; The detection module inputs the multispectral image into a pre-trained multispectral fusion model to obtain a multispectral feature map, and inputs the multispectral feature map into a pre-trained multispectral image foreign body detection model to obtain a foreign body detection result of the GIS switch to be detected.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the GIS switch foreign object detection method based on multi-spectral image fusion as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the GIS switch foreign body detection method based on multispectral image fusion described in any one of claims 1 to 7 is implemented.

Images