Substation fire identification method based on image fusion, electronic device

Abstract

The application discloses a substation fire identification method and electronic equipment based on image fusion. The method comprises the following steps: fully fusing an infrared image and a visible light image, perceiving substation fire occurrence probability and degree from multiple dimensions, improving substation fire perception accuracy through Bayes theorem and D-S evidence theory reasoning, and realizing rapid positioning of a specific position where temperature exceeds a threshold value and fire occurs according to a simulation model; and calculating detailed data in a substation operation process according to time stamps of different time periods, recording operation calculation data, forming machine operation and machine experience of a substation fire identification system through fusion of visible light images and infrared light images, and quickly identifying and positioning a substation fire occurrence position in a subsequent use process. The application solves the technical problem of low identification accuracy caused by single infrared light image or visible light image for fire identification in the prior art.

Description

Technical Field

[0001] This invention relates to the field of substation fire identification technology, and more specifically, to a substation fire identification method and electronic device based on image fusion. Background Technology

[0002] Visible light image and infrared light image fusion is a technique in the field of image processing. It aims to combine two types of images to improve image quality and the accuracy of information extraction. This fusion method mainly has three key steps: feature extraction, fusion strategy, and image reconstruction.

[0003] For methods applied to substation fire identification, substation fire identification is crucial to ensuring the safe operation of power equipment. Since fires are usually accompanied by rising temperatures, and infrared images can reflect the temperature distribution of objects, infrared images play an important role in substation fire identification. However, infrared images have limited target identification capabilities in low-light environments. In contrast, visible light images can clearly display the appearance of equipment, such as its shape, color, and texture.

[0004] Existing technologies mostly employ single infrared or visible light image recognition. Furthermore, current substation fire identification methods rely on the judgment of professionals with extensive experience. Since different personnel have varying levels of experience and different judgment standards, the accuracy of substation fire identification is low. In addition, different environmental and lighting conditions may affect image quality, resulting in problems with poor timeliness and poor detection environment in substation fire identification.

[0005] As can be seen from the above analysis, there is currently no effective solution to the problem of low identification accuracy caused by using a single infrared or visible light image for fire identification in the existing technologies. Summary of the Invention

[0006] This invention provides a substation fire identification method and electronic device based on image fusion, which at least solves the technical problem of low identification accuracy caused by the use of a single infrared or visible light image for fire identification in the prior art.

[0007] According to one aspect of the present invention, a substation fire identification method based on image fusion is provided, comprising:

[0008] The process involves acquiring visible light and infrared images of a substation and establishing a substation simulation model. The visible light and infrared images are then mapped onto the substation simulation model to obtain the target image. The visible light image is preprocessed to obtain a first image, and the infrared image is preprocessed to obtain a second image. Feature extraction is performed on the first image to obtain visible light image features, and on the second image to obtain infrared image features. The first image, the second image, and the target image are segmented to obtain segmentation results, which include multiple equally segmented regions. Temperature anomaly analysis is performed on any equally segmented region to obtain analysis results, which are used to determine multiple probability regions, including edge probability regions. Finally, the substation is simulated based on the substation simulation model. In the simulation, multiple sets of edge probability regions are extracted to obtain extraction results. Probability calculations are performed on these edge probability regions using Bayes' theorem to obtain calculation results, which are used to determine conflict weight regions. Theoretical reasoning is then performed on the conflict weight regions to obtain reasoning results, which include multiple perception results of the target based on visible light image features and infrared image features. Decision fusion is performed on the multiple perception results based on the target neural network to obtain fused perception results. First and second images with the same timestamp interval are processed to determine the substation fire identification level. Fire identification is performed based on the fused perception results and the substation fire identification level to obtain fire identification results, which are used to determine whether a fire has occurred in the substation.

[0009] Optionally, preprocessing includes: noise reduction, contrast enhancement, and histogram equalization.

[0010] Optionally, visible light image features and infrared image features include: temperature, temperature changes with timestamps, and the degree of change of device components with timestamps.

[0011] Optionally, the above-mentioned substation fire identification method based on image fusion further includes: weighting the visible light image features and infrared image features and marking them with a timestamp; or, stitching the visible light image features and infrared image features together and marking them with a timestamp.

[0012] Optionally, the segmentation includes: threshold segmentation, edge detection-based image segmentation, and substation morphological change-based image segmentation.

[0013] Optionally, the decision fusion of multiple perception results based on the target neural network to obtain the fused perception result includes: standardizing the multiple perception results to obtain a standardized perception result; and performing a nonlinear mapping on the standardized perception results based on the target neural network to obtain the fused perception result.

[0014] Optionally, processing the first and second images within the same timestamp interval to determine the substation fire identification level includes: performing multiple convolution processes on the first image to obtain visible light image convolution features; weighting and concatenating the parameters of the visible light image convolution features with the parameters of the convolution features of the second image to obtain concatenated convolution features; and extracting and classifying the concatenated convolution features to obtain the substation fire identification level.

[0015] Optionally, the identification results include: the location where the substation fire temperature exceeds the threshold and the predicted time of fire occurrence. The above-mentioned substation fire identification method based on image fusion further includes: in response to the identification results determining that a fire has occurred in the substation, determining the fire morphology based on the first image and the second image; determining the fire source area according to the fire morphology; performing morphological processing on the fire source area to obtain the processing result, wherein the processing result is used to determine the fire source location and the fire spread; and issuing a fire alarm based on the processing result.

[0016] Optionally, morphological processing includes: noise and interference elimination based on any morphological operation, where the morphological operation is one of the following: opening operation, closing operation, or dilatational erosion.

[0017] According to another aspect of the present invention, a substation fire identification device based on image fusion is also provided, comprising:

[0018] The system comprises the following modules: an acquisition module for acquiring visible light and infrared images of the substation and establishing a substation simulation model; a mapping module for mapping the visible light and infrared images to the substation simulation model to obtain the target image; a preprocessing module for preprocessing the visible light image to obtain a first image and the infrared image to obtain a second image; a feature extraction module for extracting features from the first image to obtain visible light image features and from the second image to obtain infrared image features; an image segmentation module for segmenting the first image, the second image, and the target image to obtain segmentation results, which include multiple equally segmented regions; an analysis module for performing temperature anomaly analysis on any equally segmented region to obtain analysis results, which are used to determine multiple probability regions, including edge probability regions; and a simulation module for performing simulation based on the substation simulation model. The simulation model operates on the substation, extracting multiple sets of edge probability regions during the simulation. The calculation module uses Bayes' theorem to calculate the probability within these edge probability regions, obtaining calculation results used to determine conflict weight regions. The inference module performs theoretical inference on the conflict weight regions, obtaining inference results including multiple perception results of the target based on visible light and infrared image features. The fusion module performs decision fusion on multiple perception results based on a target neural network, obtaining a fused perception result. The processing module processes first and second images from the same timestamp interval to determine the substation fire identification level. The identification module performs fire identification based on the fused perception results and the substation fire identification level, obtaining a fire identification result used to determine whether a fire has occurred at the substation.

[0019] Optionally, the above preprocessing module includes: preprocessing including: noise reduction, contrast enhancement and histogram equalization.

[0020] Optionally, the feature extraction module includes: visible light image features and infrared image features including: temperature, temperature changes with timestamps, and the degree of change of device components with timestamps.

[0021] Optionally, the above-mentioned substation fire identification device based on image fusion further includes: a marking module, used to perform a weighted average of visible light image features and infrared image features and mark a timestamp; or, to stitch visible light image features and infrared image features together and mark a timestamp.

[0022] Optionally, the above segmentation module includes: segmentation including: threshold segmentation, edge detection-based image segmentation, and substation morphological change-based image segmentation.

[0023] Optionally, the fusion module includes: standardizing multiple perception results to obtain standardized perception results; and performing nonlinear mapping on the standardized perception results based on the target neural network to obtain fused perception results.

[0024] Optionally, the above processing module includes: performing multiple convolution processes on the first image to obtain visible light image convolution features; weighting and concatenating the parameters of the visible light image convolution features with the parameters of the convolution features of the second image to obtain concatenated convolution features; and extracting and classifying the concatenated convolution features to obtain the substation fire identification level.

[0025] Optionally, the identification results include: the location where the substation fire temperature exceeds the threshold and the predicted fire occurrence time. The above-mentioned substation fire identification device based on image fusion further includes: a determination module, used to determine that a fire has occurred in the substation in response to the identification results; determine the fire morphology based on the first image and the second image; determine the fire source area according to the fire morphology; perform morphological processing on the fire source area to obtain a processing result, wherein the processing result is used to determine the fire source location and the fire spread; and issue a fire alarm according to the processing result.

[0026] Optionally, the above-mentioned substation fire identification device based on image fusion includes: morphological processing including: noise and interference elimination based on any morphological operation, wherein the morphological operation is one of the following: opening operation, closing operation, or dilatational erosion.

[0027] According to another aspect of the present invention, an electronic device is also provided, the electronic device including a memory and a processor; the memory is used to store a program; the processor executes the program to implement the method of any of the foregoing.

[0028] According to another aspect of the present invention, a computer-readable storage medium is also provided, the storage medium storing a computer program that, when executed by a processor, implements the method of any of the foregoing.

[0029] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the method described in any of the foregoing.

[0030] In this embodiment of the invention, by fully fusing infrared and visible light images, the probability and severity of substation fires are perceived from multiple dimensions. The accuracy of substation fire perception is improved through Bayes' theorem and Dempster-Shafer (DS) evidence theory. Based on a simulation model, the system can quickly locate the location of fires where temperatures exceed thresholds. Furthermore, detailed data from the substation's operation is calculated based on timestamps of different time periods, and this rapid calculation data is recorded, forming machine experience for the substation fire identification system based on the fusion of visible and infrared images. In subsequent use, this allows for rapid identification and location of substation fires, achieving accurate and rapid identification and location of substation fires. This improves the accuracy of substation fire identification and solves the technical problem of low accuracy caused by using a single infrared or visible light image for fire identification in existing technologies. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0032] Figure 1 This is a hardware structure block diagram of a mobile terminal for a substation fire identification method based on image fusion, according to an embodiment of the present invention.

[0033] Figure 2 This is a flowchart of a substation fire identification method based on image fusion according to an embodiment of the present invention;

[0034] Figure 3 This is a schematic diagram of the segmentation of an optional mapped simulation model image according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of an optional edge probability region and a corresponding probability intersection region according to an embodiment of the present invention;

[0036] Figure 5 This is a structural block diagram of a substation fire identification device based on image fusion according to an embodiment of the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] According to an embodiment of the present invention, a method embodiment for substation fire identification based on image fusion is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 1 This is a hardware structure block diagram of a mobile terminal for an image fusion-based substation fire identification method according to an embodiment of the present invention, such as... Figure 1 As shown, the mobile terminal 10 (or mobile device 10) may include one or more processors 102 (processor 102 may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA) processing device), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display device 110, an input / output device 108 (i.e., I / O device), a Universal Serial Bus (USB) port (which may be included as one of the ports of a computer bus, not shown in the figure), a network interface (not shown in the figure), a power supply (not shown in the figure), and / or a camera (not shown in the figure). Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal 10 described above. For example, the mobile terminal 10 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0041] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits may be embodied, in whole or in part, as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be wholly or partially integrated into any other element within the mobile terminal 10 (or mobile device 10).

[0042] The memory 104 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the image fusion-based substation fire identification method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned image fusion-based substation fire identification method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0043] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the mobile terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0044] Under the above operating environment, the embodiments of the present invention provide as follows: Figure 2 The image fusion-based substation fire identification method shown is as follows: Figure 2 This is a flowchart of a substation fire identification method based on image fusion according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following implementation steps:

[0045] Step S201: Acquire visible light and infrared images of the substation and establish a substation simulation model;

[0046] Step S202: Map the visible light image and infrared image onto the substation simulation model to obtain the target image;

[0047] Step S203: Preprocess the visible light image to obtain the first image, and preprocess the infrared image to obtain the second image;

[0048] Step S204: Perform feature extraction on the first image to obtain visible light image features, and perform feature extraction on the second image to obtain infrared image features;

[0049] Step S205: Segment the first image, the second image, and the target image to obtain a segmentation result, wherein the segmentation result includes multiple equally segmented regions;

[0050] Step S206: Perform temperature anomaly analysis on any equally divided region to obtain analysis results. The analysis results are used to determine multiple probability regions, including marginal probability regions.

[0051] Step S207: Based on the substation simulation model, the substation is simulated and run. During the simulation, multiple sets of edge probability regions are extracted to obtain the extraction results.

[0052] Step S208: Probability calculation is performed on multiple sets of marginal probability regions using Bayes' theorem to obtain the calculation results, wherein the calculation results are used to determine the conflict weight region.

[0053] Step S209: Perform theoretical reasoning on the conflict weight region to obtain the reasoning result, wherein the reasoning result includes multiple perception results of the target based on visible light image features and infrared image features;

[0054] Step S210: Based on the target neural network, perform decision fusion on multiple perception results to obtain a fused perception result;

[0055] Step S211: Process the first and second images with the same timestamp interval to determine the substation fire identification level;

[0056] Step S212: Fire identification is performed based on the fusion sensing results and the substation fire identification level to obtain fire identification results, which are used to determine whether a fire has occurred in the substation.

[0057] The preprocessing described above may include: noise reduction, contrast enhancement, and histogram equalization. The visible light image features and infrared image features described above may include: temperature, temperature variations with timestamps, and the degree of variation of device components with timestamps.

[0058] During the feature extraction process of the preprocessed visible light image and infrared image (i.e., the first image and the second image), the two types of images can be processed through the following steps S213 or S214 to ensure that the visible light image and the infrared image are time-aligned, thereby accurately identifying substation fires based on the fusion result of the two types of images:

[0059] Step S213: Perform a weighted average of the visible light image features and the infrared image features, and mark the timestamp.

[0060] Alternatively, in step S214, the visible light image features and infrared image features are stitched together and a timestamp is added.

[0061] The aforementioned segmentation can include: threshold segmentation, edge detection-based image segmentation, and image segmentation based on substation morphological changes. It should be noted that substation morphological changes refer to changes in the appearance and shape of a substation under different times or conditions.

[0062] The following combination Figure 3 , Figure 4 The above methods will be further explained.

[0063] like Figure 3 As shown, this invention segments the preprocessed visible light image, infrared image, and mapped simulation model image. By constructing x, y, and z-axis image data and equally segmenting the xy, yz, and xz planes, multiple equally segmented regions are obtained. Then, temperature anomaly threshold analysis is performed on the fusion features within a segmented region. It should be noted that this fusion feature is a fusion of visible light image features and infrared image features. During the analysis, when the fusion feature exceeds the temperature anomaly threshold, a fire is identified. The fire is located within the simulation model through the intersection of the xy, yz, and xz planes, forming multiple probability regions P(x), P(y), and P(z). Furthermore, P(x), P(y), and P(z) have a probability intersection region P(x, y, z).

[0064] Still as Figure 3 As shown, further, the substation is simulated using a simulation model. Multiple P(x), P(y), and P(z) regions with temperatures exceeding the threshold point are extracted during the simulation operation. The weight of a certain region in any set of P(x), P(y), and P(z) regions is increased, and P(x), P(y), P(z), and P(x, y, z) are further segmented and weighted.

[0065] Furthermore, by using Bayes' theorem to calculate the probability in the regions P(x), P(y), and P(z), the Bayes formula can be expressed as shown in the following formula (1):

[0066]

[0067] In the above formula (1), P(A|B) is the posterior probability, which is the probability that event A occurs given that event B has occurred; P(A) and P(B) represent the probabilities of event A and event B occurring, respectively, also known as marginal probabilities.

[0068] This invention simulates operational probabilities using a Bayesian theorem and a simulation model. It also increases the weight of specific regions P(x), P(y), and P(z) by adding factors related to actual temperature anomalies (temperature exceeding thresholds) or fire locations. The resulting probability intersection region P(x, y, z) is then calculated to obtain the probability of temperature exceeding thresholds and the probability of fire occurrence. Figure 4 The P(x, y, z) region shown is used as a reference.

[0069] In this invention, after calculating the probability of multiple marginal probability regions using Bayes' theorem, the conflict weight region is determined based on the calculation results. Then, the conflict weight region can be reasoned based on the DS evidence theory to correct abnormal conclusions in the fire identification results.

[0070] It should be noted that the DS evidence theory reasoning is a further expansion of the measurement of P(x), P(y), P(z), and the P(x, y, z) region. DS evidence theory can be used for multi-parameter fusion. First, the target is perceived separately by multiple parameters, and the perception results are represented by random sets or probabilistic methods. Then, DS evidence theory is used to fuse the results from each heterogeneous parameter to achieve a comprehensive perception of the target. In this invention, each heterogeneous parameter can refer to visible light image features and infrared image features, and the target can refer to a substation fire.

[0071] The DS evidence theory can represent the set of propositions of interest using an identification frame (or hypothesis space), which can be represented by Θ, assuming Θ = θ1, θ2, θ3, ..., θ n The power set of Θ is 2 Θ ={φ,Θ,{θ1},{θ2},…,{θ n When the set function m:2 Θ →Evidence analysis can be performed when [0,1] satisfies the following formula (2):

[0072] And ∑ A∈Θ Formula (2) = m(A) = 1

[0073] In the above formula (2), m is the mass function on the recognition frame, or m is called the basic probability assignment.

[0074] It should be noted here that evidence is a method of assigning weights to different hypotheses based on observation. Evidence may include the probability of simulation model operation, the probability of actual use of temperature exceeding the threshold and the probability of fire. By analyzing the evidence, the credibility evaluation of hypothesis A can be obtained (i.e. m(A) in formula (2)). In the above formula (2), hypothesis A that makes m(A) > 0 is called the focal element.

[0075] For the recognition framework Θ, m:2 Θ →[0,1] represents the basic probability assignment on Θ, Bel:2 Θ →If [0,1] is the confidence function on Θ, then:

[0076]

[0077] For the recognition frame Θ, pl:2 Θ →[0,1] is the likelihood function on Θ, Bel:2 Θ →If [0,1] is the confidence function on Θ, then:

[0078]

[0079] In the above formula (4), pl represents the degree to which there is no doubt about hypothesis A, which is a more lenient estimate than Bel, and the two satisfy the following formula (5):

[0080] [Bel(A),pl(A)]∈[0,1]Formula (5)

[0081] Formula (5) above is the trust interval of assumption A, Bel(A) is the lower limit of the trust function, and pl(A) is the upper limit of the trust function. The upper and lower limits of this trust interval represent the upper and lower limits of the probability of temperature exceeding the temperature anomaly threshold and the probability of fire occurring in the region.

[0082] Regarding the DS evidence combination rules, the composition rule of DS evidence theory is the key and difficult point of DS evidence theory, directly related to the practical application of the theory. A finite number of mass functions m1, m2, m3, ..., m on the recognition framework Θ n Then its Dempster synthesis rule can be shown in the following formula (6):

[0083]

[0084] In the above formula (6), K satisfies the following formula (7):

[0085]

[0086] In the above formula (7), K is the normalization factor, and 1-K reflects the degree of conflict of evidence.

[0087] For mass functions based on different evidence within the same identification framework, if the evidence for the temperature exceeding the temperature anomaly threshold in regions P(x), P(y), P(z), and P(x, y, z) is not completely conflicting with the evidence for the probability of fire occurrence, then the DS evidence combination rule can be used for synthesis. However, if there is a conflict in the evidence, then applying the DS evidence combination rule will result in obvious abnormal results.

[0088] It should also be noted that conflict weight refers to the weighting of conflict when calculating the direct sum of two pieces of evidence, assuming Bel1 and Bel2 are confidence functions on the identification frame Θ, based on the DS evidence combination rule, if A i ∩B j =Φ when m1(A i )m2(B j The total credibility discarded during the synthesis of Bel1 and Bel2 is shown in the following formula (8):

[0089]

[0090] That is, when A i ∩B j When k = Φ, Bel1 and Bel2 decibels assign confidence to two incompatible hypotheses, and the two pieces of evidence conflict. The larger the value of k, the greater the degree of conflict.

[0091] In summary, the DS evidence theory is a method for integrating evidence to make comprehensive decisions. Evidence can come from cognitive or sensor results. For a problem, all possible outcomes recognized are represented by the identification frame Θ. The mass function is the probability of each possibility, the confidence function is the degree of credibility of a hypothesis under all evidence, the likelihood function is the degree to which there is no doubt about the possibility under all evidence, and the DS evidence combination rule is the probability of a hypothesis under all evidence.

[0092] In an optional embodiment, in step S210, decision fusion is performed on multiple perception results based on the target neural network to obtain fused perception results, including:

[0093] Step S215: Standardize the multiple perception results to obtain standardized perception results;

[0094] Step S216: Perform nonlinear mapping on the standardized perception results based on the target neural network to obtain the fused perception results.

[0095] Specifically, the decision fusion based on neural networks represents the perception results of the above heterogeneous parameters on the target in a unified form. Then, each perception result is used as the input of the neural network. The neural network performs nonlinear mapping on the perception results of each heterogeneous parameter to obtain the fused perception result. In substation fire identification, each heterogeneous parameter can refer to visible light image features and infrared image features, including temperature, temperature changes with timestamps, and the degree of change of equipment components with timestamps.

[0096] In an optional embodiment, step S211, processing the first and second images with the same timestamp interval to determine the substation fire identification level includes:

[0097] Step S217: Perform multiple convolution processes on the first image to obtain visible light image convolution features;

[0098] Step S218: The parameters of the convolutional features of the visible light image and the parameters of the convolutional features of the second image are weighted and concatenated to obtain the concatenated convolutional features.

[0099] Step S219: Extract and classify the convolutional features after splicing to obtain the substation fire identification level.

[0100] Specifically, a visible light image with a timestamp interval is captured, and a visible light image of the target pixel size is generated using a recursive graph method. Then, features of the visible light image are extracted through one layer of convolution and two layers of convolution. Next, a convolutional feature layer of the visible light image is generated through the convolutional neural operation ROIPooling. The parameters of the convolutional feature layer of the visible light image are weighted and concatenated with the parameters of the infrared image features. The concatenated features are then further extracted through two layers of convolution in Faster R-CNN. Finally, the features extracted in the last extraction are classified using softmax, and the substation fire identification level is obtained based on the classification results.

[0101] In an optional embodiment, the identification results include: the location where the substation fire temperature exceeds the threshold and the predicted time of fire occurrence. The above-mentioned substation fire identification method based on image fusion further includes:

[0102] Step S221: In response to the identification result determining that a fire has occurred in the substation, the fire mode is determined based on the first image and the second image;

[0103] Step S222: Determine the fire source area based on the fire pattern;

[0104] Step S223: Perform morphological processing on the fire source area to obtain the processing result, which is used to determine the location of the fire source and the direction of the fire.

[0105] Step S224: Activate fire alarm based on processing results.

[0106] The aforementioned morphological processing includes: noise and interference elimination based on any morphological operation, which is one of the following: opening operation, closing operation, or dilatational erosion.

[0107] In this invention, by combining fused sensing results with the substation fire identification level, the location where the substation fire temperature exceeds the abnormal temperature threshold and the predicted fire occurrence time are determined. Then, based on the judgment and prediction results, it is determined whether a fire has occurred in the substation. Once a fire (fire source) is confirmed, the fire area is morphologically processed using fire patterns obtained from training on visible light and infrared images to determine the fire source location and its trend, and a fire alarm signal is sent.

[0108] This invention extracts features from infrared sensor data and visible light image data through two separate processes. A timestamp sequence from the visible light image corresponding to the infrared image capture time is extracted as input. Considering the temporal and spatial characteristics of the data, a recursive graph method is used to generate an image of the target pixel size. Features are then extracted through one and two convolutional layers. A convolutional neural operation (ROIPooling) is used to generate a single convolutional feature layer for the substation image, which is then weighted and concatenated with the parameters of the infrared image features. The concatenated features are then further extracted using two convolutional layers of Faster R-CNN. Finally, softmax is used for classification to determine the substation fire identification level. Furthermore, by combining the fused perception results with the identified substation fire identification level, the location where the substation fire temperature exceeds the threshold and the predicted fire occurrence time are determined to identify whether a fire has occurred.

[0109] In this embodiment of the invention, by fully fusing infrared and visible light images, the probability and severity of substation fires are perceived from multiple dimensions. The accuracy of substation fire perception is improved through Bayes' theorem and Dempster-Shafer (DS) evidence theory. Based on a simulation model, the system can quickly locate the location of fires where temperatures exceed thresholds. Furthermore, detailed data from the substation's operation is calculated based on timestamps of different time periods, and this rapid calculation data is recorded, forming machine experience for the substation fire identification system based on the fusion of visible and infrared images. In subsequent use, this allows for rapid identification and location of substation fires, achieving accurate and rapid identification and location of substation fires. This improves the accuracy of substation fire identification and solves the technical problem of low accuracy caused by using a single infrared or visible light image for fire identification in existing technologies.

[0110] In this embodiment, a substation fire identification device based on image fusion is also provided. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, a "module" is a combination of software and / or hardware that can perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] Figure 5 This is a structural block diagram of a substation fire identification device based on image fusion according to an embodiment of the present invention, such as... Figure 5 As shown, the device includes:

[0112] The acquisition module 501 is used to acquire visible light and infrared images of the substation and to establish a simulation model of the substation.

[0113] The mapping module 502 is used to map visible light images and infrared images onto the substation simulation model to obtain the target image;

[0114] The preprocessing module 503 is used to preprocess the visible light image to obtain the first image, and to preprocess the infrared image to obtain the second image;

[0115] The feature extraction module 504 is used to extract features from the first image to obtain visible light image features, and to extract features from the second image to obtain infrared image features;

[0116] The image segmentation module 505 is used to segment the first image, the second image, and the target image to obtain a segmentation result, wherein the segmentation result includes multiple equally segmented regions;

[0117] Analysis module 506 is used to perform temperature anomaly analysis on any equally divided region and obtain analysis results. The analysis results are used to determine multiple probability regions, including edge probability regions.

[0118] Simulation module 507 is used to simulate the operation of a substation based on a substation simulation model. During the simulation operation, it extracts multiple sets of edge probability regions and obtains the extraction results.

[0119] The calculation module 508 is used to perform probability calculations on multiple sets of marginal probability regions using Bayes' theorem to obtain calculation results, wherein the calculation results are used to determine the conflict weight region.

[0120] The reasoning module 509 is used to perform theoretical reasoning on the conflict weight region and obtain reasoning results, which include multiple perception results of the target based on visible light image features and infrared image features.

[0121] The fusion module 510 is used to perform decision fusion on multiple perception results based on the target neural network to obtain a fused perception result.

[0122] Processing module 511 is used to process the first image and the second image with the same timestamp interval to determine the substation fire identification level;

[0123] The identification module 512 is used to identify fires based on the fusion sensing results and the fire identification level of the substation, and obtain fire identification results, which are used to determine whether a fire has occurred in the substation.

[0124] Optionally, the preprocessing module 503 includes: preprocessing including: noise reduction, contrast enhancement and histogram equalization.

[0125] Optionally, the feature extraction module 504 includes: visible light image features and infrared image features including: temperature, temperature change with timestamp, and degree of change of device components with timestamp.

[0126] Optionally, the above-mentioned substation fire identification device based on image fusion further includes: a marking module 513 (not shown in the figure), used to perform a weighted average of visible light image features and infrared image features and mark a timestamp; or, to stitch visible light image features and infrared image features together and mark a timestamp.

[0127] Optionally, the image segmentation module 505 includes: segmentation including: threshold segmentation, edge detection-based image segmentation, and substation morphological change-based image segmentation.

[0128] Optionally, the fusion module 510 includes: standardizing multiple perception results to obtain standardized perception results; and performing nonlinear mapping on the standardized perception results based on the target neural network to obtain fused perception results.

[0129] Optionally, the above processing module 511 includes: performing multiple convolution processes on the first image to obtain visible light image convolution features; weighting and concatenating the parameters of the visible light image convolution features with the parameters of the convolution features of the second image to obtain concatenated convolution features; and extracting and classifying the concatenated convolution features to obtain the substation fire identification level.

[0130] Optionally, the identification results include: the location where the substation fire temperature exceeds the threshold and the predicted fire occurrence time. The above-mentioned substation fire identification device based on image fusion further includes: a determination module 514 (not shown in the figure), used to determine that a fire has occurred in the substation in response to the identification results; determine the fire pattern based on the first image and the second image; determine the fire source area according to the fire pattern; perform morphological processing on the fire source area to obtain the processing result, wherein the processing result is used to determine the fire source location and the fire spread; and issue a fire alarm according to the processing result.

[0131] Optionally, the aforementioned determining module 514 (not shown in the figure) includes: morphological processing including: noise and interference elimination based on any morphological operation, wherein the morphological operation is one of the following: opening operation, closing operation, or dilatational erosion.

[0132] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0133] According to another aspect of the present invention, an electronic device is also provided, the electronic device including a memory and a processor; the memory is used to store a program; the processor executes the program to implement the method of any of the foregoing.

[0134] According to another aspect of the present invention, a computer-readable storage medium is also provided, the storage medium storing a computer program that, when executed by a processor, implements the method of any of the foregoing.

[0135] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0136] Step S1: Acquire visible light and infrared images of the substation and establish a simulation model of the substation;

[0137] Step S2: Map the visible light image and infrared image onto the substation simulation model to obtain the target image;

[0138] Step S3: Preprocess the visible light image to obtain the first image, and preprocess the infrared image to obtain the second image;

[0139] Step S4: Extract features from the first image to obtain visible light image features, and extract features from the second image to obtain infrared image features;

[0140] Step S5: Segment the first image, the second image, and the target image to obtain a segmentation result, wherein the segmentation result includes multiple equally segmented regions;

[0141] Step S6: Perform temperature anomaly analysis on any equally divided region to obtain analysis results. The analysis results are used to determine multiple probability regions, including edge probability regions.

[0142] Step S7: Simulate the substation based on the substation simulation model. During the simulation, extract multiple sets of edge probability regions to obtain the extraction results.

[0143] Step S8: Probability calculation is performed on multiple sets of marginal probability regions using Bayes' theorem to obtain the calculation results, which are used to determine the conflict weight region.

[0144] Step S9: Perform theoretical reasoning on the conflict weight region to obtain the reasoning results, which include multiple perception results of the target based on visible light image features and infrared image features.

[0145] Step S10: Based on the target neural network, perform decision fusion on multiple perception results to obtain fused perception results;

[0146] Step S11: Process the first and second images with the same timestamp interval to determine the substation fire identification level;

[0147] Step S12: Fire identification is performed based on the fusion sensing results and the substation fire identification level to obtain fire identification results, which are used to determine whether a fire has occurred in the substation.

[0148] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0149] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the method described in any of the foregoing.

[0150] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and their optional implementations, which will not be repeated here.

[0151] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0152] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces; the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0157] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A substation fire identification method based on image fusion, characterized in that, include: Acquire visible light and infrared images of the substation and establish a simulation model of the substation; The visible light image and the infrared image are mapped onto the substation simulation model to obtain the target image; The visible light image is preprocessed to obtain a first image, and the infrared image is preprocessed to obtain a second image; Visible light image features are obtained by performing feature extraction on the first image, and infrared image features are obtained by performing feature extraction on the second image; The first image, the second image, and the target image are segmented to obtain a segmentation result, wherein the segmentation result includes multiple equally segmented regions; Temperature anomaly analysis is performed on any equally divided region to obtain analysis results, wherein the analysis results are used to determine multiple probability regions, including edge probability regions; The substation is simulated based on the substation simulation model. During the simulation, multiple sets of edge probability regions are extracted to obtain the extraction results. The probability of multiple marginal probability regions is calculated using Bayes' theorem to obtain the calculation results, wherein the calculation results are used to determine the conflict weight region. Theoretical reasoning is performed on the conflict weight region to obtain the reasoning result, wherein the reasoning result includes multiple perception results of the target by the visible light image features and the infrared image features; The multiple perception results are fused based on the target neural network to obtain a fused perception result. The first and second images with the same timestamp interval are processed to determine the substation fire identification level; Fire identification is performed based on the fused sensing results and the substation fire identification level to obtain fire identification results, wherein the fire identification results are used to determine whether a fire has occurred in the substation.

2. The substation fire identification method based on image fusion according to claim 1, characterized in that, The preprocessing includes: noise reduction, contrast enhancement, and histogram equalization.

3. The substation fire identification method based on image fusion according to claim 1, characterized in that, The visible light image features and infrared image features include: temperature, temperature changes with timestamps, and the degree of change of device components with timestamps.

4. The substation fire identification method based on image fusion according to claim 1, characterized in that, The substation fire identification method based on image fusion also includes: The visible light image features and the infrared image features are weighted and averaged, and then timestamped. Alternatively, the visible light image features and the infrared image features can be stitched together and marked with a timestamp.

5. The substation fire identification method based on image fusion according to claim 1, characterized in that, The segmentation includes: threshold segmentation, edge detection-based image segmentation, and substation morphological change-based image segmentation.

6. The substation fire identification method based on image fusion according to claim 1, characterized in that, The multiple perception results are fused based on the target neural network to obtain the fused perception result, which includes: The multiple perception results are standardized to obtain standardized perception results. The standardized perception result is nonlinearly mapped based on the target neural network to obtain the fused perception result.

7. The substation fire identification method based on image fusion according to claim 4, characterized in that, Processing the first and second images within the same timestamp interval to determine the substation fire identification level includes: The first image is subjected to multiple convolution processes to obtain visible light image convolution features; The parameters of the convolutional features of the visible light image and the parameters of the convolutional features of the second image are weighted and concatenated to obtain the concatenated convolutional features. The convolutional features after splicing are extracted and classified to obtain the fire identification level of the substation.

8. The substation fire identification method based on image fusion according to claim 1, characterized in that, The identification results include: the location where the substation fire temperature exceeds the threshold and the predicted fire occurrence time. The substation fire identification method based on image fusion also includes: In response to the identification result determining that a fire has occurred in the substation, the fire pattern is determined based on the first image and the second image; The fire source area is determined based on the described fire pattern; The fire source area is subjected to morphological processing to obtain processing results, wherein the processing results are used to determine the location of the fire source and the direction of the fire. A fire alarm will be triggered based on the processing results.

9. The substation fire identification method based on image fusion according to claim 8, characterized in that, The morphological processing includes: eliminating noise and interference based on any morphological operation, wherein the morphological operation is one of the following: opening operation, closing operation, or dilatational erosion.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor; The memory is used to store programs; The processor executes the program to implement the method of any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

Images