Remote intelligent inspection method and system for booster station based on digital twinning

Abstract

The embodiment of the invention provides a remote intelligent inspection method and system for a booster station based on digital twinning, and the method comprises the steps: carrying out the analysis and processing of an infrared image and a color image of booster station equipment through a machine vision technology based on deep learning, and respectively capturing the deep features and shallow features of the infrared image and the color image; the multi-level and multi-mode information of the surface state of the booster station equipment is excavated, so that intelligent oil leakage detection of the booster station equipment is realized. Therefore, the accuracy of oil leakage inspection of equipment can be effectively improved, the manual inspection cost is reduced, and safe and stable operation of booster station equipment is guaranteed.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent inspection technology, and more specifically, to a remote intelligent inspection method and system for booster stations based on digital twins. Background Art

[0002] With the rapid development of smart grid and IoT technologies, booster stations are playing an increasingly important role in power grid operations. The safe and stable operation of their equipment is directly related to the reliability of the entire power grid. Oil leak detection is a crucial task during booster station inspections. Oil leaks can not only damage equipment but also cause serious accidents such as fires. Traditional booster station inspections rely primarily on manual observation and empirical judgment, which is not only inefficient but also difficult to detect in real time.

[0003] In recent years, digital twin technology has been widely applied in various fields due to its unique advantages. By integrating the real and virtual worlds, digital twin technology enables real-time monitoring, analysis, and optimization of physical objects. In the field of booster station inspections, the introduction of digital twin technology can significantly improve inspection efficiency and accuracy, while reducing the costs and risks of manual inspections.

[0004] However, some existing remote intelligent inspection methods for booster stations rely primarily on single monitoring methods, such as visible light imaging, which makes it difficult to fully reflect the operating status of booster station equipment, resulting in certain limitations on the accuracy and reliability of inspection results. Therefore, an optimized remote intelligent inspection method and system for booster stations based on digital twins is desired. Summary of the Invention

[0005] To address the aforementioned technical issues, the present invention provides a remote intelligent inspection method and system for booster stations based on digital twins. This method uses deep learning-based machine vision technology to analyze and process infrared and color images of booster station equipment, capturing both deep and shallow features of the infrared and color images, respectively. This method extracts multi-layered, multimodal information about the surface conditions of the booster station equipment, thereby enabling intelligent oil leak detection for booster station equipment. This effectively improves the accuracy of equipment oil leak inspections, reduces manual inspection costs, and ensures the safe and stable operation of booster station equipment.

[0006] In a first aspect, an embodiment of the present invention provides a remote intelligent inspection method for a booster station based on digital twins, which includes:

[0007] Obtaining infrared images of the surface status of the booster station equipment and color images of the surface status of the booster station equipment collected by the infrared camera and the RGB camera;

[0008] Performing shallow and deep feature extraction on the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment, respectively, to obtain an infrared shallow feature map of the surface state of the booster station equipment, an infrared deep feature map of the surface state of the booster station equipment, a color shallow feature map of the surface state of the booster station equipment, and a color deep feature map of the surface state of the booster station equipment;

[0009] Performing deep-shallow feature compensation fusion on the booster station equipment surface state infrared shallow feature map and the booster station equipment surface state infrared deep feature map, as well as the booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map, respectively, to obtain a booster station equipment surface state infrared multi-scale compensated fusion feature map and a booster station equipment surface state color multi-scale compensated fusion feature map;

[0010] Determine the inspection result of the booster station equipment based on the semantic association fusion feature between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment;

[0011] In response to the inspection result indicating an oil leak, the digital twin module displays an infrared image of the surface status of the booster station equipment, a color image of the surface status of the booster station equipment, and an alarm prompt indicating that an oil leak has occurred.

[0012] In some possible embodiments, the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment are respectively subjected to shallow and deep feature extraction to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment, including: passing the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment through a booster station equipment surface state feature extractor based on a bidirectional pyramid network to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment.

[0013] In some possible embodiments, the infrared shallow layer feature map of the surface state of the booster station equipment and the infrared deep layer feature map of the surface state of the booster station equipment, as well as the color shallow layer feature map of the surface state of the booster station equipment and the color deep layer feature map of the surface state of the booster station equipment are respectively subjected to shallow-depth feature compensation fusion to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment, including: inputting the infrared shallow layer feature map of the surface state of the booster station equipment and the infrared deep layer feature map of the surface state of the booster station equipment into an information transmission loss compensation network to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment; inputting the color shallow layer feature map of the surface state of the booster station equipment and the color deep layer feature map of the surface state of the booster station equipment into the information transmission loss compensation network to obtain the color multi-scale compensation fusion feature map of the surface state of the booster station equipment.

[0014] In some possible embodiments, the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment are input into an information transmission loss compensation network to obtain the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment, including: calculating the feature difference between the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment to obtain the infrared deep-shallow difference feature map of the surface state of the booster station equipment; passing the infrared deep-shallow difference feature map of the surface state of the booster station equipment through a loss information capture module to obtain the infrared deep-shallow information loss feature map of the surface state of the booster station equipment, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer; based on the infrared deep-shallow information loss feature map of the surface state of the booster station equipment, performing information compensation fusion on the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment to obtain the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment.

[0015] In some possible embodiments, the characteristic difference between the infrared shallow layer characteristic map of the surface state of the booster station equipment and the infrared deep layer characteristic map of the surface state of the booster station equipment is calculated to obtain the infrared shallow-deep difference characteristic map of the surface state of the booster station equipment, including: downsampling the infrared shallow layer characteristic map of the surface state of the booster station equipment to obtain the downsampled infrared shallow layer characteristic map of the surface state of the booster station equipment; calculating the positional subtraction between the downsampled infrared shallow layer characteristic map of the surface state of the booster station equipment and the infrared deep layer characteristic map of the surface state of the booster station equipment to obtain the infrared shallow-deep difference characteristic map of the surface state of the booster station equipment.

[0016] In some possible embodiments, based on the infrared depth information loss characteristic map of the surface state of the boost station equipment, the infrared shallow layer characteristic map of the surface state of the boost station equipment and the infrared deep layer characteristic map of the surface state of the boost station equipment are subjected to information compensation fusion to obtain the infrared multi-scale compensation fusion characteristic map of the surface state of the boost station equipment, including: calculating the position point multiplication of the infrared depth information loss characteristic map of the surface state of the boost station equipment and the infrared shallow layer characteristic map of the surface state of the boost station equipment to obtain the information compensated infrared shallow layer characteristic map of the surface state of the boost station equipment; calculating the position weighted sum of the information compensated infrared shallow layer characteristic map of the surface state of the boost station equipment and the infrared deep layer characteristic map of the surface state of the boost station equipment to obtain the infrared multi-scale compensation fusion characteristic map of the surface state of the boost station equipment.

[0017] In some possible embodiments, based on the semantic association fusion features between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment, the inspection result of the booster station equipment is determined, including: the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment are fused through a semantic association mask module to obtain a multi-scale multi-modal fusion representation feature map of the surface state of the booster station; the multi-scale multi-modal fusion representation feature map of the surface state of the booster station is input into a classifier-based inspection result generator to obtain the inspection result, and the inspection result is used to indicate whether there is an oil leak.

[0018] In some possible embodiments, the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment and the color multi-scale compensated fusion feature map of the surface state of the booster station equipment are fused through a semantic association mask fusion module to obtain a multi-scale multimodal fusion representation feature map of the surface state of the booster station, including: cascading the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment and the color multi-scale compensated fusion feature map of the surface state of the booster station equipment to obtain a multi-scale multimodal cascade feature map of the surface state of the booster station; performing convolution processing on the multi-scale multimodal cascade feature map of the surface state of the booster station to obtain a multi-scale multimodal semantic association feature map of the surface state of the booster station; using a predetermined weight matrix to perform weighted multiplication on each feature matrix of the multi-scale multimodal semantic association feature map of the surface state of the booster station along the channel dimension, calculating the cumulative value of each eigenvalue in the feature map obtained by the weighted multiplication, and The accumulated value is added to the bias parameter to obtain a semantic association degree; the semantic association degree is nonlinearly processed by a sigmoid function to obtain a semantic association mask value; the semantic association mask value is used as a first weight parameter to perform weighted processing on the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment to obtain a weighted infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment; the difference between - and the semantic association mask value is used as a second weight parameter to perform weighted processing on the color multi-scale compensation fusion feature map of the surface state of the booster station equipment to obtain a weighted color multi-scale compensation fusion feature map of the surface state of the booster station equipment; the weighted infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the weighted color multi-scale compensation fusion feature map of the surface state of the booster station equipment are calculated and added according to position to obtain the multi-scale and multi-modal fusion representation feature map of the surface state of the booster station.

[0019] In a second aspect, an embodiment of the present invention provides a remote intelligent inspection system for booster stations based on digital twins, which includes:

[0020] The multi-dimensional monitoring module for the surface condition of the booster station equipment is used to obtain infrared images and color images of the surface condition of the booster station equipment collected by the infrared camera and the RGB camera;

[0021] a depth feature extraction module for performing depth feature extraction on the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment, respectively, to obtain an infrared shallow feature map of the surface state of the booster station equipment, an infrared deep feature map of the surface state of the booster station equipment, a color shallow feature map of the surface state of the booster station equipment, and a color deep feature map of the surface state of the booster station equipment;

[0022] a compensation fusion module, configured to perform compensation fusion of the shallow infrared feature map and the deep infrared feature map of the booster station equipment surface state, as well as the shallow color feature map and the deep color feature map of the booster station equipment surface state, respectively, to obtain a multi-scale compensation fusion feature map of the booster station equipment surface state infrared and a multi-scale compensation fusion feature map of the booster station equipment surface state color;

[0023] A semantic association fusion module, configured to determine an inspection result of the booster station equipment based on semantic association fusion features between the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state;

[0024] The inspection result display module is used to respond to the inspection result of oil leakage by displaying the infrared image of the surface status of the booster station equipment, the color image of the surface status of the booster station equipment and an alarm prompt indicating that an oil leakage has occurred through the digital twin module.

[0025] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0026] one or more processors;

[0027] The storage unit is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the method described above.

[0028] Compared to existing technologies, the digital twin-based remote intelligent inspection method and system for booster stations provided in embodiments of the present invention employs deep learning-based machine vision technology to analyze and process infrared and color images of booster station equipment, capturing both deep and shallow features of the infrared and color images, respectively. This method extracts multi-layered, multimodal information about the surface conditions of the booster station equipment, thereby enabling intelligent oil leak detection for booster station equipment. This effectively improves the accuracy of equipment oil leak inspections, reduces manual inspection costs, and ensures the safe and stable operation of booster station equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1Flowchart of a remote intelligent inspection method for a booster station based on digital twins according to an embodiment of the present invention;

[0031] Figure 2 Schematic diagram of the architecture of a remote intelligent inspection method for a booster station based on digital twins according to an embodiment of the present invention;

[0032] Figure 3 A flowchart of performing deep-shallow feature compensation fusion on the infrared shallow feature map and the infrared deep feature map of the booster station equipment surface state, as well as the color shallow feature map and the color deep feature map of the booster station equipment surface state in the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention, to obtain the infrared multi-scale compensated fusion feature map and the color multi-scale compensated fusion feature map of the booster station equipment surface state;

[0033] Figure 4 A flowchart of inputting the infrared shallow feature map and the infrared deep feature map of the surface state of the booster station equipment into an information transmission loss compensation network in the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention to obtain the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment;

[0034] Figure 5 A flowchart of determining an inspection result of a booster station device based on semantic association fusion features between an infrared multi-scale compensation fusion feature map of the booster station device surface state and a color multi-scale compensation fusion feature map of the booster station device surface state in a remote intelligent inspection method of a booster station based on a digital twin according to an embodiment of the present invention;

[0035] Figure 6 This is a flowchart of obtaining a multi-scale and multi-modal fusion representation feature map of the booster station surface state by fusing the infrared multi-scale compensated fusion feature map of the booster station equipment surface state and the color multi-scale compensated fusion feature map of the booster station equipment surface state through a semantic association mask fusion module in a remote intelligent inspection method for booster stations based on digital twins according to an embodiment of the present invention;

[0036] Figure 7 The block diagram of the remote intelligent inspection system for booster stations based on digital twins according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It is apparent that the described embodiments are only a portion of the embodiments of the present invention, rather than all of them. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without requiring creative effort are within the scope of protection of the present invention.

[0038] Unless otherwise specified, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The terms "including" or "comprising" used in the embodiments of the present invention neither limit the shapes, numbers, steps, actions, operations, components, originals and / or their groups mentioned, nor exclude the appearance or addition of one or more other different shapes, numbers, steps, actions, operations, components, originals and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number and order of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0039] Unless otherwise specifically stated, the relative arrangements of the components and steps, numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn in accordance with actual proportional relationships, and that the techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices shown should be considered part of the authorized specification. In all examples shown and discussed herein, any specific other examples may have different values. It should be noted that similar symbols and letters represent similar items in the following figures, and therefore, once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0040] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in the embodiments of the present invention and the features of different embodiments or examples, unless they are mutually inconsistent.

[0041] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0042] As mentioned in the above background technology, some existing remote intelligent inspection methods for booster stations rely too much on a single monitoring method, such as visual images, and this single monitoring method is usually unable to fully reveal the actual operating conditions of the booster station equipment, thereby affecting the accuracy and reliability of the inspection results. In response to the above technical problems, the technical concept of the present invention is to use machine vision technology based on deep learning to analyze and process the infrared images and color images of the booster station equipment, respectively capturing the deep features and shallow features of the infrared images and color images, so as to mine the multi-level and multi-modal information of the surface state of the booster station equipment, thereby realizing intelligent booster station equipment oil leakage detection. In this way, the accuracy of equipment oil leakage inspection can be effectively improved, and the cost of manual inspection can be reduced, thereby ensuring the safe and stable operation of the booster station equipment.

[0043] Figure 1 The figure is a flow chart of a remote intelligent inspection method for a substation based on digital twin according to an embodiment of the present invention. Figure 2 FIG is a schematic diagram of the architecture of a remote intelligent inspection method for a booster station based on digital twins according to an embodiment of the present invention. Figure 1 and Figure 2As shown, the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention includes the steps of: S110, acquiring the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment collected by the infrared camera and the RGB camera; S120, performing depth and shallow feature extraction on the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment respectively to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment; S130, performing depth and shallow feature extraction on the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment, as well as the The shallow color feature map of the surface state of the booster station equipment and the deep color feature map of the surface state of the booster station equipment are respectively fused in a compensation manner with deep and shallow features to obtain an infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and a color multi-scale compensation fusion feature map of the surface state of the booster station equipment; S140, based on the semantic association fusion features between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment, the inspection result of the booster station equipment is determined; S150, in response to the inspection result being an oil leak, the infrared image of the surface state of the booster station equipment, the color image of the surface state of the booster station equipment and an alarm prompt for indicating that an oil leak has occurred are displayed through the digital twin module.

[0044] In the aforementioned digital twin-based remote intelligent inspection method for booster stations, step S110 acquires infrared images and color images of the booster station equipment surface conditions, captured by infrared and RGB cameras. It should be understood that the infrared and color images of the booster station equipment surface conditions provide different types of information about the equipment surface conditions. Infrared images primarily reflect the temperature distribution on the equipment surface, and temperature anomalies are often a significant sign of equipment failure or oil leaks. Color images, on the other hand, provide visual information such as the color, texture, and shape of the equipment surface, which is crucial for identifying stains, oil stains, or other abnormalities on the equipment surface. Furthermore, in environments with poor lighting conditions, color images captured by RGB cameras often struggle to capture sufficient detail, resulting in low image quality and potentially inaccurate identification of the equipment surface conditions. However, due to their unique imaging principles, infrared cameras are insensitive to lighting conditions. Therefore, they can still display the temperature distribution on the equipment surface in low light or at night, enabling all-weather inspections of booster station equipment. Based on this, in the technical solution of the present invention, by simultaneously analyzing the infrared image and color image of the surface status of the booster station equipment, the complementary information of the multimodal data can be comprehensively utilized, thereby improving the accuracy of oil leak detection.

[0045] In the above-mentioned remote intelligent inspection method for booster stations based on digital twins, step S120 performs depth and shallow feature extraction on the infrared image of the booster station equipment surface state and the color image of the booster station equipment surface state to obtain an infrared shallow feature map of the booster station equipment surface state, an infrared deep feature map of the booster station equipment surface state, a color shallow feature map of the booster station equipment surface state, and a color deep feature map of the booster station equipment surface state. It should be understood that in the detection of oil leaks in booster station equipment, oil leaks may manifest as abnormal phenomena of different scales, including tiny oil droplets, oil stains, and larger oil-stained areas. Therefore, in order to achieve multi-scale perception of the surface state of the booster station equipment, it is necessary to further perform depth and shallow feature extraction on the infrared image of the booster station equipment surface state and the color image of the booster station equipment surface state.

[0046] In a specific example of the present invention, the processing method for extracting shallow and deep features from the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment is to pass the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment through a booster station equipment surface state feature extractor based on a bidirectional pyramid network to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment, and the color deep feature map of the surface state of the booster station equipment. Specifically, the bidirectional pyramid network is composed of multiple scale levels, and through convolution and pooling operations at different scales, it can simultaneously capture shallow local detail features and deep global structural features in the image. Among them, shallow features mainly reflect the detailed information of the image such as edges and textures, and play an important role in identifying small-scale oil leaks; while deep features mainly describe the overall structure and global abstract semantic information of the image, and have better characterization capabilities for identifying large-scale oil pollution areas.

[0047] In the above-mentioned remote intelligent inspection method of the booster station based on digital twin, the step S130 is to perform deep-shallow feature compensation fusion on the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment, as well as the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment. Figure 3The present invention provides a flowchart of performing deep-shallow feature compensation fusion on the booster station equipment surface state infrared shallow feature map and the booster station equipment surface state infrared deep feature map, as well as the booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map in the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention to obtain the booster station equipment surface state infrared multi-scale compensation fusion feature map and the booster station equipment surface state color multi-scale compensation fusion feature map. Figure 3 As shown, the step S130 includes: S131, inputting the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment into the information transmission loss compensation network to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment; S132, inputting the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment into the information transmission loss compensation network to obtain the color multi-scale compensation fusion feature map of the surface state of the booster station equipment.

[0048] Specifically, in step S131, the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment are input into the information transmission loss compensation network to obtain the infrared multi-scale compensated fusion feature map of the surface state of the booster station equipment. It should be understood that, considering that in the feature extraction process of the infrared image of the surface state of the booster station equipment, as the number of network layers increases, the image information may gradually be lost or blurred during the transmission process, resulting in differences between the deep features and shallow features of the image, directly performing simple feature splicing and other fusion operations on the two may result in information loss and poor fusion effect. Therefore, in order to achieve effective fusion of the deep features and shallow features of the infrared image of the surface state of the booster station equipment, in the technical solution of the present invention, an information transmission loss compensation network is used to process the two. Specifically, the information transmission loss compensation network calculates the feature difference between the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment to mine the information loss between the two, and compensates the infrared shallow feature map of the surface state of the booster station equipment based on the feature difference between the two, and transfers the global semantic information of the surface state of the booster station equipment to the shallow feature map, realizing information supplementation and effective fusion between the two different levels of features, so that the fused feature map retains the detailed information and overall structural semantic information of the image at the same time, thereby enabling a more comprehensive understanding of the surface temperature state of the equipment.

[0049] Figure 4The present invention provides a flowchart of inputting the infrared shallow feature map of the booster station equipment surface state and the infrared deep feature map of the booster station equipment surface state into the information transmission loss compensation network to obtain the infrared multi-scale compensation fusion feature map of the booster station equipment surface state in the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention. Figure 4 As shown, the step S131 includes: S1311, calculating the feature difference between the infrared shallow feature map of the surface state of the boost station equipment and the infrared deep feature map of the surface state of the boost station equipment to obtain the infrared deep-shallow difference feature map of the surface state of the boost station equipment; S1312, passing the infrared deep-shallow difference feature map of the surface state of the boost station equipment through a loss information capture module to obtain the infrared deep-shallow information loss feature map of the surface state of the boost station equipment, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer; S1313, based on the infrared deep-shallow information loss feature map of the surface state of the boost station equipment, performing information compensation fusion on the infrared shallow feature map of the surface state of the boost station equipment and the infrared deep feature map of the surface state of the boost station equipment to obtain the infrared multi-scale compensated fusion feature map of the surface state of the boost station equipment.

[0050] Specifically, the step S1311 includes: downsampling the infrared shallow feature map of the surface state of the booster station equipment to obtain a downsampled infrared shallow feature map of the surface state of the booster station equipment; calculating the positional subtraction between the downsampled infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment to obtain the infrared deep-shallow differential feature map of the surface state of the booster station equipment.

[0051] Specifically, the step S1313 includes: calculating the infrared deep and shallow information loss characteristic map of the surface state of the booster station equipment and multiplying the infrared shallow characteristic map of the surface state of the booster station equipment by the position point to obtain the information compensated infrared shallow characteristic map of the surface state of the booster station equipment; calculating the position weighted sum of the infrared shallow characteristic map of the surface state of the information compensated booster station equipment and the infrared deep characteristic map of the surface state of the booster station equipment to obtain the infrared multi-scale compensated fusion characteristic map of the surface state of the booster station equipment.

[0052] That is, step S131 includes: processing the infrared deep layer feature map of the surface state of the booster station equipment and the infrared shallow layer feature map of the surface state of the booster station equipment using the following information compensation fusion formula to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment, wherein the information compensation fusion formula is:

[0053] Loss=UpSampling((tanh(Conv 1×1 (DownSampling(F1))-F2)))

[0054] F a =αF1*Loss+βF2

[0055] Among them, F1 represents the infrared shallow feature map of the surface state of the boost station equipment, F2 represents the infrared deep feature map of the surface state of the boost station equipment, DownSampling(·) represents downsampling processing, UpSampling(·) represents upsampling processing, Conv 1×1 (·) represents 1×1 convolution processing, tanh is a nonlinear activation function, Loss represents the infrared depth information loss feature map of the surface state of the booster station equipment, α and β are predetermined weight parameters, * represents dot product, + represents the addition of corresponding elements, - represents the subtraction of corresponding elements, F a An infrared multi-scale compensation fusion feature map representing the surface status of the booster station equipment.

[0056] Specifically, in step S132, the booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map are input into the information transmission loss compensation network to obtain the booster station equipment surface state color multi-scale compensation fusion feature map. It should be understood that since the booster station equipment surface state color image also faces the problem of information loss and feature difference during the feature extraction process. Therefore, in the technical solution of the present invention, the deep features and shallow features of the booster station equipment surface state color image are processed with information compensation fusion in the same processing method. The information transmission loss compensation network compensates for the shallow features by utilizing the feature difference between the booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map to make up for the information loss, thereby making full use of the multi-level information in the booster station equipment surface state color image and improving the accuracy of inspection.

[0057] In the above-mentioned remote intelligent inspection method for booster stations based on digital twins, the step S140 determines the inspection result of the booster station equipment based on the semantic association fusion features between the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state. Figure 5 This is a flowchart of determining the inspection result of the booster station equipment based on the semantic association fusion feature between the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state in the remote intelligent inspection method of the booster station based on digital twin according to an embodiment of the present invention. Figure 5As shown, the step S140 includes: S141, the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment are fused through a semantic association mask fusion module to obtain a multi-scale multi-modal fusion representation feature map of the surface state of the booster station; S142, the multi-scale multi-modal fusion representation feature map of the surface state of the booster station is input into the inspection result generator based on the classifier to obtain the inspection result, and the inspection result is used to indicate whether there is an oil leak.

[0058] Specifically, in step S141, the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment are fused through a semantic association mask fusion module to obtain a multi-scale multimodal fusion representation feature map of the surface state of the booster station. It should be understood that since the infrared image and the color image of the surface state of the booster station equipment respectively provide different types of information about the surface state of the same equipment, there is a natural semantic association between the two. Therefore, in the technical solution of the present invention, in order to improve the multimodal feature fusion effect by utilizing the semantic association between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment, a semantic association mask fusion module is introduced to process the two. Specifically, the semantic association mask fusion module generates a semantic association mask by learning the semantic correlation between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment, and then performs weighted fusion on the two based on the semantic association mask, so that the fused feature map can fully utilize the complementarity of multimodal data, enhance the perception and understanding ability of the surface state of the booster station equipment, and thus further improve the accuracy of oil leak detection.

[0059] Figure 6 This is a flowchart of the method for remote intelligent inspection of booster stations based on digital twins according to an embodiment of the present invention, which uses the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state through a semantic association mask fusion module to obtain a multi-scale multi-modal fusion representation feature map of the booster station surface state. Figure 6As shown, the step S141 includes: S1411, cascading the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment to obtain a multi-scale multimodal cascade feature map of the surface state of the booster station; S1412, performing convolution processing on the multi-scale multimodal cascade feature map of the surface state of the booster station to obtain a multi-scale multimodal semantic association feature map of the surface state of the booster station; S1413, using a predetermined weight matrix to weightedly multiply each feature matrix of the multi-scale multimodal semantic association feature map of the surface state of the booster station along the channel dimension, calculating the cumulative value of each eigenvalue in the feature map obtained by the weighted multiplication, and adding the cumulative value to the bias parameter to obtain the semantic association degree; S1414, the semantic association degree is calculated by sig The moid function is nonlinearly processed to obtain a semantic association mask value; S1415, the semantic association mask value is used as a first weight parameter to perform weighted processing on the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment to obtain a weighted infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment, and the difference between one and the semantic association mask value is used as a second weight parameter to perform weighted processing on the color multi-scale compensation fusion feature map of the surface state of the booster station equipment to obtain a weighted color multi-scale compensation fusion feature map of the surface state of the booster station equipment; S1416, the weighted infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the weighted color multi-scale compensation fusion feature map of the surface state of the booster station equipment are calculated and added by position to obtain the multi-scale multimodal fusion representation feature map of the surface state of the booster station.

[0060] That is, step S141 includes: processing the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment using the following semantic association mask fusion formula to obtain the multi-scale multi-modal fusion representation feature map of the surface state of the booster station; wherein the semantic association mask fusion formula is:

[0061] F c =t·F a +(1-t)F b

[0062]

[0063] Among them, F a is the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment, F b is the multi-scale compensation fusion feature map of the surface state of the booster station equipment, concat(·,·) represents the cascade operation, f N×NIndicates a convolution operation based on an N×N convolution kernel, W is a predetermined weight matrix, b is a bias parameter, sigmoid is a nonlinear activation function, t is a semantic association mask value, and F c The multi-scale and multi-modal fusion representation feature map of the surface state of the booster station.

[0064] Specifically, in step S142, the multi-scale multi-modal fusion representation feature map of the booster station surface state is input into the inspection result generator based on the classifier to obtain the inspection result, and the inspection result is used to indicate whether there is an oil leak. It should be understood that a classifier is a trained machine learning model whose design goal is to automatically assign input data to predefined categories. In the technical solution of the present invention, the inspection result generator uses a trained classifier model to perform feature recognition on the multi-scale multi-modal fusion representation feature map of the booster station equipment surface state, and maps it to the corresponding category label according to the feature pattern of the multi-scale multi-modal fusion representation feature map of the booster station equipment surface state, thereby generating an inspection result indicating whether the booster station equipment is leaking oil, so as to realize intelligent oil leakage detection of the booster station equipment.

[0065] In the above scheme, the booster station equipment surface state color multi-scale compensation fusion feature map and the booster station equipment surface state infrared multi-scale compensation fusion feature map represent the cross-scale and cross-depth image semantic fusion features of the booster station equipment surface state infrared image and the booster station equipment surface state color image, respectively. However, considering the content differences between the booster station equipment surface state infrared image and the booster station equipment surface state color image in the image source domain, the image semantic feature distribution differences between the booster station equipment surface state color multi-scale compensation fusion feature map and the booster station equipment surface state infrared multi-scale compensation fusion feature map will be caused. As a result, when the booster station equipment surface state infrared multi-scale compensation fusion feature map and the booster station equipment surface state color multi-scale compensation fusion feature map are subjected to semantic association mask fusion through the semantic association mask fusion module, the differentiated image semantic feature distribution will cause the resulting booster station surface state multi-scale multimodal fusion representation feature map to have insufficient feature distribution aggregation, thereby affecting the classification convergence efficiency of the classifier, that is, affecting the efficiency of classification training and the accuracy of classification results.

[0066] Therefore, in a preferred embodiment of the present invention, when the multi-scale multimodal fusion representation feature map of the booster station surface state is input into the inspection result generator based on the classifier to obtain the inspection result, the multi-scale multimodal fusion representation feature map of the booster station surface state is optimized to obtain the optimized multi-scale multimodal fusion representation feature map of the booster station surface state, which specifically includes the following optimization steps: multiplying the multi-scale multimodal fusion representation feature map of the booster station surface state with the scale point of the multi-scale multimodal fusion representation feature map of the booster station surface state, and then multiplying the multi-scale multimodal fusion representation feature map with the scale point of the multi-scale multimodal fusion representation feature map of the booster station surface state. The sum of the absolute values of the eigenvalues of the modal fusion representation feature map is subtracted to obtain a multi-scale multi-modal fusion full semantic feature map of the surface state of the booster station, wherein the scale of the multi-scale multi-modal fusion representation feature map of the surface state of the booster station is the width of the feature matrix of the multi-scale multi-modal fusion representation feature map of the surface state of the booster station multi-scale multi-modal fusion representation feature map multiplied by the height and then multiplied by the number of channels of the multi-scale multi-modal fusion representation feature map of the surface state of the booster station; after the multi-scale multi-modal fusion representation feature map of the surface state of the booster station is multiplied by the square root of the scale of the multi-scale multi-modal fusion representation feature map of the surface state of the booster station, Then, the square root of the sum of the squares of the eigenvalues of the multi-scale multimodal fusion representation feature map of the booster station surface state is subtracted to obtain the multi-scale multimodal fusion semi-semantic feature map of the booster station surface state; the square root of the absolute value of each eigenvalue of the multi-scale multimodal fusion full semantic feature map of the booster station surface state and the multi-scale multimodal fusion semi-semantic feature map of the booster station surface state are calculated respectively to obtain the multi-scale multimodal fusion full semantic modulation feature map of the booster station surface state and the multi-scale multimodal fusion semi-semantic modulation feature map of the booster station surface state; the multi-scale multimodal fusion full semantic modulation feature map of the booster station surface state and the multi-scale multimodal fusion semi-semantic modulation feature map of the booster station surface state are calculated respectively. The base 2 logarithm of each eigenvalue of the multimodal fusion full semantic modulation feature map and the multi-scale multimodal fusion semi-semantic modulation feature map of the boost station surface state is obtained to obtain the multi-scale multimodal fusion full semantic information feature map of the boost station surface state and the multi-scale multimodal fusion semi-semantic information feature map of the boost station surface state; the weighted sum of the multi-scale multimodal fusion full semantic information feature map of the boost station surface state and the multi-scale multimodal fusion semi-semantic information feature map of the boost station surface state is calculated using the weighting coefficient as a hyperparameter to obtain the optimized multi-scale multimodal fusion representation feature map of the boost station surface state.

[0067] That is, for the multi-scale multimodal fusion representation feature map of the surface state of the booster station as a feature set, wherein the change semantic representation is based on the feature value of the multi-scale multimodal fusion representation feature map of the surface state of the booster station, in order to dynamically aggregate the semantic set composed of different change semantics of the multi-scale multimodal fusion representation feature map of the surface state of the booster station without ignoring the individual semantic change information, the individual features of the multi-scale multimodal fusion representation feature map of the surface state of the booster station and the collective expression of its feature map scale are used as full-scale and half-scale feature representations, and the low-rank norms of the different dimensions of the overall semantics of the feature set of the multi-scale multimodal fusion representation feature map of the surface state of the booster station are used as phases through negative correlation calculation and amplitude-phase information representation modulation is performed to dynamically adjust the semantic information change relationship of the multi-scale multimodal fusion representation feature map of the surface state of the booster station, thereby improving the overall semantic information expression aggregation of the multi-scale multimodal fusion representation feature map of the surface state of the booster station, and improving the classification convergence efficiency of the multi-scale multimodal fusion representation feature map of the surface state of the booster station when it is classified by the inspection result generator based on the classifier.

[0068] In the above-mentioned remote intelligent inspection method for booster stations based on digital twins, in step S150, in response to the inspection result indicating an oil leak, an infrared image of the surface status of the booster station equipment, a color image of the surface status of the booster station equipment, and an alarm prompt indicating that an oil leak has occurred are displayed through the digital twin module. That is, when the inspection result generator determines that an oil leak has occurred in the booster station equipment, the system immediately triggers the alarm mechanism. As the visual interface of the system, the digital twin module will display relevant image data and alarm prompts, and visualize the inspection results so that operation and maintenance personnel can quickly understand the equipment status and take appropriate measures to deal with the oil leak, thereby ensuring the normal operation, safety and stability of the booster station equipment.

[0069] In summary, a remote intelligent inspection method for booster stations based on digital twins according to an embodiment of the present invention has been described. This method uses deep learning-based machine vision technology to analyze and process infrared and color images of booster station equipment, capturing the deep and shallow features of the infrared and color images, respectively, to mine multi-level, multi-modal information about the surface state of the booster station equipment, thereby enabling intelligent detection of oil leaks in booster station equipment. This effectively improves the accuracy of equipment oil leak inspections, reduces manual inspection costs, and ensures the safe and stable operation of booster station equipment.

[0070] Figure 7 FIG is a block diagram of a remote intelligent inspection system for a booster station based on digital twins according to an embodiment of the present invention. Figure 7As shown, the remote intelligent inspection system 100 of the booster station based on digital twin according to an embodiment of the present invention includes: a multi-dimensional monitoring module 110 for the surface state of the booster station equipment, which is used to obtain the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment collected by the infrared camera and the RGB camera; a depth feature extraction module 120, which is used to extract the depth and shallow features of the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment respectively to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment; a compensatory fusion module 130, which is used to extract the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment The deep feature map, the shallow color feature map of the surface state of the booster station equipment and the deep color feature map of the surface state of the booster station equipment are respectively subjected to deep and shallow feature compensation fusion to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment; the semantic association fusion module 140 is used to determine the inspection result of the booster station equipment based on the semantic association fusion features between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment; the inspection result display module 150 is used to display the infrared image of the surface state of the booster station equipment, the color image of the surface state of the booster station equipment and the alarm prompt for indicating the occurrence of oil leakage through the digital twin module in response to the inspection result being oil leakage.

[0071] Here, those skilled in the art will understand that the specific operations of each module in the above-mentioned remote intelligent inspection system for booster stations based on digital twins have been referred to above. Figures 1 to 6 It has been introduced in detail in the description of the remote intelligent inspection method of the substation based on digital twin, and therefore, its repeated description will be omitted.

[0072] Based on the same inventive concept, an embodiment of the present invention provides an electronic device, including:

[0073] one or more processors;

[0074] A storage unit is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors can implement the method described in the above description. For details, please refer to the relevant records in the above description and will not be repeated here.

[0075] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0076] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0077] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0078] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0079] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0080] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0081] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

[0082] It will be understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present invention, and the present invention is not limited thereto. Those skilled in the art will appreciate that various modifications and improvements can be made without departing from the spirit and substance of the present invention, and such modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A remote intelligent inspection method for booster stations based on digital twins, characterized in that: include: Obtaining infrared images of the surface status of the booster station equipment and color images of the surface status of the booster station equipment collected by the infrared camera and the RGB camera; Performing shallow and deep feature extraction on the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment, respectively, to obtain an infrared shallow feature map of the surface state of the booster station equipment, an infrared deep feature map of the surface state of the booster station equipment, a color shallow feature map of the surface state of the booster station equipment, and a color deep feature map of the surface state of the booster station equipment; Performing deep-shallow feature compensation fusion on the booster station equipment surface state infrared shallow feature map and the booster station equipment surface state infrared deep feature map, as well as the booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map, respectively, to obtain a booster station equipment surface state infrared multi-scale compensated fusion feature map and a booster station equipment surface state color multi-scale compensated fusion feature map; Determine the inspection result of the booster station equipment based on the semantic association fusion feature between the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment and the color multi-scale compensation fusion feature map of the surface state of the booster station equipment; In response to the inspection result indicating an oil leak, the digital twin module displays an infrared image of the surface status of the booster station equipment, a color image of the surface status of the booster station equipment, and an alarm prompt indicating that an oil leak has occurred.

2. The remote intelligent inspection method for booster stations based on digital twins according to claim 1 is characterized in that: The infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment are respectively subjected to shallow and deep feature extraction to obtain an infrared shallow feature map of the surface state of the booster station equipment, an infrared deep feature map of the surface state of the booster station equipment, a color shallow feature map of the surface state of the booster station equipment, and a color deep feature map of the surface state of the booster station equipment, including: The infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment are respectively passed through a booster station equipment surface state feature extractor based on a bidirectional pyramid network to obtain the infrared shallow feature map of the surface state of the booster station equipment, the infrared deep feature map of the surface state of the booster station equipment, the color shallow feature map of the surface state of the booster station equipment and the color deep feature map of the surface state of the booster station equipment.

3. The remote intelligent inspection method for booster stations based on digital twins according to claim 2 is characterized in that: The infrared shallow feature map of the booster station equipment surface state and the infrared deep feature map of the booster station equipment surface state, as well as the color shallow feature map of the booster station equipment surface state and the color deep feature map of the booster station equipment surface state are respectively fused in a deep-shallow feature compensation manner to obtain an infrared multi-scale compensated fused feature map of the booster station equipment surface state and a color multi-scale compensated fused feature map of the booster station equipment surface state, including: Inputting the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment into an information transmission loss compensation network to obtain an infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment; The booster station equipment surface state color shallow feature map and the booster station equipment surface state color deep feature map are input into the information transmission loss compensation network to obtain the booster station equipment surface state color multi-scale compensation fusion feature map.

4. The remote intelligent inspection method for booster stations based on digital twins according to claim 3 is characterized in that: Inputting the infrared shallow layer feature map of the surface state of the booster station equipment and the infrared deep layer feature map of the surface state of the booster station equipment into an information transmission loss compensation network to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment, including: Calculating the characteristic difference between the infrared shallow layer characteristic map of the surface state of the booster station equipment and the infrared deep layer characteristic map of the surface state of the booster station equipment to obtain the infrared shallow and deep difference characteristic map of the surface state of the booster station equipment; The infrared depth difference feature map of the surface state of the booster station equipment is passed through a loss information capture module to obtain an infrared depth information loss feature map of the surface state of the booster station equipment, wherein the loss information capture module includes a point convolution layer, a nonlinear activation layer based on a tanh function, and an upsampling layer; Based on the infrared deep and shallow information loss characteristic map of the surface state of the booster station equipment, the infrared shallow characteristic map of the surface state of the booster station equipment and the infrared deep characteristic map of the surface state of the booster station equipment are information compensated and fused to obtain the infrared multi-scale compensated fusion characteristic map of the surface state of the booster station equipment.

5. The remote intelligent inspection method for booster stations based on digital twins according to claim 4 is characterized in that: Calculating the characteristic difference between the infrared shallow layer characteristic map of the surface state of the booster station equipment and the infrared deep layer characteristic map of the surface state of the booster station equipment to obtain the infrared deep-shallow difference characteristic map of the surface state of the booster station equipment, including: Downsampling the infrared shallow layer characteristic image of the surface state of the booster station equipment to obtain a downsampled infrared shallow layer characteristic image of the surface state of the booster station equipment; The position-based subtraction between the down-sampled infrared shallow layer feature map of the booster station equipment surface state and the infrared deep layer feature map of the booster station equipment surface state is calculated to obtain the infrared shallow-deep differential feature map of the booster station equipment surface state.

6. The remote intelligent inspection method for booster stations based on digital twins according to claim 5 is characterized in that: Based on the infrared deep and shallow information loss feature map of the surface state of the booster station equipment, information compensation fusion is performed on the infrared shallow feature map of the surface state of the booster station equipment and the infrared deep feature map of the surface state of the booster station equipment to obtain the infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment, including: Calculate the infrared deep and shallow information loss characteristic map of the booster station equipment surface state and multiply the infrared shallow characteristic map of the booster station equipment surface state by the position point to obtain the information compensation infrared shallow characteristic map of the booster station equipment surface state; The position-weighted sum of the infrared shallow layer feature map of the information-compensated booster station equipment surface state and the infrared deep layer feature map of the booster station equipment surface state is calculated to obtain the infrared multi-scale compensated fusion feature map of the booster station equipment surface state.

7. The remote intelligent inspection method for booster stations based on digital twins according to claim 6 is characterized in that: Determining an inspection result of the booster station equipment based on semantic association fusion features between the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state includes: The infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state are fused through a semantic association mask fusion module to obtain a multi-scale multi-modal fusion representation feature map of the booster station surface state; The multi-scale and multi-modal fusion representation feature map of the booster station surface state is input into a classifier-based inspection result generator to obtain the inspection result, which is used to indicate whether there is an oil leak.

8. The remote intelligent inspection method for booster stations based on digital twins according to claim 7 is characterized in that: The infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state are fused through a semantic association mask fusion module to obtain a multi-scale multi-modal fusion representation feature map of the booster station surface state, including: Cascading the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state to obtain a multi-scale multi-modal cascade feature map of the booster station surface state; Performing convolution processing on the multi-scale multi-modal cascade feature map of the booster station surface state to obtain a multi-scale multi-modal semantic association feature map of the booster station surface state; After weighted multiplication of each feature matrix of the multi-scale and multi-modal semantic association feature map of the booster station surface state along the channel dimension using a predetermined weight matrix, the cumulative value of each eigenvalue in the feature map obtained by the weighted multiplication is calculated, and the cumulative value is added to the bias parameter to obtain the semantic association degree; The semantic association degree is nonlinearly processed by a sigmoid function to obtain a semantic association mask value; The infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment is weighted by using the semantic association mask value as a first weight parameter to obtain a weighted infrared multi-scale compensation fusion feature map of the surface state of the booster station equipment; the color multi-scale compensation fusion feature map of the surface state of the booster station equipment is weighted by using the difference between - and the semantic association mask value as a second weight parameter to obtain a weighted color multi-scale compensation fusion feature map of the surface state of the booster station equipment; The infrared multi-scale compensation fusion feature map of the weighted booster station equipment surface state and the color multi-scale compensation fusion feature map of the weighted booster station equipment surface state are calculated and added according to position to obtain the multi-scale multi-modal fusion representation feature map of the booster station surface state.

9. A remote intelligent inspection system for booster stations based on digital twins, characterized in that: include: A multi-dimensional monitoring module for the surface condition of booster station equipment, which is used to obtain infrared images and color images of the surface condition of booster station equipment collected by infrared cameras and RGB cameras; a depth feature extraction module for performing depth feature extraction on the infrared image of the surface state of the booster station equipment and the color image of the surface state of the booster station equipment, respectively, to obtain an infrared shallow feature map of the surface state of the booster station equipment, an infrared deep feature map of the surface state of the booster station equipment, a color shallow feature map of the surface state of the booster station equipment, and a color deep feature map of the surface state of the booster station equipment; a compensation fusion module, configured to perform compensation fusion of the shallow infrared feature map and the deep infrared feature map of the booster station equipment surface state, as well as the shallow color feature map and the deep color feature map of the booster station equipment surface state, respectively, to obtain a multi-scale compensation fusion feature map of the booster station equipment surface state infrared and a multi-scale compensation fusion feature map of the booster station equipment surface state color; A semantic association fusion module, configured to determine an inspection result of the booster station equipment based on semantic association fusion features between the infrared multi-scale compensation fusion feature map of the booster station equipment surface state and the color multi-scale compensation fusion feature map of the booster station equipment surface state; The inspection result display module is used to respond to the inspection result of oil leakage by displaying the infrared image of the surface status of the booster station equipment, the color image of the surface status of the booster station equipment and an alarm prompt indicating that an oil leakage has occurred through the digital twin module.

10. An electronic device, characterized in that: include: one or more processors; A storage unit, configured to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 8.

Images