Image processing method, system and application in unmanned vehicle inspection of closed converter stations

Abstract

The present application relates to the field of intelligent image processing technology, and specifically discloses an image processing method, system, and application in unmanned vehicle inspection of closed converter stations. The color image and thermal image of the target device are synchronously collected by an infrared thermal imager and a visible light camera, and the material of the color image of the target device is identified by using deep learning-based image processing technology to obtain a probability map of the material distribution of the device. The device material distribution probability map is spatially aligned with the thermal image of the target device, and the original temperature value in the thermal image is compensated at the pixel level using a material thermal conductivity correction algorithm. Hot spot detection is performed based on the corrected device temperature distribution map, and an alarm prompt signal is automatically generated when an abnormal temperature area is detected. Through intelligent image processing, the present application can realize automated inspection and fault warning of equipment in closed converter stations, improve inspection efficiency and accuracy, reduce operation and maintenance costs, and ensure the safe and stable operation of the power grid.

Description

Technical Field

[0001] The present application relates to the field of intelligent image processing technology, and more specifically, to an image processing method, system, and application in unmanned vehicle inspection of closed converter stations. Background Art

[0002] As power systems continue to expand in scale and become more intelligent, the safe and stable operation of enclosed converter stations, core components of HVDC transmission systems, is directly linked to the reliability and economic viability of the power grid. Due to the numerous devices within converter stations and their complex operating environment, these devices are constantly exposed to high voltage and high current conditions, making them susceptible to potential faults such as overheating and aging. However, due to the complex structure of the equipment within converter stations, the closed operating environment, and strong electromagnetic interference, traditional converter station inspections rely primarily on close-range manual operation. This not only poses personal safety risks, but also struggles to meet the operational and maintenance needs of modern power grids due to long inspection cycles and low efficiency.

[0003] In recent years, while some automated inspection systems have employed infrared thermal imagers or visible light cameras to capture images of equipment operating conditions and then analyze them for further monitoring, they still face several limitations. First, the limitations of a single sensor are significant. While infrared thermal imagers can capture the surface temperature distribution of equipment, they are insensitive to information such as the equipment's material and surface condition. Visible light cameras, while capable of providing high-resolution image detail, cannot directly reflect temperature anomalies. Second, because differences in thermal conductivity among surface materials (such as metals, ceramics, and polymers) directly influence the heat transfer process, the radiation signatures of the same temperature change on different materials vary significantly. Traditional thermal analysis methods typically assume homogeneous material and fail to consider the impact of material distribution on the temperature field, resulting in significant deviations in the correction values. For example, due to the poor thermal conductivity of the insulator surface, the actual temperature may be higher than indicated by the thermal image. Without temperature compensation, potential hot spots may be missed.

[0004] Therefore, an image processing method, system and application in unmanned vehicle inspection of closed converter stations are needed to solve the above technical problems. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an image processing method, system and application in unmanned vehicle inspection of closed converter stations, which uses an infrared thermal imager and a visible light camera to synchronously collect color images and thermal images of target equipment, and uses deep learning-based image processing technology to perform material recognition on the color image of the target equipment to obtain a probability map of the material distribution of the equipment. Subsequently, the equipment material distribution probability map is spatially aligned with the thermal image of the target equipment, and the material thermal conductivity correction algorithm is used to perform pixel-level compensation on the original temperature value in the thermal image, and then hot spot detection is performed based on the corrected equipment temperature distribution map, and an alarm prompt signal is automatically generated when an abnormal temperature area is detected, which can realize automated inspection and fault warning of equipment in the closed converter station, improve inspection efficiency and accuracy, reduce operation and maintenance costs, and ensure the safe and stable operation of the power grid.

[0006] According to one aspect of the present application, there is provided an image processing method, comprising:

[0007] receiving a color image and a thermal image of a target device captured by an infrared thermal imager and a visible light camera;

[0008] Inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category identification based on the enhanced image features to obtain the device material distribution probability map;

[0009] Registering the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map;

[0010] Performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result;

[0011] In response to the hot spot detection result indicating that there is a temperature abnormality area, a temperature abnormality alarm prompt signal is generated.

[0012] According to another aspect of the present application, there is provided an image processing system, comprising:

[0013] A target device image receiving module is used to receive color images and thermal images of the target device captured by the infrared thermal imager and the visible light camera;

[0014] a device material recognition module, configured to input the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is configured to perform image feature-level explicit enhancement on the color image of the target device and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map;

[0015] An image registration and temperature correction module is used to register the device material distribution probability map with the thermal image of the target device, and to compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map;

[0016] a hot spot detection module, configured to perform hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result;

[0017] The abnormality alarm prompt generating module is used to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result indicating that there is a temperature abnormality area.

[0018] According to another aspect of the present application, a closed converter station unmanned vehicle is provided, wherein the closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the above-mentioned image processing system, and can execute the above-mentioned image processing method.

[0019] This application has at least the following technical effects:

[0020] Compared with the existing technology, this application uses an infrared thermal imager and a visible light camera mounted on an unmanned vehicle in a closed converter station to synchronously capture color images and thermal images of the target device, and uses deep learning-based image processing technology to perform material recognition on the color image of the target device to obtain a device material distribution probability map. Subsequently, the device material distribution probability map is spatially aligned with the thermal image of the target device, and a material thermal conductivity correction algorithm is used to perform pixel-level compensation on the original temperature value in the thermal image. Hot spots are then detected based on the corrected device temperature distribution map, and an alarm signal is automatically generated when an abnormal temperature area is detected. This enables automated inspection and fault warning of equipment in the closed converter station, improves inspection efficiency and accuracy, reduces operation and maintenance costs, and ensures the safe and stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 Flowchart of an image processing method according to an embodiment of the present application.

[0023] Figure 2 Schematic diagram of data flow of an image processing method according to an embodiment of the present application.

[0024] Figure 3 Flowchart of sub-step S2 of the image processing method according to an embodiment of the present application.

[0025] Figure 4 Flowchart of sub-step S22 of the image processing method according to an embodiment of the present application.

[0026] Figure 5 Flowchart of sub-step S222 of the image processing method according to an embodiment of the present application.

[0027] Figure 6 is a block diagram of an image processing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0029] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0030] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0031] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0032] It should be noted that all data acquisition actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0033] Figure 1 Flowchart of an image processing method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the image processing method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the image processing method includes the following steps: S1, receiving a color image and a thermal image of a target device captured by an infrared thermal imager and a visible light camera; S2, inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; S3, aligning the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; S4, performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; S5, in response to the hot spot detection result indicating the presence of a temperature abnormality area, generating a temperature abnormality alarm prompt signal.

[0034] In the above-mentioned image processing method, step S1 receives color images and thermal images of the target device captured by an infrared thermal imager and a visible light camera. Specifically, closed converter station equipment (such as converter transformers, circuit breakers, disconnectors, etc.) has complex surface materials (metal conductors, insulating materials, polymer coatings, etc.), and the operating environment is subject to strong electromagnetic interference, strong light reflection, and shadow obstruction. A single sensor (such as infrared or visible light) has difficulty in simultaneously capturing the thermal radiation characteristics and visual details of the equipment. By receiving color images and thermal images of the target device captured by an infrared thermal imager and a visible light camera, the present application can overcome the perception limitations of a single sensor and provide a high-precision, multi-dimensional data foundation for subsequent analysis.

[0035] Specifically, to ensure data acquisition quality, infrared thermal imagers and visible light cameras must be calibrated before image capture. For infrared thermal imagers, resolution, temperature range, and accuracy are three key considerations. High resolution allows for more accurate capture of subtle temperature variations on the equipment surface, which is crucial for identifying potential fault points. A wide temperature measurement range ensures proper operation even in extreme environments, adapting to the various operating conditions that may occur within closed converter stations. Furthermore, high temperature measurement accuracy reduces the possibility of misjudgment, providing a reliable basis for subsequent material identification and temperature correction. For visible light cameras, lens quality is equally important, in addition to pixel count. A high-quality lens not only improves image clarity but also effectively mitigates the effects of strong light reflections and shadows, ensuring that captured images more accurately reflect the actual condition of the equipment surface.

[0036] Environmental factors are also crucial when collecting data. Strong electromagnetic interference is often present inside closed converter stations, posing a challenge to the operational stability of electronic equipment. Therefore, when selecting infrared thermal imagers and visible light cameras, prioritize those with excellent anti-interference capabilities. Furthermore, given the potential for high humidity and dust levels within the station, necessary protective measures must be taken for these devices, such as using housings with a high waterproof and dustproof rating. Furthermore, since lighting conditions vary significantly across time periods, it is crucial to schedule data collection appropriately. Ideally, operations should be conducted during periods of relatively uniform natural light to avoid image distortion caused by uneven lighting.

[0037] To further improve the effectiveness of data collection, advanced technologies can be introduced. For example, multi-view imaging technology can simultaneously capture the same target from multiple angles, increasing the amount of information while also helping to eliminate blind spots. Furthermore, automatic exposure control technology can automatically adjust camera parameters based on on-site lighting conditions, ensuring optimal results with every shot. Furthermore, for infrared thermal imagers, dynamic range extension plays a crucial role in handling scenes with high temperature differences, maintaining excellent image quality even in the presence of drastic temperature fluctuations.

[0038] In actual operation, to ensure the high consistency and accuracy of collected data, detailed operating procedures must be developed. This includes, but is not limited to: determining a specific shooting path and sequence to ensure coverage of all key areas; setting fixed shooting distances and angles to facilitate unified analysis during post-processing; and establishing a strict quality inspection mechanism to promptly identify and correct potential issues. This approach not only improves work efficiency but also enhances data reliability, laying a solid foundation for subsequent analysis and processing.

[0039] In the above-mentioned image processing method, in step S2, the color image of the target device is input into the trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map. Specifically, the difference in thermal conductivity of the device material (such as the thermal conductivity of metal conductors ≥100 W / m·K, and the thermal conductivity of ceramic insulators ≤10 W / m·K) directly affects the heat conduction process. Traditional thermal analysis methods often do not consider the impact of different material distributions of the device on the temperature field, which may lead to deviations in temperature anomaly detection. Therefore, the present application processes the color image of the target device using the trained material recognition model to identify various materials on the surface of the device, such as metals, ceramics, polymers, etc., and generates a device material distribution probability map, thereby achieving pixel-level temperature compensation through material recognition and improving the accuracy of device thermal analysis. Among them, Figure 3 FIG. 1 is a flowchart of sub-step S2 of the image processing method according to an embodiment of the present application. Figure 3 As shown, the step S2 includes the steps of: S21, extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map; S22, performing image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain a target device surface state feature enhanced coding feature map; S23, performing material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map.

[0040] Specifically, in a specific example of the present application, the step S21 includes: using an image feature extractor based on the FPN model to extract surface state features from the color image of the target device to obtain a surface state feature coding feature map of the target device. Specifically, since the surface state features of the device (such as texture, color, roughness) are an important basis for material identification, in order to fully mine the multi-scale surface state information of the color image of the target device and more accurately identify the material, the present application uses the FPN (Feature Pyramid Network) model as an image feature extractor to mine the multi-scale surface state information in the color image of the target device. Specifically, FPN can effectively extract and fuse low-level detail features and high-level semantic information in the color image of the target device by combining bottom-up and top-down paths, and generate a surface state feature coding feature map of the target device, so as to more comprehensively reflect the surface material characteristics of the device and provide strong support for subsequent material identification.

[0041] Specifically, the step S22 performs image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain a target device surface state feature enhanced coding feature map. Specifically, the present application takes into account that the surface of the closed converter station equipment is susceptible to electromagnetic interference, uneven lighting and stain coverage, resulting in blurred or lost local features. Traditional image enhancement methods (such as histogram equalization) only rely on the statistical characteristics of the image and are difficult to restore the real details in complex scenes. To this end, the present application adopts an image feature enhancement method based on spatial-semantic collaborative constraints, which performs feature enhancement processing on local pixel positions by performing dual constraints at the spatial and semantic levels on the features of each pixel position in the target device surface state feature coding feature map, so as to optimize the spatial consistency (such as edge preservation) and semantic rationality (such as material category distribution) of the features, restore the local feature blur caused by electromagnetic interference or uneven lighting, and ensure that the enhanced feature representation can accurately reflect the real material characteristics of the device surface, thereby providing more reliable data support for subsequent material category identification. Among them, Figure 4 FIG. 1 is a flowchart of sub-step S22 of the image processing method according to an embodiment of the present application. Figure 4As shown, the step S22 includes the steps of: S221, extracting the channel feature vector of the (i, j)th pixel position from the target device surface state feature coding feature map as the target device surface state channel feature vector to be enhanced; S222, based on the target device surface state feature coding feature map, performing spatial-semantic dual-domain information compensation encoding on the target device surface state channel feature vector to be enhanced to obtain an enhanced component implicit coding vector of the target device surface state channel feature to be enhanced; S223, fusing the enhanced component implicit coding vector of the target device surface state channel feature to be enhanced and the target device surface state channel feature vector to be enhanced to obtain an enhanced target device surface state channel feature vector, wherein the enhanced target device surface state channel feature vector is the channel feature vector of the pixel position (i, j) of the target device surface state feature enhancement coding feature map.

[0042] More specifically, the step S221 is expressed as follows:

[0043]

[0044]

[0045] in, is a real number, 、 and Respectively represent the height, width and number of channels of the target device surface state feature encoding feature map, Represents the target device surface state feature coding feature map, Indicates the channel feature vector of the (i, j)th pixel position of the target device surface state feature encoding feature map, Represents the channel feature vector to be enhanced of the target device surface state.

[0046] Specifically, a multidimensional channel feature vector is extracted from a specific pixel position in the target device's surface state feature encoding feature map, representing the local feature distribution at that location within the device's surface state. Each pixel position in the target device's surface state feature encoding feature map corresponds to a multidimensional channel feature vector. Extracting a channel feature vector at a specific location essentially selects a local feature representation within the feature map. By locating the (i, j) coordinate, subsequent enhancement processing can be performed on the local features at that location, ensuring fine-grained feature enhancement.

[0047] Figure 5 FIG. 1 is a flowchart of sub-step S222 of the image processing method according to an embodiment of the present application. Figure 5As shown, the step S222 includes the steps of: S2221, performing n random scans on the target device surface state feature coding feature map to obtain n channel feature vectors as a sparse set of target device surface state reference feature vectors; S2222, calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of target device surface state reference feature vectors to construct a target device surface state channel feature space modulation matrix to be enhanced; S2223, calculating the target device surface state channel feature vector to be enhanced The implicit semantic association between the eigenvector and each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vectors is used to obtain a set of target device surface state channel feature semantic association coding matrices to be enhanced; S2224, based on the target device surface state channel feature spatial modulation matrix and the target device surface state channel feature semantic association coding matrix to be enhanced, the target device surface state channel feature vector to be enhanced is subjected to information compensation explicit modeling modulation to obtain the target device surface state channel feature enhancement component implicit coding vector.

[0048] In a specific example of the present application, step S2221 is expressed as follows:

[0049]

[0050] in, 、 、 and They represent the first, second, and third features in the sparse set of reference feature vectors of the target device surface state. and target device surface state reference feature vector, A sparse set of reference feature vectors representing the surface state of the target device.

[0051] Specifically, the target device surface state feature encoding feature map is randomly scanned n times to generate a sparsely distributed set of target device surface state reference feature vectors by randomly sampling different locations in the feature map. This introduces spatial diversity and randomness to avoid local overfitting caused by fixed sampling. Furthermore, the construction of a sparse set not only reduces computational complexity but also provides rich contextual references for subsequent information compensation by covering representative local patterns in the target device surface state feature encoding feature map.

[0052] In a specific example of the present application, step S2222 is expressed as follows:

[0053]

[0054]

[0055] in, Represents the channel feature vector to be enhanced on the surface state of the target device. represents the square of the norm of the vector, represents the inverse hyperbolic cosine function, express and The Poincare distance between express and The Poincare distance between Represents the spatial modulation matrix of the channel features to be enhanced for the surface state of the target device.

[0056] Specifically, by calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and the reference feature vectors of each target device surface state, in order to quantify their local correlation in the spatial domain, a spatial modulation matrix of the target device surface state channel feature to be enhanced is constructed. The spatial modulation matrix of the target device surface state channel feature to be enhanced provides a spatial attention weight for subsequent feature enhancement. Its value reflects the correlation between the features at different positions in the target device surface state feature encoding feature map and the target device surface state channel feature vector to be enhanced, which helps the model to pay more attention to the complementary information with a strong correlation with the channel feature to be enhanced in the reference feature set during the subsequent feature enhancement process, thereby enhancing the spatial consistency of the channel feature to be enhanced and improving the distinguishing ability of the feature representation. In addition, compared with the Euclidean distance, the Poincare distance is more suitable for capturing the nonlinear distribution law of the device surface state features. Using the Poincare distance to calculate the similarity between the target device surface state channel feature vector to be enhanced and the reference feature vectors of the target device surface state can better model the hierarchical relationship in the feature space.

[0057] In a specific example of the present application, step S2223 is expressed as follows:

[0058]

[0059] in, represents the weight matrix, represents the transpose of the matrix, represents the matrix multiplication operation, represents the normalized exponential function, express and The semantic association encoding matrix of the channel features to be enhanced for the target device surface state between the two channels.

[0060] Specifically, by mining the abstract semantic similarity between the channel feature vector to be enhanced of the target device surface state and the reference feature vectors of each target device surface state, the implicit semantic association between the channel feature to be enhanced and each reference feature is captured, so that the generated semantic association coding matrix of the channel feature to be enhanced of the target device surface state can effectively reveal the complementarity between the reference feature and the channel feature to be enhanced at the high-level semantic level, providing a semantic-level modulation basis for subsequent information compensation.

[0061] In particular, in a preferred example of the present application, the step S2224 includes: calculating the information compensation coding vector between each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced to obtain a set of target device surface state channel information compensation coding vectors to be enhanced, wherein the target device surface state channel information compensation coding vector is a positional difference vector between the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced; performing double-layer nesting constraints on each target device surface state channel information compensation coding vector in the set of target device surface state channel information compensation coding vectors to be enhanced to obtain an optimized set of target device surface state channel information compensation coding vectors to be enhanced, which is expressed by the formula:

[0062]

[0063]

[0064]

[0065] in, Indicates the target device surface state channel information compensation coding vector to be enhanced, Represents a nested optimization target device surface state to be enhanced channel information compensation coding vector, The channel information compensation coding vector representing the optimized target device surface state to be enhanced.

[0066] Finally, each semantic association coding matrix of the target device surface state channel feature to be enhanced in the set of the target device surface state channel feature semantic association coding matrices is used as a first-level mask modulation unit, and the target device surface state channel feature spatial modulation matrix is ​​used as a second-level mask modulation unit. The set of the optimized target device surface state channel feature to be enhanced information compensation coding vectors is subjected to explicit modeling modulation aggregation to obtain the target device surface state channel feature enhancement component implicit coding vector, which is expressed as follows:

[0067]

[0068] in, The implicit coding vector representing the enhanced component of the channel feature to be enhanced of the surface state of the target device.

[0069] Among them, the target device surface state to be modulated channel information compensation coding vector is used as the modulation field density distribution, then each target device surface state channel feature semantic association coding matrix in the set of the target device surface state channel feature semantic association coding matrix as the first-level mask modulation unit can be regarded as a gauge field, and the target device surface state channel feature spatial modulation matrix as the second-level mask modulation unit can be regarded as a covariant field. Therefore, when the covariant field is used to process the curved space and the gauge field is used for the derivative-level space geometric metric under the density structure, it can be first used as To constrain the canonical transformation to be invariant, To constrain the covariant transformation to be invariant, specifically, the ordinary partial derivative form in the field space is used to perform strong and weak coupling of field interactions, thereby further enhancing the correlation between the double-layer mask and information compensation, and improving the effectiveness of information encoding.

[0070] Furthermore, a double-layer mask modulation mechanism is used to achieve refined screening and fusion of the information compensation coding vectors, generating an implicit coding vector for the enhanced component of the target device surface state channel feature to be enhanced. Specifically, the double-layer mask modulation adopts a hierarchical screening mechanism with semantic masking first and spatial masking later, effectively balancing the contributions of global semantic information and local spatial information, improving the effectiveness and robustness of the enhancement. The implicit coding vector for the enhanced component of the target device surface state channel feature to be enhanced serves as an intermediate representation, encoding the enhancement components extracted from the reference vector, and will be further used to enhance and modulate the target device surface state channel feature to be enhanced.

[0071] More specifically, step S223 is expressed as follows:

[0072]

[0073] in, and represents the weight parameter, Represents the enhanced target device surface state channel feature vector.

[0074] Specifically, a weighted strategy is employed to perform a weighted fusion of the target device surface state channel feature vectors and the implicit coding vectors of the enhanced components of the target device surface state channel features. This generates an implicit coding vector of the enhanced components of the target device surface state channel features that combines the original local features with contextual enhancement information. Specifically, this fusion process retains the discriminative nature of the original features while introducing compensation information to enhance sensitivity to subtle changes in the device surface state. Ultimately, this results in an enhanced feature map for subsequent material recognition tasks.

[0075] Specifically, the step S23 performs material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map. In a specific example of the present application, the step S23 includes: inputting the target device surface state feature enhanced coding feature map into a material category identification engine based on a classifier to obtain the device material distribution probability map. Specifically, the present application adopts a classifier based on deep learning as a material category identification engine. The classifier has been trained with a large amount of sample data and has the ability to recognize the surface state features of various materials. In actual applications, the target device surface state feature enhanced coding feature map is used as the input of the classifier. The classifier analyzes each pixel position feature in the target device surface state feature enhanced coding feature map one by one, and combines the mapping relationship between the material feature representation and the category label learned during the training process. It can accurately identify the material category of the device surface, output the probability distribution of each pixel point belonging to different material categories, and then generate a device material distribution probability map to provide the necessary material information for subsequent thermal image correction.

[0076] In the above-mentioned image processing method, step S3 aligns the device material distribution probability map with the thermal image of the target device, and based on the device material distribution probability map, compensates and corrects the original temperature values ​​of each pixel in the thermal image to obtain a device temperature distribution map. Specifically, because infrared thermal imagers and visible light cameras may cause spatial misalignment of images due to differences in installation angle and focal length, it is necessary to align the two to establish a pixel-to-pixel correspondence and ensure accurate superposition of the material probability map and the thermal image. In an embodiment of the present application, a feature matching algorithm, such as the scale-invariant feature transform (SIFT) algorithm, is first employed to extract feature points from the image and calculate descriptors of the feature points. Based on the similarity of the descriptors, corresponding matching point pairs are found in the material distribution probability map and the thermal image. These matching point pairs are then used to calculate a transformation matrix to spatially align the material distribution probability map and the thermal image. After the registration is completed, due to the significant differences in the absorption and emission characteristics of thermal radiation by different materials, For example, metal surfaces dissipate heat quickly due to their high thermal conductivity, resulting in a lower actual temperature than indicated by the infrared image. Insulator surfaces also accumulate heat due to their low thermal conductivity, requiring temperature compensation to avoid missing hot spots. Therefore, this application further extracts the corresponding thermal conductivity from a database based on the material type of each pixel in the thermal image. This is then substituted into the temperature compensation formula to calculate the original temperature value of each pixel in the thermal image to produce a corrected device temperature distribution map.

[0077] Specifically, during the implementation process, a scale-invariant feature transformation algorithm is used to extract feature points from the material distribution probability map and thermal image. By detecting significant areas such as edges and corners in the image, a feature descriptor with rotation and scale invariance is generated. During the feature matching stage, a bidirectional nearest neighbor search strategy is used to screen out matching point pairs with high similarity. A random sampling consistency algorithm is then used to eliminate abnormal matching points caused by noise or local deformation. Based on the remaining high-quality matching point pairs, an affine transformation model is constructed to calculate the rotation, translation, and scaling parameters. Finally, the material distribution probability map is mapped to the coordinate system of the thermal image using a bilinear interpolation algorithm, completing sub-pixel spatial alignment.

[0078] Post-registration fusion data processing must prioritize addressing temperature measurement errors caused by differences in the thermal radiation properties of multi-material surfaces. In the material distribution probability map, the value of each pixel represents the probability that the location belongs to a specific material category. Parameters such as thermal conductivity and thermal emissivity of different materials directly impact the temperature measurement results of the infrared thermal imager. To this end, the system incorporates a built-in material property database, storing key parameters such as thermal conductivity, specific heat capacity, and thermal emissivity for typical converter station equipment materials (including metal conductors, ceramic insulators, and polymer bushings). After spatial registration of the material distribution probability map with the thermal image, multi-level temperature compensation is performed for each pixel in the thermal image based on the probability distribution of the corresponding material category. For regions dominated by a single material (such as metal connectors with a probability value exceeding 90%), the thermal conductivity correction model for that material in the database is directly invoked. For transitional regions with mixed materials (such as metal-ceramic junctions with similar probability distributions), a weighted fusion algorithm is used to calculate the equivalent thermal conductivity.

[0079] The specific implementation of temperature compensation correction is divided into two dimensions: radiation characteristic compensation and thermal conduction compensation. At the radiation characteristic compensation level, the original temperature value of the infrared sensor is inverted and calculated based on the thermal emissivity parameters of the material to eliminate measurement deviations caused by differences in the emissivity of the material surface. By establishing a thermal emissivity-temperature mapping relationship, the original temperature value of each pixel in the thermal image is converted into a standardized blackbody radiation equivalent temperature. At the thermal conduction compensation level, the environmental parameters during equipment operation (including ambient temperature, wind speed, and sunlight intensity) are combined with the thermal conductivity of the material to construct a three-dimensional heat conduction partial differential equation to dynamically compensate for the surface temperature field. In particular, for areas covered by low thermal conductivity materials (such as silicone rubber composite insulators), the surface temperature gradient is abnormal due to the inability to conduct heat in a timely manner. The measured temperature needs to be nonlinearly amplified by calculating the internal thermal resistance of the material to restore the actual heating state inside the device.

[0080] To achieve real-time calculations for the aforementioned compensation process, the system uses a parallel processing architecture to process thermal images in blocks. Computing resources are dynamically allocated to each image block based on the material distribution characteristics: In homogeneous material areas, a quick table lookup method is used to directly apply preset compensation coefficients, while in complex mixed material areas, a finite element algorithm is used to reconstruct the local temperature field. The compensated pixel temperature values ​​are Gaussian filtered to eliminate edge noise caused by registration residuals, ultimately generating a physically consistent device temperature distribution map. This map not only contains the corrected absolute temperature values ​​but also superimposes material distribution information through pseudo-color coding technology, forming a dual-dimensional temperature-material visualization map, providing a high-precision data foundation for subsequent hot spot detection.

[0081] In the above-mentioned image processing method, step S4 performs hotspot detection on the target device based on the device temperature distribution map to obtain a hotspot detection result. Specifically, the device temperature distribution map intuitively presents the temperature conditions of various parts of the device. To promptly detect and resolve potential overheating issues, the present application performs hotspot detection on the device temperature distribution map to ensure the safe operation of the converter station. Specifically, a temperature threshold is first set based on the temperature range of the device during normal operation. Then, each pixel in the device temperature distribution map is traversed and its temperature value is compared with the preset temperature threshold. If the temperature of a pixel exceeds the threshold, the pixel is determined to be a potential hotspot. Simultaneously, a temperature statistical analysis is performed on pixels within a certain range surrounding the potential hotspot, such as calculating the average temperature and maximum temperature. Statistical features are then used to further determine whether the potential hotspot is a true temperature anomaly area. If the temperatures of the surrounding pixels are also generally elevated and the statistical features exceed the preset determination criteria, the pixel is confirmed to be a true hotspot, and its location, temperature value, and other information are recorded in the hotspot detection result.

[0082] Specifically, the reference temperature range for metallic conductor components is typically set at 40-70°C, while the reference temperature for composite insulation materials is controlled within the range of 30-50°C. When setting the initial temperature threshold, the temperature data from 72 hours of continuous operation of the equipment at rated load is used as a reference. The mean μ and standard deviation σ are calculated using a Gaussian distribution model. The upper threshold is set to μ + 3σ to cover 99.7% of normal operating conditions. An ambient temperature compensation factor is also introduced to adjust the threshold in real time based on indoor temperature monitoring data from the converter station to eliminate the impact of ambient temperature fluctuations on the detection results.

[0083] Pixel-level traversal analysis of the temperature distribution map is implemented using a parallel computing architecture. The temperature value of each pixel is mapped to the device's three-dimensional model using a spatial indexing algorithm to ensure that the temperature data accurately corresponds to the device's physical location. When the detection system scans a pixel whose temperature value exceeds the dynamic threshold, the spatial correlation analysis module is immediately activated. This module constructs a circular analysis window with an adjustable radius, centered on the outlier. The window size is dynamically adjusted based on the device's thermal conductivity characteristics: for metal components with good thermal conductivity, such as copper busbars, an analysis window with a radius of 5 pixels (corresponding to an actual size of approximately 10 cm) is used; for components with poor thermal conductivity, such as epoxy resin insulation, the window is expanded to a radius of 15 pixels. Multi-dimensional temperature statistics are performed within the window area, including calculation of parameters such as the regional average temperature, temperature standard deviation, and maximum temperature gradient. At the same time, the second-order derivative characteristics of the temperature distribution curve are extracted to identify the severity of temperature changes.

[0084] The generation of hot spot detection results follows standardized data packaging specifications. Each confirmed hot spot record contains core parameters such as spatial coordinates, maximum temperature value, abnormal area, temperature change rate, etc., as well as material composition information of the corresponding area in the material distribution probability map. Data encapsulation uses the JSON-LD format to ensure seamless integration with the equipment asset management system. For the coexistence of multiple hot spots, the system automatically performs cluster analysis to determine whether there are related faults based on the spatial distribution characteristics of the heat source. In the converter valve component detection scenario, a chain propagation detection module is specially set up. When multiple adjacent thyristors are found to have temperature anomalies at the same time, the alarm level is automatically increased and the possible fault propagation path is marked.

[0085] Quality control of the detection process is achieved through a closed-loop verification mechanism. The system has a built-in virtual heat source generator that can inject simulated temperature signals at specified locations during equipment downtime and maintenance to verify the sensitivity and positioning accuracy of the detection algorithm. The infrared thermal imaging system is regularly calibrated on-site using a blackbody radiation source to ensure the accuracy of the temperature measurement benchmark. After each inspection task is completed, the system automatically generates a detection confidence report, marking any possible detection blind spots to provide data support for subsequent algorithm optimization. In view of the strong electromagnetic environment characteristics of the converter station, the detection system uses key hardware components with electromagnetic shielding design and embeds digital filtering algorithms in the signal processing link to effectively suppress the impact of pulse interference generated by the switching operation of the converter valve on the temperature measurement accuracy.

[0086] In the above-mentioned image processing method, in step S5, in response to the hot spot detection result indicating the presence of an abnormal temperature area, a temperature abnormality alarm prompt signal is generated. Specifically, when the hot spot detection result confirms the presence of an abnormal temperature area, the temperature abnormality alarm mechanism is immediately triggered, and a clear and intuitive alarm prompt signal is generated to notify the operation and maintenance personnel to take timely measures to deal with the problem. The temperature abnormality alarm prompt signal may include information such as the specific location of the hot spot and the degree of temperature abnormality, so that the operation and maintenance personnel can quickly locate the problem and take corresponding solutions. In this way, a basis is provided for timely discovery of hidden dangers of equipment failure, which helps to take measures in advance to avoid serious accidents caused by equipment failure and ensure the stable operation of the closed converter station.

[0087] In summary, the image processing method based on the embodiment of the present application is explained, which uses the infrared thermal imager and visible light camera carried by the unmanned vehicle of the closed converter station to synchronously collect the color image and thermal image of the target device, and uses the image processing technology based on deep learning to identify the material of the color image of the target device to obtain the device material distribution probability map. Subsequently, the device material distribution probability map is spatially aligned with the thermal image of the target device, and the material thermal conductivity correction algorithm is used to perform pixel-level compensation on the original temperature value in the thermal image. Then, based on the corrected device temperature distribution map, hot spot detection is performed, and when an abnormal temperature area is detected, an alarm prompt signal is automatically generated. This can realize automated inspection and fault warning of equipment in the closed converter station, improve inspection efficiency and accuracy, reduce operation and maintenance costs, and ensure the safe and stable operation of the power grid.

[0088] Furthermore, an image processing system is provided.

[0089] Figure 6 FIG is a block diagram of an image processing system according to an embodiment of the present application. Figure 6 As shown, the image processing system 100 according to an embodiment of the present application includes: a target device image receiving module 110, configured to receive a color image and a thermal image of a target device captured by an infrared thermal imager and a visible light camera; a device material recognition module 120, configured to input the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is configured to perform image feature-level explicit enhancement on the color image of the target device and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; an image registration and temperature correction module 130, configured to register the device material distribution probability map with the thermal image of the target device and compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; a hot spot detection module 140, configured to perform hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; and an abnormal alarm prompt generation module 150, configured to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result indicating the presence of a temperature abnormality area.

[0090] Furthermore, a closed converter station unmanned vehicle is also provided. The closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the above-mentioned image processing system, and can execute the above-mentioned image method.

[0091] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0092] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0093] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0094] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0095] Finally, it should be noted that the above description has been provided for purposes of illustration and description. Furthermore, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art will appreciate that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A closed converter station unmanned vehicle autonomous inspection method, characterized in that: include: receiving a color image and a thermal image of a target device captured by an infrared thermal imager and a visible light camera; Inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category identification based on the enhanced image features to obtain the device material distribution probability map; Registering the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; Performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; In response to the hot spot detection result indicating that a temperature abnormality area exists, generating a temperature abnormality alarm prompt signal; The method of inputting the color image of the target device into the trained material recognition model to obtain a device material distribution probability map includes: Extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map; Performing image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain a target device surface state feature enhanced coding feature map; Performing material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map; The step of performing image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain the target device surface state feature enhanced coding feature map includes: Extracting the channel feature vector at the (i, j)th pixel position from the target device surface state feature coding feature map as the target device surface state channel feature vector to be enhanced; Based on the target device surface state feature coding feature map, spatial-semantic dual-domain information compensation coding is performed on the target device surface state channel feature vector to be enhanced to obtain an implicit coding vector of the target device surface state channel feature enhancement component, including: based on a set of the target device surface state channel feature spatial modulation matrix and the target device surface state channel feature semantic association coding matrix, information compensation explicit modeling modulation is performed on the target device surface state channel feature vector to be enhanced to obtain an implicit coding vector of the target device surface state channel feature enhancement component; The target device surface state to be modulated channel information compensation coding vector is used as the modulation field density distribution, each target device surface state channel feature semantic association coding matrix in the set of the target device surface state channel feature semantic association coding matrix to be enhanced as the first-level mask modulation unit is regarded as a gauge field, and the target device surface state channel feature spatial modulation matrix to be enhanced as the second-level mask modulation unit is regarded as a covariant field. When the covariant field is used to process the curved space and the gauge field is used for the derivative-level space geometric metric under the density structure, the gauge transformation is first constrained to be invariant, and then the covariant transformation is constrained to be invariant, and the ordinary partial derivative form in the field space is used to perform strong and weak coupling of the field interaction; The implicit coding vector of the enhanced component of the channel feature to be enhanced of the target device surface state and the channel feature vector to be enhanced of the target device surface state are fused to obtain an enhanced target device surface state channel feature vector, wherein the enhanced target device surface state channel feature vector is the channel feature vector of the pixel position (i, j) of the enhanced coding feature map of the target device surface state feature.

2. The closed converter station unmanned vehicle autonomous inspection method according to claim 1 is characterized in that: Extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map includes: An image feature extractor based on an FPN model is used to extract surface state features from the color image of the target device to obtain a surface state feature coding feature map of the target device.

3. The closed converter station unmanned vehicle autonomous inspection method according to claim 2, characterized in that: Based on the target device surface state feature coding feature map, spatial-semantic dual-domain information compensation coding is performed on the target device surface state channel feature vector to be enhanced to obtain an enhanced component implicit coding vector of the target device surface state channel feature to be enhanced, including: Performing n random scans on the target device surface state feature coding feature map to obtain n channel feature vectors as a sparse set of target device surface state reference feature vectors; Calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of target device surface state reference feature vectors to construct a target device surface state channel feature space modulation matrix to be enhanced; Calculating implicit semantic associations between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of target device surface state reference feature vectors to obtain a set of target device surface state channel feature semantic association coding matrices to be enhanced; Based on the set of the spatial modulation matrix of the target device surface state channel feature to be enhanced and the semantic association coding matrix of the target device surface state channel feature to be enhanced, the target device surface state channel feature vector to be enhanced is subjected to information compensation explicit modeling modulation to obtain an implicit coding vector of the enhanced component of the target device surface state channel feature to be enhanced.

4. The closed converter station unmanned vehicle autonomous inspection method according to claim 3 is characterized in that: Based on a set of a spatial modulation matrix of the target device surface state channel feature to be enhanced and a semantic association coding matrix of the target device surface state channel feature to be enhanced, performing information compensation explicit modeling modulation on the target device surface state channel feature vector to be enhanced to obtain an implicit coding vector of an enhanced component of the target device surface state channel feature to be enhanced, including: Calculating an information compensation coding vector between each target device surface state reference feature vector in the sparse set of target device surface state reference feature vectors and the target device surface state channel feature vector to be enhanced to obtain a set of target device surface state channel information compensation coding vectors; Performing double-layer nesting constraints on each target device surface state channel information compensation code vector in the set of target device surface state channel information compensation code vectors to obtain an optimized set of target device surface state channel information compensation code vectors to be enhanced; Using each target device surface state channel feature semantic association coding matrix in the set of target device surface state channel feature semantic association coding matrices as a first-level mask modulation unit, and using the target device surface state channel feature spatial modulation matrix as a second-level mask modulation unit, the set of optimized target device surface state channel feature information compensation coding vectors to be enhanced is explicitly modeled, modulated and aggregated to obtain the target device surface state channel feature enhancement component implicit coding vector.

5. The closed converter station unmanned vehicle autonomous inspection method according to claim 4 is characterized in that: The target device surface state channel information compensation coding vector is a position difference vector between the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced.

6. The closed converter station unmanned vehicle autonomous inspection method according to claim 5, characterized in that: Performing material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map includes: The target device surface state feature enhanced coding feature map is input into a classifier-based material category recognition engine to obtain the device material distribution probability map.

7. A closed converter station unmanned vehicle autonomous inspection system, used to execute the closed converter station unmanned vehicle autonomous inspection method according to any one of claims 1 to 6, characterized in that: include: A target device image receiving module is used to receive color images and thermal images of the target device captured by the infrared thermal imager and the visible light camera; a device material recognition module, configured to input the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is configured to perform image feature-level explicit enhancement on the color image of the target device and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; An image registration and temperature correction module is used to register the device material distribution probability map with the thermal image of the target device, and to compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; a hot spot detection module, configured to perform hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; The abnormality alarm prompt generating module is used to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result indicating that there is a temperature abnormality area.

8. An unmanned vehicle for a closed converter station, characterized in that: The closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the closed converter station unmanned vehicle autonomous inspection system according to claim 7, and can execute the closed converter station unmanned vehicle autonomous inspection method according to any one of claims 1 to 6.

Images