Image processing method and system, and application of image processing method and system in closed converter station unmanned vehicle inspection

Abstract

The invention relates to the technical field of intelligent image processing, and particularly discloses an image processing method and system and application of the image processing method and system in closed converter station unmanned vehicle inspection. And carrying out material identification on the color image of the target equipment by using an image processing technology based on deep learning to obtain an equipment material distribution probability graph. And carrying out spatial registration on the equipment material distribution probability graph and the thermal image of the target equipment, carrying out pixel-level compensation on an original temperature value in the thermal image by adopting a material heat conductivity coefficient correction algorithm, carrying out heating point detection based on the corrected equipment temperature distribution graph, and automatically generating an alarm prompt signal when a temperature abnormal region is detected. Through intelligent image processing, automatic inspection and fault early warning of equipment in the closed converter station can be realized, the inspection efficiency and accuracy are improved, the operation and maintenance cost is reduced, and safe and stable operation of a power grid is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of intelligent image processing technology, and more specifically, to an image processing method and system and its application in unmanned vehicle inspection of closed converter stations. Background Art

[0002] As the scale of the power system continues to expand and the level of intelligence increases, the safe and stable operation of the closed converter station, as a core component of the high-voltage direct current transmission system, is directly related to the reliability and economy of the power grid. Due to the large number of equipment in the converter station and the complex operating environment, the equipment is under high voltage and high current conditions for a long time, which is prone to potential faults such as heating and aging. However, the internal equipment structure of the converter station is complex, the operating environment is closed, and the electromagnetic interference is strong. The traditional inspection method of the converter station mainly relies on manual close-range operation, which not only poses a personal safety risk, but also has problems such as long inspection cycles and low efficiency, which makes it difficult to meet the operation and maintenance needs of modern power grids.

[0003] In recent years, although some automated inspection systems have used infrared thermal imagers or visible light cameras to collect images of the working status of equipment, and then analyze the images to monitor the equipment status; there are still some limitations. First, the limitations of a single sensor are significant. Infrared thermal imagers can capture the surface temperature distribution of equipment, but are not sensitive to information such as the material and surface status of the equipment; although visible light cameras can provide high-resolution image details, they cannot directly reflect temperature anomalies. Secondly, since the difference in thermal conductivity of the surface material of the equipment (such as metal, ceramic, polymer, etc.) directly affects the heat conduction process, the radiation characteristics of the same temperature change on the surface of different materials are significantly different. Traditional thermal analysis methods usually assume that the equipment is a homogeneous material and do not consider the impact of material distribution on the temperature field, resulting in a large deviation in the correction value. For example, due to the poor thermal conductivity of the material on the surface of the insulator, its actual temperature may be higher than the value displayed by the thermal image. If temperature compensation correction is not performed, potential hot spots may be missed.

[0004] Therefore, an image processing method and system and its application in unmanned vehicle inspection of closed converter stations are needed to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an image processing method, system and application in unmanned vehicle inspection of closed converter stations, which synchronously collects color images and thermal images of target equipment through infrared thermal imagers and visible light cameras, and uses deep learning-based image processing technology to perform material recognition on the color images of target equipment to obtain equipment material distribution probability maps. Subsequently, the equipment material distribution probability map is spatially registered with the thermal image of the target equipment, and the original temperature value in the thermal image is pixel-level compensated using a material thermal conductivity correction algorithm, and then hot spot detection is performed based on the corrected equipment temperature distribution map, and an alarm prompt signal is automatically generated when an abnormal temperature area is detected, which can realize automated inspection and fault warning of equipment in closed converter stations, improve inspection efficiency and accuracy, reduce operation and maintenance costs, and ensure safe and stable operation of the power grid.

[0006] According to one aspect of the present application, there is provided an image processing method, which includes:

[0007] Receiving color images and thermal images of target devices acquired by infrared thermal imagers and visible light cameras;

[0008] Inputting the color image of the target device into the trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map;

[0009] Registering the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map;

[0010] Performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result;

[0011] In response to the hot spot detection result indicating that there is an abnormal temperature area, a temperature abnormality alarm prompt signal is generated.

[0012] According to another aspect of the present application, there is provided an image processing system, comprising:

[0013] A target device image receiving module is used to receive a color image and a thermal image of the target device acquired by an infrared thermal imager and a visible light camera;

[0014] A device material recognition module, used for inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used for performing image feature-level explicit enhancement on the color image of the target device, and performing material category recognition based on the enhanced image features to obtain the device material distribution probability map;

[0015] An image registration and temperature correction module, used to register the device material distribution probability map with the thermal image of the target device, and to compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map;

[0016] A hot spot detection module, used for performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result;

[0017] The abnormal alarm prompt generation module is used to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result that there is a temperature abnormality area.

[0018] According to another aspect of the present application, a closed converter station unmanned vehicle is provided, wherein the closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the above-mentioned image processing system, and can execute the above-mentioned image processing method.

[0019] This application has at least the following technical effects:

[0020] Compared with the prior art, this application uses an infrared thermal imager and a visible light camera carried by an unmanned vehicle in a closed converter station to synchronously collect color images and thermal images of the target device, and uses image processing technology based on deep learning to identify the material of the color image of the target device to obtain a probability map of the material distribution of the device. Subsequently, the probability map of the material distribution of the device is spatially registered with the thermal image of the target device, and the original temperature value in the thermal image is compensated at the pixel level using a material thermal conductivity correction algorithm, and then the hot spot detection is performed based on the corrected device temperature distribution map, and an alarm signal is automatically generated when an abnormal temperature area is detected, which can realize the automated inspection and fault warning of the equipment in the closed converter station, improve the inspection efficiency and accuracy, reduce the operation and maintenance costs, and ensure the safe and stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 Flowchart of an image processing method according to an embodiment of the present application.

[0023] Figure 2 Schematic diagram of data flow of an image processing method according to an embodiment of the present application.

[0024] Figure 3 4 is a flowchart of sub-step S2 of the image processing method according to an embodiment of the present application.

[0025] Figure 4 Flow chart of sub-step S22 of the image processing method according to an embodiment of the present application.

[0026] Figure 5 Flow chart of sub-step S222 of the image processing method according to an embodiment of the present application.

[0027] Figure 6 is a block diagram of an image processing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0029] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0030] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0031] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0032] It should be noted that all data acquisition actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization given by the owner of the corresponding device.

[0033] Figure 1 Flowchart of an image processing method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the image processing method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the image processing method includes the following steps: S1, receiving a color image and a thermal image of a target device collected by an infrared thermal imager and a visible light camera; S2, inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; S3, aligning the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; S4, performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; S5, in response to the hot spot detection result indicating the presence of an abnormal temperature area, generating a temperature abnormality alarm prompt signal.

[0034] In the above-mentioned image processing method, the step S1 receives the color image and thermal image of the target device captured by the infrared thermal imager and the visible light camera. Specifically, the surface material of the closed converter station equipment (such as converter transformers, circuit breakers, disconnectors, etc.) is complex (metal conductors, insulating materials, polymer coatings, etc.), and the operating environment has strong electromagnetic interference, strong light reflection and shadow occlusion problems. It is difficult for a single sensor (such as infrared or visible light) to simultaneously obtain the thermal radiation characteristics and visual details of the equipment. The present application can make up for the perception limitations of a single sensor by receiving the color image and thermal image of the target device captured by the infrared thermal imager and the visible light camera, and provide a high-precision, multi-dimensional data foundation for subsequent analysis.

[0035] Specifically, in order to ensure the quality of data collection, the infrared thermal imager and visible light camera must be calibrated before image acquisition. For infrared thermal imagers, resolution, temperature range and accuracy are three core considerations. High resolution means that subtle temperature differences on the surface of the equipment can be captured more accurately, which is crucial for discovering potential fault points. The wide temperature measurement range ensures normal operation even in extreme environments and adapts to various working conditions that may occur in closed converter stations. In addition, higher temperature measurement accuracy can reduce the possibility of misjudgment and provide a reliable basis for subsequent material identification and temperature correction. For visible light cameras, in addition to the number of pixels, lens quality is equally important. High-quality lenses can not only improve image clarity, but also effectively reduce the impact of strong light reflection and shadow occlusion, so that the photos taken can more realistically reflect the actual condition of the equipment surface.

[0036] When collecting data, environmental factors are also an important part that cannot be ignored. There is usually strong electromagnetic interference inside the closed converter station, which poses a challenge to the working stability of electronic equipment. Therefore, when choosing infrared thermal imagers and visible light cameras, priority should be given to products with good anti-interference capabilities. At the same time, considering that there may be high humidity or heavy dust in the station, necessary protective measures need to be taken for these devices, such as using a casing with a high waterproof and dustproof grade. In addition, since the lighting conditions vary greatly in different time periods, it is also particularly important to arrange the time period for data collection reasonably. Ideally, operations should be carried out during periods when natural light is more uniform to avoid image distortion caused by uneven lighting.

[0037] In order to further improve the effectiveness of data collection, some advanced technical means can also be introduced. For example, the use of multi-view imaging technology can simultaneously shoot the same target from multiple angles, which not only increases the amount of information, but also helps to eliminate visual blind spots. In addition, the use of automatic exposure control technology can automatically adjust the camera parameters according to the lighting conditions on site to ensure that each shot can achieve the best effect. At the same time, for infrared thermal imagers, the dynamic range extension function can play an important role in processing high temperature difference scenes, so that even in the case of drastic temperature changes, good imaging quality can be maintained.

[0038] In the actual operation process, in order to ensure the high consistency and accuracy of the collected data, detailed operation specifications need to be formulated. This includes but is not limited to: determining the specific shooting path and sequence to ensure that all key parts are covered; setting a fixed shooting distance and angle to facilitate unified analysis during post-processing; establishing a strict quality inspection mechanism to promptly discover and correct possible problems. In this way, not only can work efficiency be improved, but also the reliability of data can be enhanced, laying a solid foundation for subsequent analysis and processing.

[0039] In the above-mentioned image processing method, in step S2, the color image of the target device is input into the trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map. Specifically, the difference in thermal conductivity of device materials (such as thermal conductivity of metal conductors ≥100 W / m·K, ceramic insulators ≤10 W / m·K) directly affects the heat conduction process. Traditional thermal analysis methods often do not consider the impact of different material distributions of equipment on the temperature field, which may lead to deviations in temperature anomaly detection. Therefore, the present application processes the color image of the target device using a trained material recognition model to identify various materials on the surface of the device, such as metals, ceramics, polymers, etc., and generates a device material distribution probability map, thereby achieving pixel-level temperature compensation through material recognition and improving the accuracy of device thermal analysis. Among them, Figure 3 FIG. 4 is a flowchart of sub-step S2 of the image processing method according to an embodiment of the present application. Figure 3 As shown, the step S2 includes the steps of: S21, extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map; S22, performing image feature enhancement based on space-semantic collaborative constraints on the target device surface state feature coding feature map to obtain a target device surface state feature enhanced coding feature map; S23, performing material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map.

[0040] Specifically, in a specific example of the present application, the step S21 includes: using an image feature extractor based on the FPN model to extract surface state features of the color image of the target device to obtain a surface state feature coding feature map of the target device. Specifically, since the surface state features of the device (such as texture, color, and roughness) are an important basis for material identification, in order to fully mine the multi-scale surface state information of the color image of the target device and more accurately identify the material, the present application uses the FPN (Feature Pyramid Network) model as an image feature extractor to mine the multi-scale surface state information in the color image of the target device. Specifically, FPN can effectively extract and fuse low-level detail features and high-level semantic information in the color image of the target device by combining bottom-up and top-down paths, and generate a surface state feature coding feature map of the target device, so as to more comprehensively reflect the surface material characteristics of the device and provide strong support for subsequent material identification.

[0041] Specifically, in step S22, the target device surface state feature coding feature map is subjected to image feature enhancement based on spatial-semantic collaborative constraints to obtain a target device surface state feature enhanced coding feature map. Specifically, the present application takes into account that the surface of the closed converter station equipment is susceptible to electromagnetic interference, uneven lighting, and stain coverage, resulting in blurred or lost local features. However, traditional image enhancement methods (such as histogram equalization) only rely on the statistical characteristics of the image and are difficult to restore the real details in complex scenes. In this regard, the present application adopts an image feature enhancement method based on spatial-semantic collaborative constraints, which performs feature enhancement processing on local pixel positions by performing dual constraints at the spatial and semantic levels on the features of each pixel position in the target device surface state feature coding feature map, so as to optimize the spatial consistency (such as edge retention) and semantic rationality (such as material category distribution) of the features, restore local feature blur caused by electromagnetic interference or uneven lighting, and ensure that the enhanced feature representation can accurately reflect the real material characteristics of the device surface, thereby providing more reliable data support for subsequent material category identification. Among them, Figure 4 FIG. 4 is a flowchart of sub-step S22 of the image processing method according to an embodiment of the present application. Figure 4As shown, the step S22 includes the steps of: S221, extracting the channel feature vector of the (i, j)th pixel position from the target device surface state feature coding feature map as the target device surface state channel feature vector to be enhanced; S222, based on the target device surface state feature coding feature map, performing spatial-semantic dual-domain information compensation encoding on the target device surface state channel feature vector to be enhanced to obtain an enhanced component implicit coding vector of the target device surface state channel feature to be enhanced; S223, fusing the enhanced component implicit coding vector of the target device surface state channel feature to be enhanced and the target device surface state channel feature vector to be enhanced to obtain an enhanced target device surface state channel feature vector, wherein the enhanced target device surface state channel feature vector is the channel feature vector of the pixel position (i, j) of the target device surface state feature enhancement coding feature map.

[0042] More specifically, the step S221 is expressed by the formula:

[0043]

[0044]

[0045] in, is a real number, , and Respectively represent the height, width and number of channels of the target device surface state feature encoding feature map, Represents the target device surface state feature encoding feature map, Indicates the channel feature vector of the (i, j)th pixel position of the target device surface state feature encoding feature map, Represents the channel feature vector to be enhanced of the surface state of the target device.

[0046] Specifically, the multidimensional channel feature vector extracted from the specific pixel position of the target device surface state feature coding feature map characterizes the local feature distribution of the position in the device surface state. Each pixel position of the target device surface state feature coding feature map corresponds to a multidimensional channel feature vector, and extracting the channel feature vector at a specific position is essentially selecting a local feature representation in the feature map. By locating the (i, j) coordinates, the local features of the position can be subsequently enhanced in a targeted manner to ensure the fine granularity of the feature enhancement.

[0047] Figure 5 FIG. 2 is a flowchart of sub-step S222 of the image processing method according to an embodiment of the present application. Figure 5As shown, the step S222 comprises the steps of: S2221, performing n random scans on the target device surface state feature coding feature map to obtain n channel feature vectors as a sparse set of target device surface state reference feature vectors; S2222, calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vector to construct a target device surface state channel feature space modulation matrix to be enhanced; S2223, calculating the target device surface state channel feature vector to be enhanced The implicit semantic association between the feature vector and each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vectors is used to obtain a set of target device surface state channel feature semantic association coding matrices to be enhanced; S2224, based on the target device surface state channel feature spatial modulation matrix and the set of target device surface state channel feature semantic association coding matrices to be enhanced, the target device surface state channel feature vector to be enhanced is subjected to information compensation explicit modeling modulation to obtain an implicit coding vector of the target device surface state channel feature enhancement component to be enhanced.

[0048] In a specific example of the present application, step S2221 is expressed by the formula:

[0049]

[0050] in, , , and They represent the first, second, and third features in the sparse set of reference feature vectors of the surface state of the target device. and The target device surface state reference feature vector, A sparse set of reference feature vectors representing the surface state of the target device.

[0051] Specifically, the target device surface state feature encoding feature map is randomly scanned n times to generate a set of sparsely distributed target device surface state reference feature vectors by randomly sampling different positions in the feature map multiple times. In this way, local overfitting caused by fixed sampling is avoided by introducing spatial diversity and randomness. In addition, the construction of sparse sets not only reduces the computational complexity, but also provides rich contextual references for subsequent information compensation by covering representative local patterns in the target device surface state feature encoding feature map.

[0052] In a specific example of the present application, step S2222 is expressed by the formula:

[0053]

[0054]

[0055] in, Represents the channel feature vector to be enhanced on the surface state of the target device, represents the square of the norm of a vector, represents the inverse hyperbolic cosine function, express and The Poincare distance between express and The Poincare distance between Represents the spatial modulation matrix of the channel features to be enhanced of the surface state of the target device.

[0056] Specifically, by calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and the reference feature vectors of each target device surface state, the local correlation in the spatial domain is quantified to construct a spatial modulation matrix of the target device surface state channel feature to be enhanced. The spatial modulation matrix of the target device surface state channel feature to be enhanced provides a spatial attention weight for subsequent feature enhancement, and its value reflects the correlation between the features at different positions in the target device surface state feature encoding feature map and the target device surface state channel feature vector to be enhanced, which helps the model to pay more attention to the complementary information with strong correlation with the channel feature to be enhanced in the reference feature set during the subsequent feature enhancement process, thereby enhancing the spatial consistency of the channel feature to be enhanced and improving the distinguishing ability of the feature representation. In addition, compared with the Euclidean distance, the Poincare distance is more suitable for capturing the nonlinear distribution law of the device surface state features. Using the Poincare distance to calculate the similarity between the target device surface state channel feature vector to be enhanced and the reference feature vectors of the surface state of each target device can better model the hierarchical relationship in the feature space.

[0057] In a specific example of the present application, step S2223 is expressed by the formula:

[0058]

[0059] in, represents the weight matrix, represents the transpose of a matrix, represents the matrix multiplication operation, represents the normalized exponential function, express and The semantic association encoding matrix of the channel features to be enhanced for the target device surface state between the two channels.

[0060] Specifically, the implicit semantic association between the channel features to be enhanced and the reference features is captured by mining the abstract semantic similarity between the feature vectors of the channel to be enhanced of the target device surface state and the reference feature vectors of each target device surface state. This enables the generated semantic association coding matrix of the channel features to be enhanced of the target device surface state to effectively reveal the complementarity between the reference features and the channel features to be enhanced at the high-level semantic level, thereby providing a semantic-level modulation basis for subsequent information compensation.

[0061] In particular, in a preferred example of the present application, the step S2224 includes: calculating the information compensation coding vector between each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced to obtain a set of target device surface state channel information compensation coding vectors to be enhanced, wherein the target device surface state channel information compensation coding vector to be enhanced is a position difference vector between the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced; performing double-layer nesting constraints on each target device surface state channel information compensation coding vector in the set of target device surface state channel information compensation coding vectors to be enhanced to obtain an optimized set of target device surface state channel information compensation coding vectors to be enhanced, which is expressed by the formula:

[0062]

[0063]

[0064]

[0065] in, Indicates the target device surface state channel information compensation coding vector to be enhanced, It represents a nested optimization target device surface state to be enhanced channel information compensation coding vector, The channel information compensation coding vector representing the optimized target device surface state to be enhanced.

[0066] Finally, each target device surface state channel feature semantic association coding matrix in the set of the target device surface state channel feature semantic association coding matrix to be enhanced is used as a primary mask modulation unit, and the target device surface state channel feature spatial modulation matrix to be enhanced is used as a secondary mask modulation unit, and the set of the optimized target device surface state channel information compensation coding vector to be enhanced is subjected to explicit modeling modulation aggregation to obtain the target device surface state channel feature enhancement component implicit coding vector, which is expressed by the formula:

[0067]

[0068] in, The implicit coding vector representing the enhanced component of the channel feature to be enhanced of the surface state of the target device.

[0069] Among them, the target device surface state to be modulated channel to be enhanced information compensation coding vector is used as the modulation field density distribution, then each target device surface state channel feature semantic association coding matrix in the set of the target device surface state channel feature semantic association coding matrix to be enhanced as the first-level mask modulation unit can be regarded as a canonical field, and the target device surface state channel feature spatial modulation matrix to be enhanced as the second-level mask modulation unit can be regarded as a covariant field. Therefore, when the covariant field is used to process the curved space and the canonical field is used for the derivative-level space geometric metric under the density structure, it can be firstly used as To constrain the canonical transformation to be invariant, To constrain the covariant transformation to be invariant, specifically, the ordinary partial derivative form in the field space is used to perform strong and weak coupling of field interactions, thereby further enhancing the correlation between the double-layer mask and information compensation and improving the effectiveness of information coding.

[0070] Furthermore, the information compensation coding vector is refinedly screened and fused through a double-layer mask modulation mechanism to generate an implicit coding vector of the enhanced component of the channel feature of the target device surface state to be enhanced. Specifically, the double-layer mask modulation adopts a hierarchical screening mechanism of semantic mask first and spatial mask later, which effectively balances the contribution of global semantic information and local spatial information, and improves the effectiveness and robustness of the enhancement. The implicit coding vector of the enhanced component of the channel feature of the enhanced surface state of the target device to be enhanced is used as an intermediate representation to encode the enhancement component extracted from the reference vector, which will be further used for the enhancement and modulation of the channel feature of the target device surface state to be enhanced.

[0071] More specifically, the step S223 is expressed by the formula:

[0072]

[0073] in, and represents the weight parameter, Represents the enhanced target device surface state channel feature vector.

[0074] Specifically, a weighted strategy is adopted to weight the feature vector of the target device surface state channel to be enhanced and the implicit coding vector of the enhanced component of the target device surface state channel to be enhanced, generating an implicit coding vector of the enhanced component of the target device surface state channel to be enhanced with both the original local features and context enhancement information. Specifically, the fusion process retains the discriminability of the original features, while introducing compensation information to improve the sensitivity to subtle changes in the device surface state, and finally forms an enhanced feature map for subsequent material recognition tasks.

[0075] Specifically, the step S23 performs material category identification based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map. In a specific example of the present application, the step S23 includes: inputting the target device surface state feature enhanced coding feature map into a material category identification engine based on a classifier to obtain the device material distribution probability map. Specifically, the present application adopts a classifier based on deep learning as a material category recognition engine. The classifier has the ability to recognize surface state features of various materials after being trained with a large amount of sample data. In practical applications, the target device surface state feature enhanced coding feature map is used as the input of the classifier. The classifier analyzes each pixel position feature in the target device surface state feature enhanced coding feature map one by one, and combines the mapping relationship between the material feature representation and the category label learned during the training process. It can accurately identify the material category of the device surface, output the probability distribution of each pixel belonging to different material categories, and then generate a device material distribution probability map to provide the necessary material information for subsequent thermal image correction.

[0076] In the above-mentioned image processing method, the step S3 is to register the device material distribution probability map with the thermal image of the target device, and to compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain the device temperature distribution map. Specifically, since the infrared thermal imager and the visible light camera may cause image space misalignment due to differences in installation angle and focal length, it is necessary to register the two to establish a corresponding relationship between pixels to ensure the accurate superposition of the material probability map and the thermal image. In an embodiment of the present application, first, a feature matching algorithm, such as a scale-invariant feature transform (SIFT) algorithm, is used to extract feature points in the image, calculate the descriptor of the feature points, find the corresponding matching point pairs in the material distribution probability map and the thermal image according to the similarity of the descriptors, and then use these matching point pairs to calculate the transformation matrix, and align the material distribution probability map with the thermal image in space. After the registration is completed, due to the significant differences in the absorption and emission characteristics of thermal radiation of different materials. For example, the metal surface dissipates heat quickly due to its high thermal conductivity, and its actual temperature is lower than the infrared image display value; the insulator surface accumulates heat due to its low thermal conductivity, and temperature compensation is required to avoid missing hot spots. Therefore, this application further obtains the corresponding thermal conductivity from the database according to the material type corresponding to each pixel in the thermal image, substitutes it into the temperature compensation correction formula, and calculates the original temperature value of each pixel in the thermal image to obtain the corrected device temperature distribution map.

[0077] Specifically, in the implementation process, the scale-invariant feature transformation algorithm is used to extract feature points from the material distribution probability map and thermal image, and feature descriptors with rotation and scale invariance are generated by detecting significant areas such as edges and corners in the image. In the feature matching stage, a bidirectional nearest neighbor search strategy is used to screen out matching point pairs with high similarity, and a random sampling consistency algorithm is used to remove abnormal matching points caused by noise or local deformation. Based on the remaining high-quality matching point pairs, an affine transformation model is constructed to calculate the rotation, translation and scaling parameters, and finally the material distribution probability map is mapped to the coordinate system of the thermal image through a bilinear interpolation algorithm to complete sub-pixel spatial alignment.

[0078] The fusion data processing after registration needs to focus on solving the temperature measurement errors caused by the differences in thermal radiation characteristics of multi-material surfaces. In the material distribution probability map, the value of each pixel represents the probability value of the location belonging to a specific material category, and the thermal conductivity, thermal emissivity and other parameters of different materials directly affect the temperature measurement results of the infrared thermal imager. To this end, the system has a built-in material property database, which stores key parameters such as thermal conductivity, specific heat capacity, and thermal emissivity of typical equipment materials in the converter station (including metal conductors, ceramic insulators, polymer bushings, etc.). After the material distribution probability map and the thermal image are spatially registered, multi-level temperature compensation correction is performed for each pixel in the thermal image according to the probability distribution of the material category at the corresponding position. For areas dominated by a single material (such as metal connectors with a probability value of more than 90%), the thermal conductivity correction model of the material in the database is directly called; for transition areas of mixed materials (such as metal-ceramic joints with similar probability distributions), a weighted fusion algorithm is used to calculate the equivalent thermal conductivity.

[0079] The specific implementation of temperature compensation correction is divided into two dimensions: radiation characteristic compensation and thermal conduction compensation. At the radiation characteristic compensation level, the original temperature value of the infrared sensor is inverted and calculated according to the thermal emissivity parameters of the material to eliminate the measurement deviation caused by the difference in the surface emissivity of the material. By establishing a thermal emissivity-temperature mapping relationship, the original temperature value of each pixel in the thermal image is converted into a standardized blackbody radiation equivalent temperature. At the thermal conduction compensation level, a three-dimensional thermal conduction partial differential equation is constructed by combining the environmental parameters (including ambient temperature, wind speed, and sunlight intensity) during the operation of the equipment with the thermal conductivity of the material to dynamically compensate for the surface temperature field. In particular, for areas covered by low thermal conductivity materials (such as silicone rubber composite insulators), the surface temperature gradient is abnormal because the heat cannot be conducted in time. It is necessary to perform nonlinear amplification processing on the measured temperature by calculating the internal thermal resistance of the material to restore the actual heating state inside the equipment.

[0080] In order to realize the real-time calculation of the above compensation process, the system uses a parallel processing architecture to process the thermal image in blocks. Each image block dynamically allocates computing resources according to the material distribution characteristics: the homogeneous material area uses the fast table lookup method to directly apply the preset compensation coefficient, and the complex mixed material area uses the finite element algorithm to reconstruct the local temperature field. The compensated pixel temperature value is Gaussian filtered to eliminate the edge noise caused by the registration residual, and finally generates a device temperature distribution map with physical consistency. This distribution map not only contains the corrected absolute temperature value, but also superimposes the material distribution information through pseudo-color coding technology to form a temperature-material dual-dimensional visualization map, providing a high-precision data basis for subsequent hot spot detection.

[0081] In the above-mentioned image processing method, the step S4 performs a hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result. Specifically, the device temperature distribution map intuitively presents the temperature conditions of various parts of the device. In order to timely discover and solve potential overheating problems, the present application performs a hot spot detection on the device temperature distribution map to ensure the safe operation of the converter station. Specifically, a temperature threshold is first set, and the threshold is set based on the temperature range when the device is in normal operation. Then, each pixel point in the device temperature distribution map is traversed, and its temperature value is compared with the preset temperature threshold. If the temperature of a certain pixel point exceeds the threshold, the point is determined to be a potential hot spot. At the same time, by performing temperature statistical analysis on the pixels within a certain range around the potential hot spot, such as calculating the average temperature, the maximum temperature, etc., further judging whether the potential hot spot is a real temperature abnormal area through statistical features. If the temperature of the surrounding pixels is also generally high, and the statistical features exceed the preset judgment conditions, the point is confirmed to be an actual hot spot, and its position, temperature value and other information are recorded in the hot spot detection result.

[0082] Specifically, the reference temperature range of metal conductor parts is usually set at 40-70°C, while the reference temperature of composite insulation materials is controlled in the range of 30-50°C. When setting the initial temperature threshold, the temperature data of the equipment running continuously for 72 hours under rated load is used as the reference. The mean μ and standard deviation σ are calculated by the Gaussian distribution model, and the upper limit of the threshold is set to μ+3σ to cover 99.7% of the normal operating data. At the same time, the ambient temperature compensation factor is introduced to correct the threshold in real time according to the indoor temperature monitoring data of the converter station to eliminate the influence of ambient temperature fluctuations on the detection results.

[0083] The pixel-level traversal analysis of the temperature distribution map is implemented using a parallel computing architecture. The temperature value of each pixel is mapped to the three-dimensional model of the device through a spatial index algorithm to ensure that the temperature data accurately corresponds to the physical location of the device. When the detection system scans that the temperature value of a certain pixel exceeds the dynamic threshold, the spatial correlation analysis module is immediately started. This module is centered on the abnormal point and constructs a circular analysis window with an adjustable radius. The window size is dynamically adjusted according to the thermal conductivity characteristics of the equipment: for metal parts with good thermal conductivity such as copper bars, an analysis window with a radius of 5 pixels (corresponding to an actual size of about 10 cm) is used; for parts with poor thermal conductivity such as epoxy resin insulation, the window is expanded to a radius of 15 pixels. Multi-dimensional temperature statistics are performed in the window area, including calculating parameters such as the average temperature of the area, the standard deviation of the temperature, and the maximum temperature gradient. At the same time, the second-order derivative characteristics of the temperature distribution curve are extracted to identify the severity of the temperature change.

[0084] The generation of hot spot detection results follows standardized data packaging specifications. Each confirmed hot spot record contains core parameters such as spatial coordinates, maximum temperature value, abnormal area, temperature change rate, and material composition information of the corresponding area in the material distribution probability map. Data encapsulation uses the JSON-LD format to ensure seamless integration with the equipment asset management system. For the coexistence of multiple hot spots, the system automatically performs cluster analysis to determine whether there are related faults based on the spatial distribution characteristics of the heat source. In the converter valve assembly detection scenario, a chain propagation detection module is specially set up. When multiple adjacent thyristors are found to have temperature anomalies at the same time, the alarm level is automatically increased and the possible fault propagation path is marked.

[0085] The quality control of the detection process is achieved through a closed-loop verification mechanism. The system has a built-in virtual heat source generator, which can inject simulated temperature signals at specified locations during equipment downtime maintenance to verify the sensitivity and positioning accuracy of the detection algorithm. The infrared thermal imaging system is regularly calibrated on-site using a blackbody radiation source to ensure the accuracy of the temperature measurement benchmark. After each inspection task is completed, the system automatically generates a detection confidence report, marks possible detection blind areas, and provides data support for subsequent algorithm optimization. In view of the strong electromagnetic environment characteristics of the converter station, the detection system uses key hardware components with electromagnetic shielding design, and embeds a digital filtering algorithm in the signal processing link to effectively suppress the impact of pulse interference generated by the switching operation of the converter valve on the temperature measurement accuracy.

[0086] In the above-mentioned image processing method, in step S5, in response to the hot spot detection result that there is a temperature abnormality area, a temperature abnormality alarm prompt signal is generated. Specifically, when the hot spot detection result confirms that there is a temperature abnormality area, the temperature abnormality alarm mechanism is immediately triggered, and a clear and intuitive alarm prompt signal is generated to notify the operation and maintenance personnel to take timely measures to deal with it. The temperature abnormality alarm prompt signal may include information such as the specific location of the hot spot and the degree of temperature abnormality, so that the operation and maintenance personnel can quickly locate the problem and take corresponding solutions. In this way, it provides a basis for timely discovering hidden dangers of equipment failure, helps to take measures in advance, avoid serious accidents caused by equipment failure, and ensure the stable operation of the closed converter station.

[0087] In summary, the image processing method based on the embodiment of the present application is explained, which uses the infrared thermal imager and visible light camera carried by the unmanned vehicle of the closed converter station to synchronously collect the color image and thermal image of the target device, and uses the image processing technology based on deep learning to identify the material of the color image of the target device to obtain the equipment material distribution probability map. Subsequently, the equipment material distribution probability map is spatially registered with the thermal image of the target device, and the original temperature value in the thermal image is compensated at the pixel level using the material thermal conductivity correction algorithm, and then the hot spot detection is performed based on the corrected equipment temperature distribution map, and an alarm prompt signal is automatically generated when an abnormal temperature area is detected, which can realize the automated inspection and fault warning of the equipment in the closed converter station, improve the inspection efficiency and accuracy, reduce the operation and maintenance cost, and ensure the safe and stable operation of the power grid.

[0088] Furthermore, an image processing system is also provided.

[0089] Figure 6 FIG. 1 is a block diagram of an image processing system according to an embodiment of the present application. Figure 6 As shown, according to the embodiment of the present application, the image processing system 100 includes: a target device image receiving module 110, which is used to receive a color image and a thermal image of a target device collected by an infrared thermal imager and a visible light camera; a device material recognition module 120, which is used to input the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; an image registration and temperature correction module 130, which is used to register the device material distribution probability map with the thermal image of the target device, and compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; a hot spot detection module 140, which is used to perform hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; and an abnormal alarm prompt generation module 150, which is used to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result indicating that there is a temperature abnormality area.

[0090] Furthermore, a closed converter station unmanned vehicle is also provided. The closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the above-mentioned image processing system, and can execute the above-mentioned image method.

[0091] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0092] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0093] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0094] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0095] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An image processing method, characterized in that: include: Receiving color images and thermal images of target devices acquired by infrared thermal imagers and visible light cameras; Inputting the color image of the target device into the trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used to perform image feature-level explicit enhancement on the color image of the target device, and perform material category recognition based on the enhanced image features to obtain the device material distribution probability map; Registering the device material distribution probability map with the thermal image of the target device, and compensating and correcting the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; Performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; In response to the hot spot detection result indicating that there is an abnormal temperature area, a temperature abnormality alarm prompt signal is generated.

2. The image processing method according to claim 1, characterized in that: Inputting the color image of the target device into the trained material recognition model to obtain a device material distribution probability map, including: Extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map; Performing image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain a target device surface state feature enhanced coding feature map; The material category is identified based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map.

3. The image processing method according to claim 2, characterized in that: Extracting image features from the color image of the target device to obtain a target device surface state feature coding feature map includes: An image feature extractor based on the FPN model is used to extract surface state features from the color image of the target device to obtain a surface state feature coding feature map of the target device.

4. The image processing method according to claim 3, characterized in that: Performing image feature enhancement based on spatial-semantic collaborative constraints on the target device surface state feature coding feature map to obtain the target device surface state feature enhanced coding feature map, including: Extracting a channel feature vector at the (i, j)th pixel position from the target device surface state feature coding feature map as a target device surface state channel feature vector to be enhanced; Based on the target device surface state feature coding feature map, spatial-semantic dual-domain information compensation coding is performed on the target device surface state channel feature vector to be enhanced to obtain a target device surface state channel feature enhancement component implicit coding vector; The implicit coding vector of the enhanced component of the channel feature to be enhanced of the target device surface state and the channel feature vector to be enhanced of the target device surface state are fused to obtain an enhanced target device surface state channel feature vector, wherein the enhanced target device surface state channel feature vector is the channel feature vector of the pixel position (i, j) of the target device surface state feature enhanced coding feature map.

5. The image processing method according to claim 4, characterized in that: Based on the target device surface state feature coding feature map, spatial-semantic dual-domain information compensation coding is performed on the target device surface state channel feature vector to be enhanced to obtain the target device surface state channel feature enhancement component implicit coding vector, including: Performing n random scans on the target device surface state feature coding feature map to obtain n channel feature vectors as a sparse set of target device surface state reference feature vectors; Calculating the Poincare distance between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vectors to construct a target device surface state channel feature space modulation matrix to be enhanced; Calculating implicit semantic associations between the target device surface state channel feature vector to be enhanced and each target device surface state reference feature vector in the sparse set of target device surface state reference feature vectors to obtain a set of target device surface state channel feature semantic association coding matrices to be enhanced; Based on a set of the spatial modulation matrix of the target device surface state channel feature to be enhanced and the semantic association coding matrix of the target device surface state channel feature to be enhanced, the feature vector of the target device surface state channel to be enhanced is subjected to information compensation explicit modeling modulation to obtain an implicit coding vector of the enhanced component of the target device surface state channel feature to be enhanced.

6. The image processing method according to claim 5, characterized in that: Based on the set of the spatial modulation degree matrix of the channel feature of the surface state of the target device to be enhanced and the semantic association coding matrix of the channel feature of the surface state of the target device to be enhanced, the feature vector of the channel feature of the surface state of the target device to be enhanced is subjected to information compensation explicit modeling modulation to obtain an enhanced component implicit coding vector of the channel feature of the surface state of the target device to be enhanced, including: Calculating the information compensation coding vector between each target device surface state reference feature vector in the sparse set of the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced to obtain a set of target device surface state channel information compensation coding vectors to be enhanced; Performing double-layer nesting constraints on each target device surface state channel information compensation coding vector in the set of target device surface state channel information compensation coding vectors to be enhanced to obtain an optimized set of target device surface state channel information compensation coding vectors to be enhanced; Using each target device surface state channel feature semantic association coding matrix in the set of target device surface state channel feature semantic association coding matrices to be enhanced as a first-level mask modulation unit, and using the target device surface state channel feature spatial modulation matrix to be enhanced as a second-level mask modulation unit, the set of optimized target device surface state channel feature information compensation coding vectors to be enhanced is explicitly modeled, modulated and aggregated to obtain an implicit coding vector of the target device surface state channel feature enhancement component to be enhanced.

7. The image processing method according to claim 6, characterized in that: The target device surface state channel information compensation coding vector is a position difference vector between the target device surface state reference feature vector and the target device surface state channel feature vector to be enhanced.

8. The image processing method according to claim 7, characterized in that: The material category identification is performed based on the target device surface state feature enhanced coding feature map to obtain the device material distribution probability map, including: The target device surface state feature enhanced coding feature map is input into a classifier-based material category recognition engine to obtain the device material distribution probability map.

9. An image processing system, characterized in that: include: A target device image receiving module is used to receive a color image and a thermal image of the target device acquired by an infrared thermal imager and a visible light camera; A device material recognition module, used for inputting the color image of the target device into a trained material recognition model to obtain a device material distribution probability map, wherein the material recognition model is used for performing image feature-level explicit enhancement on the color image of the target device, and performing material category recognition based on the enhanced image features to obtain the device material distribution probability map; An image registration and temperature correction module, used to register the device material distribution probability map with the thermal image of the target device, and to compensate and correct the original temperature value of each pixel in the thermal image based on the device material distribution probability map to obtain a device temperature distribution map; A hot spot detection module, used for performing hot spot detection on the target device based on the device temperature distribution map to obtain a hot spot detection result; The abnormal alarm prompt generation module is used to generate a temperature abnormality alarm prompt signal in response to the hot spot detection result that there is a temperature abnormality area.

10. An unmanned vehicle for a closed converter station, characterized in that: The closed converter station unmanned vehicle is equipped with an infrared thermal imager, a visible light camera and the image processing system according to claim 9, and can execute the image processing method according to any one of claims 1 to 8.

Images