Infrared image power transmission equipment target identification method and system based on YOLOv7

Abstract

The invention discloses an infrared image power transmission equipment target identification method and system based on YOLOv7, and relates to the technical field of electric power automation, and the method comprises the following steps: obtaining continuous frames of infrared images and visible light images of power transmission equipment, and carrying out the preprocessing; based on the processed continuous frame infrared image and visible light image, constructing a space-time multi-mode input tensor; processing the input tensor through an improved YOLOv7 model, and outputting a first target frame and a corresponding frame-level feature vector; performing cross-modal feature fusion optimization based on the frame-level feature vector to obtain a joint optimization feature; performing confidence coefficient correction on the first target frame by using the joint optimization feature to obtain a second target frame; and outputting a target recognition result based on the second target frame. The method is used for solving the defects of a traditional power transmission equipment identification method in the aspects of information fusion effect, detection result stability and result credibility.

Description

Technical Field

[0001] This invention relates to the field of power system automation technology, and more specifically, to a method and system for infrared image-based power transmission equipment target recognition based on YOLOv7. Background Technology

[0002] In the operation, maintenance, and intelligent inspection of power systems, the status identification of transmission equipment is a crucial link in ensuring power grid safety and improving inspection efficiency. With the continuous expansion of transmission line scale and the frequent occurrence of complex operating conditions, single-modal image detection methods are insufficient to meet the demands of intelligent identification in all weather and environments. Traditional transmission equipment identification methods mostly rely on single infrared or visible light images for detection. While these methods can provide some recognition results under certain conditions, they have significant shortcomings when facing complex environments such as changes in lighting, equipment obstruction, noise interference, and differences in thermal feature distribution. In existing research on image recognition of power transmission equipment, although infrared and visible light information are widely used to assist in inspection, most methods are still insufficient in the fusion of multimodal data, often failing to fully leverage the complementary effects between different modalities. At the same time, most existing detection methods use static images as input, lacking in-depth modeling of the temporal correlation in video sequences. When the external environment changes or image quality is disturbed, the detection results are prone to instability. Furthermore, existing methods still rely on relatively fixed processing methods in the evaluation and application of result confidence, making it difficult to flexibly reflect the reliability of prediction results in different scenarios, thus affecting the accuracy and stability of the final recognition.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of existing technologies, this invention addresses the shortcomings of traditional power transmission equipment identification methods in terms of information fusion effect, detection result stability, and result reliability by constructing a spatiotemporal multimodal input tensor, introducing temperature consistency constraints and temporal attention mechanisms into the improved YOLOv7 model, and performing uncertainty dynamic correction based on joint optimization features.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for target recognition of power transmission equipment based on infrared images using YOLOv7 includes the following steps: acquiring continuous frame infrared and visible light images of the power transmission equipment and preprocessing them; constructing a spatiotemporal multimodal input tensor based on the processed continuous frame infrared and visible light images; processing the input tensor using an improved YOLOv7 model to output a first target box and its corresponding frame-level feature vector; performing cross-modal feature fusion optimization based on the frame-level feature vector to obtain a joint optimized feature; using the joint optimized feature to perform confidence correction on the first target box to obtain a second target box; and outputting the target recognition result based on the second target box.

[0006] In a preferred embodiment, the preprocessing includes: performing histogram equalization on the infrared image, and performing noise filtering and edge enhancement processing on the equalized infrared image; and performing contrast enhancement and color normalization processing on the visible light image.

[0007] In a preferred embodiment, the step of constructing a spatiotemporal multimodal input tensor based on the processed consecutive frames of infrared and visible light images specifically involves: performing spatial alignment processing on the continuously acquired infrared and visible light images to obtain a registered image sequence; stacking the registered image sequence along the channel dimension and time dimension to generate an input tensor containing spatiotemporal multimodal information.

[0008] In a preferred embodiment, the spatial alignment processing of continuously acquired infrared and visible light images includes: extracting candidate feature points from the infrared and visible light images and establishing an initial matching relationship; applying a modal mutual information maximization constraint to the initial matching relationship to generate an optimized matching result; calculating image transformation parameters based on the optimized matching result; and performing geometric correction on the infrared image according to the image transformation parameters to generate a sequence of registered image pairs spatially aligned with the visible light image.

[0009] In a preferred embodiment, processing the input tensor using the improved YOLOv7 model includes: extracting multi-scale spatiotemporal features of the spatiotemporal multimodal input tensor; aggregating the multi-scale spatiotemporal features across frames using a temporally enhanced feature pyramid network to generate cross-frame aggregated features, wherein the temporally enhanced feature pyramid network introduces a temporal attention mechanism to achieve weighted aggregation of features from consecutive frames; calibrating the cross-frame aggregated features based on temperature consistency constraints to obtain calibrated multimodal features; and fusing the calibrated multimodal features through cross-layer residual connections to output a first target bounding box.

[0010] In a preferred embodiment, the calibration of cross-frame aggregated features based on temperature consistency constraints to obtain calibrated multimodal features specifically involves: generating a preliminary result of a first target bounding box based on temporal fusion features, the preliminary result including target location information and class confidence, which is obtained through a detection head including a regression branch and a classification branch; performing confidence calibration on the class confidence to obtain a calibrated class probability distribution; and jointly optimizing the cross-frame aggregated features based on the target location information and the calibrated class probability distribution through multimodal consistency constraints to obtain the calibrated multimodal features.

[0011] In a preferred embodiment, the step of performing cross-modal feature fusion optimization based on frame-level feature vectors to obtain joint optimized features specifically involves: extracting feature vectors of the first target box in infrared and visible light modes to form cross-modal feature pairs; calculating the similarity index of the cross-modal feature pairs to generate constraint weights that reflect modal consistency; performing feature alignment processing on the cross-modal feature pairs based on the constraint weights to obtain cross-modal aligned feature representations; and fusing the cross-modal aligned feature representations with the temporal features of the target box to generate joint optimized features.

[0012] In a preferred embodiment, the step of using joint optimization features to correct the confidence of the first target box specifically involves: estimating the prediction uncertainty of the first target box based on the joint optimization features, wherein the estimation of the prediction uncertainty includes: calculating the probability distribution variance and information entropy of the prediction result of the first target box; generating a confidence correction coefficient based on the prediction uncertainty, wherein the correction coefficient is determined by comparing the uncertainty measurement result with a preset threshold; and adaptively adjusting the original confidence of the first target box using the correction coefficient to obtain the second target box.

[0013] In a preferred embodiment, estimating the prediction uncertainty of the first target box based on joint optimization features specifically involves: inputting the joint optimization features into a multi-path computation structure to obtain the preliminary probability distribution of each path; performing uncertainty-weighted fusion on the preliminary probability distribution to obtain a fused probability distribution; and performing perturbation sampling and smoothing on the fused probability distribution to generate the prediction probability distribution of the first target box.

[0014] Secondly, this application provides an infrared image power transmission equipment target recognition system based on YOLOv7, comprising: an image acquisition and preprocessing module for acquiring and preprocessing continuous frame infrared and visible light images of the power transmission equipment; a multimodal tensor construction module for constructing a spatiotemporal multimodal input tensor based on the processed continuous frame infrared and visible light images; a feature fusion and detection module for processing the input tensor using an improved YOLOv7 model and outputting a first target box and a corresponding frame-level feature vector; a cross-modal optimization module for performing cross-modal feature fusion optimization based on the frame-level feature vector to obtain jointly optimized features; a confidence correction module for performing confidence correction on the first target box using the jointly optimized features to obtain a second target box; and a result output module for outputting target recognition results based on the second target box.

[0015] As can be seen from the above technical solutions, this invention solves the limitations of traditional methods in multi-source information fusion by constructing a spatiotemporal multimodal input tensor and introducing cross-modal consistency constraints; by integrating a temporal attention mechanism into the improved YOLOv7 model, it solves the problem that existing detection methods fail to fully utilize the temporal information of continuous frames and that the recognition results are prone to fluctuation; and by using uncertainty estimation and dynamic correction based on joint optimization features, it solves the problems of a single confidence determination method and insufficient reliability of detection results. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the infrared image target recognition method for power transmission equipment based on YOLOv7 according to the present invention.

[0017] Figure 2 This is a schematic diagram of the infrared image power transmission equipment target recognition system based on YOLOv7 of the present invention.

[0018] Figure 3 This is a schematic diagram of the entire confidence level correction process. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1, Figure 1 This invention presents a method for intelligently preventing errors in generating power grid operation tickets based on the principle of source compatibility, comprising the following steps: S1: Acquire continuous frame infrared and visible light images of the power transmission equipment and perform preprocessing.

[0021] Continuous frame image data of power transmission equipment are acquired simultaneously using an infrared thermal imager and a visible light camera. Histogram equalization is performed on the infrared image, and noise filtering and edge enhancement are applied to the equalized infrared image. The visible light image is subjected to contrast enhancement and color normalization processing; The preprocessed image will be used for the construction of subsequent spatiotemporal multimodal input tensors.

[0022] S2, based on the processed consecutive frames of infrared and visible light images, constructs a spatiotemporal multimodal input tensor, specifically as follows: Spatial alignment processing is performed on continuously acquired infrared and visible light images to obtain a registered image sequence; The registered image sequences are stacked along the channel and time dimensions to generate an input tensor containing spatiotemporal multimodal information.

[0023] Specifically: In this embodiment, the spatial alignment processing of continuously acquired infrared and visible light images includes: Extract candidate feature points from infrared and visible light images and establish initial matching relationships; Apply modal mutual information maximization constraints to the initial matching relationship to generate optimized matching results; Calculate image transformation parameters based on the optimized matching results; Geometric correction is performed on the infrared image based on the image transformation parameters to generate a sequence of registered image pairs spatially aligned with the visible light image.

[0024] Specifically: Preprocessed infrared images Corresponding visible light image Candidate feature points are extracted separately, denoted as... and ,in It is an infrared image. The number of candidate feature points extracted. It is a visible light image. The number of candidate feature points extracted; Calculate matching similarity based on candidate feature points and establish an initial set of matching pairs:

[0025] in, Indicating infrared feature points With visible light feature points similarity, This is the matching threshold.

[0026] In the initial matching pair Based on this, a modal mutual information maximization constraint is introduced to generate an optimal matching set. :

[0027] in, Indicates the set of matching point pairs Mutual information between infrared and visible light images; This constraint can prioritize feature point pairs that have the best information content in both modalities, thus solving the difficulty of cross-modal matching caused by differences in grayscale and texture.

[0028] The constrained matching results Input a robust estimation algorithm, such as RANSAC, to obtain the registration transformation matrix from the infrared image to the visible light image. ; The matrix is ​​used to perform geometric correction and eliminate abnormal matching points, thereby improving the stability and accuracy of registration.

[0029] The obtained transformation matrix Acting on infrared images Obtain the geometrically corrected infrared image :

[0030] in, Represents the transformation matrix The inverse transformation is used to transform points in the visible light image coordinate system. Map back to the infrared image coordinate system; Geometrically corrected infrared image With visible light images The images are stitched together along the channel dimension, and the stitched multimodal images are then subjected to size unification and normalization processing to obtain a multimodal image sequence. ; The stitching refers to aligning the geometrically corrected infrared image and the visible light image according to their pixel positions, and then stacking them as different channels to form a multi-channel tensor. The stitching result has three dimensions, namely the height of the image, the width of the image, and the number of channels. Furthermore, the multimodal image sequences are stacked in the time dimension to form a spatiotemporal multimodal input tensor; The stacking in the time dimension refers to stacking multimodal image sequences acquired in chronological order frame-by-frame to form a four-dimensional tensor. This tensor has four dimensions: image height, image width, number of channels, and number of time frames. Through the above steps, the generated spatiotemporal multimodal input tensor can be directly input into the improved YOLOv7 model for infrared-visible joint target detection, achieving accurate cross-modal recognition.

[0031] S3, the input tensor is processed by the improved YOLOv7 model to output the first target box and the corresponding frame-level feature vector.

[0032] In this embodiment, processing the input tensor using the improved YOLOv7 model includes: Extracting multi-scale spatiotemporal features from spatiotemporal multimodal input tensors; The temporally enhanced feature pyramid network performs cross-frame aggregation of multi-scale spatiotemporal features to generate cross-frame aggregated features. The temporally enhanced feature pyramid network introduces a temporal attention mechanism to achieve weighted aggregation of features from consecutive frames. The cross-frame aggregated features are calibrated based on temperature consistency constraints to obtain calibrated multimodal features; The first target bounding box is output by fusing the calibrated multimodal features through cross-layer residual connections.

[0033] Specifically: First, a ResNet structure is used as the backbone network. The spatiotemporal multimodal input tensor is input into the network, and multi-level feature maps are output at different residual blocks. These feature maps have differences in spatial resolution and semantic abstraction level, thus obtaining spatiotemporal feature maps at different scales. Subsequently, the spatiotemporal feature maps at different scales are input into the feature pyramid network, and a temporal attention mechanism is introduced into the network to model the dependency relationship of cross-frame features. The temporal attention mechanism can adaptively allocate attention weights based on the similarity between consecutive frames, thereby generating cross-frame aggregated features to characterize the dynamic changes of the target over time. The Feature Pyramid Network (FPN) is a typical multi-scale feature fusion structure. Its basic principle is to upsample and fuse high-level semantic features extracted from the backbone network with low-level spatial detail features layer by layer, thereby generating a multi-scale feature representation that combines spatial resolution and semantic expressiveness. In this embodiment, a standard FPN structure or an improved path aggregation feature pyramid network can be used to further improve the efficiency of cross-scale feature transfer. In this embodiment, the calibration of cross-frame aggregated features based on temperature consistency constraints to obtain calibrated multimodal features specifically involves: The preliminary result of generating the first target bounding box based on cross-frame aggregated features includes target location information and category confidence. This preliminary result is obtained through a detection head, which includes a regression branch and a classification branch. The category confidence scores are calibrated to obtain the calibrated category probability distributions. Based on the target location information and the calibrated category probability distribution, the cross-frame aggregated features are jointly optimized through multimodal consistency constraints to obtain the calibrated multimodal features.

[0034] Specifically: In this embodiment, the preliminary result of the first target bounding box refers to the target detection result output by the detection head, including target location information and category confidence. Among them, the target location information comes from the regression results of the regression branch, and the category confidence refers to the probability value of the predicted category in the category probability vector obtained after normalizing the output of the classification branch by the softmax function; Cross-frame aggregated features are input into the detection head, which includes a classification branch and a regression branch: The classification branch processes the input cross-frame aggregated features through convolutional layers and uses the softmax function to output a class probability vector. , where represents the predicted probability of each target belonging to a category, where The total number of categories, Class probability vector The One component; The regression branch outputs the regression results. This includes the center coordinates, width, and height of the target bounding box, used to determine the position and size of the target in the image; The class probability vector output by the classification branch Temperature scaling is applied to generate a category probability distribution:

[0035] in, Indicates the first The target belongs to the category The probability of; This is a temperature coefficient used to adjust the smoothness of the probability distribution; This can make the distribution smoother. This can make the distribution more acute; Indicates the total number of categories; Indicates the first The complete category probability distribution vector of each target; The temperature scaling process described above is a calibration process for the category confidence. By adjusting the temperature coefficient, the confidence distribution of the classification results can be effectively improved, making it more consistent with the uncertainty characteristics of the model under cross-modal fusion. Next, in the regression results Based on the temperature-scaled class probability distribution, construct a multimodal consistency constraint function:

[0036] in, Indicates the first The target in the modality The following category probability distribution, Indicates the first The target in the modality The following category probability distribution, , These are the regression boxes for the modality and the corresponding target, respectively. For measuring the difference in probability distributions, cross-entropy or KL divergence can be selected. This represents the squared error of the regression box parameters, used to constrain positional consistency. These are weighting coefficients used to balance the importance of classification consistency and regression consistency; During the training phase, the multimodal consistency constraint function is used as the optimization objective to perform gradient descent updates on the network parameters of the detection head, thereby enabling cross-frame aggregated features to be gradually constrained and achieving intermodal calibration during the training process. After training, the detection head can perform constraint correction on the input cross-frame aggregated features during the inference phase, output calibrated multimodal features, and fuse the calibrated multimodal features through cross-layer residual connections to output the first target box in consecutive frames. After outputting the first target box in the consecutive frames, RoI features are extracted from the fused feature map according to the target box position. The features of the corresponding region of the target box are segmented, and after pooling and vectorization, the frame-level feature vector is obtained. The frame-level feature vector can characterize the appearance features of the target in the current frame and provide input for subsequent cross-modal consistency constraints and uncertainty estimation. It should be noted that the improved YOLOv7 model is a detection model enhanced on the YOLOv7 basic structure, and its main improvements are: Temperature consistency constraints and temporal attention mechanisms are introduced into the feature fusion layer, enabling the network to calibrate cross-modal features while extracting multi-scale features, and to model the temporal dependencies between consecutive frames.

[0037] Unlike the original YOLOv7 model, which only performs target detection on a spatial scale, the improved model of this invention can output target boxes with greater temporal stability and cross-modal consistency, thereby improving the accuracy and robustness of target recognition in infrared and visible light fusion scenarios.

[0038] It should be noted that the detection head is the output structure of an improved YOLOv7 model, comprising two parts: a regression branch and a classification branch. The regression branch is used to predict the coordinate parameters of the target box, and the classification branch is used to predict the probability distribution of the target class. In this embodiment, cross-frame aggregated features are input into the detection head to obtain the regression results and softmax classification results of the candidate target boxes.

[0039] S4, based on frame-level feature vectors, performs cross-modal feature fusion optimization to obtain joint optimized features, specifically: The feature vectors of the first target box in the infrared mode and the visible light mode are extracted to form cross-modal feature pairs; Calculate the similarity index of cross-modal feature pairs and generate constraint weights that reflect modal consistency; Based on the constraint weights, feature alignment processing is performed on cross-modal feature pairs to obtain cross-modal aligned feature representations; Cross-modal alignment feature representations are fused with temporal features of the target bounding box to generate joint optimized features.

[0040] Specifically: For the first target bounding box detected in consecutive frames, the corresponding region features are extracted from the infrared modal feature map and the visible light modal feature map, respectively, and then the infrared modal feature vector is obtained through pooling and vectorization. With visible light mode eigenvectors The two constitute a cross-modal feature pair:

[0041] Based on the aforementioned cross-modal feature pairs, a similarity index, such as cosine similarity, is calculated between them:

[0042] Modal consistency constraint weights are generated based on similarity levels. :

[0043] in, This is the scaling factor. This is the Sigmoid function, used to normalize the weights to the (0,1) interval; Using the constraint weights Weighted alignment of infrared and visible light mode features:

[0044] This generates cross-modal aligned feature representations. This makes the feature distributions of the two modes tend to be consistent; Representing cross-modal alignment features Temporal features of the target bounding box To achieve fusion, for example, by using weighted splicing:

[0045] in, This represents aggregation operations on temporal feature sequences, such as attention-weighted or average pooling. The fusion weighting coefficient can be a preset constant, and its value can be determined through experiments; The target bounding box temporal features It is a frame-level feature vector extracted based on S3, which is obtained by weighted aggregation of the features of the same target in consecutive frames through a temporal attention mechanism, and is used to characterize the dynamic changes of the target over time. The final result This is the joint optimization feature.

[0046] S5, using joint optimization features to perform confidence correction on the first target box, to obtain the second target box, specifically: The prediction uncertainty of the first target box is estimated based on joint optimization features. The estimation of the prediction uncertainty includes: calculating the probability distribution variance and information entropy of the prediction result of the first target box. A confidence correction coefficient is generated based on the prediction uncertainty. The correction coefficient is determined by comparing the uncertainty measurement result with a preset threshold. The original confidence level of the first target box is adaptively adjusted using correction coefficients to obtain the second target box.

[0047] Specifically: In this embodiment, the prediction uncertainty of the first target box based on joint optimization feature estimation specifically includes: The joint optimization features are input into the multi-path computation structure to obtain the preliminary probability distribution of each path; Uncertainty-weighted fusion is performed on the preliminary probability distribution to obtain the fused probability distribution; The fusion probability distribution is perturbed and smoothed to generate the predicted probability distribution of the first target box.

[0048] Specifically: By inputting the joint optimization features into the multi-path computational structure of the uncertainty estimation algorithm, different paths differ in parameter initialization, regularization, or random deactivation strategies, thus enabling the output of multiple sets of preliminary class probability distributions under the same input. A reliability assessment is performed on the preliminary probability distribution of each channel's output, for example, by judging its stability based on the magnitude of variance or the level of entropy. For pathways with high stability, a greater fusion weight is assigned; For pathways with high uncertainty, assign smaller weights; Based on the fusion results, further perturbation sampling and multiple smoothing operations are performed: Slight random perturbations are introduced during the calculation of the probability distribution, and the results of multiple samplings are averaged or smoothed. After the above steps, the resulting predicted probability distribution can be used as the final probability representation of the first target box, providing a basis for subsequent uncertainty measurement and confidence dynamic correction.

[0049] After obtaining the predicted probability distribution, its variance and entropy indices are further calculated, and the uncertainty measurement results of the first target box are generated accordingly. The uncertainty measurement result can be represented by a weighted combination of variance and entropy or a comprehensive index, or any one of these indicators can be used alone. Those skilled in the art can set it according to application requirements. Variance measures the consistency among different predicted samples, while entropy measures the concentration of the probability distribution. Larger variance and entropy values ​​indicate greater uncertainty in the model's judgment of the bounding box.

[0050] The result of this uncertainty measurement will serve as the basis for subsequent corrections; The uncertainty measurement result is compared with a preset threshold: If the value exceeds the threshold, a lower correction coefficient is generated; if the value is below the threshold, a higher correction coefficient is generated, thus reflecting the reliability of the target box prediction. The confidence level of the first target box is dynamically adjusted based on the correction coefficient, and the adjusted confidence level is assigned to the first target box to obtain the second target box.

[0051] The second target bounding box not only contains location and category information, but also confidence level after dynamic uncertainty correction, which serves as the basis for the final target recognition result; The dynamic adjustment can employ multiplicative scaling, linear correction, or other equivalent methods to apply the correction coefficient to the original confidence level, thereby obtaining the updated confidence level. The specific confidence correction process is as follows: Figure 3 As shown; It should be noted that the uncertainty estimation algorithm can be implemented based on Monte Carlo dropout, deep ensemble, or Bayesian approximation methods. What they have in common is that they estimate the variance and entropy of the output distribution by introducing multiple random inferences or multi-model structures, thereby measuring the prediction uncertainty of the target box.

[0052] S6, based on the second target bounding box, outputs the target recognition result, specifically: The spatial parameters of the second bounding box are used to represent the location of the target in the image. The output of the classification branch is used to determine the target category label. The confidence level after uncertainty estimation correction is used as the confidence level of the category. The target recognition results include the location information of the target box, the category label, and the corresponding confidence level, which are used to characterize the detection status of power transmission equipment in consecutive frame images.

[0053] Example 2, Figure 2 This invention discloses an infrared image-based power transmission equipment target recognition system based on YOLOv7, comprising: The image acquisition and preprocessing module is used to acquire continuous frame infrared and visible light images of the power transmission equipment and perform preprocessing. The multimodal tensor construction module is used to construct a spatiotemporal multimodal input tensor based on the processed consecutive frames of infrared and visible light images; The feature fusion detection module is used to process the input tensor through the improved YOLOv7 model and output the first target box and the corresponding frame-level feature vector. The cross-modal optimization module is used to perform cross-modal feature fusion optimization based on frame-level feature vectors to obtain joint optimized features; The confidence correction module is used to perform confidence correction on the first target box using joint optimization features to obtain the second target box; The results output module is used to output the target recognition results based on the second target bounding box.

[0054] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0055] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application of the technical solution and the design electrical constraints. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0056] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0057] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0058] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A YOLOv7-based infrared image power transmission equipment target identification method and system, characterized in that, The method comprises the following steps: obtaining continuous frames of infrared images and visible light images of power transmission equipment and performing preprocessing; based on the processed continuous frames of infrared images and visible light images, constructing a spatio-temporal multi-modal input tensor; processing the input tensor through an improved YOLOv7 model to output a first target box and a corresponding frame-level feature vector; based on the frame-level feature vector, performing cross-modal feature fusion optimization to obtain a joint optimization feature; using the joint optimization feature to correct the confidence of the first target box to obtain a second target box; outputting a target recognition result based on the second target box.

2. The YOLOv7-based infrared image power transmission equipment target identification method according to claim 1, characterized in that, The preprocessing comprises: performing histogram equalization on the infrared image, and performing noise filtering and edge enhancement processing on the equalized infrared image; performing contrast enhancement and color normalization processing on the visible light image.

3. The YOLOv7-based infrared image power transmission equipment target identification method according to claim 2, characterized in that, Based on the processed continuous frames of infrared images and visible light images, the spatio-temporal multi-modal input tensor is constructed, specifically: performing spatial alignment processing on the continuously collected infrared images and visible light images to obtain a registered image sequence; stacking the registered image sequence along the channel dimension and the time dimension to generate an input tensor containing spatio-temporal multi-modal information.

4. The YOLOv7-based infrared image power transmission device target identification method according to claim 3, characterized in that, The spatial alignment processing on the continuously collected infrared images and visible light images comprises: extracting candidate feature points of the infrared image and the visible light image, and establishing an initial matching relationship; applying a modal mutual information maximization constraint to the initial matching relationship to generate an optimized matching result; calculating image transformation parameters based on the optimized matching result; performing geometric correction on the infrared image according to the image transformation parameters to generate a registered image pair sequence that is spatially aligned with the visible light image.

5. The YOLOv7-based infrared image power transmission device target identification method according to claim 4, characterized in that, The processing of the input tensor through the improved YOLOv7 model comprises: extracting multi-scale spatio-temporal features of the spatio-temporal multi-modal input tensor; performing cross-frame aggregation on the multi-scale spatio-temporal features through a time-series enhanced feature pyramid network to generate cross-frame aggregated features, the time-series enhanced feature pyramid network realizes weighted aggregation of continuous frame features by introducing a time-series attention mechanism; calibrating the cross-frame aggregated features based on a temperature consistency constraint to obtain calibrated multi-modal features; fusing the calibrated multi-modal features through cross-layer residual connection to output the first target box.

6. The YOLOv7-based infrared image power transmission device target identification method according to claim 5, characterized in that, The calibration of the cross-frame aggregated features based on the temperature consistency constraint to obtain the calibrated multi-modal features specifically comprises: generating a preliminary result of the first target box based on the time-series fusion features, the preliminary result including target position information and class confidence, the preliminary result being obtained through a detection head, the detection head including a regression branch and a classification branch; performing confidence calibration processing on the class confidence to obtain a calibrated class probability distribution; based on the target position information and the calibrated class probability distribution, performing joint optimization on the cross-frame aggregated features through a multi-modal consistency constraint to obtain the calibrated multi-modal features.

7. The YOLOv7-based infrared image power transmission device target recognition method according to claim 6, characterized in that, The cross-modal feature fusion optimization based on the frame-level feature vector to obtain the joint optimization feature specifically comprises: extracting feature vectors of the first target box under infrared and visible light modalities to form a cross-modal feature pair; A similarity index of the cross-modal feature pair is calculated to generate a constraint weight reflecting modality consistency; Based on the constraint weight, the cross-modal feature pair is processed for feature alignment to obtain a cross-modal aligned feature representation; The cross-modal aligned feature representation is fused with the target box time sequence feature to generate a joint optimization feature.

8. The YOLOv7-based infrared image power transmission device target identification method according to claim 7, characterized in that, The joint optimization feature is used to correct the confidence of the first target box, specifically: The prediction uncertainty of the first target box is estimated based on the joint optimization feature, and the estimation of the prediction uncertainty includes: calculating the probability distribution variance and information entropy of the first target box prediction result; A confidence correction coefficient is generated based on the prediction uncertainty, and the correction coefficient is determined based on the comparison result by comparing the uncertainty measure result with a preset threshold; The original confidence of the first target box is adaptively adjusted using the correction coefficient to obtain a second target box.

9. The YOLOv7-based infrared image power transmission device target identification method according to claim 8, characterized in that, The prediction uncertainty of the first target box is estimated based on the joint optimization feature, and the estimation of the prediction uncertainty includes: The joint optimization feature is input into a multi-path calculation structure to obtain a preliminary probability distribution of each path; The preliminary probability distribution is subjected to uncertainty weighted fusion to obtain a fused probability distribution; The fused probability distribution is subjected to perturbation sampling and smoothing processing to generate a prediction probability distribution of the first target box.

10. A system using the YOLOv7-based infrared image power transmission equipment target identification method according to any one of claims 1-9, comprising: An image acquisition and preprocessing module for acquiring continuous frame infrared images and visible light images of power transmission equipment and performing preprocessing; A multi-modal tensor construction module for constructing a spatio-temporal multi-modal input tensor based on the processed continuous frame infrared images and visible light images; A feature fusion detection module for processing the input tensor through an improved YOLOv7 model to output a first target box and a corresponding frame-level feature vector; A cross-modal optimization module for performing cross-modal feature fusion optimization based on the frame-level feature vector to obtain a joint optimization feature; A confidence correction module for correcting the confidence of the first target box using the joint optimization feature to obtain a second target box; A result output module for outputting a target identification result based on the second target box.

Images