A method, system, and electronic device for identifying small-sample dual-light temperature faults based on improved meta-learning

Abstract

This application provides a few-sample dual-photothermal fault recognition algorithm based on improved meta-learning. By optimizing the meta-learning framework and combining it with the YOLOv7 deep learning algorithm, it enables training and testing of the target under limited sample conditions, solving the target recognition problem under small sample conditions. An improved non-subsampled shear wave transform algorithm is used to fuse visible light and infrared images, improving detection accuracy and precision. Temperature values ​​in the target recognition result area are automatically extracted and compared with their corresponding temperature fault thresholds to determine the presence of a fault, improving detection efficiency, avoiding human error, and reducing the difficulty of the detection operation.

Description

Technical Field

[0001] This application relates to the field of optical detection technology, and in particular to a small-sample dual-light temperature fault identification method, system and electronic equipment based on improved meta-learning. Background Technology

[0002] Conventional deep learning-based image recognition algorithms can automatically learn image features from massive amounts of data, exhibiting strong feature representation capabilities and high detection accuracy. However, in actual equipment maintenance operations, acquiring a large number of training images is extremely time-consuming and labor-intensive, resulting in high time and economic costs. Furthermore, the training accuracy of the model decreases when new categories of samples are introduced. Currently, thermal fault detection in equipment mainly relies on infrared thermal imagers to collect infrared image data, with temperature data manually read and the presence of thermal faults determined based on the detection results. This approach is prone to misjudgment, and infrared images have low resolution and lack texture and detail information, making them unsuitable for accurately locating faults. Summary of the Invention

[0003] This application provides a few-sample dual-light temperature fault identification method based on improved meta-learning to improve the above-mentioned problems.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, this application proposes a few-sample dual-light temperature fault identification method based on improved meta-learning, the method comprising:

[0006] Obtain the dataset and expand it;

[0007] The model is built based on the dataset. The process includes a target detection algorithm based on the improved MAML meta-learning framework and YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset.

[0008] The original visible light image and the original infrared image to be fused are acquired, and the original visible light image and the original infrared image are registered using the feature point matching method. The two images are fused using the non-subsampled shear wave transform algorithm to obtain a dual-light fused image. The target is identified in the original visible light image based on the target detection model. According to the output of the target detection model, the coordinates of the detection box at the upper left and lower right corners are obtained, and the detection box coordinates are mapped to the original infrared image to obtain the corresponding temperature box coordinates.

[0009] The maximum temperature of each pixel in the temperature frame is obtained, and the maximum temperature of each pixel is compared with the temperature fault threshold of the detected target, that is, the value after a fault occurs. Based on the comparison result, it is determined whether a thermal fault has occurred.

[0010] In conjunction with the first aspect, in some implementations, a dataset is acquired and expanded, including:

[0011] The image dataset is augmented using the Mosaic data augmentation method. Four images are randomly selected from the dataset, each image is scaled and randomly cropped, and then stitched together to generate a new training image.

[0012] Multiple training images were used as the augmented dataset.

[0013] In conjunction with the first aspect, in some implementations, an object detection model is built based on a dataset. The building process includes an object detection algorithm based on a YOLOv7 deep learning algorithm combined with an improved MAML meta-learning framework, performing meta-training and meta-testing on a visible light image dataset, including:

[0014] The dataset is divided into a base class dataset and a small sample new class dataset;

[0015] Initialize the parameters of the meta-learning model optimizer θ ;

[0016] Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Train the model parameters;

[0017] Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ;

[0018] The original visible light image and the original infrared image are decomposed into three parts using the NSST algorithm (Non-Subsampled Shear Wave Transform) to obtain the low-frequency subband system of the infrared image. Low-frequency subband system of visible light images High-frequency subband system of infrared images High-frequency subband system of visible light images ;

[0019] The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

[0020] In conjunction with the first aspect, in some implementations, the base class dataset is constructed. I d Support set data for individual training tasks DS i and query set data DQ i Training the model parameters includes:

[0021] For each support set task DS i Calculation loop j Adaptive parameters for subsequent gradient descent ,satisfy:

[0022]

[0023]

[0024]

[0025] in, Indicates the first i Task iterations J D The parameters after the last update α The learning rate parameter for the inner loop. J D To update the step count, j This is a loop variable, taking values ​​from 0 to... J D -1, Indicates the first i The first task j Model parameters for the next iteration It is determined by parameters The network model that is composed of for i Training task support set DS i loss function, loss function Regarding parameters The gradient; calculate the query set of all training tasks. DQ i In their respective The loss function is summed, and then the model parameters are updated to obtain the parameters. :

[0026]

[0027] in, The learning rate parameter for the outer loop. For the updated optimizer parameters, forI d Group training task query set DQi The loss function and, Represents query set data DQi The loss function and the initial parameters θ Gradient in the direction.

[0028] In conjunction with the first aspect, in some implementations, a support set for a meta-test task is constructed using a small sample dataset of new classes. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ,include:

[0029]

[0030]

[0031] in, This represents the updated model parameters after testing the set. α The learning rate parameter for the inner loop. To consider the loss function for domain adaptation, To support the collection TS loss function, It is determined by parameters The network model that is composed of To account for the equilibrium parameters of domain adaptation, The number of samples in the base class data. For the number of samples in the new class of data with a small sample size, Data samples for the base class data, This is a data sample for a small sample of new types of data. For sample data Feature representation after feature extraction Let it be the probability of the corresponding discriminator. For sample data Feature representation after feature extraction The probability of the corresponding discriminator.

[0032] In conjunction with the first aspect, in some implementations, the high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0033]

[0034]

[0035]

[0036] in, To fuse the low-frequency subband coefficients of the image, The local average gradient of the original infrared image. The local average gradient of the original visible light image. For pixels ( i, j The sub-band coefficient value.

[0037] In conjunction with the first aspect, in some implementations, the high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0038]

[0039]

[0040]

[0041] In the formula To fuse the high-frequency subband coefficients of the image, The sum of the local window Laplacian values ​​of the original infrared image, The sum of the local window Laplacian values ​​of the original visible light image, For pixels ( i, j The Laplace value of ) For pixels ( i, j The sub-band coefficient values ​​of ), where w is the weight coefficient matrix.

[0042] In conjunction with the first aspect, in some implementations, target recognition is performed on the original visible light image based on a target detection model. Based on the output of the target detection model, the coordinates of the top-left and bottom-right detection boxes are obtained, and these coordinates are mapped onto the original infrared image to obtain the corresponding temperature box coordinates, satisfying the following:

[0043]

[0044] in, These are the coordinate values ​​from the original infrared image. These are the coordinate values ​​in the original visible light image. These are the coefficients of the homography transformation matrix.

[0045] Secondly, this application proposes a few-sample dual-light temperature fault identification system based on improved meta-learning, which is configured as follows:

[0046] Obtain the dataset and expand it;

[0047] The model is built based on the dataset. The process includes a target detection algorithm based on the improved MAML meta-learning framework and YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset.

[0048] The original visible light image and the original infrared image to be fused are acquired, and the original visible light image and the original infrared image are registered using the feature point matching method.

[0049] The two images are fused using a non-subsampled shear wave transform algorithm to obtain a dual-light fused image. The original visible light image is then used to identify targets based on a target detection model. The coordinates of the detection boxes at the top left and bottom right corners are obtained based on the output of the target detection model. These detection box coordinates are then mapped to the original infrared image to obtain the corresponding temperature box coordinates.

[0050] The maximum temperature of each pixel in the temperature frame is obtained, and the maximum temperature of each pixel is compared with the temperature fault threshold of the target. Based on the comparison result, it is determined whether a thermal fault has occurred.

[0051] In conjunction with the first aspect, in some implementations, a dataset is acquired and expanded, including:

[0052] The image dataset is augmented using the Mosaic data augmentation method. Four images are randomly selected from the dataset, each image is scaled and randomly cropped, and then stitched together to generate a new training image.

[0053] Multiple training images were used as the augmented dataset.

[0054] In conjunction with the second aspect, in some implementations, the system is configured as follows:

[0055] A target detection model is built based on the dataset. The building process includes a target detection algorithm based on the improved MAML meta-learning framework combined with the YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset, including:

[0056] The dataset is divided into a base class dataset and a small sample new class dataset;

[0057] Initialize the parameters of the meta-learning model optimizer θ ;

[0058] Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Train the model parameters;

[0059] Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ;

[0060] The NSST algorithm is used to perform three decompositions on the original visible light image and the original infrared image to obtain the low-frequency subband system of the infrared image. Low-frequency subband system of visible light images High-frequency subband system of infrared images High-frequency subband system of visible light images ;

[0061] The low-frequency sub-band images are fused using a local average gradient weighting method, and the fused low-frequency sub-band coefficients are... and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

[0062] In conjunction with the first aspect, in some implementations, the base class dataset is constructed. I d Support set data for individual training tasks DS i and query set data DQ i Training the model parameters includes:

[0063] For each support set task DS i Calculation loop j Adaptive parameters for subsequent gradient descent ,satisfy:

[0064]

[0065]

[0066]

[0067] in, Indicates the first i Task iterations J D The parameters after the last updateα The learning rate parameter for the inner loop. J D To update the step count, j This is a loop variable, taking values ​​from 0 to... J D -1, Indicates the first i The first task j Model parameters for the next iteration It is determined by parameters The network model that is composed of for i Training task support set DS i loss function, For loss function Regarding parameters The gradient; calculate the query set of all training tasks. DQ i In their respective The loss function is summed, and then the model parameters are updated to obtain the parameters. :

[0068]

[0069] in, The learning rate parameter for the outer loop. For the updated optimizer parameters, for I d Group training task query set DQi The loss function and, Represents query set data DQi The loss function and the initial parameters θ Gradient in the direction.

[0070] In conjunction with the second aspect, in some implementations, the system is configured as follows:

[0071] Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ,include:

[0072]

[0073]

[0074] in, This represents the updated model parameters after testing the set. α The learning rate parameter for the inner loop. To consider the loss function for domain adaptation, To support the collection TS loss function, It is determined by parameters The network model that is composed of To account for the equilibrium parameters of domain adaptation, The number of samples in the base class data. For the number of samples in the new class of data with a small sample size, Data samples for the base class data, This is a data sample for a small sample of new types of data. For sample data Feature representation after feature extraction The probability of its corresponding discriminator. For sample data Feature representation after feature extraction The probability of the corresponding discriminator.

[0075] In conjunction with the second aspect, in some implementations, the system is configured as follows:

[0076] The low-frequency sub-band images are fused using a local average gradient weighting method, and the fused low-frequency sub-band coefficients are... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0077]

[0078]

[0079]

[0080] in, To fuse the low-frequency subband coefficients of the image, The local average gradient of the original infrared image. The local average gradient of the original visible light image. For pixels ( i, j The sub-band coefficient value.

[0081] In conjunction with the second aspect, in some implementations, the system is configured as follows:

[0082] The low-frequency sub-band images are fused using a local average gradient weighting method, and the fused low-frequency sub-band coefficients are... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0083]

[0084]

[0085]

[0086] In the formula To fuse the high-frequency subband coefficients of the image, The sum of the local window Laplacian values ​​of the original infrared image, The sum of the local window Laplacian values ​​of the original visible light image, For pixels ( i, j The Laplace value of ) For pixels ( i, j The sub-band coefficient values ​​of ), where w is the weight coefficient matrix.

[0087] In conjunction with the second aspect, in some implementations, the system is configured as follows:

[0088] Target recognition is performed on the original visible light image based on the target detection model. The coordinates of the top-left and bottom-right detection boxes are obtained from the model's output. These coordinates are then mapped onto the original infrared image to obtain the corresponding temperature box coordinates, satisfying the following conditions:

[0089]

[0090] in, These are the coordinate values ​​from the original infrared image. These are the coordinate values ​​in the original visible light image. These are the coefficients of the homography transformation matrix.

[0091] A third aspect of this invention provides an electronic device, which includes:

[0092] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method proposed in the first aspect of the present invention.

[0093] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect of the present invention.

[0094] In summary, the above method has the following technical effects:

[0095] This invention proposes a small-sample dual-photothermal fault identification method based on improved meta-learning. By optimizing the meta-learning framework and combining it with the YOLOv7 deep learning algorithm, it enables training and testing of the target under limited sample conditions, solving the target identification problem under small sample conditions. An improved non-subsampled shear wave transform algorithm is used to fuse visible light and infrared images, improving detection accuracy and precision. Temperature values ​​in the target identification result area are automatically extracted and compared with their corresponding temperature fault thresholds to determine the presence of a fault, improving detection efficiency, avoiding human error, and reducing the difficulty of the detection operation. Attached Figure Description

[0096] Figure 1 This is a flowchart illustrating a few-sample dual-light temperature fault identification method based on improved meta-learning proposed in this application. Detailed Implementation

[0097] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0098] This application proposes a few-sample dual-light temperature fault identification method based on improved meta-learning. Please refer to [link to relevant documentation]. Figure 1 This includes the following steps:

[0099] S101: Obtain the dataset and expand it.

[0100] Specifically, in this embodiment, the Mosaic data augmentation method can be used to expand the image dataset. Four images are randomly selected from the dataset, and each image is scaled and randomly cropped before being stitched together to generate a new training image. Then, multiple training images are used as the expanded dataset.

[0101] As is understandable, Mosaic data augmentation is a data augmentation technique used for image datasets, particularly common in object detection tasks. It creates new synthetic images by combining multiple images. In this embodiment, four images are randomly selected from the original dataset; then, each image is scaled (resized) and randomly cropped (removing a portion of the image); these processed images are then stitched together to form a new synthetic image; finally, this process is repeated to generate multiple such new images, which are then added to the original dataset to form the augmented dataset. This method aims to increase data diversity, helping machine learning models (such as object detection models) generalize better during training and avoiding overfitting.

[0102] S102: The two images are fused using a non-subsampled shear wave transform algorithm to obtain a dual-light fused image. A target detection model is then established based on the dataset. The establishment process includes a target detection algorithm based on the YOLOv7 deep learning algorithm combined with the improved MAML meta-learning framework, and meta-training and meta-testing are performed on the visible light image dataset.

[0103] Understandably, the object detection model was built using an improved MAML (Model-Agnostic Meta-Learning) framework, combined with the advanced deep learning algorithm YOLOv7, to achieve meta-training and meta-testing on a visible light image dataset. This process not only involved optimizing existing techniques but also fine-tuning algorithmic details to ensure the model could efficiently and accurately identify and locate targets in images.

[0104] Specifically, as one implementation method, step S102 may include the following steps:

[0105] S1021: Divide the dataset into a base class dataset and a small sample new class dataset.

[0106] S1022: Initialize meta-learning model optimizer parameters θ.

[0107] S1023: Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i The model parameters are trained.

[0108] Specifically, for each support set task DS i Calculation loop j Adaptive parameters for subsequent gradient descent ,satisfy:

[0109]

[0110]

[0111]

[0112] in, Indicates the first i Task iterations J D The parameters after the last update α The learning rate parameter for the inner loop. J D To update the step count, j This is a loop variable, taking values ​​from 0 to... J D -1, Indicates the first i The first task j Model parameters for the next iteration It is determined by parameters The network model that is composed of for i Training task support set DS i loss function, loss function Regarding parameters The gradient; calculate the query set of all training tasks. DQ i In their respective The loss function is summed, and then the model parameters are updated to obtain the parameters. :

[0113]

[0114] in, The learning rate parameter for the outer loop. For the updated optimizer parameters, for I d Group training task query set DQi The loss function and, Represents query set data DQi The loss function and the initial parameters θ Gradient in the direction.

[0115] S1024: Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. .

[0116] Specifically:

[0117]

[0118]

[0119] in, This represents the updated model parameters after testing the set. α The learning rate parameter for the inner loop. To consider the loss function for domain adaptation, To support the collection TS loss function, It is determined by parameters The network model that is composed of To account for the equilibrium parameters of domain adaptation, The number of samples in the base class data. For the number of samples in the new class of data with a small sample size, Data samples for the base class data, This is a data sample for a small sample of new types of data. For sample data Feature representation after feature extraction The probability of its corresponding discriminator. For sample data Feature representation after feature extraction The probability of the corresponding discriminator.

[0120] S1025: The NSST algorithm is used to perform three decompositions on the original visible light image and the original infrared image to obtain the low-frequency subband system of the infrared image. Low-frequency subband system of visible light images High-frequency subband system of infrared images High-frequency subband system of visible light images .

[0121] Specifically,

[0122]

[0123] in, To fuse the low-frequency subband coefficients of the image, The local average gradient of the original infrared image. The local average gradient of the original visible light image. For pixels ( i, j The sub-band coefficient value.

[0124] S1026: Fuse the high-frequency sub-band images according to spatial frequency weighting, and fuse the low-frequency sub-band images according to local average gradient weighting. Then fuse the fused low-frequency sub-band coefficients. and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

[0125] Specifically,

[0126]

[0127]

[0128]

[0129] in, To fuse the high-frequency subband coefficients of the image, The sum of the Laplacian values ​​of the local window of the original infrared image. The sum of the local window Laplacian values ​​of the original visible light image, For pixels ( i, j The Laplace value of ) For pixels ( i, j The sub-band coefficient values ​​of ), where w is the weight coefficient matrix.

[0130] S103: Acquire the original visible light image and the original infrared image to be fused, and register the original visible light image and the original infrared image using the feature point matching method.

[0131] Specifically, the first step is to acquire the original visible light image and the original infrared image to be fused. After acquisition, an efficient feature point matching method will be used to accurately register these original visible light and infrared images. This method ensures that the two different types of images maintain a high degree of consistency and accuracy during the fusion process, thus providing a solid foundation for subsequent image processing and analysis.

[0132] S104: Based on the target detection model, target recognition is performed on the original visible light image. According to the output of the target detection model, the coordinates of the detection boxes at the upper left and lower right corners are obtained, and the detection box coordinates are mapped to the original infrared image to obtain the corresponding temperature box coordinates.

[0133] Specifically, it satisfies:

[0134]

[0135] in, These are the coordinate values ​​from the original infrared image. These are the coordinate values ​​in the original visible light image. These are the coefficients of the homography transformation matrix.

[0136] S105: Obtain the maximum temperature of each pixel in the temperature frame, compare the maximum temperature of each pixel with the temperature fault threshold of the detected target, and determine whether a thermal fault has occurred based on the comparison result.

[0137] Understandably, the maximum temperature of each pixel in the temperature frame is extracted and compared with the temperature fault threshold of the target. If the value is greater than or equal to the threshold, a thermal fault is determined to have occurred; if the value is less than the threshold, no thermal fault has occurred.

[0138] This invention proposes a small-sample dual-photothermal fault identification method based on improved meta-learning. By optimizing the meta-learning framework and combining it with the YOLOv7 deep learning algorithm, it enables training and testing of the target under limited sample conditions, solving the target identification problem under small sample conditions. An improved non-subsampled shear wave transform algorithm is used to fuse visible light and infrared images, improving detection accuracy and precision. Temperature values ​​in the target identification result area are automatically extracted and compared with their corresponding temperature fault thresholds to determine the presence of a fault, improving detection efficiency, avoiding human error, and reducing the difficulty of the detection operation.

[0139] Based on the same inventive concept, this application also proposes a small-sample dual-light temperature fault identification system based on improved meta-learning, which is configured as follows:

[0140] Obtain the dataset and expand it;

[0141] The model is built based on the dataset. The process includes a target detection algorithm based on the improved MAML meta-learning framework and YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset.

[0142] The original visible light image and the original infrared image to be fused are acquired, and the original visible light image and the original infrared image are registered using the feature point matching method.

[0143] The two images are fused using a non-subsampled shear wave transform algorithm to obtain a dual-light fused image. The original visible light image is then used to identify targets based on a target detection model. The coordinates of the detection boxes at the top left and bottom right corners are obtained based on the output of the target detection model. These detection box coordinates are then mapped to the original infrared image to obtain the corresponding temperature box coordinates.

[0144] The maximum temperature of each pixel in the temperature frame is obtained, and the maximum temperature of each pixel is compared with the temperature fault threshold of the target. Based on the comparison result, it is determined whether a thermal fault has occurred.

[0145] In some implementations, acquiring a dataset and expanding the dataset includes:

[0146] The image dataset is augmented using the Mosaic data augmentation method. Four images are randomly selected from the dataset, each image is scaled and randomly cropped, and then stitched together to generate a new training image.

[0147] Multiple training images were used as the augmented dataset.

[0148] In some implementations, the system is configured as follows:

[0149] A target detection model is built based on the dataset. The building process includes a target detection algorithm based on the improved MAML meta-learning framework combined with the YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset, including:

[0150] The dataset is divided into a base class dataset and a small sample new class dataset;

[0151] Initialize the parameters of the meta-learning model optimizer θ ;

[0152] Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Train the model parameters;

[0153] Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ;

[0154] The NSST algorithm is used to perform three decompositions on the original visible light image and the original infrared image to obtain the low-frequency subband system of the infrared image. Low-frequency subband system of visible light images High-frequency subband system of infrared images High-frequency subband system of visible light images ;

[0155] The low-frequency sub-band images are fused using a local average gradient weighting method, and the fused low-frequency sub-band coefficients are... and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

[0156] Construct the base class datasetI d Support set data for individual training tasks DS i and query set data DQ i Training the model parameters includes:

[0157] For each support set task DS i Calculation loop j Adaptive parameters for subsequent gradient descent ,satisfy:

[0158]

[0159]

[0160]

[0161] in, Indicates the first i Task iterations J D The parameters after the last update α The learning rate parameter for the inner loop. J D To update the step count, j This is a loop variable, taking values ​​from 0 to... J D -1, Indicates the first i The first task j Model parameters for the next iteration It is determined by parameters The network model that is composed of for i Training task support set DS i loss function, loss function Regarding parameters The gradient; calculate the query set of all training tasks. DQ i In their respective The loss function is summed, and then the model parameters are updated to obtain the parameters. :

[0162]

[0163] in, The learning rate parameter for the outer loop. For the updated optimizer parameters, for I dGroup training task query set DQi The loss function and, Represents query set data DQi The loss function and the initial parameters θ Gradient in the direction.

[0164] In some implementations, the system is configured as follows:

[0165] Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ,include:

[0166]

[0167]

[0168] in, This represents the updated model parameters after testing the set. α The learning rate parameter for the inner loop. To consider the loss function for domain adaptation, To support the collection TS loss function, It is determined by parameters The network model that is composed of To account for the equilibrium parameters of domain adaptation, The number of samples in the base class data. For the number of samples in the new class of data with a small sample size, Data samples for the base class data, This is a data sample for a small sample of new types of data. For sample data Feature representation after feature extraction The probability of its corresponding discriminator. For sample data Feature representation after feature extraction The probability of the corresponding discriminator.

[0169] In some implementations, the system is configured as follows:

[0170] The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0171]

[0172]

[0173]

[0174] in, To fuse the low-frequency subband coefficients of the image, The local average gradient of the original infrared image. The local average gradient of the original visible light image. For pixels ( i, j The sub-band coefficient value.

[0175] In some implementations, the system is configured as follows:

[0176] The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Performing the inverse NSST transform yields the final fused image, including:

[0177]

[0178]

[0179]

[0180] In the formula To fuse the low-frequency subband coefficients of the image, The sum of the Laplacian values ​​of the local window of the original infrared image. The sum of the local window Laplacian values ​​of the original visible light image, For pixels ( i, j The Laplace value of ) For pixels ( i, j The sub-band coefficient values ​​of ), where w is the weight coefficient matrix.

[0181] In some implementations, the system is configured as follows:

[0182] Target recognition is performed on the original visible light image based on the target detection model. The coordinates of the top-left and bottom-right detection boxes are obtained from the model's output. These coordinates are then mapped onto the original infrared image to obtain the corresponding temperature box coordinates, satisfying the following conditions:

[0183]

[0184] in, These are the coordinate values ​​from the original infrared image. These are the coordinate values ​​in the original visible light image. These are the coefficients of the homography transformation matrix.

[0185] This invention proposes a small-sample dual-photothermal fault identification system based on improved meta-learning. By optimizing the meta-learning framework and combining it with the YOLOv7 deep learning algorithm, it enables training and testing of the target under limited sample conditions, solving the target identification problem under small sample conditions. An improved non-subsampled shear wave transform algorithm is used to fuse visible light and infrared images, improving detection accuracy and precision. Temperature values ​​in the target identification result area are automatically extracted and compared with their corresponding temperature fault thresholds to determine the presence of a fault, improving detection efficiency, avoiding human error, and reducing the difficulty of the detection operation.

[0186] Based on the same inventive concept, embodiments of this application also propose an electronic device, which includes:

[0187] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the small-sample dual-photothermal fault identification method based on improved meta-learning of the embodiments of this application.

[0188] Furthermore, to achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the small-sample dual-photothermal fault identification method based on improved meta-learning of embodiments of this application.

[0189] The following is a detailed introduction to the various components of the electronic device:

[0190] In this context, the processor is the control center of the electronic device. It can be a single processor or a collective term for multiple processing elements. For example, a processor can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0191] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0192] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can be referred to the above method embodiment, which will not be repeated here.

[0193] Optionally, the memory can be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory can be integrated with the processor or exist independently and coupled to the processor through the interface circuit of the electronic device; the embodiments of the present invention do not specifically limit this.

[0194] A transceiver is used to communicate with network devices or with terminal devices.

[0195] Optionally, the transceiver may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0196] Optionally, the transceiver can be integrated with the processor or exist independently and coupled to the processor through the router's interface circuit. This embodiment of the invention does not specifically limit this.

[0197] Furthermore, the technical effects of the electronic device can be referred to the technical effects of the data transmission method in the above method embodiments, and will not be repeated here.

[0198] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0199] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0200] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0201] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0202] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0203] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0204] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

Claims

1. A few-sample dual-light temperature fault identification method based on improved meta-learning, characterized in that, The method includes: Obtain the dataset and expand the dataset; A target detection model is established based on the dataset. The establishment process includes a target detection algorithm based on the YOLOv7 deep learning algorithm combined with the improved MAML meta-learning framework, and meta-training and meta-testing are performed on the visible light image dataset. Acquire the original visible light image and the original infrared image to be fused, and use the feature point matching method to register the original visible light image and the original infrared image; The two images are fused using a non-subsampled shear wave transform algorithm to obtain a dual-light fused image. Target recognition is then performed on the original visible light image based on the target detection model. According to the output of the target detection model, the coordinates of the detection boxes at the upper left and lower right corners are obtained, and these coordinates are mapped onto the original infrared image to obtain the corresponding temperature box coordinates. The maximum temperature of each pixel in the temperature frame is obtained, and the maximum temperature of each pixel is compared with the temperature fault threshold of the detected target. Based on the comparison result, it is determined whether a thermal fault has occurred. Obtain the dataset and expand the dataset, including: The image dataset is augmented using the Mosaic data augmentation method. Four images are randomly selected from the dataset, and each image is scaled, randomly cropped, and then stitched together to generate a new training image. The training images are used as the augmented dataset. An object detection model is established based on the dataset. The establishment process includes an object detection algorithm based on the improved MAML meta-learning framework combined with the YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset, including: The dataset is divided into a base class dataset and a small sample new class dataset; Initialize the parameters of the meta-learning model optimizer θ ; Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Train the model parameters; Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ; The NSST algorithm is used to perform a three-stage decomposition on the original visible light image and the original infrared image to obtain the low-frequency subband coefficients of the infrared image. Low-frequency subband coefficients of visible light images High-frequency subband coefficients of infrared images High-frequency subband coefficients of visible light images ; The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

2. The method for identifying small-sample dual-light temperature faults based on improved meta-learning according to claim 1, characterized in that, Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Training the model parameters includes: For each support set task DS i Calculation loop j Adaptive parameters for subsequent gradient descent ,satisfy: in, Indicates the first i Task iterations J D The parameters after the last update α The learning rate parameter for the inner loop. J D To update the step count, j This is a loop variable, taking values ​​from 0 to... J D -1, Indicates the first i The first task j Model parameters for the next iteration It is determined by parameters The network model that is composed of for i Training task support set DS i loss function, For loss function Regarding parameters The gradient; calculate the query set of all training tasks. DQ i In their respective The loss function is summed, and then the model parameters are updated to obtain the parameters. : in, The learning rate parameter for the outer loop. For the updated optimizer parameters, for I d Group training task query set DQi The loss function and, Represents query set data DQi The loss function and the initial parameters θ Gradient in the direction.

3. The method for identifying small-sample dual-light temperature faults based on improved meta-learning according to claim 2, characterized in that, Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ,include: in, This represents the updated model parameters after testing the set. α The learning rate parameter for the inner loop. To consider the loss function for domain adaptation, To support the collection TS loss function, It is determined by parameters The network model that is composed of To account for the equilibrium parameters of domain adaptation, The number of samples in the base class data. For the number of samples in the new class of data with a small sample size, Data samples for the base class data, This is a data sample for a small sample of new types of data. For sample data Feature representation after feature extraction The probability of its corresponding discriminator. For sample data Feature representation after feature extraction The probability of the corresponding discriminator.

4. The method for identifying small-sample dual-light temperature faults based on improved meta-learning according to claim 3, characterized in that, The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Perform the inverse NSST transform to obtain the final fused image. include: in, To fuse the low-frequency subband coefficients of the image, The local average gradient of the original infrared image. The local average gradient of the original visible light image. For pixels ( i,j The sub-band coefficient value.

5. The method for identifying small-sample dual-light temperature faults based on improved meta-learning according to claim 4, characterized in that, The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient Perform the inverse NSST transform to obtain the final fused image. include: in, To fuse the high-frequency subband coefficients of the image, The sum of the local window Laplacian values ​​of the original infrared image, The sum of the local window Laplacian values ​​of the original visible light image, For pixels ( i,j The Laplace value of ) For pixels ( i,j The sub-band coefficient values ​​of ), where w is the weight coefficient matrix.

6. The method for identifying small-sample dual-light temperature faults based on improved meta-learning according to claim 1, characterized in that, Based on the target detection model, target recognition is performed on the original visible light image. According to the output of the target detection model, the coordinates of the upper left and lower right detection boxes are obtained, and these coordinates are mapped to the original infrared image to obtain the corresponding temperature box coordinates, satisfying the following: in, These are the coordinate values ​​from the original infrared image. These are the coordinate values ​​in the original visible light image. These are the coefficients of the homography transformation matrix.

7. A few-sample dual-light temperature fault identification system based on improved meta-learning, characterized in that, The system is configured as follows: Obtain the dataset and expand the dataset; A target detection model is established based on the dataset. The establishment process includes a target detection algorithm based on the YOLOv7 deep learning algorithm combined with the improved MAML meta-learning framework, and meta-training and meta-testing are performed on the visible light image dataset. Acquire the original visible light image and the original infrared image to be fused, and use the feature point matching method to register the original visible light image and the original infrared image; The two images are fused using a non-subsampled shear wave transform algorithm to obtain a dual-light fused image. Target recognition is then performed on the original visible light image based on the target detection model. According to the output of the target detection model, the coordinates of the detection boxes at the upper left and lower right corners are obtained, and these coordinates are mapped onto the original infrared image to obtain the corresponding temperature box coordinates. The maximum temperature of each pixel in the temperature frame is obtained, and the maximum temperature of each pixel is compared with the temperature fault threshold of the detected target. Based on the comparison result, it is determined whether a thermal fault has occurred. Obtain the dataset and expand the dataset, including: The image dataset is augmented using the Mosaic data augmentation method. Four images are randomly selected from the dataset, and each image is scaled, randomly cropped, and then stitched together to generate a new training image. The training images are used as the augmented dataset. An object detection model is established based on the dataset. The establishment process includes an object detection algorithm based on the improved MAML meta-learning framework combined with the YOLOv7 deep learning algorithm, and meta-training and meta-testing on the visible light image dataset, including: The dataset is divided into a base class dataset and a small sample new class dataset; Initialize the parameters of the meta-learning model optimizer θ ; Construct the base class dataset I d Support set data for individual training tasks DS i and query set data DQ i Train the model parameters; Construct a support set for a meta-test task using a small sample new class dataset. TS and query set data TQ, The inner loop loss function is optimized, and the model parameters are trained to obtain fine-tuned model parameters. ; The NSST algorithm is used to perform a three-stage decomposition on the original visible light image and the original infrared image to obtain the low-frequency subband coefficients of the infrared image. Low-frequency subband coefficients of visible light images High-frequency subband coefficients of infrared images High-frequency subband coefficients of visible light images ; The high-frequency sub-band images are fused using a spatial frequency weighting method, and the low-frequency sub-band images are fused using a local average gradient weighting method. The fused low-frequency sub-band coefficients are then... and high-frequency subband coefficient The final fused image is obtained by performing the inverse NSST transform.

8. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to at least one of the processors; The memory stores instructions executable by at least one of the processors, which are executed by at least one of the processors to enable the at least one of the processors to perform the method as claimed in any one of claims 1-6.

Images