A new energy adaptive detection method with built-in lightweight

Abstract

The present invention discloses a built-in lightweight adaptive detection method for new energy, which relates to the field of object detection technology, step S1: building a database; step S2: data preprocessing; step S3: model architecture construction; step S4: attention module integration; step S5: self-supervised contrast learning preparation; step S6: pre-training preparation; step S7: self-supervised contrast learning training; step S8: unsupervised adaptive training; step S9: model optimization and parameter adjustment; step S10: model verification and evaluation. The present invention adopts the above-mentioned built-in lightweight adaptive detection method for new energy, which not only improves the real-time monitoring capability and detection accuracy of the system, but also enhances the robustness of the system under changeable weather and environmental conditions, improves the user's operating efficiency and experience, and provides a new technical path and solution for the sustainable development of small hydropower, photovoltaic and wind power projects.

Description

Technical Field

[0001] The present invention relates to the technical field of object detection, and in particular to a built-in lightweight new energy adaptive detection method. Background Art

[0002] Object detection technology plays a vital role in devices such as surveillance systems, helping these systems perceive and understand their surroundings. In recent years, the application of deep learning technology has significantly improved the ability to detect objects, especially in normal weather conditions. Although deep learning has significantly improved the ability to detect objects, ensuring the accuracy of these systems in adverse weather conditions such as rain and snowfall remains an ongoing challenge. Currently, the use of synthetic datasets has become a common approach, and fully annotated adverse weather images can be obtained by generating synthetic adverse weather effects on clear weather images. These synthetic images are used to train robust models that enable reliable object detection in various weather conditions. Although existing deep learning techniques have made significant progress in object detection, their accuracy in adverse weather conditions remains challenging. Real-world adverse weather, such as rain and snowfall, has complex and diverse effects on images, and it is difficult to fully capture and process these complex features based solely on prior knowledge in existing technologies. In addition, the process of obtaining real labeled data under adverse weather conditions is very laborious, resulting in a lack of datasets, which makes it impossible for models to fully cover all possible weather changes and effects when training. To overcome these shortcomings, it is necessary to develop a new learning method that enables object detectors to better adapt to adverse weather conditions. Specific measures include: (1) Utilizing synthetic datasets: Continue to use synthetic datasets to generate high-quality synthetic bad weather effects for clear weather images to make up for the lack of real data. These synthetic images can be used to train models in a supervised manner. (2) Training robust models: Using synthetic datasets, train robust models that can adapt to different bad weather conditions and improve their performance in real environments. (3) Developing removal networks: Train removal networks to restore clear images from bad weather images, so that detectors can detect objects on clear images and improve detection accuracy. (4) Enhance the application of prior knowledge: Improve the application of prior knowledge to better capture the complex characteristics of bad weather conditions in the real world and enhance the model's adaptability in complex weather conditions. Through these measures, the reliability and accuracy of object detection systems in bad weather conditions can be significantly improved, and the environmental perception and understanding capabilities of equipment such as surveillance systems can be further improved.

[0003] In recent years, with the increase in global demand for renewable energy, small hydropower, wind power and photovoltaic industries have developed rapidly in China. During the construction of these renewable energy projects, the management and monitoring of construction sites have become particularly important. In the operation and management of renewable energy projects, hidden danger video monitoring systems are widely used to monitor the safety status of construction sites in real time. However, in actual applications, monitoring faces problems such as equipment jitter, foggy weather, and rainy weather, which lead to blurred video images or reduced clarity, thus affecting the accuracy of hidden danger detection and making it difficult to accurately monitor the operation status and potential risks of renewable energy project construction sites.

[0004] At present, some small hydropower, wind power and photovoltaic projects in China have begun to explore and apply image recognition technology, but there are still some challenges in large-scale promotion and application, such as how to build in lightweight algorithms to carry out adaptive detection in the field of renewable energy, so as to adapt to more construction sites in bad weather. With the continuous advancement of technology and the reduction of costs, the application prospects of image recognition technology in wind power and photovoltaic construction sites will be broader, which is expected to significantly improve the management level of construction sites and promote the high-quality development of the renewable energy industry.

[0005] Although deep learning technology has made significant progress in the field of object detection, it still has many shortcomings in adverse weather conditions:

[0006] (1) Decreased detection accuracy: Under adverse weather conditions such as rain and snow, the accuracy of existing object detection systems decreases significantly. This is because the adverse weather affects the image quality, making it difficult for the detector to correctly identify the target.

[0007] (2) Data scarcity: It is extremely difficult and time-consuming to obtain labeled data under adverse weather conditions. Therefore, existing technologies often lack sufficient high-quality data to train and validate models, resulting in poor performance of the models in practical applications.

[0008] (3) Limitations of prior knowledge: Existing technologies rely on prior knowledge of weather conditions, but this knowledge is difficult to fully capture the complex characteristics of severe weather in the real world. Each type of severe weather has different and complex effects on images, and existing methods cannot fully cope with these changes.

[0009] (4) Differences between synthetic data and real data: Although synthetic datasets can partially make up for the problem of insufficient data, there are still differences between synthetic images and real severe weather images. This difference may cause the trained model to perform worse than expected in practical applications.

[0010] (5) Insufficient model robustness: Existing models are not robust enough to maintain consistent performance in the face of multiple severe weather conditions. Models often perform well only in certain types of severe weather, but perform poorly in other types of severe weather.

[0011] By identifying and understanding these shortcomings, we can provide direction for developing more effective object detection techniques that can improve detection accuracy and reliability in adverse weather conditions. Summary of the invention

[0012] The purpose of the present invention is to provide a built-in lightweight new energy adaptive detection method to solve the problems of weak adaptability of monitoring equipment and poor robustness of target detection methods under severe weather conditions such as rain, snow and fog. The attention module is used to extract relevant information of high-level features to reduce the style gap between clear images and blurred images; self-supervised contrast learning is used to reduce the weather gap between clear images and blurred images.

[0013] To achieve the above object, the present invention provides a built-in lightweight new energy adaptive detection method, comprising the following steps:

[0014] Step S1: Building a database: Collect a large number of clear images and blurred images and make them into data sets, divide the data, and build a small hydropower, wind power and photovoltaic construction site database;

[0015] Step S2: Data preprocessing: standardize the clear image and the blurred image, including resizing, color normalization, and data enhancement to ensure the consistency of the input data;

[0016] Step S3: Model architecture construction: Build a deep learning model including an attention module to extract high-level features from the image;

[0017] Step S4: Attention module integration: Integrate the attention module into the feature extraction layer of the deep learning model, use the feature extraction layer to extract important feature information in the image, and reduce the style gap between the clear image and the blurred image;

[0018] Step S5: Self-supervised contrastive learning preparation: construct a self-supervised contrastive learning mechanism to provide a contrastive learning basis for the deep learning model by generating positive sample pairs and negative sample pairs;

[0019] Among them, the positive sample pair is a clear image and an enhanced version of the clear image, and the negative sample pair is a clear image and an image of a different category;

[0020] Step S6: Pre-training preparation: extract features from the clear image and the blurred image through the attention module to generate high-level feature representation;

[0021] Step S7: self-supervised contrastive learning training: using a self-supervised contrastive learning method, the deep learning model is trained to maximize the similarity between the clear image and the enhanced version of the clear image, while minimizing the difference between the clear image and the blurred image;

[0022] Step S8: Unsupervised adaptive training: In the absence of explicit labels, the deep learning model is adapted to the image features under severe weather conditions through adaptive training methods to improve detection accuracy and robustness;

[0023] Step S9: Model optimization and parameter adjustment: perform hyperparameter tuning on the deep learning model, optimize the learning rate and batch size parameters during training, and ensure the robustness of the model under different weather conditions;

[0024] Step S10: Model verification and evaluation: Use an independent verification set to evaluate the deep learning model, test the target detection performance of the deep learning model under different severe weather conditions, and further optimize and improve the model based on the evaluation results.

[0025] Preferably, the clear image in step S1 includes images of small reservoirs, photovoltaic panels and wind power generation equipment taken under clear weather conditions; the blurred image includes images of small reservoirs, photovoltaic panels and wind power generation equipment taken under adverse weather conditions such as rain, snow and fog.

[0026] Preferably, the data division in step S1 includes dividing the data set into a training set, a validation set and a test set, and the training set, the validation set and the test set all include clear images and blurred images.

[0027] Preferably, the size adjustment processing in step S2 includes adjusting all images to a uniform size; the color normalization processing includes making all images have a consistent color distribution; the data enhancement processing includes enhancing data diversity, specifically including rotation, flipping, cropping, and color jittering.

[0028] Preferably, the architecture of the deep learning model in step S3 is as follows:

[0029] Given labeled images of clean conditions from source domain S and unlabeled images of unfavorable conditions from target domain T, each mini-batch of images consists of the same number of source and target data, where the source and target data are different in terms of weather conditions and surrounding environments;

[0030] The FasterR-CNN workflow and FPN backbone are used during training. The FPN backbone adopts a pyramid architecture to generate multi-scale feature maps in an image, including P2, P3, P4, and P5 layers, allowing objects of different scales to be effectively detected.

[0031] The regional proposal network RPN proposes RoIs for these features and extracts instance features from each RoI. The final class and bounding box prediction is made based on the regional proposal network RCN, and the supervision loss is The data obtained from the RPN backbone and the region proposal network (RCN) are only applied to the source data.

[0032] Preferably, extracting important feature information from the image using the feature extraction layer in step S4 specifically includes performing image-level alignment processing on the P4 layer and the P5 layer of the FPN backbone.

[0033] Preferably, the image-level alignment loss , the calculation formula is as follows:

[0034]

[0035] in, ∈{ , } indicates the feature level, Indicates the position after the convolutional block attention module CBAM layer Each feature of ∈{0,1} represents the position The domain label of each feature, Represents the discriminator.

[0036] Preferably, the specific steps of step S8 are as follows:

[0037] Step S81, assuming that an instance feature of the target domain image consists of an object and an arbitrary weather damage pattern;

[0038] Step S82, encouraging similarity between instance features of object proposals within the same category, so that the instance features are invariant to weather damage;

[0039] Step S83, obtain instance features from the source domain and target domain training images, and use the class score of each instance provided by the region proposal network RCN to pseudo-label each instance feature as kind;

[0040] Step S84: The instance features are forwarded to the multi-layer perceptron of the deep learning model to generate D-dimensional instance embeddings. ,in, is the total number of instances;

[0041] Step S85: Based on the contrastive learning method of learnable prototypes, each category is represented by some learnable prototype vectors, instance embeddings are assigned to the prototype vectors, and the deep learning model increases the similarity between instance embeddings and their prototypes through learning. All classes use A learnable prototype Considering instance features and intra-class variations of each prototype, each prototype As the first Cluster centers;

[0042] Step S86: Calculate The class assignment matrix is ​​used to assign each class with a pseudo label Instances of are assigned to the prototype of the same class;

[0043] Step S87, obtain the allocation matrix through the assignment matrix, and train the network so that the similarity between the prototype and the instance embedding corresponds to the allocation matrix;

[0044] Step S88: After obtaining the assignment matrix, the prototype and instance embeddings are optimized by minimizing the cross entropy loss between the similarity and the assignment matrix.

[0045] Preferably, by adding an entropy regularization term and a parameter controlling the stationarity of the distribution ,get Class assignment matrix The goal is as follows:

[0046]

[0047] in, Pseudo-marking The number of instances of the class, and Indicates that all dimensions are and The vector of represents the smoothness parameter, which can be 0.5 or 0.9. represents entropy, represents the trace of the matrix, represents the transposed matrix of the allocation matrix, Represents constraints, limited by ;

[0048] The solution to the above equation is calculated iteratively using the Sinkhorn-Knopp algorithm, which outputs the renormalized vector and , and obtain the allocation matrix , the expression is as follows:

[0049]

[0050] Get the allocation matrix After that, the expression of cross entropy loss is as follows:

[0051]

[0052] in, It is Pseudo labels for instances, It is for Assignment of prototype instances, is the temperature parameter, represents the number of classes, including background classes, Indicates a category. represents the total number of samples, Represents the total number of prototype instances, Represents a prototype instance, for each instance , minimize Add the specified prototype , and reduce its similarity to all other classes.

[0053] Preferably, the goals of the adaptive training method in step S8 are as follows:

[0054]

[0055] in, represents the supervision loss, and Represents the adjustment coefficient, which is 0.5.

[0056] Therefore, the present invention adopts the above-mentioned built-in lightweight new energy adaptive detection method, which has the following beneficial effects:

[0057] (1) This method has significant advantages in small hydropower, photovoltaic and wind power projects. It can achieve fast reasoning on resource-constrained devices, improve real-time monitoring and detection capabilities, and significantly reduce computing resource requirements and energy consumption. It is suitable for embedded systems and edge computing devices.

[0058] (2) This method requires low computing resources, significantly reducing the system's power consumption and hardware costs. Through the built-in adaptive detection mechanism, the system can maintain high detection accuracy in various environments and weather conditions, enhancing the system's robustness and reliability.

[0059] (3) The lightweight algorithm of this method is extremely flexible, making it easy to deploy in a variety of scenarios such as drones, fixed cameras, and mobile devices. It has high data transmission efficiency and is easy to maintain, which helps to obtain reliable detection results in a timely manner, improve user experience, and enhance the overall benefits of small hydropower, photovoltaic, and wind power projects.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a structural schematic diagram of an embodiment of a built-in lightweight new energy adaptive detection method of the present invention. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0063] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0064] Example

[0065] See also Figure 1 The present invention provides a built-in lightweight new energy adaptive detection method, which specifically includes the following steps:

[0066] Step S1: Build a database: collect a large number of clear images and blurred images and make them into data sets, divide the data, and build a small hydropower, wind power and photovoltaic construction site database.

[0067] Clear images include high-quality clear images of small reservoirs, photovoltaic panels and wind power generation equipment taken under clear weather conditions, ensuring that the images cover various angles, different types of equipment and different time periods. Blurred images include images of small reservoirs, photovoltaic panels and wind power generation equipment taken under adverse weather conditions such as rain, snow and fog, ensuring the diversity and representativeness of the data set. Data partitioning includes dividing the data set into training set, validation set and test set, and the training set, validation set and test set all include clear images and blurred images. Small hydropower; wind power and photovoltaic construction site databases include image data of wind turbine blades, tower transportation site or wind turbine hoisting site; image data of safety belts for operation inside wind turbine tower or photovoltaic field; access monitoring and early warning of vehicles and personnel at photovoltaic field site or access monitoring data of special operation personnel inside wind turbine tower.

[0068] Step S2: Data preprocessing: Standardize the clear and blurred images, including resizing, color normalization, and data enhancement to ensure the consistency of the input data. The resizing process includes adjusting all images to a uniform size, such as 256×256 pixels, to ensure the consistency of the input data. The color normalization process includes making all images have a consistent color distribution to facilitate model learning. The data enhancement process includes enhancing data diversity, including rotation, flipping, cropping, and color jittering to enhance data diversity.

[0069] Step S3: Model architecture construction: Build a deep learning model including an attention module, such as a convolutional neural network (CNN)-based architecture, to extract high-level features from images.

[0070] The architecture of the deep learning model is as follows:

[0071] Given labeled images of clean conditions from a source domain S and unlabeled images of unfavorable conditions from a target domain T, each mini-batch of images consists of the same number of source and target data, where the source and target data differ in both weather conditions and surrounding environments.

[0072] The FasterR-CNN workflow and FPN backbone are used during the training process. The FPN backbone adopts a pyramid architecture to generate multi-scale feature maps in an image, including P2 layer, P3 layer, P4 layer and P5 layer, allowing objects of different scales to be effectively detected.

[0073] The regional proposal network RPN proposes RoIs for these features and extracts instance features from each RoI. The final class and bounding box prediction is made based on the regional proposal network RCN, and the supervision loss is The data obtained from the RPN backbone and the region proposal network (RCN) are only applied to the source data.

[0074] The goal of this detection method is to train a robust object detector that performs well in real-world environments with device jitter, fog, and rain. Based on the architecture of the convolutional neural network framework with FPN backbone, two domain-adaptive components are proposed to handle style gap and weather gap. First, image-level style alignment is used to reduce the style gap through adversarial training, and then instance-level weather alignment is used to reduce the weather gap and learn corruption-invariant features. The entire model is trained simultaneously in an end-to-end manner.

[0075] Step S4: Attention module integration: Integrate the attention module into the feature extraction layer of the deep learning model, use the feature extraction layer to extract important feature information in the image, and reduce the style gap between the clear image and the blurred image. The different styles of the source domain S and the target domain T environment lead to different feature distributions. The style gap between the domains is solved by aligning the image-level features.

[0076] The feature extraction layer is implemented as follows: a domain classifier is attached to each layer of the FPN backbone to distinguish domains, the domain classifiers are confused through adversarial training, and the backbone is trained to generate domain-invariant features.

[0077] The identification of clear images and blurred images is achieved through a weight coefficient of The gradient reversal layer (GRL) is implemented in one back-propagation step to perform image-level feature adaptation by focusing only on the style attributes of the image. However, due to weather damage, some features are severely degraded, making it difficult to focus on style differences.

[0078] In order to enable the network to emphasize style-related features during the alignment process, a convolutional block attention module (CBAM) is used to emphasize features necessary for domain alignment. CBAM is attached to each feature map and channel and spatial attention modules are applied to obtain refinement at the feature level. Features .

[0079] The participating features are fed into the discriminator The domain of a feature is predicted, leading to it to align features through the gradient reversal layer GRL by focusing on the essential information. Since low-level features with fine-grained details are more susceptible to weather corruption, only high-level features are used for alignment.

[0080] Therefore, using the feature extraction layer to extract important feature information in the image specifically includes performing image-level alignment processing on the P4 layer and the P5 layer of the FPN backbone.

[0081] Image-level alignment loss , the calculation formula is as follows:

[0082]

[0083] in, ∈{ , } indicates the feature level, Indicates the position after the convolutional block attention module CBAM layer Each feature of ∈{0,1} represents the position The domain label of each feature, Represents the discriminator.

[0084] Step S5: Self-supervised contrastive learning preparation: construct a self-supervised contrastive learning mechanism to provide a contrastive learning basis for the deep learning model by generating positive sample pairs and negative sample pairs;

[0085] Among them, the positive sample pair is a clear image and an enhanced version of the clear image, and the negative sample pair is a clear image and an image of a different category.

[0086] Step S6: Pre-training preparation: Extract features of the clear image and blurred image through the attention module respectively to generate high-level feature representation.

[0087] Step S7: Self-supervised contrastive learning training: Using a self-supervised contrastive learning method, the deep learning model is trained to maximize the similarity between the clear image and the enhanced version of the clear image, while minimizing the difference between the clear image and the blurred image.

[0088] Step S8: Unsupervised Adaptive Training: In the absence of explicit labels, an adaptive training method is used to adapt the deep learning model to image features under adverse weather conditions to improve detection accuracy and robustness. Weather damage has a local effect on the image and significantly degrades instance features that are critical for object detection. The goal of this detection method is to obtain corruption-invariant features and reduce weather gaps through prototype-based contrastive learning. "Corruption-invariant features" refers to features that remain stable and reliable in the face of various data corruptions or interferences.

[0089] Data corruption can take various forms, such as noise, occlusion, blur, distortion, etc. Features with corruption-invariance can maintain their discriminability when processing such corrupted data without significantly affecting the performance of the model.

[0090] The specific steps of step S8 are as follows:

[0091] Step S81: Assume that an instance feature of the target domain image consists of an object and an arbitrary weather damage pattern.

[0092] Step S82: Encourage similarity between instance features of object proposals within the same category, so that the instance features are invariant to weather damage.

[0093] Step S83, obtain instance features from the source domain and target domain training images, and use the class score of each instance provided by the region proposal network RCN to pseudo-label each instance feature as kind.

[0094] Step S84: The instance features are forwarded to the multi-layer perceptron of the deep learning model to generate D-dimensional instance embeddings. ,in, is the total number of instances.

[0095] Step S85, in the deep learning model, "MLP head" refers to a multi-layer perceptron (MLP) as the last few layers of the model, which is used to process features from the previous layers and make final task-specific predictions. Based on the contrastive learning method of learnable prototypes, each category is represented by some learnable prototype vectors, and instance embeddings are assigned to the prototype vectors. The deep learning model increases the similarity between the instance embedding and its prototype through learning. Classes all use A learnable prototype Considering instance features and intra-class variations of each prototype, each prototype As Class In addition, to further improve the performance of contrastive learning, instance embeddings with background classes are also constructed with the same number of learnable prototypes, which can be used as negative samples for other instances.

[0096] Step S86: Calculate The class assignment matrix is ​​used to assign each class with a pseudo label The instances of are assigned to the prototype of the same class. The assignment matrix satisfies the condition that the sum of the soft assignment probabilities of each instance is 1, that is, ,in and Indicates that all dimensions are and Vector.

[0097] Step S87, obtain the allocation matrix through the assignment matrix, and train the network so that the similarity between the prototype and the instance embedding corresponds to the allocation matrix.

[0098] The assignment matrix can be obtained by maximizing the similarity between instance embedding and class prototype, as shown in the following formula:

[0099]

[0100] in, and Denote instance embedding and pseudo labeling as The number of instances of the class, express The transposed matrix of represents instance embedding ( Dimension) in the prototype ( Dimension), belongs to The characteristics of instances of a class change.

[0101] To avoid all instance embeddings being assigned to a single prototype, an equal distribution constraint is added to ensure that instances are evenly distributed among prototypes of a certain class. By adding an entropy regularization term and a parameter that controls the stationarity of the distribution ,get Class assignment matrix The goal is as follows:

[0102]

[0103] in, and Indicates that all dimensions are parameters and The vector of represents the smoothness parameter (0.5 or 0.9), represents entropy, represents the trace of the matrix, represents the transposed matrix of the allocation matrix, (subject to) indicates a constraint, limited by .

[0104] The solution to the above equation is calculated iteratively using the Sinkhorn-Knopp algorithm, which outputs the renormalized vector and , and obtain the allocation matrix , the expression is as follows:

[0105]

[0106] Step S88: After obtaining the assignment matrix, the prototype and instance embeddings are optimized by minimizing the cross entropy loss between the similarity and the assignment matrix.

[0107] Get the allocation matrix After that, the expression of cross entropy loss is as follows:

[0108]

[0109] in, It is Pseudo labels for instances, It is for Assignment of prototype instances, is the temperature parameter, represents the number of classes, including background classes, Indicates a category. represents the total number of samples, Represents the total number of prototype instances, Represents a prototype instance, for each instance , minimize Add the specified prototype , and reduce its similarity to all other classes.

[0110] Therefore, instance embeddings will be grouped around the prototypes they are assigned to. This produces corruption-resistant instance properties by promoting instance embeddings with similar semantics and variable weather damage to become closer. The goals of the adaptive training method are as follows:

[0111]

[0112] in, represents the supervision loss, and Represents the adjustment coefficient, which is 0.5.

[0113] Step S9: Model optimization and parameter adjustment: Perform hyperparameter tuning on the deep learning model, optimize the learning rate and batch size parameters during training, and ensure the robustness of the model under different weather conditions. Use methods such as grid search or random search to find the best hyperparameter configuration.

[0114] Step S10: Model verification and evaluation: Use an independent validation set to evaluate the deep learning model, test the target detection performance of the deep learning model under different severe weather conditions, and further optimize and improve the model based on the evaluation results. Use common evaluation indicators such as accuracy, recall, F1 score, etc. to comprehensively evaluate the model performance.

[0115] Therefore, the present invention adopts the above-mentioned built-in lightweight new energy adaptive detection method, which not only improves the real-time monitoring capability and detection accuracy of the system, but also enhances the robustness of the system under changeable weather and environmental conditions, improves the user's operating efficiency and experience, and provides a new technical path and solution for the sustainable development of small hydropower, photovoltaic and wind power projects.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A built-in lightweight new energy adaptive detection method, characterized in that: The following steps are involved: Step S1: Building a database: Collect a large number of clear images and blurred images and make them into data sets, divide the data, and build a small hydropower, wind power and photovoltaic construction site database; Step S2: Data preprocessing: standardize the clear image and the blurred image, including resizing, color normalization, and data enhancement to ensure the consistency of the input data; Step S3: Model architecture construction: Build a deep learning model including an attention module to extract high-level features from the image; The architecture of the deep learning model in step S3 is as follows: Given labeled images of clean conditions from source domain S and unlabeled images of unfavorable conditions from target domain T, each mini-batch of images consists of the same number of source and target data, where the source and target data are different in terms of weather conditions and surrounding environments; The FasterR-CNN workflow and FPN backbone are used during training. The FPN backbone adopts a pyramid architecture to generate multi-scale feature maps in an image, including P2, P3, P4, and P5 layers, allowing objects of different scales to be effectively detected. The regional proposal network RPN proposes RoIs for these features and extracts instance features from each RoI. The final class and bounding box prediction is made based on the regional proposal network RCN, and the supervision loss The data obtained from the RPN backbone and the region proposal network RCN are only applied to the source data; Step S4: Attention module integration: Integrate the attention module into the feature extraction layer of the deep learning model, use the feature extraction layer to extract important feature information in the image, and reduce the style gap between the clear image and the blurred image; The step S4 of extracting important feature information from the image using the feature extraction layer specifically includes performing image-level alignment processing on the P4 layer and the P5 layer of the FPN backbone; The image-level alignment loss , the calculation formula is as follows: ； in, ∈{ , } indicates the feature level, Indicates the position after the convolutional block attention module CBAM layer Each feature of ∈{0,1} represents the position The domain label of each feature, represents the discriminator; Step S5: Self-supervised contrastive learning preparation: construct a self-supervised contrastive learning mechanism to provide a contrastive learning basis for the deep learning model by generating positive sample pairs and negative sample pairs; Among them, the positive sample pair is a clear image and an enhanced version of the clear image, and the negative sample pair is a clear image and an image of a different category; Step S6: Pre-training preparation: extract features from the clear image and the blurred image through the attention module to generate high-level feature representation; Step S7: self-supervised contrastive learning training: using a self-supervised contrastive learning method, the deep learning model is trained to maximize the similarity between the clear image and the enhanced version of the clear image, while minimizing the difference between the clear image and the blurred image; Step S8: Unsupervised adaptive training: In the absence of explicit labels, the deep learning model is adapted to the image features under severe weather conditions through adaptive training methods to improve detection accuracy and robustness; Step S9: Model optimization and parameter adjustment: perform hyperparameter tuning on the deep learning model, optimize the learning rate and batch size parameters during training, and ensure the robustness of the model under different weather conditions; Step S10: Model verification and evaluation: Use an independent verification set to evaluate the deep learning model, test the target detection performance of the deep learning model under different severe weather conditions, and further optimize and improve the model based on the evaluation results.

2. A built-in lightweight new energy adaptive detection method according to claim 1, characterized in that: The clear images in step S1 include images of small reservoirs, photovoltaic panels and wind power generation equipment taken under clear weather conditions; the blurred images include images of small reservoirs, photovoltaic panels and wind power generation equipment taken under adverse weather conditions such as rain, snow and fog.

3. A built-in lightweight new energy adaptive detection method according to claim 2, characterized in that: The data division in step S1 includes dividing the data set into a training set, a verification set and a test set, and the training set, the verification set and the test set all include clear images and blurred images.

4. The method for adaptive detection of new energy with built-in lightweight according to claim 3, characterized in that: The size adjustment processing in step S2 includes adjusting all images to a uniform size; the color normalization processing includes making all images have a consistent color distribution; the data enhancement processing includes enhancing data diversity, specifically including rotation, flipping, cropping, and color jittering.

5. The built-in lightweight new energy adaptive detection method according to claim 1, characterized in that: The specific steps of step S8 are as follows: Step S81, assuming that an instance feature of the target domain image consists of an object and an arbitrary weather damage pattern; Step S82, encouraging similarity between instance features of object proposals within the same category, so that the instance features are invariant to weather damage; Step S83, obtain instance features from the source domain and target domain training images, and use the class score of each instance provided by the region proposal network RCN to pseudo-label each instance feature as kind; Step S84: The instance features are forwarded to the multi-layer perceptron of the deep learning model to generate D-dimensional instance embeddings. ,in, is the total number of instances; Step S85: Based on the contrastive learning method of learnable prototypes, each category is represented by some learnable prototype vectors, instance embeddings are assigned to the prototype vectors, and the deep learning model increases the similarity between instance embeddings and their prototypes through learning. Classes all use A learnable prototype Considering instance features and intra-class variations of each prototype, each prototype As the first cluster centers; among them, Indicates that the instance is embedded in the prototype The instance characteristics of the class change, express Class The cluster centers of prototype vectors, represents a D-dimensional matrix, The dimension is D, Denotes D dimension A learnable class prototype matrix; Step S86: Calculate The class assignment matrix is ​​used to assign each class with a pseudo label Instances of are assigned to the prototype of the same class; Step S87, obtain the allocation matrix through the assignment matrix, and train the network so that the similarity between the prototype and the instance embedding corresponds to the allocation matrix; Step S88: After obtaining the assignment matrix, the prototype and instance embeddings are optimized by minimizing the cross entropy loss between the similarity and the assignment matrix.

6. A built-in lightweight new energy adaptive detection method according to claim 5, characterized in that: By adding an entropy regularization term and a parameter that controls the stationarity of the distribution ,get Class assignment matrix The goal is as follows: ； in, Indicates that the pseudo-mark is The number of instances of the class, and Indicates that all dimensions are and The vector of represents the smoothness parameter, which can be 0.5 or 0.

9. represents entropy, represents the trace of the matrix, represents the transposed matrix of the allocation matrix, Represents constraints, limited by ; The solution to the above equation is calculated iteratively using the Sinkhorn-Knopp algorithm, which outputs the renormalized vector u and v , and obtain the allocation matrix , the expression is as follows: ； Get the allocation matrix After that, the expression of cross entropy loss is as follows: ； in, It is The pseudo labels of instances, It is for Assignment of prototype instances, is the temperature parameter, represents the number of classes, including background classes, Indicates a category. represents the total number of samples, Represents the total number of prototype instances, Represents a prototype instance, for each instance , minimize Add the specified prototype , and reduce its similarity to all other classes, express Class Cluster centers.

7. A built-in lightweight new energy adaptive detection method according to claim 6, characterized in that: The goals of the adaptive training method in step S8 are as follows: ； in, represents the supervision loss, and Represents the adjustment coefficient, which is 0.5.

Images