Branch detection method and device, model creation method and device, storage medium and terminal

Abstract

The invention discloses a branch detection method, a model creation method, a model creation device, a storage medium and a terminal, an improved YOLOv5 algorithm is adopted, and the technology of an unmanned aerial vehicle and a monitoring camera is combined to realize all-weather automatic monitoring of the surrounding environment of a photovoltaic power plant. By introducing a convolution attention module and a detection layer for a small target, the detection capability of the algorithm on a small target branch is enhanced; meanwhile, multi-scale feature fusion is carried out through residual connection, a context enhancement module CAM and an SPPCSPCgroup pyramid structure, the sensing and recognition capability of an algorithm on small targets in a complex environment is enhanced, it is ensured that branches can be accurately detected under different illumination and weather conditions, the accuracy and processing speed of branch recognition can be remarkably improved, and the method is suitable for large-scale application and popularization. Early warning can be given out in time, and equipment damage caused by natural factors is effectively avoided or minimized.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a tree branch detection method, a model creation method, a device, a storage medium and a terminal. Background Art

[0002] Photovoltaic power plants are usually located in open areas outdoors and face various challenges from the natural environment, especially the potential threat posed by fallen branches to photovoltaic panels and other key equipment. Traditional equipment protection methods mainly rely on manual inspections and monitoring systems, but these methods have obvious shortcomings. Manual inspections are not only time-consuming, labor-intensive and costly, but also far from sufficient response speed and preventive measures when faced with sudden events such as fallen branches. Monitoring systems are often limited by technical levels and environmental conditions, making it difficult to accurately identify and quickly respond to potential threats.

[0003] In the complex environment of photovoltaic power plants, natural factors such as tree branches often become the cause of equipment damage. In particular, small branches are difficult to accurately identify and locate with traditional technology due to their small size and easy obstruction by leaves. Existing detection methods such as ultrasonic technology or old deep learning models are insufficient in recognition accuracy and real-time performance. Although ultrasonic technology can detect branches approaching the equipment, it is easily interfered with in complex environments, and it is difficult to accurately determine the specific location and size of the branches. Due to the limitations of the algorithm, the old deep learning model is insufficient in feature extraction and target detection, and it is difficult to meet the needs of photovoltaic power plant equipment protection.

[0004] Therefore, how to solve the deficiencies and shortcomings of existing technologies through effective photovoltaic power plant tree branch detection methods has become an important issue that researchers in this field need to solve urgently. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and provide a branch detection method, a model creation method, a device, a storage medium and a terminal.

[0006] The technical solution of the present invention is: a method for creating a tree branch detection model for photovoltaic power plant equipment, comprising the following steps: obtaining training samples, the training samples comprising a plurality of training images and identification results corresponding to each training image, the identification results being used to characterize whether the training image is blocked by tree branches and the position of the tree branches blocking the training image, and each training image containing the photovoltaic power plant equipment; creating a YOLOv5 model, the YOLOv5 model comprising: an input end, a backbone network, a neck network, a detection head and an output end; the input end is used to receive images to be detected and to perform adaptive scaling, normalization and image enhancement processing on the images to be detected; the backbone network is used to extract features of the images to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end; the YOLOv5 model is trained based on the training samples to obtain a tree branch detection model.

[0007] As an improvement of an embodiment of the present invention, the YOLOv5 model further includes: adding a convolutional attention module CBAM to the backbone network of the YOLOv5 model; the convolutional attention module CBAM includes a first submodule and a second submodule, and a residual connection is embedded in the convolutional attention module CBAM; the calculation formula of the convolutional attention module CBAM is F′=M c (F i )⊙F i , F″=M s (F′)⊙F′, where ⊙ is the dot product, M c (F i ) is the output of the first submodule, M s (F′) is the output of the second submodule; F′ is the feature map after the output of the first submodule is fused with the image to be detected, and F″ is the output result of the convolutional attention module CBAM.

[0008] As an improvement of the embodiment of the present invention, the YOLOv5 model further includes: a context enhancement module CAM added to the backbone network of the YOLOv5 model.

[0009] As an improvement of the embodiment of the present invention, the YOLOv5 model further includes: an SPPC SPC_group pyramid structure set by the neck network, and the SPPCSPC_group pyramid structure is used to enhance the receptive field.

[0010] As an improvement of an embodiment of the present invention, the YOLOv5 model further includes: the detection head outputs N types of size feature maps, where N=4.

[0011] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a tree branch detection method for photovoltaic power plant equipment, and the tree branch detection method specifically includes: obtaining a picture of the photovoltaic power plant equipment, and detecting the picture of the photovoltaic power plant equipment based on a tree branch detection model created as described in any of the above items to obtain a detection result.

[0012] As an improvement to an embodiment of the present invention, when the detection result is that the photovoltaic power plant equipment is blocked by tree branches, an early warning is issued.

[0013] To achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a device for creating a tree branch detection model for a photovoltaic power plant, comprising the following modules: an information acquisition module, used to acquire training samples, wherein the training samples include a plurality of training images and an identification result corresponding to each training image, wherein the identification result is used to characterize whether the training image is blocked by tree branches and the position of the tree branches, and each training image includes the photovoltaic power plant equipment; a model creation module, used to create a YOLOv5 model, wherein the YOLOv5 model includes: an input end, a backbone network, a neck network, a detection head and an output end; the input end is used to receive the image to be detected and to perform adaptive scaling, normalization and image enhancement processing on the image to be detected; the backbone network is used to extract features of the image to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end; a training module is used to train the YOLOv5 model based on the training samples and obtain a tree branch detection model.

[0014] To achieve one of the above objects of the invention, an embodiment of the present invention provides a storage medium storing program instructions, wherein the program instructions are executed by a processor to implement any of the above branch detection methods.

[0015] To achieve one of the above objects of the invention, an embodiment of the present invention provides a terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement any of the above branch detection methods.

[0016] The branch detection method, model creation method, device, storage medium and terminal provided by the embodiment of the present invention have the following advantages: The present invention adopts the improved YOLOv5 algorithm, combined with the technology of drones and surveillance cameras, to achieve all-weather automatic monitoring of the surrounding environment of photovoltaic power plants. By introducing the convolutional attention module and the detection layer for small targets, the algorithm's detection ability for small target branches is enhanced; at the same time, through the residual connection, the context enhancement module CAM and the use of the SPPCSPC_group pyramid structure for multi-scale feature fusion, the algorithm's perception and recognition ability for small targets in complex environments is enhanced, ensuring that branches can be accurately detected under different lighting and weather conditions, which can not only significantly improve the accuracy and processing speed of branch recognition, but also issue early warnings in time to effectively avoid or minimize equipment damage caused by natural factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the method for creating a tree branch detection model for photovoltaic power plant equipment according to the present invention;

[0018] Figure 2 It is a schematic diagram of the YOLOv5 model structure of the present invention;

[0019] Figure 3 It is a schematic diagram of the structure of the convolutional attention module CBAM of the present invention;

[0020] Figure 4 It is a schematic diagram of the structure of the context enhancement module CAM described in the present invention. DETAILED DESCRIPTION

[0021] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0022] If the present invention involves directions (for example, up, down, left, right, front, back, outside, inside, etc.) when describing, it is necessary to define the directions involved.

[0023] The scope of the embodiments herein includes the entire scope of the claims, and all available equivalents of the claims.Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements.In fact, the first element can also be called the second element, and vice versa.Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements not only include those elements, but also include other elements not clearly listed, or also include elements inherent to such structure, device or equipment.In the absence of more restrictions, the elements limited by the statement "include one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements.Each embodiment herein is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same similar parts between each embodiment can be referred to each other.

[0024] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. in this document indicate the orientation or position relationship based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this document and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description of this document, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal communication of two elements, it can be a direct connection, or it can be indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0025] Embodiment 1 of the present invention provides a method for creating a tree branch detection model for photovoltaic power plant equipment, such as Figure 1 As shown, the method for creating the branch detection model includes the following steps:

[0026] Step 101: obtaining training samples, wherein the training samples include a plurality of training images and identification results corresponding to each training image, wherein the identification results are used to indicate whether the training image is blocked by tree branches and the position of the tree branches, and each training image includes the photovoltaic power plant equipment;

[0027] Step 102: Create a YOLOv5 model, the YOLOv5 model includes: an input end, a backbone network, a neck network, a detection head and an output end; the input end is used to receive the image to be detected and perform adaptive scaling, normalization and image enhancement processing on the image to be detected; the backbone network is used to extract features of the image to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end;

[0028] Step 103: Train the YOLOv5 model based on the training samples and obtain a tree branch detection model.

[0029] Here, in the model training phase, data augmentation techniques are first used, such as randomly rotating, scaling, and flipping images, as well as adjusting the color saturation and contrast of images, to increase the diversity of training samples, thereby improving the model's ability to recognize branches in different environments. The Adam optimizer is used to accelerate the convergence of model training and reduce the need to manually adjust the learning rate. An early stopping mechanism is used to avoid model overfitting, that is, when the accuracy on the validation set does not improve significantly within several consecutive epochs, the training is automatically stopped. In the model evaluation phase, in addition to accuracy, precision, recall, and F1 scores are introduced as measurement indicators. It is understandable that this allows a comprehensive evaluation of the model's performance. When the model meets the predetermined criteria, the trained model and its dependent environment are packaged into a Docker container, including model files, inference scripts, necessary libraries, and runtime environments, and the Docker container is deployed to the local cloud platform to ensure that the model can run stably and efficiently. A monitoring mechanism is set up to regularly check the model's operating status and performance indicators. At the same time, a process for model update and maintenance is prepared to respond quickly when needed.

[0030] In this embodiment, the YOLOv5 model further includes: adding a convolutional attention module CBAM to the backbone network of the YOLOv5 model; the convolutional attention module CBAM includes a first submodule and a second submodule, and a residual connection is embedded in the convolutional attention module CBAM; the calculation formula of the convolutional attention module CBAM is F′=M c (F i )⊙F i , F″=M s (F′)⊙F′, where ⊙ is the dot product, M c (F i ) is the output of the first submodule, M s (F′) is the output of the second submodule; F′ is the feature map after the output of the first submodule is fused with the image to be detected, and F″ is the output result of the convolutional attention module CBAM.

[0031] Here, if Figure 3 As shown in the figure, the CBAM module integrates the first submodule and the second submodule in a sequential manner, wherein the first submodule is the channel attention module and the second submodule is the spatial attention module. The channel attention module obtains two feature descriptors by performing global average pooling and global maximum pooling on each channel of the input feature map, which respectively capture the global average information and global maximum information of each channel in the input feature map. Then, the two descriptors are fed into the multi-layer perceptron MLP to generate channel attention weights, which are then dot-multiplied with the original input feature map to achieve weighting in the channel dimension. The spatial attention module first performs average pooling and maximum pooling on the feature map in the channel dimension to generate two two-dimensional spatial descriptors, which respectively contain important information at different positions. Then, the two descriptors are concatenated and convolved to generate a spatial attention weight map. Finally, this weight map is dot-multiplied with the input feature map to achieve weighting in the spatial dimension. The input and output of the CBAM module are connected by a residual connection, which helps to stabilize the gradient during training and promote the flow of information, thereby accelerating the convergence of the model.

[0032] It is understandable that by introducing the CBAM module into the backbone network of the YOLOv5 model, the model's ability to capture target features such as tree branches can be effectively improved. The combination of the channel attention and spatial attention mechanisms of the CBAM module provides the model with a more powerful feature representation capability, which helps to achieve more efficient and accurate tree branch detection.

[0033] In this embodiment, the YOLOv5 model further includes: a context enhancement module CAM added to the backbone network of the YOLOv5 model.

[0034] Here, if Figure 4 As shown in the figure, the context augmentation module (CAM) processes the input through dilated convolution at dilated convolution rates of 1, 3, and 5, respectively, to fuse features of different receptive fields. The present invention selects a convolution kernel with a kernel size of 3×3, compresses the number of channels to 1 / 4, reduces the number of parameters, and then obtains three outputs of the same size but different receptive fields through convolution expansion, which are finally fused.

[0035] In this embodiment, the YOLOv5 model also includes: a SPPCSPC_group pyramid structure set by the neck network, and the SPPCSPC_group pyramid structure is used to enhance the receptive field.

[0036] Here, the SPP layer is one of the cores of the SPP_group structure. By applying pooling windows of different sizes on the feature map, the feature map is divided into multiple grids of different scales, and pooling operations are performed on each grid. In this way, the SPP layer can capture multi-scale information on the feature map, thereby enhancing the model's receptive field for targets of different sizes. After the SPP layer, the SPP_group structure uses a variant of FPN to further fuse multi-scale features. SPP_group performs feature fusion and enhancement on different scales or feature maps. It is understandable that this grouping processing method helps the model to process features of different scales more flexibly and reduce the waste of computing resources.

[0037] In this embodiment, the YOLOv5 model also includes: the detection head outputs N types of size feature maps, where N=4.

[0038] Here, after the original YOLOv5 completes feature fusion, it outputs three sizes of feature maps of 20×20, 40×40, and 80×80 through the detection layer, which are used to detect objects of different sizes. In order to enhance the network's ability to detect small objects, a small object detection head is added, and its output feature map size is 160×160.

[0039] Embodiment 2 of the present invention provides a tree branch detection method for photovoltaic power plant equipment, and the tree branch detection method specifically includes: obtaining a picture of the photovoltaic power plant equipment, and detecting the picture of the photovoltaic power plant equipment based on a tree branch detection model created as described in any of the above items to obtain a detection result.

[0040] In practice, firstly, real-time images of the photovoltaic power plant and its surrounding environment are obtained through high-definition cameras or drones, and these images are used as the data to be detected, which includes photovoltaic panels, brackets, ground and possible branches. Next, the obtained images are input into the pre-trained branch detection model.

[0041] At the input end, the model will perform adaptive scaling, normalization and image enhancement on the image to ensure the consistency of the image data and enhance the generalization ability of the model. A convolutional attention module CBAM is added to the backbone network to focus on the spatial information and channel information in the image, effectively enhancing the model's ability to focus on key areas. At the same time, the residual connection in the CBAM module ensures the stability and convergence of network training. Subsequently, the neck network fuses the features extracted by the backbone network to generate more representative fusion features. In the detection head part, the model uses these fusion features to detect targets of photovoltaic power plant equipment and branches. Through multi-scale prediction and bounding box regression and other technologies, the model can accurately identify the branches in the image and give their position and size information. Finally, the detection results are output through the output end for use by the managers or automatic control systems of the photovoltaic power plant. Managers can take measures such as clearing branches in time according to the detection results to ensure the normal operation and power generation efficiency of the photovoltaic power plant equipment. At the same time, the automatic control system can also trigger warnings or take corresponding protection measures based on the detection results to further improve the safety and stability of the photovoltaic power plant.

[0042] In this embodiment, when the detection result is that the photovoltaic power plant equipment is blocked by tree branches, an early warning is issued.

[0043] In practice, when the branch detection method of the photovoltaic power plant equipment identifies that the photovoltaic panel is blocked by the branch, the system will immediately start the early warning mechanism. After the detection is completed, the system will analyze the detected branch blocking situation, including the proportion of the branch blocking area to the total area of ​​the photovoltaic panel and the specific location of the blocking. Based on the analysis results of the blocking degree and position, the system will automatically set the early warning level. Preferably, three early warning levels are set in the present invention: low, medium and high. When the blocking area is small and located at the edge of the photovoltaic panel, only a low-level early warning may be triggered; when the blocking area is large or located in a key area such as the center of the photovoltaic panel, a medium or high-level early warning will be triggered. After the early warning level is determined, the system will automatically generate corresponding early warning information, including but not limited to the location coordinates of the branch, the blocking area, etc. In order to ensure that the problem is effectively solved, the system can also automatically re-detect the area within a period of time after the early warning is issued to confirm whether the branch has been cleared. If the problem is still not solved, the system can send a reminder again or upgrade the early warning level to attract a higher level of attention.

[0044] Embodiment 3 of the present invention provides a device for creating a tree branch detection model for a photovoltaic power plant, comprising the following modules: an information acquisition module, used to acquire training samples, the training samples comprising a plurality of training images and identification results corresponding to each training image, the identification results being used to characterize whether the training image is blocked by tree branches and the position of the tree branches blocking the training image, and each training image comprising the photovoltaic power plant equipment; a model creation module, used to create a YOLOv5 model, the YOLOv5 model comprising: an input end, a backbone network, a neck network, a detection head and an output end; the input end is used to receive images to be detected and to perform adaptive scaling, normalization and image enhancement processing on the images to be detected; the backbone network is used to extract features of the images to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end; a training module is used to train the YOLOv5 model based on the training samples and obtain a tree branch detection model.

[0045] A fourth embodiment of the present invention provides a storage medium storing program instructions, wherein the program instructions are executed by a processor to implement any of the above branch detection methods.

[0046] A fifth embodiment of the present invention provides a terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement any of the above branch detection methods.

[0047] Storage medium can be a tangible device that keeps and stores the instructions used by the instruction execution device. Storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0048] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0049] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for creating a tree branch detection model for photovoltaic power plant equipment, characterized in that: The following steps are involved: Acquire training samples, wherein the training samples include a plurality of training images and identification results corresponding to each training image, wherein the identification results are used to characterize whether the training image is blocked by tree branches and the position of the tree branches blocking the training image, and each training image includes the photovoltaic power plant equipment; A YOLOv5 model is created, wherein the YOLOv5 model includes: an input end, a backbone network, a neck network, a detection head, and an output end; the input end is used to receive a picture to be detected and to perform adaptive scaling, normalization, and image enhancement processing on the picture to be detected; the backbone network is used to extract features of the picture to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end; The YOLOv5 model is trained based on the training samples to obtain a branch detection model.

2. The method for creating a branch detection model according to claim 1, characterized in that: The YOLOv5 model further includes: adding a convolutional attention module CBAM to the backbone network of the YOLOv5 model; The convolutional attention module CBAM includes a first submodule and a second submodule, and a residual connection is embedded in the convolutional attention module CBAM; the calculation formula of the convolutional attention module CBAM is F ′ =M c (F i )⊙F i , F″=M s (F′)⊙F′, where ⊙ is the dot product, M c (F i ) is the output of the first submodule, M s (F′) is the output of the second submodule; F ′ is the feature map after the output of the first submodule is fused with the image to be detected, and F″ is the output result of the convolutional attention module CBA M.

3. The method for creating a branch detection model according to claim 1, characterized in that: The YOLOv5 model also includes: a context enhancement module CAM added to the backbone network of the YOLOv5 model.

4. The method for creating a branch detection model according to claim 1, characterized in that: The YOLOv5 model also includes: a SPPCSPC_group pyramid structure set by the neck network, and the SPPCSPC_group pyramid structure is used to enhance the receptive field.

5. The method for creating a branch detection model according to claim 1, characterized in that: The YOLOv5 model also includes: the detection head outputs N types of size feature maps, where N=4.

6. A tree branch detection method for photovoltaic power plant equipment, characterized in that: The tree branch detection method specifically comprises: obtaining a picture of the photovoltaic power plant equipment, and detecting the picture of the photovoltaic power plant equipment based on a tree branch detection model created according to any one of claims 1 to 5 to obtain a detection result.

7. The branch detection method according to claim 6, characterized in that: When the detection result shows that the photovoltaic power plant equipment is blocked by tree branches, an early warning is issued.

8. A device for creating a tree branch detection model for a photovoltaic power plant, characterized in that: Includes the following modules: An information acquisition module, used to acquire training samples, wherein the training samples include a plurality of training images and an identification result corresponding to each training image, wherein the identification result is used to characterize whether the training image is blocked by a branch and the position of the branch blocking, and each training image includes the photovoltaic power plant equipment; A model creation module is used to create a YOLOv5 model, wherein the YOLOv5 model includes: an input end, a backbone network, a neck network, a detection head and an output end; the input end is used to receive the image to be detected and perform adaptive scaling, normalization and image enhancement processing on the image to be detected; the backbone network is used to extract features of the image to be detected; the neck network is used to fuse the features extracted by the backbone network to obtain fused features; the detection head is used to detect photovoltaic power plant equipment and tree branches and output the detection results through the output end; The training module is used to train the YOLOv5 model based on the training samples and obtain a branch detection model.

9. A storage medium storing program instructions, characterized in that: When the program instructions are executed by a processor, the branch detection method according to any one of claims 6 to 7 is implemented.

10. A terminal, characterized in that: The method comprises a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the branch detection method according to any one of claims 6 to 7.

Images