A deep learning photovoltaic panel recognition method based on multi-scale Retinex algorithm

Abstract

The present invention discloses a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm in the field of new generation information technology, aiming to solve the problem that the detection accuracy of the existing technology needs to be improved. It includes: obtaining a remote sensing image data set; performing data annotation on the remote sensing image data set to obtain a labeled data set; performing data enhancement on the labeled data set to obtain a data enhanced data set; performing target detection based on the trained photovoltaic panel recognition model according to the data enhanced data set to obtain a recognition result. This patent can realize the rapid recognition of photovoltaic panels, help improve the planning of photovoltaic panel power generation, and understand the capacity and coverage of photovoltaic panels in various regions; it overcomes the phenomenon of unclear remote sensing images due to weather factors, has stronger feature recognition capabilities, and higher accuracy.

Description

Technical Field

[0001] The present invention relates to the field of new generation information technology, and in particular to a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm. Background Art

[0002] With the growth of human demand for energy and the decreasing energy resources, the attention paid to renewable energy at home and abroad has gradually increased. China is in a period of rapid industrialization and urbanization, and the field of new energy plays an extremely important role. Photovoltaic panels can better realize the configuration and utilization of electricity. For example, during peak electricity consumption, photovoltaics can provide more power supply and relieve the pressure on the power grid, while during low electricity consumption, photovoltaics can use excess power to charge or provide other household electricity. Considering the problems of low efficiency and poor accuracy of traditional manual detection methods, and the wide range and strong generalization of satellite images, deep learning technology and high-resolution remote sensing images can be used to effectively extract the location information and installed capacity information of photovoltaic panels. This method is not restricted by terrain and can easily detect the coverage of photovoltaic panels in various complex terrain areas, helping to optimize local energy planning. However, in practical applications, complex weather such as rain and fog will have a negative impact on target detection, causing a decrease in detection accuracy.

[0003] Therefore, the present invention proposes a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm, which can realize the rapid recognition of photovoltaic panels, help improve the planning of photovoltaic panel power generation, and understand the capacity and coverage of photovoltaic panels in various regions; it overcomes the phenomenon of unclear remote sensing images caused by weather factors, has stronger feature recognition capabilities, and higher accuracy. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm, which overcomes the phenomenon of unclear remote sensing images due to the limitations of weather factors, has stronger feature recognition capabilities, higher accuracy, and has practical significance and good application prospects.

[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] On the one hand, the present invention provides a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm, comprising:

[0007] Acquire remote sensing image datasets;

[0008] Annotating the remote sensing image data set to obtain an annotated data set;

[0009] Performing data enhancement on the labeled data set to obtain a data enhanced data set;

[0010] According to the data enhanced data set, based on the trained photovoltaic panel recognition model, target detection is performed to obtain a recognition result;

[0011] The training of the photovoltaic panel recognition model includes:

[0012] Construct a training dataset based on the collected remote sensing image dataset;

[0013] A photovoltaic panel recognition model is built, and the training data set is used to perform model training to obtain a trained photovoltaic panel recognition model.

[0014] Optionally, the data enhancement adopts a Mosaic method, which mainly includes the following steps:

[0015] Randomly select the coordinates of the reference points for image stitching , and then randomly select four pictures;

[0016] The four pictures are mapped according to the reference point coordinates. , after resizing and scaling, they are placed at the upper left, upper right, lower left, and lower right positions of the large image of the specified size;

[0017] According to the size transformation method of each image, the mapping relationship is mapped to the image label;

[0018] According to the specified horizontal and vertical coordinates, the large images are spliced;

[0019] Process the coordinates of the detection box that exceed the boundary.

[0020] Optionally, use the Retinex algorithm to perform image enhancement, including Gaussian blur processing and color restoration. The specific steps include:

[0021] Calculated by formula (1) The maximum value of and minimum value :

[0022] (1)

[0023] in, Represents the position coordinates of image elements, represents the image after image enhancement, Represents the remote sensing image data that has been obtained, , represents the image after Gaussian blur processing, represents the Gaussian function;

[0024] Each value calculated by formula (1) is Perform linear quantization:

[0025] (2)

[0026] Each scale is accumulated and calculated using formula (3):

[0027] (3)

[0028] in, Represents the weight corresponding to each scale, and the sum of the weights of each scale is 1. Indicates the scale number.

[0029] Optionally, the photovoltaic panel recognition model adopts a single-stage detection method to convert the target detection task into an end-to-end regression problem; the photovoltaic panel recognition model adopts an FPN+PAN structure and uses Darknet-53 as the backbone network.

[0030] Optionally, the photovoltaic panel recognition model uses a C2f module, and adds a SE attention mechanism to the C2f module.

[0031] Optionally, the working steps of the photovoltaic panel identification model are as follows:

[0032] The input dimension is The feature map X;

[0033] The feature map X is Operation, generate feature map U;

[0034] pass The function performs global average pooling on the feature map U to generate a A vector of

[0035] conduct Incentive operation, through function and two fully connected layers, the dimensions are The global description features of

[0036] pass The function multiplies the two-dimensional matrix of the corresponding channel of the global description feature with the feature map X to obtain a dimension of Output.

[0037] Optionally, the global average pooling is implemented by formula (4):

[0038] (4)

[0039] in, represents the global description feature, and Represents the dimension parameter, Indicates the first OK Elements of a column.

[0040] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on a multi-scale Retinex algorithm described in any one of the first aspects are implemented.

[0041] In a third aspect, the present invention provides a computer device / equipment / system, comprising:

[0042] Memory, for storing computer programs / instructions;

[0043] A processor is used to execute the computer program / instructions to implement the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm described in any one of the first aspects.

[0044] In a fourth aspect, the present invention provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on a multi-scale Retinex algorithm described in any one of the first aspects are implemented.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm provided by the present invention can realize the rapid recognition of photovoltaic panels, help improve the planning of photovoltaic panel power generation, and understand the capacity and coverage of photovoltaic panels in various regions; it overcomes the phenomenon of unclear remote sensing images caused by weather factors, has stronger feature recognition ability and higher accuracy;

[0047] 2. The computer-readable storage medium, computer device / equipment / system and computer program product provided by the present invention can execute the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm provided by the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a recognition effect diagram of a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm provided according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of an SE model of a deep learning photovoltaic panel identification method based on a multi-scale Retinex algorithm provided according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a Retinex algorithm of a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm provided according to an embodiment of the present invention;

[0051] Figure 4 This is a diagram of the image enhancement effect under complex weather conditions of the deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0053] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0054] Embodiment 1:

[0055] The embodiment of the present invention discloses a deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm, which specifically includes the following steps:

[0056] S1, obtain remote sensing image dataset;

[0057] S2, annotating the remote sensing image data set to obtain an annotated data set;

[0058] S3, performing data enhancement on the labeled data set to obtain a data enhanced data set;

[0059] S4, performing target detection according to the data enhanced data set and based on the trained photovoltaic panel recognition model to obtain a recognition result.

[0060] Specifically,

[0061] In step S1, GPS is used for field positioning. In this example, the remote sensing image of Xinchang County is taken as an example. The longitude and latitude boundaries of the remote sensing image are between 120°41′34″ and 121°13′34″ east longitude and 29°13′55″ and 29°33′52″ north latitude. The remote sensing image of the corresponding area is obtained by the Sentinel-2A satellite. Figure 1As shown, the recognition of photovoltaic panels in the remote sensing image of Xinchang County by the deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm described in this embodiment is shown.

[0062] In step S2, the LabelImg tool is used to annotate the photovoltaic areas in the remote sensing image and create corresponding category labels, convert the annotations into a format that the YOLO algorithm can understand, and organize the data set according to the requirements of the YOLO algorithm, which usually requires train, val, and labels folders.

[0063] In step S3, the data set is enhanced using the Mosaic method, which combines multiple images by random cropping and mixing to generate data enhancement similar to that of a single image, aiming to improve the detection of small objects. This allows the data of four images to be directly calculated, which increases the efficiency of training and improves the detection capability of the model. The main steps are as follows:

[0064] Randomly select the coordinates of the reference points for image stitching , and then randomly select four pictures;

[0065] The four pictures are mapped according to the reference point coordinates. , after resizing and scaling, they are placed at the upper left, upper right, lower left, and lower right positions of the large image of the specified size;

[0066] According to the size transformation method of each image, the mapping relationship is mapped to the image label;

[0067] According to the specified horizontal and vertical coordinates, the large images are spliced;

[0068] Process the coordinates of the detection box that exceed the boundary.

[0069] refer to Figure 3 and Figure 4 As shown, for target recognition images in complex weather conditions, this embodiment uses the Retinex algorithm to enhance the remote sensing image and improve the image contrast and visual effect by performing Gaussian blur processing and color restoration operations on the image.

[0070] Retinex theory believes that an image is composed of an illumination image and a reflection image, and its mathematical expression is: ,in is the position coordinate of a pixel in the image, represents the image signal being observed or received by the camera, Represents the illumination component of ambient light, Represents the reflection component of the target object that carries image detail information.

[0071] In order to facilitate calculation, the above formula is transformed to obtain formula (1):

[0072] (1)

[0073] in, Represents the position coordinates of image elements, represents the image after image enhancement, Represents the remote sensing image data that has been obtained, , represents the image after Gaussian blur processing, represents a Gaussian function.

[0074] The main principle of Retinex is to remove the influence of irradiation light and retain the reflective properties of the object itself. , calculate the corresponding ,but This is the enhanced image.

[0075] In this embodiment, the calculation The maximum value of and minimum value, then for each value Perform linear quantization and Quantized to pixel values ​​ranging from 0 to 255 as the final output, as shown in formula (2):

[0076] (2)

[0077] The multi-scale Retinex algorithm used in this embodiment is based on the above method to perform Gaussian blur on the original image at each scale to obtain the blurred image. , among which the small represents the number of scales; then each scale is accumulated and calculated using formula (3):

[0078] (3)

[0079] in, Represents the weight corresponding to each scale, and the sum of the weights of each scale is 1. Indicates the scale number.

[0080] The basic calculation principle above is as follows:

[0081] in, is the original input image, is the Gaussian blur function, is the number of scales, is the weight for each scale.

[0082] In step S4, the training of the photovoltaic panel recognition model includes:

[0083] Construct a training dataset based on the collected remote sensing image dataset;

[0084] A photovoltaic panel recognition model is built, and the training data set is used to perform model training to obtain a trained photovoltaic panel recognition model.

[0085] The photovoltaic panel recognition model uses Darknet-53 as the backbone network, with the aim of processing large-scale image data and extracting relatively rich feature information; Darknet-53 is a convolutional neural network model, and its structure can be divided into three groups: the front section is mainly composed of convolutional layers and max-pooling layers, the middle section is mainly composed of residual blocks, and the back section is mainly composed of global average pooling layers and fully connected layers.

[0086] The photovoltaic panel recognition model adopts the FPN+PAN structure to enhance the network feature fusion capability. The main goal of FPN is to solve the problems of information loss and resolution mismatch during target detection and segmentation at different scales. In order to establish high-level semantic feature mapping at multiple scales, it is a top-down framework with horizontal connections. PAN expands the idea of ​​FPN and improves the feature fusion method. It mainly improves the target detection effect by fusing high- and low-level features, especially the detection effect of small-sized targets.

[0087] refer to Figure 2 As shown in the figure, the C2f module is used in the photovoltaic panel recognition model, which can combine global semantic information and local target information to better understand the image and make the target detector pay more attention to meaningful areas; and the SE attention mechanism is added to the C2f module to achieve more efficient and accurate image processing; the working steps of the photovoltaic panel recognition model are as follows:

[0088] The input dimension is The feature map X;

[0089] The feature map X is Operation, generate feature map U;

[0090] pass The function performs global average pooling on the feature map U to generate a The global average pooling is implemented by formula (4):

[0091] (4)

[0092] in, represents the global description feature, and Represents the dimension parameter, Indicates the first OK Elements of a column;

[0093] conduct Incentive operation, through function and two fully connected layers, the dimensions are There is a hyperparameter r between the two fully connected layers. The vector is reduced to , normalized to a set of 0~1 real numbers through the Sigmoid activation function. This set of real numbers corresponds to the importance of each channel, 1 is very important, 0 is not important, and the final output is ;

[0094] pass The function multiplies the two-dimensional matrix of the corresponding channel of the global description feature with the feature map X to obtain a dimension of Output.

[0095] The loss function of the photovoltaic panel recognition model is divided into classification loss and regression loss, and its formula is as follows:

[0096] Binary classification function:

[0097]

[0098] in Representation sample The label of the positive class is 1 and the negative class is 0.

[0099] The multi-classification function is an extension of the binary classification function. The specific formula is as follows:

[0100]

[0101] in is the number of categories, is a symbolic function, if the sample The true category is equal to Take 1, otherwise take 0; For the observed sample Belongs to category The predicted probability of .

[0102] VFL LOSS function:

[0103]

[0104] in is the intersection-over-union ratio of the predicted box and the true box, that is, the intersection of the predicted box and the true box divided by the union of the two boxes, is probability; is a positive sample, is a negative sample.

[0105] Conventional coordinate point regression assumes that the probability of a certain point is infinite, while the probability of other points is 0. This method assumes that the label is absolutely correct. However, in reality, there is a certain degree of uncertainty for targets in occluded and blurred scenes. At this time, conventional regression methods cannot solve this problem. In this case, it is more reasonable to use boundary distribution. Considering that the real distribution is usually not too far from the labeled position, the DFL LOSS method used in this method makes the network concentrate the value on the label Nearby, increase and The specific function is as follows:

[0106]

[0107] in: .

[0108] CIoU is an improved version based on the IoU loss function. It takes into account the complete intersection between target boxes and introduces a correction factor to more accurately measure the similarity between target boxes. The formula involved is as follows:

[0109] , ,

[0110] in is the distance between the center point of the predicted box and the real box, is the diagonal distance of the minimum enclosing rectangle, Is a correction factor, which is used to further adjust the loss function. The specific calculation method is:

[0111]

[0112] in and are the width and height of the target box and the predicted box respectively.

[0113] Embodiment 2:

[0114] This embodiment provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm as described in any one of the first embodiments are implemented.

[0115] Embodiment three:

[0116] This embodiment provides a computer device / equipment / system, including:

[0117] Memory, for storing computer programs / instructions;

[0118] A processor is used to execute the computer program / instructions to implement the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm as described in any one of the first embodiments.

[0119] Embodiment 4:

[0120] This embodiment provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm as described in any one of the first embodiments are implemented.

[0121] In summary, the deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm provided by the present invention has the following beneficial effects:

[0122] 1. It can realize the rapid identification of photovoltaic panels, help improve the planning of photovoltaic panel power generation, and understand the capacity and coverage of photovoltaic panels in various regions;

[0123] 2. It overcomes the problem of unclear remote sensing images due to weather factors, has stronger feature recognition capabilities and higher accuracy;

[0124] 3. The present invention can be implemented by pure software or hardware.

[0125] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A deep learning photovoltaic panel recognition method based on a multi-scale Retinex algorithm, characterized in that: include: Acquire remote sensing image datasets; Annotating the remote sensing image data set to obtain an annotated data set; Performing data enhancement on the labeled data set to obtain a data enhanced data set; According to the data enhanced data set, based on the trained photovoltaic panel recognition model, target detection is performed to obtain a recognition result; The training of the photovoltaic panel recognition model includes: Construct a training dataset based on the collected remote sensing image dataset; Building a photovoltaic panel recognition model, using the training data set to perform model training, and obtaining a trained photovoltaic panel recognition model; The photovoltaic panel recognition model adopts a single-stage detection method to transform the target detection task into an end-to-end regression problem; the photovoltaic panel recognition model adopts an FPN+PAN structure and uses Darknet-53 as the backbone network; The photovoltaic panel recognition model uses the C2f module and adds the SE attention mechanism to the C2f module; The working steps of the photovoltaic panel identification model are as follows: The input dimension is The feature map X; The feature map X is Operation, generate feature map U; pass The function performs global average pooling on the feature map U to generate a A vector of conduct Incentive operation, through function and two fully connected layers, the dimensions are The global description features of pass The function multiplies the two-dimensional matrix of the corresponding channel of the global description feature with the feature map X to obtain a dimension of Output: It also includes image enhancement using the Retinex algorithm, including Gaussian blur processing and color restoration. The specific steps include: Calculated by formula (1) The maximum value of and minimum value : （1） in, Represents the position coordinates of image elements, represents the image after image enhancement, Represents the remote sensing image data that has been obtained, , represents the image after Gaussian blur processing, represents the Gaussian function; Each value calculated by formula (1) is Perform linear quantization: （2） Each scale is accumulated and calculated using formula (3): （3） in, Represents the weight corresponding to each scale, and the sum of the weights of each scale is 1. Indicates the scale number.

2. The deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm according to claim 1 is characterized in that: The data enhancement adopts the Mosaic method, which mainly includes the following steps: Randomly select the coordinates of the reference points for image stitching , and then randomly select four pictures; The four pictures are mapped according to the reference point coordinates. , after resizing and scaling, they are placed at the upper left, upper right, lower left, and lower right positions of the large image of the specified size; According to the size transformation method of each image, the mapping relationship is mapped to the image label; According to the specified horizontal and vertical coordinates, the large images are spliced; Process the coordinates of the detection box that exceed the boundary.

3. The deep learning photovoltaic panel recognition method based on the multi-scale Retinex algorithm according to claim 1 is characterized in that: The global average pooling is implemented by formula (4): （4） in, represents the global description feature, and Represents the dimension parameter, Indicates the first OK Elements of a column.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm described in any one of claims 1 to 3 are implemented.

5. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm described in any one of claims 1 to 3.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the deep learning photovoltaic panel identification method based on the multi-scale Retinex algorithm described in any one of claims 1 to 3 are implemented.

Images