A hot spot category identification method and device, electronic equipment, medium and product

Abstract

The application discloses a hot spot category identification method and device, electronic equipment, medium and product. The method comprises the following steps: acquiring a visible light picture of a photovoltaic module and an infrared picture corresponding to the visible light picture; detecting a first position coordinate of a hot spot component from the infrared picture; positioning the first position coordinate to the visible light picture to determine a second position coordinate of the first position coordinate in the visible light picture; identifying a target hot spot component picture corresponding to the second position coordinate to determine a hot spot category corresponding to the hot spot component. The method can identify the category of the hot spot component from the visible light picture by positioning the position of the hot spot component to the visible light picture, so that the maintenance personnel can maintain the faulty component according to the category of the hot spot component.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a hot spot category identification method and device, electronic equipment, medium and product. BACKGROUND

[0002] In the process of operation of a solar photovoltaic power station, hot spots are easily formed on solar cell modules, and operation and maintenance personnel need to maintain the photovoltaic modules according to the causes of the hot spots.

[0003] The prior art scheme can only detect hot spots according to images of photovoltaic modules, but cannot determine the hot spot category, and thus cannot effectively maintain the photovoltaic modules.

[0004] Therefore, how to identify the hot spot category is a technical problem to be solved at present. SUMMARY

[0005] Embodiments of the present application provide a hot spot category identification method and device, electronic equipment, medium and product to solve the problem that the prior art can only identify hot spot modules and cannot identify hot spot categories.

[0006] According to an aspect of the present application, a hot spot category identification method is provided, comprising:

[0007] obtaining a visible light picture of a photovoltaic module and an infrared picture corresponding to the visible light picture;

[0008] detecting a first position coordinate of a hot spot module from the infrared picture;

[0009] positioning the first position coordinate to the visible light picture to determine a second position coordinate of the first position coordinate in the visible light picture;

[0010] identifying a target hot spot module picture corresponding to the second position coordinate to determine a hot spot category corresponding to the hot spot module.

[0011] According to another aspect of the present application, a hot spot category identification device is provided, comprising:

[0012] an acquisition module configured to obtain a visible light picture of a photovoltaic module and an infrared picture corresponding to the visible light picture;

[0013] a detection module configured to detect a first position coordinate of a hot spot module from the infrared picture;

[0014] a determination module configured to position the first position coordinate to the visible light picture to determine a second position coordinate of the first position coordinate in the visible light picture;

[0015] A recognition module is configured to recognize the target hot spot component picture corresponding to the second position coordinate, and determine the hot spot category corresponding to the hot spot component.

[0016] According to another aspect of the present application, an electronic device is provided, which comprises:

[0017] at least one processor; and

[0018] a memory connected to the at least one processor in communication; wherein

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the hot spot category recognition method according to any one of the embodiments of the present application.

[0020] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform the hot spot category recognition method according to any one of the embodiments of the present application when executed by the processor.

[0021] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for enabling a processor to perform the hot spot category recognition method according to any one of the embodiments of the present application when executed by the processor.

[0022] The technical solution of the embodiments of the present application accurately locates the position of the hot spot component detected from the infrared picture in the visible light picture, and performs hot spot category recognition based on the visible light picture, thereby solving the shortcoming that the hot spot type cannot be recognized in the prior art, and achieving the beneficial effect that the hot spot category of the hot spot component is accurately recognized, and the maintenance personnel can maintain the faulty component according to the hot spot category.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 A flowchart of a hot spot category recognition method provided by Embodiment One of the present application;

[0026] Figure 2 A schematic diagram of model training and model application of a target classification network model in a hot spot classification method provided in the first embodiment of the present invention;

[0027] Figure 3 A schematic diagram of the target classification network model structure in a hot spot classification method provided in the first embodiment of the present invention;

[0028] Figure 4 A schematic flow chart of a hot spot classification identification method provided in the second embodiment of the present invention;

[0029] Figure 5 This is an example flow chart of a hot spot classification identification method provided in the third embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of an infrared image provided by the third embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of the infrared image segmentation effect provided by the third embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the visible light image segmentation effect provided by the third embodiment of the present invention;

[0033] Figure 9 A schematic diagram showing a line connecting the center point of a hot spot component in an infrared image and the center point of the infrared image provided by the third embodiment of the present invention;

[0034] Figure 10 A schematic diagram showing a line connecting the center point of a hot spot component in a visible light image and the center point of the visible light image provided by the third embodiment of the present invention;

[0035] Figure 11 This is a schematic diagram of a hot spot component provided in Example 3 of the present invention;

[0036] Figure 12 A schematic diagram of a hot spot classification device provided in the fourth embodiment of the present invention

[0037] Figure 13 This is a structural diagram of an electronic device provided in Example 5 of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method implementation mode of the present invention can be performed in different orders and / or in parallel. In addition, the method implementation mode may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0039] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0040] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0042] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0043] Example 1

[0044] Figure 1A flow chart of a hot spot category identification method provided in the first embodiment of the present invention is provided. The method can be applied to the situation of identifying the hot spot category corresponding to the hot spot components in the photovoltaic power station. The method can be executed by a hot spot category identification device, wherein the device can be implemented by software and / or hardware and is generally integrated into an electronic device. In this embodiment, the electronic device includes but is not limited to: computer equipment.

[0045] like Figure 1 As shown, a hot spot classification identification method provided by the first embodiment of the present invention includes the following steps:

[0046] S110 : Obtain a visible light image of the photovoltaic module and an infrared image corresponding to the visible light image.

[0047] The photovoltaic components may be photovoltaic panels in a photovoltaic power station.

[0048] In this embodiment, the method for obtaining the visible light image of the photovoltaic module and the corresponding infrared image of the visible light image is not limited. One feasible method is to use a drone to collect the visible light image and the infrared image of the photovoltaic module. The center point of the visible light image collected by the drone is aligned with the center point of the infrared image.

[0049] S120. Detecting the first position coordinates of the hot spot component from the infrared image.

[0050] Since the temperature distribution thermal imaging of the photovoltaic panel in different working states is different in the infrared image, the hot spot component can be detected in the infrared image and the first position coordinates of the hot spot component can be determined.

[0051] Among them, the first position coordinates are the center point coordinates of the hot spot component, and the center point coordinates of the hot spot component are determined according to the detection frame coordinates of the hot spot component in the infrared image.

[0052] In this embodiment, a preset detection algorithm can be used to detect photovoltaic modules that generate hot spots in infrared images to obtain the detection frame coordinates of the hot spot modules in the infrared images. The preset detection algorithm can be any target detection algorithm. For example, the preset detection algorithm can be a deep learning target detection algorithm, such as the YOLO series or SSD series target detection algorithms.

[0053] The coordinates of the center point of the hot spot component may be determined by calculating the coordinates of the center point of the detection frame according to the coordinates of the detection frame, and using the coordinates of the center point of the detection frame as the coordinates of the center point of the hot spot component.

[0054] S130: Position the first position coordinate in a visible light image, and determine a second position coordinate of the first position coordinate in the visible light image.

[0055] The second position coordinate can be understood as the position coordinate of the center point of the hot spot component in the visible light picture.

[0056] In this embodiment, since only the hot spot component can be detected in the infrared picture, but the hot spot category cannot be accurately identified, it is necessary to accurately position the hot spot component detected in the infrared picture to the visible light picture, so as to further identify the hot spot category corresponding to the hot spot component from the visible light picture. It should be noted that the hot spot category to be identified in this embodiment can include a hot spot caused by an occlusion and a hot spot of a broken component. The hot spot caused by the occlusion can be exemplarily understood as a hot spot caused by bird droppings. The hot spot of the broken component can be understood as a hot spot caused by the broken photovoltaic component. Therefore, in order to identify the hot spot category, it is necessary to accurately identify the occlusion and the broken component, and the broken component and the occlusion can be accurately identified only in the visible light picture.

[0057] In this embodiment, the hot spot component detected in the infrared picture can be positioned to the visible light picture by an image processing method to obtain the second position coordinate corresponding to the first position coordinate in the visible light picture.

[0058] This embodiment does not limit the way in which the first position coordinate is positioned to the visible light picture. The second position coordinate can be determined in the visible light picture by using geometric knowledge or optical knowledge. The process is not described in detail here, and a feasible implementation can refer to embodiment two.

[0059] S140, identifying a target hot spot component picture corresponding to the second position coordinate to determine the hot spot category corresponding to the hot spot component.

[0060] The target hot spot component picture can be a visible light picture in a target segmentation frame, and the target segmentation frame is a segmentation frame closest to the second position coordinate in the plurality of second segmentation frames.

[0061] In this embodiment, after the second position coordinate is determined, the target hot spot component picture needs to be further determined. The determination method of the target hot spot component picture can be that a segmentation frame closest to the second position coordinate in the plurality of second segmentation frames is taken as a target segmentation frame, and a visible light picture in the target segmentation frame is taken as the target hot spot component picture.

[0062] Furthermore, the identifying of the target hot spot component image corresponding to the second position coordinate to determine the hot spot category corresponding to the hot spot component includes: inputting the target hot spot component image corresponding to the second position coordinate into a target classification network model, identifying the target hot spot component image through the target classification network model, and determining the hot spot category corresponding to the hot spot component in the target hot spot component image; wherein, the target classification network model is obtained after model training based on multiple visible light images of hot spot components.

[0063] The hot spot categories determined in this embodiment include obstruction hot spots and broken module hot spots. Therefore, it is necessary to identify the hot spot category in the target hot spot module image. Obstruction hot spots can be understood as hot spots caused by obstructions, and broken module hot spots can be understood as hot spots caused by broken photovoltaic modules. The target hot spot module image can be identified using a classification network model. Preferably, the classification network model can be an AlexNet classification network model.

[0064] In one embodiment, if the identification result is that the hot spot component in the target hot spot component image is a fragmented component, then the hot spot category corresponding to the hot spot component is determined to be a fragmented component hot spot.

[0065] In one embodiment, if the recognition result is that there is an obstruction on the hot spot component in the target hot spot component image, the hot spot category corresponding to the hot spot component is determined to be an obstruction hot spot.

[0066] Figure 2 Schematic diagram of model training and model application of a target classification network model in a hot spot classification method provided in the first embodiment of the present invention, as shown in FIG. Figure 2 As shown, the model training process may include the following processes: collecting hot spot component images; classifying the hot spot component images to obtain a test set and a training set; constructing a classification network model, adjusting the model parameters, and training the classification network model on the training set until the model converges to obtain a trained model; evaluating the effect of the trained model on the test set, and outputting the target classification network model; inputting the target hot spot component image into the target classification network model for identification, and outputting the hot spot category of the identified hot spot component.

[0067] For example, Figure 3 This is a schematic diagram of the target classification network model structure in a hot spot classification method provided in the first embodiment of the present invention. Figure 3As shown in the figure, the target classification network model mainly consists of 5 convolutional layers, 5 pooling layers, and two fully connected layers. Among them, the first convolutional layer uses a 224*224*3 image as input and uses 96 11*11*3 convolution kernels with a stride of 4; the second convolutional layer uses the output of the first convolutional layer as input and uses 256 5*5*48 convolution kernels; the third convolutional layer uses the output of the second convolutional layer as input and uses 384 3*3*256 convolution kernels; the fourth and fifth convolutional layers use 384 3*3*192 and 256 3*3*192 convolution kernels respectively; each fully connected layer contains 4096 neurons.

[0068] It should be noted that this embodiment significantly reduces the sample size required compared to existing deep learning network models by inputting target hot spot component images into the target classification network model for identification. Only a little over 100 target hot spot component images are required to complete model training. Furthermore, since the target hot spot component images only include the hot spot components, the recognition range is smaller than that of the entire image, reducing the difficulty and improving the recognition speed.

[0069] A method for identifying hot spot categories is provided in a first embodiment of the present invention. The method first obtains a visible light image of a photovoltaic component and an infrared image corresponding to the visible light image; then detects the first position coordinates of the hot spot component from the infrared image; then locates the first position coordinates in the visible light image and determines the second position coordinates of the first position coordinates in the visible light image; finally, the target hot spot component image corresponding to the second position coordinates is identified to determine the hot spot category corresponding to the hot spot component. The above method can identify the category of the hot spot component from the visible light image by locating the position of the hot spot component in the visible light image. This method can accurately identify hot spot components such as broken components, making it easier for operation and maintenance personnel to perform corresponding maintenance on faulty components based on the hot spot category.

[0070] Example 2

[0071] Figure 4 This is a flow chart of a method for identifying hot spot categories, provided in Example 2 of the present invention. This Example 2 is optimized based on the above-mentioned examples. This Example provides a specific process for locating a first position coordinate in a visible light image. In this Example, a first average value of the length and width of all components in the infrared image and a second average value of the length and width of all components in the visible light image can be determined. Based on the first position coordinate, the first average value, and the second average value, geometric knowledge can be used to determine the second position coordinate. For details not yet fully detailed in this Example, please refer to Example 1.

[0072] like Figure 4 As shown, a hot spot classification identification method provided by the second embodiment of the present invention includes the following steps:

[0073] S210: Obtain a visible light image of the photovoltaic module and an infrared image corresponding to the visible light image.

[0074] S220: Detect the first position coordinates of the hot spot component from the infrared image.

[0075] S230 , segmenting the photovoltaic component parts in the infrared image and the visible light image respectively to obtain a plurality of first segmentation frames and a plurality of second segmentation frames.

[0076] In this embodiment, the image segmentation method can be used to segment the photovoltaic modules in the infrared image and the visible light image. The specific segmentation steps are as follows:

[0077] Step 1: grayscale the image and then perform Canny edge processing to obtain the edge image;

[0078] Step 2: Search for straight lines in the edge image using HoughLines Hough line transform;

[0079] Step 3: Determine the grid boundary line of the photovoltaic module area according to the grid size characteristics inside the photovoltaic module area;

[0080] Step 4: Calculate the coordinates of the four intersection points where the four grid boundary lines of each photovoltaic module intersect, thereby achieving the goal of segmenting all photovoltaic modules in the module area image.

[0081] The first segmentation frame may be understood as a segmentation frame obtained by segmenting the photovoltaic components in the infrared image, and the second segmentation frame may be understood as a segmentation frame obtained by segmenting the photovoltaic components in the visible light image.

[0082] S240: Determine first lengths and first widths of all components in the infrared image based on the multiple first segmentation frames, and use an average value of the first lengths and the first widths as a first average value.

[0083] The first length may be understood as the length of all components in the infrared image, the first width may be understood as the width of all components in the infrared image, and the first average value may be understood as the average value of the length and width of all components in the infrared image.

[0084] In the embodiment, the process of determining the first length and the first width of all components can be: calculating the average length of the plurality of first segmentation frames as the first length of all components, and calculating the average width of the plurality of first segmentation frames as the first width of all components; wherein the length of a first segmentation frame is the difference between the vertical coordinates of the upper frame and the lower frame of the first segmentation frame, and the width of a first segmentation frame is the difference between the horizontal coordinates of the right frame and the left frame of the first segmentation frame.

[0085] S250, determining the second length and the second width of all components in the visible light picture based on the plurality of second segmentation frames, and taking the average of the first length and the second width as a second average.

[0086] It should be noted that the execution of S240 and S250 is not in a specific order, and can be executed simultaneously.

[0087] The second length can be understood as the length of all components in the visible light picture, the second width can be understood as the width of all components in the visible light picture, and the second average can be understood as the average of the length and the width of all components in the visible light picture.

[0088] In the embodiment, the process of determining the second length of all components can be: calculating the average length of the plurality of second segmentation frames as the second length of all components. The length of a second segmentation frame is the difference between the vertical coordinates of the upper frame and the lower frame of the second segmentation frame, and the width of a second segmentation frame is the difference between the horizontal coordinates of the right frame and the left frame of the second segmentation frame.

[0089] S260, determining the second position coordinates of the hot spot component in the visible light picture based on the first average, the second average, the first position coordinates, the center point coordinates of the infrared picture, and the center point coordinates of the visible light picture.

[0090] In the embodiment, since the center points of the collected visible light picture and infrared light image are aligned, the position coordinates of the hot spot component in the visible light picture can be determined based on the first average, the second average, the first position coordinates, the center point coordinates of the infrared picture, and the center point coordinates of the visible light picture, combined with geometric knowledge.

[0091] Specifically, the second position coordinates of the hot spot component in the visible light picture are determined based on the first average value, the second average value, the first position coordinates, the coordinates of the center point of the infrared picture, and the coordinates of the center point of the visible light picture, including: determining a first distance based on the first position coordinates and the coordinates of the center point of the infrared picture, the first distance being the distance between the center point of the hot spot component and the center point of the infrared picture; determining the angle between the line and the horizontal axis; taking the ratio of the first distance to the first average value as a proportional coefficient; taking the product of the proportional coefficient and the second average value as the second distance, the second distance being the distance between the center point of the hot spot component in the visible light picture and the center point of the visible light picture; determining the second position coordinates of the hot spot component in the visible light picture based on the coordinates of the center point of the visible light picture, the angle, and the second distance.

[0092] Among them, geometric knowledge is used to calculate the proportional coefficient in the infrared light image, and the distance between the center point of the hot spot component in the visible light image and the center point of the visible light image is calculated based on the proportional coefficient and the second average value. Then, geometric knowledge is used to determine the position coordinates of the hot spot component in the visible light image based on the distance.

[0093] Furthermore, determining the second position coordinates of the hot spot component in the visible light image based on the coordinates of the center point of the visible light image, the angle and the second distance includes: determining the center point of the visible light image based on the coordinates of the center point of the visible light image; starting from the center point of the visible light image, making an extension line along the direction of the angle, the length of the extension line being the second distance; and using the end point coordinates of the extension line as the second position coordinates of the hot spot component in the visible light image.

[0094] S270. Identify the target hot spot component image corresponding to the second position coordinates, and determine the hot spot category corresponding to the hot spot component.

[0095] A second embodiment of the present invention provides a method for identifying hot spot categories, specifically locating the first position coordinates in a visible light image and determining the second position coordinates of the first position coordinates within the visible light image. This method leverages the ease with which visible light images can be used for image classification and recognition to determine the second position coordinates corresponding to the hot spot component within the visible light image, thereby accurately identifying the category of the hot spot component based on the visible light image.

[0096] Example 3

[0097] The embodiment of the present invention provides a specific implementation method based on the technical solutions of the above embodiments. Figure 5This is an example flow chart of a method for identifying hot spot categories provided by the third embodiment of the present invention. Figure 5 As shown, the hot spot category identification method may include the following steps:

[0098] Step 110: Detect the location of the faulty component in the infrared image;

[0099] The faulty component is the hot spot component, and the position of the faulty component is the first position.

[0100] In this step, the drone inspects photovoltaic modules according to the planned path, collecting visible light and infrared images. The deep learning target detection algorithm model is used to detect photovoltaic modules that produce hot spots in the infrared images to obtain the coordinates of the hot spot module detection frame.

[0101] This function can be implemented using various deep learning target detection and recognition algorithm models, such as the YOLO series or the SSD series. These models detect and identify the hot spot components that generate hot spots in infrared images and provide the pixel coordinates of the hot spot component rectangle, or detection frame. Figure 6 This is a schematic diagram of an infrared image provided by the third embodiment of the present invention. Figure 6 A hot spot assembly rectangle is shown.

[0102] Step 120: Position the hot spot components detected in the infrared image in the visible light image using image processing related methods.

[0103] In this step, the photovoltaic modules in the infrared image and the visible light image are segmented using an image segmentation method. Figure 7 This is a schematic diagram of the infrared image segmentation effect provided by the third embodiment of the present invention. Figure 8 This is a schematic diagram of the visible light image segmentation effect provided by Example 3 of the present invention.

[0104] Based on the coordinates of the rectangular frames obtained by segmentation in the infrared image and the visible light image, namely the first segmentation frame and the second segmentation frame, the average values ​​of the length and width of the photovoltaic modules in the infrared image and the visible light image, namely the first average value and the second average value, are calculated. The calculation formula is as follows:

[0105]

[0106]

[0107]

[0108] Among them, w i Indicates the width of the i-th rectangular box, h iRepresents the length of the i-th rectangular box, N represents the number of rectangular boxes, and Avg represents the average length and width of the hot spot component.

[0109] The above formulas can be used to calculate the average value Avg1 of the length and width of all components in the infrared image, i.e., the first average value, and the average value Avg2 of the length and width of all components in the visible light image, i.e., the second average value.

[0110] The distance Dis1 between the center point of the hot spot component and the center point of the infrared image is calculated from the infrared image, and the angle α between the line and the horizontal axis is calculated. Figure 9 This is a schematic diagram of the line connecting the center point of the hot spot component in the infrared image and the center point of the infrared image provided by the third embodiment of the present invention.

[0111] Calculate the proportional coefficient r between Dis1 and Avg1. The calculation formula is as follows:

[0112]

[0113] Starting from the center of the visible light image and at an angle α to the horizontal axis, draw a line of length Dis2 in the visible light image. The end point of this line is the coordinate position of the center of the hot spot component in the visible light image, which is the second position coordinate. The calculation formula for Dis2 is: Dis2 = r * Avg2. Figure 10 This is a schematic diagram of a line connecting the center point of a hot spot component in a visible light image and the center point of the visible light image provided by the third embodiment of the present invention.

[0114] Step 130: crop a hot spot component image from the corresponding position of the visible light image, and input the hot spot component image into the AlexNet classification network model to identify the hot spot category.

[0115] According to the coordinate position of the center point of the hot spot component in the visible light image, that is, the second position coordinate, find the rectangular frame closest to the center point of the hot spot component from the rectangular frame coordinate set obtained by segmenting the visible light image, and according to this rectangular frame coordinate, crop the hot spot component image, that is, the target hot spot component image, from the corresponding position of the visible light image. Figure 11 This is a schematic diagram of the hot spot component provided in Example 3 of the present invention.

[0116] A third embodiment of the present invention provides a method for identifying hot spot categories. This method locates the hot spot components detected in an infrared image in a visible light image through image processing related methods, crops the hot spot component image from the corresponding position in the visible light image, and identifies the hot spot category by inputting the hot spot component image into the AlexNet classification network model, thereby determining the category of the hot spot.

[0117] Example 4

[0118] Figure 12 This is a structural schematic diagram of a hot spot category identification device provided in Example 4 of the present invention. The device can be used to identify the hot spot category corresponding to the hot spot components in a photovoltaic power station, wherein the device can be implemented by software and / or hardware and is generally integrated on an electronic device.

[0119] like Figure 12 As shown, the device includes: an acquisition module 110 , a detection module 120 , a determination module 130 and an identification module 140 .

[0120] An acquisition module 110 is configured to acquire a visible light image of a photovoltaic module and an infrared image corresponding to the visible light image;

[0121] A detection module 120 is configured to detect a first position coordinate of a hot spot component from the infrared image;

[0122] a determination module 130 configured to locate the first position coordinate in a visible light image and determine a second position coordinate of the first position coordinate in the visible light image;

[0123] The identification module 140 is used to identify the target hot spot component image corresponding to the second position coordinates and determine the hot spot category corresponding to the hot spot component.

[0124] In this embodiment, the device first obtains a visible light image of the photovoltaic component and an infrared image corresponding to the visible light image through the acquisition module 110; then detects the first position coordinates of the hot spot component from the infrared image through the detection module 120; then locates the first position coordinates in the visible light image through the determination module 130, and determines the second position coordinates of the first position coordinates in the visible light image; finally, the recognition module 140 identifies the target hot spot component image corresponding to the second position coordinates, and determines the hot spot category corresponding to the hot spot component.

[0125] This embodiment provides a hot spot category identification device, which can identify the category of the hot spot component from the visible light image by locating the position of the hot spot component in the visible light image. This method can accurately identify hot spot components such as broken components, making it easier for operation and maintenance personnel to maintain faulty components.

[0126] Furthermore, the first position coordinates are the coordinates of the center point of the hot spot component, and the coordinates of the center point of the hot spot component are determined according to the detection frame coordinates of the hot spot component in the infrared image.

[0127] Furthermore, the determination module 130 is specifically used to: respectively divide the photovoltaic component parts in the infrared picture and the visible light picture to obtain multiple first segmentation frames and multiple second segmentation frames; determine the first length and first width of all components in the infrared picture based on the multiple first segmentation frames, and take the average of the first length and the first width as the first average; determine the second length and second width of all components in the visible light picture based on the multiple second segmentation frames, and take the average of the first length and the second width as the second average; determine the second position coordinates of the hot spot component in the visible light picture based on the first average, the second average, the first position coordinates, the coordinates of the center point of the infrared picture and the coordinates of the center point of the visible light picture.

[0128] Based on the above technical solution, the second position coordinates of the hot spot component in the visible light picture are determined based on the first average value, the second average value, the first position coordinates, the coordinates of the center point of the infrared picture and the coordinates of the center point of the visible light picture, including: determining a first distance based on the first position coordinates and the coordinates of the center point of the infrared picture, the first distance being the distance of the line between the center point of the hot spot component and the center point of the infrared picture; determining the angle between the line and the horizontal axis; taking the ratio of the first distance to the first average value as a proportional coefficient; taking the product of the proportional coefficient and the second average value as the second distance, the second distance being the distance of the line between the center point of the hot spot component in the visible light picture and the center point of the visible light picture; determining the second position coordinates of the hot spot component in the visible light picture based on the coordinates of the center point of the visible light picture, the angle and the second distance.

[0129] Furthermore, determining the second position coordinates of the hot spot component in the visible light image based on the coordinates of the center point of the visible light image, the angle and the second distance includes: determining the center point of the visible light image based on the coordinates of the center point of the visible light image; starting from the center point of the visible light image, making an extension line along the direction of the angle, the length of the extension line being the second distance; and using the end point coordinates of the extension line as the second position coordinates of the hot spot component in the visible light image.

[0130] The target hot spot component image is a visible light image within a target segmentation frame, and the target segmentation frame is the segmentation frame closest to the second position coordinate among the multiple second segmentation frames.

[0131] Furthermore, the identification module 140 is specifically used to: input the target hot spot component image corresponding to the second position coordinate into the target classification network model, identify the target hot spot component image through the target classification network model, and determine the hot spot category corresponding to the hot spot component in the target hot spot component image; wherein, the target classification network model is obtained after model training based on multiple visible light images of hot spot components.

[0132] Further, if the recognition result is that the hot spot component in the target hot spot component image is a fragmented component, then the hot spot category corresponding to the hot spot component is determined to be a fragmented component hot spot.

[0133] The above-mentioned hot spot category identification device can execute the hot spot category identification method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0134] Example 4

[0135] Figure 13 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0136] like Figure 13 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0138] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the hot spot category identification method.

[0139] In some embodiments, the hot spot category identification method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps in the hot spot category identification method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the hot spot category identification method in any other appropriate manner (for example, by means of firmware).

[0140] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0145] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0146] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0147] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for identifying hot spot categories, characterized in that: The method comprises: Obtaining a visible light image of the photovoltaic module and an infrared image corresponding to the visible light image; Detecting a first position coordinate of the hot spot component from the infrared image; Positioning the first position coordinate in a visible light image, and determining a second position coordinate of the first position coordinate in the visible light image; Identify the target hot spot component image corresponding to the second position coordinates, and determine the hot spot category corresponding to the hot spot component; The step of locating the first position coordinates in the visible light image and determining the second position coordinates of the first position coordinates in the visible light image includes: Segmenting the photovoltaic component parts in the infrared image and the visible light image respectively to obtain a plurality of first segmentation frames and a plurality of second segmentation frames; determining first lengths and first widths of all components in the infrared image based on the multiple first segmentation frames, and taking an average of the first lengths and the first widths as a first average; determining second lengths and second widths of all components in the visible light image based on the plurality of second segmentation frames, and taking an average of the first lengths and second widths as a second average; Based on the first average value, the second average value, the first position coordinates, the coordinates of the center point of the infrared image, and the coordinates of the center point of the visible light image, the second position coordinates of the hot spot component in the visible light image are determined.

2. The method according to claim 1, characterized in that The first position coordinates are the center point coordinates of the hot spot component, and the center point coordinates of the hot spot component are determined according to the detection frame coordinates of the hot spot component in the infrared image.

3. The method according to claim 1, characterized in that The determining, based on the first average value, the second average value, the first position coordinates, the coordinates of the center point of the infrared image, and the coordinates of the center point of the visible light image, of the second position coordinates of the hot spot component in the visible light image includes: Determine a first distance based on the first position coordinates and the coordinates of the center point of the infrared image, where the first distance is the distance between the center point of the hot spot assembly and the center point of the infrared image; Determining the angle between the connecting line and the horizontal axis; using a ratio of the first distance to the first average value as a proportional coefficient; multiplying the proportional coefficient by the second average value as a second distance, where the second distance is the distance between a center point of a hot spot component in the visible light image and a center point of the visible light image; The second position coordinates of the hot spot component in the visible light image are determined according to the coordinates of the center point of the visible light image, the angle, and the second distance.

4. The method according to claim 3, characterized in that The determining the second position coordinates of the hot spot component in the visible light image according to the coordinates of the center point of the visible light image, the angle, and the second distance includes: Determine the center point of the visible light image according to the coordinates of the center point of the visible light image; Starting from the center point of the visible light image, an extension line is drawn along the direction of the angle, and the length of the extension line is the second distance; The end point coordinates of the extended line are used as the second position coordinates of the hot spot component in the visible light image.

5. The method according to claim 1, wherein The target hot spot component image is a visible light image within a target segmentation frame, and the target segmentation frame is the segmentation frame closest to the second position coordinate among the multiple second segmentation frames.

6. The method according to claim 1, characterized in that The identifying the hot spot component image corresponding to the second position coordinate and determining the hot spot category corresponding to the hot spot component includes: Inputting the target hot spot component image corresponding to the second position coordinate into the target classification network model, identifying the target hot spot component image through the target classification network model, and determining the hot spot category corresponding to the hot spot component in the target hot spot component image; The target classification network model is obtained after model training based on multiple visible light images of hot spot components.

7. The method according to claim 6, characterized in that If the identification result is that the hot spot component in the target hot spot component image is a fragmented component, then the hot spot category corresponding to the hot spot component is determined to be a fragmented component hot spot.

8. A hot spot classification identification device, characterized in that: The device comprises: An acquisition module, configured to acquire a visible light image of the photovoltaic module and an infrared image corresponding to the visible light image; A detection module, configured to detect a first position coordinate of a hot spot component from the infrared image; a determination module, configured to locate the first position coordinate in a visible light image and determine a second position coordinate of the first position coordinate in the visible light image; an identification module, configured to identify the target hot spot component image corresponding to the second position coordinate, and determine the hot spot category corresponding to the hot spot component; Among them, the determination module is specifically used to: respectively divide the photovoltaic component parts in the infrared picture and the visible light picture to obtain multiple first segmentation frames and multiple second segmentation frames; determine the first length and first width of all components in the infrared picture based on the multiple first segmentation frames, and take the average of the first length and the first width as the first average; determine the second length and second width of all components in the visible light picture based on the multiple second segmentation frames, and take the average of the first length and the second width as the second average; determine the second position coordinates of the hot spot component in the visible light picture based on the first average, the second average, the first position coordinates, the coordinates of the center point of the infrared picture and the coordinates of the center point of the visible light picture.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; The one or more programs are executed by the one or more processors, so that the one or more processors are used to execute the hot spot category identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the hot spot category identification method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the hot spot category identification method according to any one of claims 1 to 7.

Images