Batch lamp fault detection method and device based on unmanned aerial vehicle inspection and medium

Abstract

The invention provides a batch lamp fault detection method and device based on unmanned aerial vehicle inspection and a medium, and relates to the technical field of computers. The batch lamp fault detection method comprises the following steps: acquiring a to-be-detected lamp effect image at a geographic coordinate of an inspection point through an unmanned aerial vehicle, and matching a corresponding reference lamp effect image group according to the geographic coordinate of the inspection point, and performing feature fusion on each reference lamp effect image including various lamp effects in the reference lamp effect image group and the lamp effect image to be detected to obtain a target lamp effect fusion image, inputting the target lamp effect fusion image into a pre-trained fault area position detection model to obtain an initial lamp fault area, screening a target lamp fault area from the initial lamp fault area, and obtaining a target lamp fault detection result. And determining the position of the fault lamp through the target lamp fault area. According to the invention, the comprehensiveness of batch lamp detection can be improved, the fault detection time of batch lamps can be shortened, the fault lamp detection efficiency can be improved, and the accuracy of the detection result can be improved.

Description

Background Art

[0002] In large-scale landscape lighting projects (for example, building landscape lighting projects, stadium landscape lighting projects, or road and bridge landscape lighting projects, etc.), the number of lamps is often tens of thousands to hundreds of thousands, and the chromaticity and brightness of the light source of the lamps in the working state will change accordingly with the needs of the scene.

[0003] In large-scale landscape lighting projects, the failure of installed batches of lamps is generally discovered through manual observation. The comprehensiveness and efficiency of manual observation of batches of lamps are low, and it is difficult to locate faulty lamps. This is a pain point in the field of landscape lighting, and there is no effective solution so far. In related technologies, RDM (Raw Device Mapping) lamps can be used to realize lamp signal collection and fault analysis. However, due to the high cost of RDM lamps, if they are used in large-scale landscape lighting projects, the cost of the project will be greatly increased.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a batch lamp fault detection method based on drone inspection, a batch lamp fault detection device based on drone inspection, and a computer-readable storage medium, which to a certain extent solve the problems of incomplete fault detection of batch lamps, high lamp fault diagnosis cost, and low accuracy and efficiency of lamp fault detection.

[0006] According to one aspect of an embodiment of the present disclosure, a method for detecting faults of batch lamps based on drone inspection is provided, comprising:

[0007] Obtain the lighting effect image to be detected collected by the drone when performing the inspection task at the geographical coordinates of the inspection point;

[0008] Matching a reference lighting effect image group in a pre-built normal lighting effect image library according to the geographical coordinates of the inspection points, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly shooting at the geographical coordinates of all inspection points corresponding to the inspection tasks by cyclically performing inspection tasks by drones when ensuring that the batch of lamps are intact and present different lighting effects;

[0009] Performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image;

[0010] Input the target lighting effect fusion image into the pre-trained fault area location detection model to determine the initial lamp fault area in the target lighting effect fusion image;

[0011] The target lamp fault area is screened in the initial lamp fault area, and the position coordinates of the faulty lamp are determined through the target lamp fault area.

[0012] According to a second aspect of an embodiment of the present disclosure, a batch lamp fault detection device based on drone inspection is provided, comprising:

[0013] A lighting effect image acquisition module is used to acquire the lighting effect image to be detected collected by the drone at the geographical coordinates to be detected;

[0014] A lighting effect image matching module, used for matching a reference lighting effect image group in a pre-built normal lighting effect image library according to the geographic coordinates to be detected, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly shooting at the geographic coordinates of all inspection points corresponding to the inspection task by the drone while ensuring that the batch of lamps are intact and present different lighting effects;

[0015] A lighting effect image fusion module is used to perform feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image;

[0016] A lamp fault area estimation module is used to input the target lighting effect fusion image into a pre-trained fault area location detection model to determine the initial lamp fault area in the target lighting effect fusion image;

[0017] The lamp fault area screening module is used to screen the target lamp fault area in the initial lamp fault area, and determine the position coordinates of the faulty lamp through the target lamp fault area.

[0018] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the batch lamp fault detection method based on drone inspection as in the first aspect is implemented.

[0019] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0020] Through the embodiments of the present disclosure, a drone can be used to collect lighting effect images to be detected at the geographic coordinates of the inspection point, and the corresponding reference lighting effect image group can be matched according to the geographic coordinates of the inspection point. The reference lighting effect image group including various lighting effects is feature fused with the lighting effect image to be detected to obtain a target lighting effect fusion image, and the image is input into a pre-trained fault area location detection model to obtain an initial lamp fault area, and the target lamp fault area is screened from the initial lamp fault area to determine the location of the faulty lamp through the target lamp fault area. On the one hand, the use of drones to collect the lighting effect images to be detected at the geographic coordinates of the inspection points can be unaffected by the height and angle of the location to be detected, thereby ensuring comprehensive monitoring of the lighting effects presented by batch lamps; on the other hand, by performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected, and then inputting the fused target lighting effect fusion image into the pre-trained fault area location detection model to obtain the corresponding fault area, the efficiency of faulty lamp detection can be improved, and at the same time, the interference of the outdoor environment on each lighting effect image can be eliminated, thereby ensuring the robustness when detecting batch lamps, thereby improving the accuracy of the predicted fault area; on the other hand, the fault area location detection model is used to detect faulty lamps, rather than relying on special lamps to perform self-inspection to determine whether a fault has occurred, thereby reducing the detection cost of faulty lamps to a certain extent.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] Figure 1 A flow chart of a method for detecting faults of batch lamps based on drone inspection in an embodiment of the present disclosure is schematically shown.

[0024] Figure 2 A flowchart of obtaining a target lighting effect fusion image using an image fusion model based on a channel attention mechanism in an embodiment of the present disclosure is schematically shown.

[0025] Figure 3 A schematic diagram schematically shows the principle of determining a target lighting effect fusion image through a channel attention network in an embodiment of the present disclosure.

[0026] Figure 4 A flowchart of obtaining a target lighting effect fusion image by image registration in an embodiment of the present disclosure is schematically shown.

[0027] Figure 5A schematic diagram schematically shows a principle of predicting a target lamp fault area in a target lighting effect fusion image through a fault area position detection model in an embodiment of the present disclosure.

[0028] Figure 6 The present invention schematically shows a composition diagram of a batch lamp fault detection device based on drone inspection in an embodiment of the present invention.

[0029] Figure 7 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown schematically.

[0030] Figure 8 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is schematically shown.

[0031] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] In the related art, there are the following technical problems:

[0036] When lighting the building landscape, the light source lamps are very small on the building facade. When a lamp fails, the overall landscape lighting effect will be affected. It is not only unsightly, but also difficult to identify the occurrence and area of ​​the fault with the naked eye, which brings an operation and maintenance bottleneck to the entire building landscape lighting work. At present, the detection of lamp faults takes a long time, the monitoring of the lighting effects presented by batch lamps is not comprehensive, the fault area cannot be quickly located, and the cost of self-inspection of special lamps is high. These are all problems that cannot be solved in the building lighting industry so far.

[0037] Based on one or more problems in the related art, the embodiment of the present disclosure first proposes a batch lamp fault detection method based on drone inspection. The method can be executed by a control terminal or server of the batch lamps. The following is an example of the server executing the method.

[0038] like Figure 1 As shown, Figure 1 The present disclosure is a flowchart of a method for detecting faults of batch lamps based on drone inspection according to an exemplary embodiment, comprising the following steps:

[0039] Step S110, obtaining the lighting effect image to be detected collected by the drone when performing the inspection task at the geographical coordinates of the inspection point;

[0040] Step S120, matching a reference lighting effect image group in a pre-built normal lighting effect image library according to the geographical coordinates of the inspection points, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly photographing the geographical coordinates of all inspection points corresponding to the inspection tasks by cyclically performing inspection tasks by drones when ensuring that the batch of lamps are intact and present different lighting effects;

[0041] Step S130, performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image;

[0042] Step S140, inputting the target lighting effect fusion image into a pre-trained fault area location detection model to determine an initial lamp fault area in the target lighting effect fusion image;

[0043] Step S150 , screening a target lamp fault area in the initial lamp fault area, and determining the position coordinates of the faulty lamp through the target lamp fault area.

[0044] Through the embodiments disclosed in the present invention, on the one hand, the use of drones to collect lighting effect images to be detected at the geographic coordinates of the inspection point can be unaffected by the height and angle of the location to be detected, thereby ensuring comprehensive monitoring of the lighting effects presented by batch lamps; on the other hand, by performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected, and then inputting the fused target lighting effect fusion image into the pre-trained fault area location detection model to obtain the corresponding fault area, the efficiency of faulty lamp detection can be improved, and at the same time, the interference of the outdoor environment on each lighting effect image can be eliminated, thereby ensuring the robustness when detecting batch lamps, thereby improving the accuracy of the predicted fault area; on the other hand, the fault area location detection model is used to detect faulty lamps, instead of relying on special lamps to perform self-inspection to determine whether a fault has occurred, thereby reducing the detection cost of faulty lamps to a certain extent.

[0045] Next, steps S110 to S150 are described in detail.

[0046] In step S110, the lighting effect image to be detected collected by the drone when performing the inspection task at the geographical coordinates of the inspection point is obtained.

[0047] In an example embodiment of the present disclosure, the geographic coordinates of the inspection point refer to the geographic coordinates of the inspection point preset by the drone when performing the inspection task, which are used to match the corresponding pre-stored complete lighting effect image. For example, a pre-stored complete lighting effect image with the same coordinates can be selected according to the geographic coordinates of the inspection point, or all pre-stored complete lighting effect images within a certain range of the geographic coordinates of each inspection point can be selected according to the geographic coordinates of the inspection point. Of course, a suitable method can also be selected according to the specific situation to select the complete lighting effect image. This embodiment does not specifically limit the method of matching the corresponding complete lighting effect image according to the geographic coordinates of the inspection point.

[0048] The lighting effect image to be detected refers to an image collected in real time at the geographical coordinates of the inspection point when the drone flies along a preset route, and is used to detect whether there are faulty lamps in the batch of lamps in the corresponding landscape lighting scene. The lighting effect image to be detected can be determined by taking a photo with the drone camera at the geographical coordinates of the inspection point, or by recording a video with the drone camera at the geographical coordinates of the inspection point. Of course, the method for the drone to obtain the lighting effect image to be detected can also be set according to the specific situation. This embodiment does not specifically limit the method for obtaining the lighting effect image to be detected.

[0049] Optionally, drones can be used to perform rough detection of fault conditions of batch lamps at the geographic coordinates of each inspection point, screen out multiple inspection point geographic coordinates with a higher probability of faulty lamps, and then use the lighting effect images taken by the drone at different angles at the screened multiple inspection point geographic coordinates as the lighting effect images to be detected. By further analyzing the lighting effect images to be detected with a higher probability of faulty lamps, the area where the faulty lamps are located can be quickly located, thereby improving the efficiency of batch lamp fault detection.

[0050] In step S120, a reference lighting effect image group is matched in a pre-constructed normal lighting effect image library according to the geographic coordinates of the inspection points, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by cyclically performing inspection tasks by drones and taking multiple shots at the geographic coordinates of all inspection points corresponding to the inspection tasks while ensuring that batch lamps are intact and present different lighting effects.

[0051] In an example implementation of the present disclosure, the normal lighting effect image library refers to the lighting effect images of the batch of lamps captured and stored at the drone inspection site when the landscape lighting project is completed, which are all normally lit, and are used as reference images for the lighting effect images to be detected. The normal lighting effect image library includes a reference lighting effect image group corresponding to each inspection point when the drone performs an inspection task, so that the reference lighting effect image group corresponding to the reference lighting effect image to be detected can be matched according to the geographic coordinates of the inspection point.

[0052] Each reference lighting effect image in the reference lighting effect image group can be collected by a drone camera when the landscape lighting project is completed and all lamps can be lit normally. Since each lamp is lit normally, it may be affected by different scenes and the chromaticity and brightness of each lamp to produce different lighting effects. When the lighting effect image at each inspection point is taken by a drone, the accuracy of the GPS positioning system may cause certain deviations in the positioning of the geographic coordinates of the inspection point and the shooting angle each time. Therefore, lighting effect images showing a variety of lighting effects taken at different angles within a certain range from each inspection point can be used as reference lighting effect images of the corresponding inspection point, thereby forming a reference lighting effect image group including multiple reference lighting effect images.

[0053] In step S130 , feature fusion is performed on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fused image.

[0054] In an example implementation of the present disclosure, a target lighting effect fusion image refers to a lighting effect image determined based on the difference features between a lighting effect image to be detected and each reference lighting effect image in a matching reference lighting effect image group, and is used as input information for further fault detection. The target lighting effect fusion image can be obtained by fusing the difference features of each reference lighting effect image with the lighting effect image to be detected in combination with the importance of each reference lighting effect image in the reference lighting effect image group to the lighting effect image to be detected, or by fusing the difference features between the lighting effect image to be detected and the reference lighting effect image in the reference lighting effect image group that has the greatest similarity to the lighting effect image to be detected, or by fusing the difference features between the local images of the lighting effect image to be detected and multiple reference lighting effect images in the reference lighting effect image group that have the greatest similarity to the reference lighting effect image to be detected. This embodiment does not particularly limit the method for determining the target lighting effect fusion image.

[0055] Optionally, when fusing the differential features between the reference lighting effect image and the lighting effect image to be detected, corresponding weight information can be set for the reference lighting effect image and the lighting effect image to be detected, and weighted summation is performed on each pixel in the reference lighting effect image and the pixel at the corresponding position in the lighting effect image to be detected according to the set weight to obtain each pixel after fusion of the differential features, thereby obtaining a target lighting effect fusion image; or each reference lighting effect image in the matched reference lighting effect image group can be input into a pre-trained image fusion model, and the differential features between each reference lighting effect image and the lighting effect image to be detected are fused through the image fusion model to obtain a target lighting effect fusion image. Of course, the fusion of differential features between lighting effect images can also be achieved in other ways. This embodiment does not specifically limit the method of performing feature fusion between lighting effect images.

[0056] Optionally, when an image fusion model is used to fuse the differential features between each reference lighting effect image and the lighting effect image to be detected, the image fusion model needs to be trained using the lighting effect images. Specifically, a large number of lighting effect images to be detected can be labeled using the labelimg program, and the corresponding reference lighting effect image group can be screened out from the pre-built normal lighting effect image library, so as to construct a sample set according to each reference lighting effect image in the reference lighting effect image group, each lighting effect image to be detected, and the corresponding annotation information, and then the sample set is split into a training set, a validation set, and a test set in a ratio of 8:1:1, and the lighting effect images to be detected with annotation information in the training set and the corresponding reference lighting effect images are input into the image fusion model for differential feature fusion, and the hyperparameters of the training image fusion model, such as the network structure and the learning rate, are adjusted according to the location of the actual faulty lamp in the validation set data and the image area obtained by the fusion of the differential features, and then the image fusion model is judged whether the training is completed by performing a comprehensive analysis on the test set data, and the training is stopped when the image fusion model is about to overfit, so as to obtain a trained image fusion model.

[0057] In step S140, the target lighting effect fusion image is input into a pre-trained fault area location detection model to determine the initial lamp fault area in the target lighting effect fusion image.

[0058] In an example implementation of the present disclosure, the fault area location detection model refers to a model constructed based on a deep learning target detection algorithm, which is used to determine the area where the faulty lamp is located when there is a faulty lamp in the target lighting effect fusion image, thereby determining the specific location of the faulty lamp. The deep learning target detection algorithm in the fault area location detection model can be a two-stage detection algorithm, such as the RCNN (Region-CNN) series algorithm, or a one-stage detection algorithm, such as the YOLO series detection algorithm. Of course, a suitable detection algorithm can also be selected according to the specific situation to construct the fault area location detection model. This embodiment does not specifically limit the type of deep learning target detection algorithm relied on when constructing the fault area location detection model.

[0059] Optionally, the features in the target lighting effect fusion image obtained by fusion of differential features are relatively obvious, so a detailed description is given by taking the fault area location detection model determined by the YOLO series detection algorithm as an example. It should be noted that if an image fusion model is used to realize the fusion of differential features between lighting effect images, the image fusion model can be independent of the fault area location detection model, that is, the output of the image fusion model is used as the input of the fault area location detection model, that is, the fused lighting effect image obtained when training the image fusion model is used to train the fault location area detection model to perform fault detection on suspicious lamps in the target lighting effect fusion image, or the image fusion model and the fault area location detection model can be used as a complete model to perform batch lamp fault detection, and the present disclosure does not make any special limitations on this.

[0060] The initial lamp fault area refers to an area where the probability of the existence of a faulty lamp is greater than a certain threshold, and is used to determine the faulty lamp. The initial lamp fault area can be determined by the neural network in the fault area location detection model. For example, when the target lighting effect fusion image is input into the fault area location detection model, the corresponding neural network will obtain multiple areas where faulty lamps may exist and the corresponding probabilities of the existence of faulty lamps through a series of analyses, so that the corresponding predicted fault area can be screened out from the area where each predicted faulty lamp is located by setting the corresponding threshold, and then the initial lamp fault area in the target lighting effect fusion image is determined based on the location information of the screened predicted fault area for further analysis.

[0061] In step S150, a target lamp fault area is screened in the initial lamp fault area, and the position coordinates of the faulty lamp are determined through the target lamp fault area.

[0062] It should be noted that there will be overlapping parts between the predicted initial lamp fault areas. When multiple initial lamp fault areas overlap and are large, the corresponding initial lamp fault areas are likely to represent the failure of the same faulty lamp. If all the initial lamp fault areas with overlapping parts are retained, it will lead to repeated detection of the same target, which will increase the calculation burden and reduce the fault detection efficiency. By deleting the initial lamp fault areas with large overlapping parts, the repeated detection can be reduced, thereby improving the fault detection speed.

[0063] In an example implementation of the present disclosure, the target lamp fault area refers to the area where the faulty lamp exists, which is used as a basis for locating the location of the faulty lamp. The faulty area can be drawn in the lighting effect image to be detected according to the position coordinates of the target lamp fault area, and then the position coordinates of the faulty lamp can be accurately located by gradually troubleshooting the faulty area in the lighting effect image to be detected. The area where the faulty lamp is located can be identified and detected by a pre-trained fault area position detection model, which can effectively improve the efficiency and accuracy of fault detection of batch lamps.

[0064] The technical solutions involved in step S110 to step S150 are explained in detail below.

[0065] In an exemplary embodiment of the present disclosure, during the inspection process, the drone may perform a rough detection process of the lamp fault condition on the geographical coordinates of each inspection point to obtain the lamp effect image to be detected at the geographical coordinates of the inspection point. For example, the rough detection process of the drone may include the following steps:

[0066] The drone can obtain the original lighting effect image collected at the geographic coordinates of the inspection point, and the original lighting effect image includes at least two lighting effect images collected by the drone at the same position at a preset time interval; the drone can perform color space conversion on the original lighting effect image to obtain the converted original lighting effect image, and the converted original lighting effect image includes a brightness channel; the drone divides the original lighting effect image according to the preset grid parameters, and determines the average brightness value corresponding to each divided image grid according to the brightness value corresponding to the brightness channel; the drone calculates the brightness difference between the average brightness value of any image grid and the average brightness value of the remaining other image grids. If there is an image grid whose brightness difference is greater than or equal to the difference threshold, the position coordinates corresponding to the image grid are used as the coordinates of the local fault area, and the lighting effect images to be detected at different shooting angles are collected at the coordinates of the local fault area using a preset fault detection route.

[0067] The original lighting effect image refers to the lighting effect image taken by the drone at each geographic coordinate to be detected when the drone performs the inspection task in a loop, which is used to preliminarily determine whether there is a faulty lamp. The original lighting effect image can be multiple lighting effect images taken multiple times at the same location at a preset interval. For example, the preset interval time can be 0.5s or 1s. As long as the original lighting effect image includes lighting effect images with different contents at the geographic coordinates of the inspection point and the normal lamps in the original lighting effect image can be lit, it can be customized according to the specific content of the lighting effect on the building. This embodiment does not specifically limit the collection interval of the original lighting effect image.

[0068] Color space conversion refers to the process of converting the color space of the original lighting effect image by the drone into a color space that can represent the brightness value. Of course, if the drone can directly capture the image brightness, there is no need to perform color space conversion. For example, if the original lighting effect image captured by the drone is in RGB color space, the RGB color space can be converted to YUV color space. Of course, the RGB color space can also be converted to HSV color space. There are brightness channels in both YUV color space and HSV color space (such as the brightness channel can be the Y channel in the YUV color space, or the brightness channel can also be the V channel in the HSV color space). This example embodiment does not specifically limit this.

[0069] The brightness value corresponding to each pixel in the original lighting effect image can be obtained through the brightness channel, and then the brightness value corresponding to each image grid in the aligned and superimposed original lighting effect image can be determined. Therefore, it is possible to preliminarily judge whether there is a faulty lamp by analyzing the difference in brightness values ​​between each image grid. For example, on the premise of ensuring that the collected original lighting effect image contains as many different lighting effect images as possible, each normal lamp has the opportunity to be lit. If a lamp has not been lit, the brightness value of the image grid where it is located will inevitably be significantly lower than the brightness value of the surrounding image grids. Therefore, it can be considered that the image grid may contain a faulty lamp.

[0070] Optionally, the image area where the faulty lamp is located can be screened out by setting a brightness difference threshold. In the application scenario of the present invention, since point light source lamps are used, the brightness of each lamp cannot be determined due to the influence of lighting in the image. The preset grid parameters can be set and the image grid can be divided by the preset grid parameters. The average brightness value of the pixel points in the image grid can be determined instead of analyzing the brightness value of each lamp, which can effectively reduce the difficulty of analysis and improve the efficiency of analysis.

[0071] For example, the original lighting effect image can be evenly divided into 100 image grids of 10*10 by setting preset grid parameters such as 10*10, and then the average brightness value corresponding to each image grid is determined according to the brightness value of each pixel in each image grid, and the brightness difference between the average brightness value of any image grid and the average brightness value of the remaining image grids is calculated, and then the image grids greater than the difference threshold can be screened out according to the determined brightness difference, that is, there may be faulty lamps in the screened image grids, and the coordinates of the local fault area corresponding to the image grid on the actual building are determined according to the position coordinates of the screened image grid in the entire image.

[0072] The preset fault detection route refers to the flight route of the pre-set drone for fault collection in the fault area. For example, the preset fault detection route can control the drone to collect lighting effect images corresponding to the coordinates of the local fault area at multiple angles and use them as the lighting effect images to be detected to improve the detection efficiency of faulty lamps. The preset fault route can be set around the coordinates of the local fault area, or it can be set according to the end points and center position coordinates of the local fault area coordinates. This embodiment does not specifically limit the setting method of the fault detection route for the drone to collect lighting effect images at the coordinates of the local fault area.

[0073] In an exemplary embodiment of the present disclosure, after the drone obtains the lighting effect image to be detected through the rough detection process, the lighting effect image to be detected can be transmitted back to the control center for further detection in real time or offline transmission. Figure 2 The steps in the above figure are used to determine the target lighting effect fusion image:

[0074] Step S210, splicing each reference lighting effect image and the lighting effect image to be detected in the channel direction to obtain a spliced ​​lighting effect image;

[0075] Step S220, inputting the spliced ​​lighting effect image into a pre-trained image fusion model based on a channel attention mechanism, and outputting an initial lighting effect fusion image;

[0076] Step S230 , splicing the initial lighting effect fusion image and the lighting effect image to be detected in the channel direction to obtain a target lighting effect fusion image.

[0077] Among them, the spliced ​​lighting effect image refers to a lighting effect image formed by aligning and splicing the lighting effect image to be detected with the corresponding reference lighting effect images, which is used to extract the difference features between the reference lighting effect images and the lighting effect image to be detected as a whole. The position of each lighting effect image in the spliced ​​lighting effect image can be set according to the specific situation. Optionally, in order to ensure the standardization of the feature map obtained by extracting features from the reference lighting effect images in the spliced ​​lighting effect image and the lighting effect image to be detected, the lighting effect image to be detected can be arranged on one side after the reference lighting effect images are arranged side by side. This embodiment does not specifically limit the position of each lighting effect image in the spliced ​​lighting effect image.

[0078] The channel attention mechanism model refers to an image fusion model that can fuse the information of each lighting effect image in the spliced ​​lighting effect image by adding an attention mechanism to multiple channels, and is used to determine the optimal fused image for the current lighting effect image to be detected, that is, the target lighting effect fused image. The fused image output by the channel attention mechanism model can be a target lighting effect fused image, or a fused lighting effect image with labeled information used to train a fault area location detection model. This embodiment does not specifically limit the specific type of the fused image output by the attention mechanism model.

[0079] The initial lighting effect fusion image refers to an image obtained by fusing the reference lighting effect images according to their importance to the lighting effect image to be detected, and is used for further information fusion to obtain the target lighting effect fusion image input into the fault area location detection model. Specifically, the initial lighting effect fusion image includes the information of each reference lighting effect image, and the neural network in the trained channel attention mechanism model can automatically determine the importance of each reference lighting effect image to the lighting effect image to be detected. The more important the reference lighting effect image is, the greater the similarity between it and the lighting effect image to be detected, and the correspondingly larger proportion in the initial fused lighting effect image, so that the target lighting effect fusion image obtained by further information fusion of the initial lighting effect fusion image can include all the information of all reference lighting effect images and the lighting effect image to be detected.

[0080] The initial lighting effect fusion image and the lighting effect image to be detected can be spliced ​​in the channel direction, and the features of the initial lighting effect fusion image and the lighting effect image to be detected can be further fused. On the one hand, it can prevent the loss of some important information caused by error accumulation of the channel attention mechanism model, data noise and potential conflicts between the reference lighting effect images, thereby enhancing the integrity and effectiveness of the target lighting effect fusion image; on the other hand, the initial lighting effect fusion image may contain some under-utilized information. By fusing the lighting effect image to be detected, the corresponding information can be further extracted and utilized.

[0081] Optionally, the image fusion model may include a channel attention network, and the determination of the initial lighting effect fusion image may be achieved through the following steps:

[0082] The spliced ​​lighting effect image can be input into the channel attention network to obtain the attention weights corresponding to each reference lighting effect image, and the information of each reference lighting effect image can be fused according to the attention weights to output the initial lighting effect fusion image.

[0083] refer to Figure 3 As shown, each reference lighting effect image (A 1 , A 2 , …, A n ) is spliced ​​with the lighting effect image to be detected in step S210 to obtain a spliced ​​lighting effect image, and then the spliced ​​lighting effect image is input into the channel attention network. The channel attention network performs convolution and downsampling operations on the spliced ​​lighting effect image to extract the features of the spliced ​​lighting effect image, thereby obtaining a corresponding feature map, and then passes through a fully connected layer to normalize the feature map into an n-dimensional vector (α 1 , α 2 , …, α n ), where α i (i=1, 2, ..., n) represents the reference lighting effect image Ai (i=1, 2, ..., n) and then use equation (1) to calculate the initial lighting effect fusion image:

[0084]

[0085] Among them, i=1, 2,..., n, A α represents the initial lighting effect fusion image, i represents the reference lighting effect image and the corresponding attention weight, n represents the number of reference lighting effect images, A i represents the i-th reference lighting effect image, α i Represents the reference lighting effect image A i The corresponding attention weight, ∑ represents the weighted summation symbol.

[0086] In an exemplary embodiment of the present disclosure, Figure 4 The steps in the above figure are used to determine the target lighting effect fusion image:

[0087] Step S410, extracting feature points from each reference lighting effect image in the reference lighting effect image group to determine first image feature points, and extracting feature points from the lighting effect image to be detected to determine second image feature points;

[0088] Step S420, performing feature matching on the first image feature points and the second image feature points to determine feature point pairs between each reference lighting effect image and the lighting effect image to be detected;

[0089] Step S430, determining the similarity and image offset parameters between each reference lighting effect image and the lighting effect image to be detected according to the feature point pairs, and determining the target reference lighting effect image according to the similarity;

[0090] Step S440, based on the image offset parameter between the target reference lighting effect image and the lighting effect image to be detected, the target reference lighting effect image and the lighting effect image to be detected are image registered and weightedly fused to obtain a target lighting effect fused image.

[0091] Among them, feature point extraction refers to some points extracted from each lighting effect image that can represent the features of the corresponding lighting effect image, which are used to determine the first image feature points corresponding to each reference lighting effect image and the second image feature points corresponding to the lighting effect image to be detected. The first image feature points and the second image feature points corresponding to each lighting effect image can be extracted by the SIFT (Scale-invariant feature transform) descriptor, or by the SURF (Speeded UpRobust Feature) descriptor. Of course, the appropriate method can be selected for feature point extraction according to the specific situation. This embodiment does not specifically limit the method for extracting feature points from each lighting effect image.

[0092] Optionally, the first image feature points and the second image feature points can still be identified under image transformation operations such as rotation and scaling. Feature matching can be performed on the first image feature points and the second image feature points to find out the feature points with the same features corresponding to each reference lighting effect image and the lighting effect image to be detected, thereby forming a feature point pair. For example, feature matching can be performed by calculating the Euclidean distance between the first image feature points and the second image feature points. The smaller the Euclidean distance, the more similar the two feature points are, thereby determining the corresponding feature point pairs.

[0093] The similarity and image offset parameters between each reference lighting effect image and the lighting effect image to be detected can be determined based on the feature point pairs. For example, the number of matching feature point pairs between each reference lighting effect image and the lighting effect image to be detected can be counted, and then normalized within the range of 0-1 to determine the similarity between each reference lighting effect image and the lighting effect image to be detected. Optionally, the image offset parameters can be determined by the position distance between the first image feature point and the second image feature point in each feature point pair, so as to achieve spatial alignment between each reference lighting effect image and the lighting effect image to be detected based on the image offset parameters, so as to perform further processing. The image offset parameters can include translation data parameters, rotation angle parameters, and of course, other parameters that can adjust the position of each lighting effect image. This embodiment does not specifically limit the type of image offset parameters, the feature matching method, and the method for determining the similarity between each lighting effect image and the image offset parameters.

[0094] Among them, the target reference lighting effect image refers to the reference lighting effect image with the greatest similarity to the lighting effect image to be detected, and is used to fuse with the lighting effect image to be detected to obtain the corresponding target lighting effect fusion image. There may be a certain distance between the position of the target reference lighting effect image and the position of the lighting effect image to be detected, so it can be adjusted through image offset data. The position of the target reference lighting effect image can be adjusted through image offset data, and the position of the lighting effect image to be detected can also be adjusted. This embodiment does not specifically limit the type of lighting effect image to be adjusted, as long as the target reference lighting effect image and the lighting effect image to be detected are aligned.

[0095] Optionally, after completing the registration of the lighting effect image, the target pixel value corresponding to the target fusion pixel point can be determined by weighting the corresponding pixel values ​​of the target reference lighting effect image and the lighting effect image to be detected, so as to determine the target lighting effect fusion image according to each target fusion pixel point, thereby improving the accuracy of lighting effect image fusion, retaining target features, and suppressing noise and interference. For example, when the similarity between the target reference lighting effect image and the lighting effect image to be detected is high, such as 99%, the weight of the pixel value corresponding to each pixel point in the lighting effect image to be detected can be set to 0.6, and the weight of the pixel value corresponding to each pixel point in the target reference lighting effect image can be set to 0.4, and then the pixel value corresponding to each target fusion pixel point is determined by weighted summation between the pixels at the same position. For example, weighted fusion can be performed through equation (2):

[0096]

[0097] Among them, P {R，G，B} is the pixel value of each target fusion pixel in the target lighting effect fusion image, {R, G, B} represents the three color channel values ​​of the lighting effect image, R represents the red channel, G represents the green channel, B represents the blue channel, a represents the weight corresponding to the target reference lighting effect image, b represents the weight corresponding to the lighting effect image to be detected, a and b are both constants greater than 0 and less than 1, and the sum of a and b is 1, such as a can be 0.4, b can be 0.6, Represents the pixel value of each pixel in the target reference lighting effect image, Indicates the pixel value of each pixel in the lighting effect image to be detected.

[0098] In an exemplary embodiment of the present disclosure, the determination of the target lighting effect fusion image can be achieved by the following steps:

[0099] The reference lighting effect images in the reference lighting effect image group are stitched together to obtain a reference lighting effect stitched image, feature points are extracted from the reference lighting effect stitched image to determine feature points of the third image, and feature points are extracted from the lighting effect image to be detected to determine feature points of the second image, feature matching is performed between the third image feature points and the second image feature points to determine feature point pairs between the reference lighting effect stitched image and the lighting effect image to be detected, a reference lighting effect local image is determined in the reference lighting effect stitched image based on the feature point pairs, and the reference lighting effect local image is weightedly fused with the lighting effect image to be detected to obtain a target lighting effect fused image.

[0100] The reference lighting effect stitching image refers to a lighting effect image formed by stitching together the reference lighting effect images, and is used to determine the local image with the greatest similarity to the lighting effect image to be detected. In landscape lighting projects, the reference lighting effect images can be stitched together according to the lighting effect presented by the entire floor during the stitching process to improve the efficiency of feature matching. The size and shape of the reference lighting effect stitching image are related to the size and number of the reference lighting effect images. The reference lighting effect stitching image may be an image of equal length and width, or an image of different lengths and widths. This embodiment does not specifically limit the shape of the reference lighting effect stitching image.

[0101] The reference lighting effect partial image refers to the regional image of each reference lighting effect image corresponding to the region with the greatest similarity to the lighting effect image to be detected in the reference stitched image, and is used for feature fusion with the lighting effect image to be detected to obtain the target lighting effect fusion image. The reference lighting effect image partial image can be the side portion of the reference lighting effect image, or the portion at the end point of the reference lighting effect image, and of course, it can also be the portion extending to the center of the reference lighting effect image. This embodiment does not specifically limit the position of the reference lighting effect partial image in each reference lighting effect image.

[0102] Optionally, the feature points of the third image can be determined by the feature point extraction method described in step S410, and the feature point pairs between the reference lighting effect stitched image and the lighting effect image to be detected can be determined by the feature matching method described in step S420. Then, the reference lighting effect local image in the reference lighting effect stitched image can be determined by the method for determining the similarity and image offset parameters between the lighting effect images in step S430, and the pixel values ​​corresponding to each pixel point in the reference lighting effect local image and the pixel values ​​corresponding to the pixel points at the same position in the lighting effect image to be detected are weightedly fused by equation (2) to obtain the target pixel values ​​corresponding to each target fused pixel point, so as to obtain the target lighting effect fused image.

[0103] Optionally, the target lighting effect fusion image can be obtained by at least one of the three image fusion methods in the embodiment of the present disclosure; the target lighting effect fusion image can also be determined separately by at least two of the three image fusion methods in the embodiment of the present disclosure, and this example embodiment does not make any special limitations on this.

[0104] Exemplarily, when the target lighting effect fusion image is determined by combining three image fusion methods, namely, an image fusion model based on an attention mechanism, screening target reference lighting effect fusion images based on similarity and weighted fusion of the lighting effect images to be detected, and splicing the reference lighting effect images and weighted fusion of the lighting effect images to be detected, the weight of the first target lighting effect fusion image determined by the image fusion model based on the attention mechanism can be set to 0.4, the weight of the second target lighting effect fusion image determined by screening target reference lighting effect fusion images based on similarity and weighted fusion of the lighting effect images to be detected can be set to 0.3, and the weight of the third target lighting effect fusion image obtained by splicing the reference lighting effect images and weighted fusion of the lighting effect images to be detected can be set to 0.3, so that the pixel values ​​corresponding to each pixel point in the corresponding target lighting effect fusion image can be weightedly summed according to the set weights to obtain the target lighting effect fusion image. Of course, the weights corresponding to each fusion method can also be set according to the specific situation. The present embodiment does not specifically limit the size of the weights corresponding to each fusion method.

[0105] In an exemplary embodiment of the present disclosure, the fault area location detection model may include a preprocessing layer, a feature extraction layer, a fully connected layer, and a priori frame generation layer, and the determination of the initial lamp fault area may be achieved through the following steps:

[0106] The target lighting effect fusion image is divided into multiple grid areas according to the preset grid division parameters through the preprocessing layer, and the target number of prior frames with different width and height values ​​are constructed in the grid area through the prior frame generation layer; the feature extraction layer uses the grid area as the analysis unit to extract features of the target lighting effect fusion image to obtain a lighting effect fusion image feature map; the lighting effect fusion image feature map is input into the fully connected layer, and the confidence estimation vector is output, which includes the position prediction value of the update frame after each prior frame is updated, and the confidence of the update frame surrounding the task area; the target update frame with a confidence greater than or equal to the preset confidence threshold is determined, and the initial lighting fixture fault area is determined in the target lighting effect fusion image according to the position prediction value corresponding to the target update frame, and the confidence of the target update frame is used as the probability value of the presence of a faulty lamp in the initial lighting fixture fault area.

[0107] Among them, the preset grid division parameter refers to a value that can evenly divide the target lighting effect fusion image into a number of grids. For example, when the preset grid division parameter is S=15, the target lighting effect fusion image can be virtually divided into 15*15 grid areas, so that the fault area location detection model constructed based on the YOLO series detection algorithm can use each virtual grid area as an analysis unit for fault detection. The size of S can be designed according to the specific situation. This embodiment does not specifically limit the specific value of S.

[0108] Optionally, the size and aspect ratio of the area where the faulty lamp is located in the target lighting effect fusion image may be different. The prior frame generation layer can be used to construct a priori frames with different width and height values ​​for the target number in the grid area, so that the prior frame can accurately locate and identify targets of different sizes, thereby improving the flexibility and accuracy of fault detection. The type B of the prior frame width and height value can be determined when the training of the fault area position detection model is completed, or it can be obtained by clustering the width and height value distribution of the target to be detected in the target lighting effect image into B types of statistics. Of course, the type of prior frame width and height value can also be determined in an appropriate manner according to the specific situation. This embodiment does not specifically limit the number of types of prior frame width and height values.

[0109] Optionally, feature extraction layers, such as convolution layers and downsampling layers, can be used to perform feature extraction on the target lighting effect fusion image with the grid area as the analysis unit to obtain a feature map corresponding to the target lighting effect fusion image, that is, a lighting effect fusion image feature map. The lighting effect fusion feature map can then be input into a fully connected layer for normalization to obtain a one-dimensional vector corresponding to the lighting effect fusion image feature map.

[0110] Optionally, when it is necessary to normalize the lighting effect fusion feature map through a fully connected layer, it is necessary to scale the target lighting effect fusion image to an equal length and width through a preprocessing layer. If the convolutional layer in the fault area location detection model is used instead of the fully connected layer to process the lighting effect fusion image feature map, there is no need to scale the target lighting effect fusion image. The appropriate preprocessing method can be selected according to the specific situation. This embodiment does not specifically limit whether to scale the target lighting effect fusion image.

[0111] The confidence estimation vector refers to the position prediction value when the preset prior frame is updated according to the fully connected layer in the fault area location detection model, and the vector composed of the confidence of the adjusted prior frame, i.e., the update frame surrounding the faulty lamp. The position of the predicted fault area can be determined according to the confidence estimation vector. For example, when the fault area location detection model is trained, the fully connected layer can adjust the position of the center point of the preset prior frame and the corresponding width and height values ​​of the prior frame according to the learning results to obtain the corresponding update frame, so that the position of each update frame in the target lighting effect fusion image can be determined based on the position of the center point of the preset prior frame, the corresponding width and height values, and the position prediction value of the preset prior frame by the fully connected layer. Then, the position of the fault area can be determined by the confidence of each update frame surrounding the faulty lamp.

[0112] Optionally, the specific content of the confidence estimation vector can be determined according to the actual application scenario of the fault detection model. For example, in the present disclosure, the fault area location detection model is applied to the fault detection of batch lamps. Therefore, the probability that each predicted prior box surrounds the area where the faulty lamp is located can be incorporated into the confidence, thereby avoiding the prediction of the probability value of each preset box surrounding the area where the faulty lamp is located.

[0113] The confidence threshold refers to a reference value with a high probability that the update frame surrounds the area where the faulty lamp is located, and is used as a basis for selecting a target update frame from the update frame. The setting of the confidence threshold is related to the confidence corresponding to each update frame. When the confidence corresponding to each update frame is large, the confidence threshold can be selected as a large value. On the contrary, if the confidence corresponding to each update frame is small, the confidence threshold can be selected as a small value. This embodiment does not specifically limit the setting of the confidence threshold.

[0114] Optionally, each update frame can be screened by a confidence threshold to obtain a target update frame with a higher probability of surrounding a faulty lamp. Then, the position of the target update frame in the target lighting effect fusion image can be calculated by the confidence estimation vector to obtain the initial lamp fault area and the corresponding probability of surrounding the faulty lamp, thereby improving the efficiency of fault area detection and quickly locating the faulty lamp.

[0115] Specifically, the position prediction value in the confidence estimation vector can be used to calculate the position of the target update box in the target lighting effect fusion image, that is:

[0116]

[0117]

[0118]

[0119]

[0120] Among them, x can represent the horizontal coordinate information of the target update box in the target lighting effect fusion image, y can represent the vertical coordinate information of the target update box in the target lighting effect fusion image, w can represent the width of the target update box, h can represent the height of the target update box, and σ(x) can identify the Sigmoid function, which is defined as S can represent the grid partitioning parameters, p x 、p y 、p w 、p h can represent the horizontal coordinate value c of the fault area location detection model for the unupdated prior frame corresponding to the target update frame. x , vertical coordinate value c y 、Width c w and high c h The adjustment value, c is the confidence level, exp(c w ) means the base is e and the exponent is c w The exponential function, exp(c h ) means the base is e and the exponent is c h The exponential function of .

[0121] In an exemplary embodiment of the present disclosure, the determination of the target lamp fault area can be achieved by the following steps:

[0122] The target initial lamp fault area is screened out according to the confidence level corresponding to each initial lamp fault area, and the intersection-and-joint ratio data between the target initial lamp fault area and other initial lamp fault areas are calculated; the initial lamp fault areas whose intersection-and-joint ratio data are greater than or equal to a preset intersection-and-joint ratio threshold are deleted until the intersection-and-joint ratio data of the remaining initial lamp fault areas are all less than the intersection-and-joint ratio threshold, and the remaining initial lamp fault areas are determined as target lamp fault areas.

[0123] Among them, the target initial lamp fault area refers to the initial lamp fault area with the largest confidence selected from the initial lamp fault area, which is used as the basis for performing redundancy removal operations on the overlapping parts between the initial lamp fault areas to obtain the target lamp fault area. For example, the overlapping parts in the initial lamp fault areas can be removed by a method based on NMS (Non-Maximum Suppression), that is, by calculating the intersection and union ratio data between the target initial lamp fault area and other initial lamp fault areas, and removing the initial lamp fault areas whose intersection and union ratio data is greater than or equal to the preset intersection and union ratio threshold, and then repeating the screening process of the target initial lamp fault area, and removing the corresponding initial lamp fault area according to the new target initial lamp fault area, and repeating it multiple times, so as to obtain multiple initial lamp fault areas without overlapping parts, and determine the remaining initial lamp fault areas as target lamp fault areas, so as to accurately locate the faulty lamp according to the position of the target lamp fault area in the lighting effect image to be detected.

[0124] For example, the intersection-and-combination ratio between the target initial lamp fault region with the maximum confidence and other initial lamp fault regions can be determined by equation (7):

[0125]

[0126] Among them, IoU can represent the intersection over union ratio, S 1 It can represent the area of ​​the intersection of the target initial lamp fault area and other initial lamp fault areas, S 2 It can represent the area of ​​the union of the target initial lamp fault area and other initial lamp fault areas.

[0127] In an exemplary embodiment of the present disclosure, the construction of a normal lighting effect image library can be achieved through the following steps:

[0128] The complete lighting effect images collected by the drone through the cyclic execution of the inspection task can be obtained. The inspection task includes the geographical coordinates of multiple inspection points. The complete lighting effect images are obtained by the drone through multiple cycles of collection at the geographical coordinates of each inspection point, while ensuring that the batch of lamps are intact and present different lighting effects. The complete lighting effect images are classified according to the geographical coordinates of the inspection points to obtain a complete lighting effect image group at the geographical coordinates of each inspection point, and the complete lighting effect image group is used as the reference lighting effect image group at the geographical coordinates of each inspection point to construct a normal lighting effect image library.

[0129] The geographic coordinates of the inspection points can be locations where batches of lamps present complex lighting effects, or locations where lighting effects change frequently. Of course, they can also be locations at equal intervals on the flight path of the drone. This embodiment does not specifically limit the setting of the geographic coordinates of the inspection points.

[0130] Optionally, due to the error of the drone positioning system and the interference of the external environment when batch lamps present lighting effects, reference lighting effect images taken at different shooting angles and when the batch lamps present a variety of different lighting effects while ensuring that the batch lamps are intact can be used as reference lighting effect images in the reference lighting effect image group of the corresponding inspection point geographic coordinates, thereby constructing a normal lighting effect image library through multiple reference lighting effect image groups corresponding to the geographic coordinates of each inspection point.

[0131] Optionally, by classifying the complete lighting effect images taken at the geographic coordinates of each inspection point and constructing a normal lighting effect image library, the drone can successfully match the corresponding reference lighting effect image group according to the geographic coordinates of the inspection point when performing the inspection task, eliminating the influence of the drone positioning accuracy, so that multiple reference lighting effect images with a large degree of similarity to the lighting effect image to be detected can be accurately matched, further ensuring the accuracy of fault detection of batch lamps at the geographic coordinates of the inspection point through the fault detection system.

[0132] In an exemplary embodiment of the present disclosure, the training of the fault area location detection model can be implemented by the following steps:

[0133] A training sample set is obtained, and the training sample set includes a sample fault lighting effect image, a reference lighting effect image, and a fault area annotation box corresponding to the sample fault lighting effect image. The sample fault lighting effect image and the reference lighting effect image are input into an image fusion model based on a channel attention mechanism to obtain a sample lighting effect fusion image. The sample lighting effect fusion image is input into a pre-built initial fault area position detection model to obtain a fault area prediction box corresponding to the fusion lighting effect image. The initial fault area position detection model includes a loss function, and the loss function includes a positive sample loss function and a negative sample loss function. The loss function is used to calculate the fault area annotation box and the fault area prediction box to obtain the corresponding loss value, and the model parameters of the initial fault area position detection model are adjusted according to the loss value until the loss function converges to obtain a trained fault area position detection model.

[0134] Optionally, the lighting effects of the sample faulty lighting effect image and the corresponding reference lighting effect image in the training sample set may be inconsistent, or they may be consistent. When the fault area location detection model is trained by using images with different lighting effects, different lighting effect conditions can be better covered, thereby improving the robustness and accuracy of the detection algorithm in detecting faulty lamps under various lighting effects.

[0135] Optionally, the image fusion model based on the channel attention mechanism and the fault area location detection model can be synchronously trained through a training sample set, or the output of the image fusion model based on the channel attention mechanism can be used as the input of the fault area location detection model to train the fault area location detection model. This embodiment does not specifically limit the training process of the image fusion model based on the channel attention mechanism and the fault area location detection model.

[0136] Exemplarily, the output of the image fusion model based on the channel attention mechanism is used as the input of the fault area location detection model to train the fault area location detection model. Specifically, the training process of the image fusion model has been described in detail in step S130, which will not be repeated here. In each training process of the image fusion model based on the channel attention mechanism, each channel will determine the corresponding attention weight according to the learning result, so as to determine the sample lighting effect fusion image, and then input the sample lighting effect fusion image with the fault area annotation frame into the fault area location detection model to obtain the fault area prediction frame. The fault area prediction frame whose intersection and union ratio with the fault area annotation frame is greater than or equal to the preset intersection and union ratio threshold can be used as a positive sample frame for calculating the positive sample loss function. Correspondingly, the fault area prediction frame whose intersection and union ratio with the fault area annotation frame is less than the preset intersection and union ratio threshold can be used as a negative sample frame for calculating the negative sample loss function. In the exemplary implementation of the present disclosure, the confidence of the positive sample frame in each sample lighting effect fusion image can be set to 1, and the confidence of the negative sample frame can be set to 0.

[0137] For example, the positive sample loss function can be expressed by equation (8):

[0138]

[0139] Among them, L 1 represents the positive sample loss function, (x i ,y i , w i ,h i ) represents the positive sample box in the fault area prediction box, (x′ i , y′ i , w′ i , h′ i ) represents the fault area labeling box, c i Represents the confidence of the positive sample box, λ 1 , 2 is the weight parameter, ∑ i∈positive Represents the weighted sum of all positive sample boxes in the positive sample set.

[0140] For example, the negative sample loss function can be expressed by equation (9):

[0141] L 2 =∑ i∈negative λ 3 c i ……………………(9)

[0142] Among them, L 2 represents the negative sample loss function, c i Represents the confidence of the negative sample box, λ 3 is the weight parameter, ∑ i∈negative Represents the weighted sum of all negative sample boxes in the negative sample set.

[0143] The loss function formula can be used to combine the results of the positive sample loss and negative sample loss in the training sample set, so that it can be judged whether the fault area location detection model has been trained by comprehensive analysis of the loss function. The loss function can be expressed as relation (10):

[0144] L 3 =L 1 +L 2 …………………………(10)

[0145] Among them, L 3 Represents the total loss function, L 1 represents the positive sample loss function, L 2 Represents the negative sample loss function.

[0146] Optionally, the fault area position detection model is trained through back propagation based on the deviation between the fault area prediction box and the fault area annotation box. After multiple back propagations, the parameters of the fault area position detection model can eventually converge, or the training is terminated after a certain number of iterations, so as to obtain a trained fault area position detection model to accurately predict the fault area in the lighting effect fusion image to be detected.

[0147] refer to Figure 5As shown in the figure, taking the YOLO series detection algorithm as an example, a brief description is given of predicting the fault area in the target lighting effect fusion image through the fault area position detection model. When the target lighting effect fusion image is input into the fault area position detection model, the fault area position detection model can first scale the target lighting effect fusion image of size W*H to an image of equal length and width such as L*L, and divide the scaled image into virtual areas of S*S grids, and obtain the feature map corresponding to the target lighting effect fusion image through a series of convolution and downsampling operations, and then normalize the feature map through a fully connected layer to obtain a vector of length S*S*B*5, where S*S*B represents the total number of preset boxes, B represents the number of preset boxes in each grid, and 5 represents the 4 position data and confidence of the preset box. The position information of the fault area is calculated using the normalized vector information, that is, the position data and confidence of the prior box, and the calculated position information can be corresponded to the target lighting effect fusion image. Finally, the NMS algorithm is used to remove redundant bounding boxes to obtain the final prediction result, that is, the fault area.

[0148] Next, refer to Figure 6 A batch lamp fault detection device based on drone inspection according to an exemplary embodiment of the present disclosure is introduced.

[0149] like Figure 6 As shown, the batch lamp fault detection device 600 based on drone inspection may include a lighting effect image acquisition module 610, a lighting effect image matching module 620, a lighting effect image fusion module 630, a lamp fault area estimation module 640 and a lamp fault area screening module 650.

[0150] The lighting effect image acquisition module 610 is used to acquire the lighting effect image to be detected collected when the drone performs the inspection task at the geographical coordinates of the inspection point;

[0151] The lighting effect image matching module 620 is used to match the reference lighting effect image group in the pre-built normal lighting effect image library according to the geographical coordinates of the inspection points, wherein the normal lighting effect image library at least includes the complete lighting effect images obtained by repeatedly shooting at the geographical coordinates of all inspection points corresponding to the inspection tasks by the drone while ensuring that the batch of lamps are intact and present different lighting effects;

[0152] A lighting effect image fusion module 630 is used to perform feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image;

[0153] A lamp fault area estimation module 640 is used to input the target lighting effect fusion image into a pre-trained fault area location detection model to determine an initial lamp fault area in the target lighting effect fusion image;

[0154] The lamp fault area screening module 650 is used to screen a target lamp fault area in the initial lamp fault area, and determine the position coordinates of the faulty lamp through the target lamp fault area.

[0155] In an example embodiment of the present disclosure, a batch lamp fault detection device 600 based on drone inspection may include a normal lighting effect image library construction module, which can be used to: obtain the complete lighting effect image collected by the drone through cyclic execution of the inspection task; classify the complete lighting effect image according to the geographical coordinates of the inspection point to obtain a complete lighting effect image group at the geographical coordinates of each inspection point, and use the complete lighting effect image group as a reference lighting effect image group at the geographical coordinates of each inspection point to construct the normal lighting effect image library.

[0156] In an example embodiment of the present disclosure, the lighting effect image to be detected is obtained by screening by a drone based on a rough detection process, and the rough detection process may include: the drone obtains the original lighting effect image collected at the geographical coordinates of the inspection point, and the original lighting effect image includes at least two lighting effect images collected by the drone at the same position at a preset time interval; the drone performs color space conversion on the original lighting effect image to obtain the converted original lighting effect image, and the converted original lighting effect image includes a brightness channel; the drone divides the original lighting effect image according to preset grid parameters, and determines the average brightness value corresponding to each divided image grid according to the brightness value corresponding to the brightness channel; the drone calculates the brightness difference between the average brightness value of any image grid and the average brightness value of the remaining image grids. If there is an image grid whose brightness difference is greater than or equal to the difference threshold, the position coordinates corresponding to the image grid are used as the local fault area coordinates, and the lighting effect images to be detected at different shooting angles are acquired at the local fault area coordinates using a preset fault detection route.

[0157] In an example embodiment of the present disclosure, the lighting effect image fusion module 630 is configured to: splice each reference lighting effect image and the lighting effect image to be detected in the channel direction to obtain a spliced ​​lighting effect image; input the spliced ​​lighting effect image into a pre-trained image fusion model based on the channel attention mechanism, and output an initial lighting effect fusion image; splice the initial lighting effect fusion image with the lighting effect image to be detected in the channel direction to obtain a target lighting effect fusion image.

[0158] In an example embodiment of the present disclosure, the image fusion model includes a channel attention network, and the lighting effect image fusion module 630 is configured to: input the spliced ​​lighting effect image into the channel attention network to obtain the attention weight corresponding to each reference lighting effect image; fuse the information of each reference lighting effect image according to the attention weight, and output an initial lighting effect fusion image.

[0159] In an example embodiment of the present disclosure, the lighting effect image fusion module 630 is configured to: extract feature points for each reference lighting effect image in the reference lighting effect image group to determine the first image feature points, and extract feature points for the lighting effect image to be detected to determine the second image feature points; perform feature matching on the first image feature points and the second image feature points to determine the feature point pairs between each reference lighting effect image and the lighting effect image to be detected; determine the similarity and image offset parameters between each reference lighting effect image and the lighting effect image to be detected based on the feature point pairs, and determine the target reference lighting effect image based on the similarity; based on the image offset parameters between the target reference lighting effect image and the lighting effect image to be detected, perform image registration and weighted fusion on the target reference lighting effect image and the lighting effect image to be detected to obtain a target lighting effect fused image.

[0160] In an example embodiment of the present disclosure, the lighting effect image fusion module 630 is configured to: stitch together the reference lighting effect images in the reference lighting effect image group to obtain a reference lighting effect stitched image; extract feature points from the reference lighting effect stitched image to determine feature points of a third image, and extract feature points from the lighting effect image to be detected to determine feature points of a second image; perform feature matching on the feature points of the third image and the feature points of the second image to determine feature point pairs between the reference lighting effect stitched image and the lighting effect image to be detected; determine a reference lighting effect local image in the reference lighting effect stitched image based on the feature point pairs; and perform weighted fusion on the reference lighting effect local image and the lighting effect image to be detected to obtain a target lighting effect fused image.

[0161] In an exemplary embodiment of the present disclosure, the fault area location detection model includes at least a preprocessing layer, a feature extraction layer, a fully connected layer, and a priori frame generation layer, and the lamp fault area estimation module 640 is configured as follows: the target lighting effect fusion image is divided into a plurality of grid areas according to a preset grid division parameter by the preprocessing layer, and a target number of priori frames with different width and height values ​​are constructed in the grid areas by the priori frame generation layer; the feature extraction layer is used to extract features of the target lighting effect fusion image with the grid area as the analysis unit to obtain a feature map of the lighting effect fusion image; the feature map of the lighting effect fusion image is input into the fully connected layer, and a confidence estimation vector is output, the confidence estimation vector includes the position prediction value of the update frame after each priori frame is updated, and the confidence of the update frame surrounding the task area; a target update frame with a confidence greater than or equal to a preset confidence threshold is determined, and an initial lamp fault area is determined in the target lighting effect fusion image according to the position prediction value corresponding to the target update frame, and the confidence of the target update frame is used as the probability value of the presence of a faulty lamp in the initial lamp fault area.

[0162] In an example embodiment of the present disclosure, the lamp fault area screening module 650 is configured to: screen out the target initial lamp fault area according to the confidence level corresponding to each initial lamp fault area, and calculate the intersection-and-joint ratio data between the target initial lamp fault area and other initial lamp fault areas; delete the initial lamp fault areas whose intersection-and-joint ratio data is greater than or equal to a preset intersection-and-joint ratio threshold, until the intersection-and-joint ratio data of the remaining initial lamp fault areas are all less than the intersection-and-joint ratio threshold, and determine the remaining initial lamp fault areas as the target lamp fault areas.

[0163] In an example embodiment of the present disclosure, a batch lamp fault detection device based on drone inspection also includes a normal lighting effect image library construction module, which is configured to obtain a complete lighting effect image collected by the drone when the project construction is completed by performing an inspection task, the inspection task includes a plurality of inspection point geographic coordinates, and the complete lighting effect image is a lighting effect image collected while ensuring that the batch lamps are intact; the complete lighting effect image is classified according to the inspection point geographic coordinates to obtain a complete lighting effect image group at each inspection point geographic coordinates, and a normal lighting effect image library is constructed through the complete lighting effect image group at each inspection point geographic coordinates.

[0164] In an example embodiment of the present disclosure, a batch lamp fault detection device 600 based on drone inspection also includes a model training module, which is configured to: obtain a training sample set, the training sample set includes a sample fault lighting effect image, a reference lighting effect image and a fault area annotation box corresponding to the sample fault lighting effect image; input the sample fault lighting effect image and the reference lighting effect image into an image fusion model based on a channel attention mechanism to obtain a sample lighting effect fusion image; input the sample lighting effect fusion image into a pre-constructed initial fault area position detection model to obtain a fault area prediction box corresponding to the fused lighting effect image, the initial fault area position detection model includes a loss function, and the loss function includes a positive sample loss function and a negative sample loss function; use the loss function to calculate the fault area annotation box and the fault area prediction box to obtain a corresponding loss value, and adjust the model parameters of the initial fault area position detection model according to the loss value until the loss function converges to obtain a trained fault area position detection model.

[0165] The specific details of each module of the above-mentioned batch lamp fault detection device based on drone inspection have been described in detail in the corresponding batch lamp fault detection method based on drone inspection, so they will not be repeated here.

[0166] It should be noted that although several modules or units of the batch lamp fault detection device based on drone inspection are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units for embodiment.

[0167] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-mentioned batch lamp fault detection method based on drone inspection is also provided.

[0168] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as devices, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0169] Refer to the following Figure 7 hereinafter describes an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0170] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0171] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of the present disclosure. For example, the processing unit 710 can perform the following steps: Figure 1 In step S110 shown in the figure, the lighting effect image to be detected, which is collected by the drone when performing the inspection task at the geographical coordinates of the inspection point, is obtained; in step S120, a reference lighting effect image group is matched in a pre-constructed normal lighting effect image library according to the geographical coordinates of the inspection point, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly photographing at the geographical coordinates of all inspection points corresponding to the inspection task by the drone cyclically performing the inspection task when ensuring that the batch of lamps are intact and present different lighting effects; in step S130, feature fusion is performed on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image; in step S140, the target lighting effect fusion image is input into a pre-trained fault area location detection model to determine an initial lamp fault area in the target lighting effect fusion image; in step S150, a target lamp fault area is screened in the initial lamp fault area, and the position coordinates of the faulty lamp are determined through the target lamp fault area.

[0172] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .

[0173] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0174] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0175] The electronic device 700 may also communicate with one or more external devices 770 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0176] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0177] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present disclosure is stored. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present disclosure.

[0178] refer to Figure 8 As shown, a program product 800 for implementing a batch lamp fault detection method based on drone inspection according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus, or device.

[0179] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0180] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0181] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0182] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0183] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0184] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0185] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0186] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting faults of batch lamps based on drone inspection, wherein the batch lamps are used to present lighting effects on buildings in landscape lighting projects, characterized in that: The batch lamp fault detection method based on drone inspection includes: Obtain the lighting effect image to be detected collected by the drone when performing the inspection task at the geographical coordinates of the inspection point; Matching a reference lighting effect image group in a pre-constructed normal lighting effect image library according to the geographical coordinates of the inspection points, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly shooting at the geographical coordinates of all inspection points corresponding to the inspection tasks by the drone while ensuring that the batch of lamps are intact and present different lighting effects; Performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image; Inputting the target lighting effect fusion image into a pre-trained fault area location detection model to determine an initial lamp fault area in the target lighting effect fusion image; A target lamp fault area is screened in the initial lamp fault area, and the position coordinates of the faulty lamp are determined through the target lamp fault area.

2. The batch lamp fault detection method according to claim 1, characterized in that: The method further comprises: Acquire the complete lighting effect image collected by the drone through cyclically executing the inspection task; The complete lighting effect images are classified according to the geographical coordinates of the inspection points to obtain a complete lighting effect image group at the geographical coordinates of each inspection point, and the complete lighting effect image group is used as a reference lighting effect image group at the geographical coordinates of each inspection point to construct the normal lighting effect image library.

3. The batch lamp fault detection method according to claim 1, characterized in that: The lighting effect image to be detected is obtained by screening the drone based on a rough detection process, and the rough detection process includes: The drone acquires an original lighting effect image collected at the geographical coordinates of the inspection point, wherein the original lighting effect image includes at least two lighting effect images collected by the drone at the same position at a preset time interval; The drone performs color space conversion on the original lighting effect image to obtain a converted original lighting effect image, wherein the converted original lighting effect image includes a brightness channel; The drone divides the original lighting effect image according to preset grid parameters, and determines an average brightness value corresponding to each divided image grid according to the brightness value corresponding to the brightness channel; The drone calculates the brightness difference between the average brightness value of any image grid and the average brightness values ​​of the remaining image grids. If there is an image grid whose brightness difference is greater than or equal to the difference threshold, the position coordinates corresponding to the image grid are used as the local fault area coordinates, and the lighting effect images to be detected at different shooting angles are collected at the local fault area coordinates using a preset fault detection route.

4. The batch lamp fault detection method according to claim 1, characterized in that: The step of performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image includes: Splicing the reference lighting effect images and the lighting effect images to be detected in the channel direction to obtain a spliced ​​lighting effect image; Inputting the spliced ​​lighting effect image into a pre-trained image fusion model based on a channel attention mechanism, and outputting an initial lighting effect fusion image; The initial lighting effect fusion image and the lighting effect image to be detected are spliced ​​in the channel direction to obtain the target lighting effect fusion image.

5. The batch lamp fault detection method according to claim 4, characterized in that: The image fusion model includes a channel attention network, and the spliced ​​lighting effect image is input into a pre-trained image fusion model based on a channel attention mechanism to output an initial lighting effect fusion image, including: Inputting the spliced ​​lighting effect image into the channel attention network to obtain the attention weight corresponding to each reference lighting effect image; The reference lighting effect images are fused according to the attention weights, and an initial lighting effect fusion image is output.

6. The batch lamp fault detection method according to claim 1, characterized in that: The step of performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image includes: Extracting feature points of each reference lighting effect image in the reference lighting effect image group to determine first image feature points, and extracting feature points of the lighting effect image to be detected to determine second image feature points; Performing feature matching on the first image feature points and the second image feature points to determine feature point pairs between each of the reference lighting effect images and the lighting effect image to be detected; Determine the similarity and image offset parameter between each of the reference lighting effect images and the lighting effect image to be detected according to the feature point pairs, and determine the target reference lighting effect image according to the similarity; Based on the image offset parameter between the target reference lighting effect image and the lighting effect image to be detected, the target reference lighting effect image and the lighting effect image to be detected are image registered and weighted fused to obtain a target lighting effect fused image.

7. The batch lamp fault detection method according to claim 1, characterized in that: The step of performing feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image includes: splicing the reference lighting effect images in the reference lighting effect image group to obtain a reference lighting effect spliced ​​image; Extracting feature points from the reference lighting effect spliced ​​image to determine feature points of a third image, and extracting feature points from the lighting effect image to be detected to determine feature points of a second image; Perform feature matching on the third image feature points and the second image feature points to determine feature point pairs between the reference lighting effect spliced ​​image and the lighting effect image to be detected; Determine a reference lighting effect partial image in the reference lighting effect stitched image according to the feature point pair; The reference lighting effect partial image and the lighting effect image to be detected are weightedly fused to obtain a target lighting effect fused image.

8. The batch lamp fault detection method according to claim 1, characterized in that: The fault area location detection model at least includes a preprocessing layer, a feature extraction layer, a fully connected layer and a priori frame generation layer; The step of inputting the target lighting effect fusion image into a pre-trained fault area location detection model to determine an initial lamp fault area in the target lighting effect fusion image includes: The target lighting effect fusion image is divided into a plurality of grid areas by the preprocessing layer according to preset grid division parameters, and a target number of a priori boxes with different width and height values ​​are constructed in the grid areas by the priori box generation layer; Using the feature extraction layer and taking the grid area as an analysis unit, feature extraction is performed on the target lighting effect fusion image to obtain a lighting effect fusion image feature map; Inputting the lighting effect fusion image feature map into the fully connected layer, and outputting a confidence estimation vector, wherein the confidence estimation vector includes a position prediction value of the update box after each of the prior boxes is updated, and a confidence that the update box surrounds the task area; Determine a target update frame whose confidence is greater than or equal to a preset confidence threshold, and determine an initial lamp fault area in the target lighting effect fusion image based on a position prediction value corresponding to the target update frame, and use the confidence of the target update frame as a probability value of the presence of a faulty lamp in the initial lamp fault area.

9. The batch lamp fault detection method according to claim 1 or 6, characterized in that: The step of screening the target lamp fault area in the initial lamp fault area comprises: Filtering out a target initial lamp fault region according to the confidence level corresponding to each of the initial lamp fault regions, and calculating intersection-over-combination ratio data between the target initial lamp fault region and other initial lamp fault regions; The initial lamp fault areas whose INR data are greater than or equal to a preset INR threshold are deleted until the INR data of the remaining initial lamp fault areas are all less than the INR threshold, and the remaining initial lamp fault areas are determined as target lamp fault areas.

10. The batch lamp fault detection method according to claim 1, characterized in that: The method further comprises: Acquire a training sample set, wherein the training sample set includes a sample faulty lighting effect image, a reference lighting effect image, and a fault area annotation box corresponding to the sample faulty lighting effect image; Inputting the sample fault lighting effect image and the reference lighting effect image into an image fusion model based on a channel attention mechanism to obtain a sample lighting effect fusion image; Inputting the sample lighting effect fusion image into a pre-built initial fault area location detection model to obtain a fault area prediction frame corresponding to the fused lighting effect image, wherein the initial fault area location detection model includes a loss function, and the loss function includes a positive sample loss function and a negative sample loss function; The fault area annotation box and the fault area prediction box are calculated using the loss function to obtain corresponding loss values, and the model parameters of the initial fault area location detection model are adjusted according to the loss values ​​until the loss function converges to obtain a trained fault area location detection model.

11. A batch lamp fault detection device based on drone inspection, wherein the batch lamps are used to present lighting effects on buildings in landscape lighting projects, characterized in that: The batch lamp fault detection device based on drone inspection includes: The lighting effect image acquisition module is used to acquire the lighting effect image to be detected collected when the drone performs the inspection task at the geographical coordinates of the inspection point; A lighting effect image matching module, used for matching a reference lighting effect image group in a pre-built normal lighting effect image library according to the geographical coordinates of the inspection points, wherein the normal lighting effect image library at least includes complete lighting effect images obtained by repeatedly shooting at the geographical coordinates of all inspection points corresponding to the inspection tasks by cyclically executing the inspection tasks by the drone when ensuring that the batch of lamps are intact and present different lighting effects; a lighting effect image fusion module, configured to perform feature fusion on each reference lighting effect image in the reference lighting effect image group and the lighting effect image to be detected to obtain a target lighting effect fusion image; A lamp fault area estimation module, used for inputting the target lighting effect fusion image into a pre-trained fault area position detection model to determine an initial lamp fault area in the target lighting effect fusion image; The lamp fault area screening module is used to screen a target lamp fault area in the initial lamp fault area, and determine the position coordinates of the faulty lamp through the target lamp fault area.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the batch lamp fault detection method based on drone inspection according to any one of claims 1 to 10 is implemented.

Images