Fault detection method and device for batch lamps installed on landscape lighting engineering

Abstract

The invention provides a fault detection method and device for batch lamps installed on landscape lighting engineering, and relates to the technical field of artificial intelligence. The fault detection method comprises the steps that feature fusion is carried out on to-be-detected lamp effect images of batch lamps on a landscape lighting project and a pre-stored reference lamp effect image group of the batch lamps to obtain a target lamp effect fusion image, and the reference lamp effect image group comprises at least one reference lamp effect image; and then inputting the target lamp effect fusion image into the trained fault detection model for prediction to obtain a fault area in the target lamp effect fusion image, thereby determining a fault lamp according to the fault area. The fault lamp detection time can be shortened, and the fault lamp detection efficiency can be improved.

Description

Background Art

[0002] In large-scale landscape lighting projects (for example, installed on the curtain walls of super-high-rise buildings), the number of light sources often ranges from tens of thousands to hundreds of thousands. In the related building lighting business, faulty or damaged building lamps cannot be discovered in a timely manner with the naked eye. Especially for super-high-rise building lighting, the efficiency of lamp fault detection is low, and the faulty area cannot be quickly located, thus affecting the lighting effect.

[0003] In the related technology, self-detection methods are usually used, such as using RDM (Raw Device Mapping) lamps to collect and analyze the status of lamps, and then determine whether the lamps are faulty. However, due to the high cost of RDM lamps, if they are used in large-scale landscape lighting projects, the cost of the project will be greatly increased.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a method and device for fault detection of batch lamps installed in landscape lighting projects, which to a certain extent solve the problems of low efficiency in detecting building lamps and high cost of lamp fault diagnosis.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for detecting faults of a batch of lamps installed in a landscape lighting project is provided, comprising:

[0007] Obtain lighting effect images of batches of lamps to be tested in landscape lighting projects;

[0008] Performing feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of a batch of lamps to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image;

[0009] The target lighting effect fusion image is input into the trained fault detection model for prediction, the fault area in the target lighting effect fusion image is obtained, and the faulty lamp is determined according to the fault area.

[0010] According to a second aspect of an embodiment of the present disclosure, there is provided a fault detection device for a batch of lamps installed in a landscape lighting project, comprising:

[0011] The lighting effect image acquisition module is used to acquire lighting effect images to be tested of batches of lamps in landscape lighting projects;

[0012] A lighting effect image fusion module, used to perform feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of batch lamps to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image;

[0013] The faulty lamp determination module is used to input the target lighting effect fusion image into the trained fault detection model for prediction, obtain the fault area in the target lighting effect fusion image, and determine the faulty lamp based on the fault area.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor, a memory, and a program stored in the memory and executable on the processor; when the program is executed by the processor, any step of the method for fault detection of batch lamps installed in a landscape lighting project in the first aspect is implemented.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step of the method for fault detection of batch lamps installed in a landscape lighting project in the first aspect is implemented.

[0016] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0017] Through the embodiment of the present disclosure, the acquired lighting effect images to be detected of the batch of lamps in the landscape lighting project and the pre-stored reference lighting effect image group of the batch of lamps can be feature fused to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image, and then the target lighting effect fusion image is input into the trained fault detection model for prediction to obtain the fault area in the target lighting effect fusion image, so as to determine the faulty lamp according to the fault area. On the one hand, the use of image information captured by the camera and the trained fault detection model to detect faulty lamps can reduce the difficulty of the human eye in judging the fault area, shorten the time for fault detection, and thus improve the efficiency of fault detection to a certain extent; on the other hand, the present disclosure improves the accuracy of the predicted fault area by feature fusion of the lighting effect image to be detected and the pre-stored reference lighting effect image group of the batch of lamps, and introduces the fused target lighting effect fusion image into the fault detection model to predict the fault area in the lighting effect image to be detected; on the other hand, the fault detection model is used to detect the faulty lamp, without relying on the special lamp to perform self-test to determine whether the fault occurs, thereby reducing the detection cost of the faulty lamp to a certain extent.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 A flow chart of a method for detecting faults of a batch of lamps installed in a landscape lighting project in an embodiment of the present disclosure is schematically shown.

[0021] Figure 2 A flowchart for obtaining a target lighting effect fusion image based on an attention mechanism model in an embodiment of the present disclosure is schematically shown.

[0022] Figure 3 A flowchart for predicting a fault area through a fault detection model in an embodiment of the present disclosure is schematically shown.

[0023] Figure 4 An example diagram of a bounding box annotation value in an embodiment of the present disclosure is schematically shown.

[0024] Figure 5 A flowchart for predicting a fault area in a target lighting effect fusion image by using a fault detection model in an embodiment of the present disclosure is schematically shown.

[0025] Figure 6 A schematic diagram of a fault detection device for a batch of lamps installed in a landscape lighting project in an embodiment of the present disclosure is schematically shown.

[0026] Figure 7 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown schematically.

[0027] Figure 8 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is schematically shown.

[0028] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0030] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0031] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0032] In the related art, there are the following technical problems:

[0033] When lighting a building, the light source lamps are very small on the building facade. When a lamp fails, the overall lighting effect will be affected. It is not only unsightly, but also difficult to identify the occurrence and area of ​​the fault with the naked eye, which brings an operation and maintenance bottleneck to the entire building lighting work. At present, the time-consuming detection of lamp faults, the inability to quickly locate the fault area, and the high cost of self-testing of special lamps are all problems that cannot be solved in the building lighting industry so far.

[0034] Based on one or more problems in the related art, the embodiment of the present disclosure first proposes a fault detection method for batch lamps installed in a landscape lighting project. The method can be executed by a control terminal or server of the lamp. The execution method is described below using a server as an example.

[0035] like Figure 1 As shown, Figure 1 The present disclosure is a flowchart of a method for detecting a fault of a batch of lamps installed in a landscape lighting project according to an exemplary embodiment, comprising the following steps:

[0036] In step S110, a lighting effect image to be detected of a batch of lamps in a landscape lighting project is obtained;

[0037] In step S120, feature fusion is performed on the lighting effect image to be detected and a pre-stored reference lighting effect image group of batch lamps to obtain a target lighting effect fusion image, where the reference lighting effect image group includes at least one reference lighting effect image;

[0038] In step S130, the target lighting effect fusion image is input into the trained fault detection model for prediction, the fault area in the target lighting effect fusion image is obtained, and the faulty lamp is determined according to the fault area.

[0039] Through the embodiment of the present disclosure, the acquired lighting effect images to be detected of the batch of lamps in the landscape lighting project and the pre-stored reference lighting effect image group of the batch of lamps can be feature fused to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image, and then the target lighting effect fusion image is input into the trained fault detection model for prediction to obtain the fault area in the target lighting effect fusion image, so as to determine the faulty lamp according to the fault area. On the one hand, the use of image information captured by the camera and the trained fault detection model to detect faulty lamps can reduce the difficulty of the human eye in judging the fault area, shorten the time for fault detection, and thus improve the efficiency of fault detection to a certain extent; on the other hand, the present disclosure improves the accuracy of the predicted fault area by feature fusion of the lighting effect image to be detected and the pre-stored reference lighting effect image group of the batch of lamps, and introduces the fused target lighting effect fusion image into the fault detection model to predict the fault area in the lighting effect image to be detected; on the other hand, the fault detection model is used to detect the faulty lamp, without relying on the special lamp to perform self-test to determine whether a fault occurs, thereby reducing the detection cost of the faulty lamp to a certain extent.

[0040] Next, steps S110 to S130 are described in detail.

[0041] In step S110, lighting effect images to be detected of a batch of lamps in a landscape lighting project are obtained.

[0042] The lighting effect image to be detected refers to an image collected in real time during the detection task, which is used to detect whether there are faulty lamps in the batch of lamps in the corresponding landscape lighting scene. Typically, the batch of lamps can refer to all or part of the lamps on a single building facade in a landscape lighting project. Of course, the solution of the present invention is not limited to this, and can be applied to multiple scenes such as stadium landscape lighting, building landscape lighting, road and bridge landscape lighting, etc. without restriction.

[0043] The lighting effect image to be detected can be acquired through a camera, such as an Eagle Eye camera. Of course, a suitable camera can also be selected according to the specific situation. This embodiment does not specifically limit the camera for acquiring the lighting effect image to be detected.

[0044] In step S120, feature fusion is performed on the lighting effect image to be detected and a pre-stored reference lighting effect image group of batch lamps to obtain a target lighting effect fusion image, where the reference lighting effect image group includes at least one reference lighting effect image.

[0045] In the example implementation of the present disclosure, the reference lighting effect image refers to a lighting effect image that is taken and pre-stored when all the batches of lamps installed in the landscape lighting project are normally lit, and is used as a reference image for the lighting effect image to be detected, so as to determine whether there is a faulty lamp in the lighting effect image to be detected based on the difference between the lighting effect image to be detected and the reference lighting effect image. The reason for forming multiple reference lighting effect images is that when each lamp in each reference lighting effect image is running at different times, its chromaticity and brightness will change according to the needs of the scene.

[0046] In one embodiment, when the reference lighting effect image group includes a reference lighting effect image, the fusion of the two lighting effect images can be achieved by extracting the differences between each first pixel point in the reference lighting effect image and each second pixel point corresponding to each first pixel point in the lighting effect image to be detected.

[0047] For example, the pixel value of each first pixel point may be directly subtracted from the pixel value of each corresponding second pixel point and the absolute value may be taken to obtain the pixel value of each fused pixel point in the target lighting effect fusion image, that is:

[0048]

[0049] Among them, P {R，G，B} is the pixel value of each fused pixel in the target lighting effect fusion image. {R, G, B} represents the three color channels of the lighting effect image, where R represents red, G represents green, and B represents blue. Represents the pixel value of each first pixel in the reference lighting effect image, Represents the pixel value of each second pixel in the lighting effect image to be detected.

[0050] The reference lighting effect image can be acquired through a camera, such as an eagle-eye camera at a fixed position, or other cameras such as a panoramic camera. Of course, the method for acquiring the reference lighting effect image can be the same as or different from the method for acquiring the lighting effect image to be detected. The present embodiment does not specifically limit the method for acquiring the reference lighting effect image. Considering the detection accuracy, preferably, the angle and position of the reference lighting effect image are substantially consistent with the image to be detected. In the event of inconsistency, the lighting effect image to be detected and the reference lighting effect image can be matched, calibrated, etc. first, and then fused to ensure the accuracy of the detection result of the faulty area in the lighting effect image to be detected.

[0051] In addition, it should be noted that in the embodiments of the present disclosure, batches of lamps displaying any lighting effect can be photographed to obtain a reference lighting effect image, that is, the lighting effect in the lighting effect image to be detected may be inconsistent with the lighting effect in the reference lighting effect image, or of course they may be consistent, and the present disclosure does not make specific limitations on this. The present disclosure can improve the flexibility of batch lamp fault detection systems in large-scale landscape lighting projects, so that it can be applied to more different environments and scenarios. Moreover, by using reference lighting effect images of different lighting effects to detect faulty lamps, different lighting effect conditions can be better covered, thereby improving the robustness and accuracy of the detection algorithm in detecting faulty lamps under various lighting effects.

[0052] Optionally, since the accuracy of detecting faulty lamps in the lighting effect image to be detected depends on the difference between the lighting effect image to be detected and the reference lighting effect image, multiple reference lighting effect images can be selected as the reference lighting effect image group, and the difference between the lighting effect image to be detected and the multiple reference lighting effect images can be extracted, thereby improving the accuracy of determining the area where the faulty lamps in the lighting effect image to be detected are located. Of course, only one reference lighting effect image can also be selected, and the faulty lamps in the lighting effect image to be detected can be determined by extracting the difference between the reference lighting effect image and the lighting effect image to be detected.

[0053] Exemplarily, when the reference lighting effect image group includes multiple reference lighting effect images, the lighting effect image to be detected and each reference lighting effect image can be fused using a model that performs feature fusion by analyzing the importance between images, such as an attention mechanism model. The attention mechanism model can be a model constructed based on an image fusion algorithm based on an attention mechanism, and is used to fuse multiple input images into one image by extracting the difference characteristics between the input images. Specifically, the attention mechanism model can be used to determine a fused lighting effect image for fault detection, and can also be used to determine a fused lighting effect image with annotation information. This embodiment does not specifically limit the category of the fused lighting effect image determined by the attention mechanism model.

[0054] Optionally, the fused lighting effect image with labeled information can be used to train a model for fault detection. When the fused lighting effect image with labeled information is input into the fault detection model, the model for fault detection will predict the area in the fused lighting effect image where the faulty lamp may exist, and then continuously adjust the parameters in the model for fault detection based on the gap between the predicted area and the labeled information, i.e., using the loss function; a comprehensive analysis of the test set data can be performed to determine whether the model for fault detection is overfitting, and training is stopped when the model for fault detection is about to overfit, thereby obtaining a model for fault detection. Training the model for fault detection with fused lighting effect images with labeled information can further improve the accuracy of faulty lamp prediction.

[0055] The target lighting effect fusion image refers to the image of the suspected lighting fixture determined by fusing the difference features between the lighting effect image to be detected and multiple reference lighting effect images, and is used as input information for further fault detection. Exemplarily, the target lighting effect fusion image can be a fusion lighting effect image for fault detection determined by an attention mechanism model. For example, by analyzing the attention weights between the lighting effect image to be detected and multiple reference lighting effect images respectively, the difference features between the reference lighting effect image with a larger attention weight and the lighting effect image to be detected can be extracted, and the extracted difference features can be fused to obtain the target lighting effect fusion image.

[0056] In step S130, the target lighting effect fusion image is input into the trained fault detection model for prediction, the fault area in the target lighting effect fusion image is obtained, and the faulty lamp is determined according to the fault area.

[0057] It should be noted that the target lighting effect fusion image obtained by the attention mechanism model will be disturbed by the outdoor environment, which will affect the detection of faulty lamps. Therefore, in order to ensure that the detection system for faulty lamps has considerable robustness, a deep learning target detection algorithm can be used for further detection.

[0058] Exemplarily, a fault detection model can be constructed based on a deep learning target detection algorithm, so that the area where the faulty lamp is located can be obtained by performing a detection task on the target lighting effect fusion image, and then the specific faulty lamp can be determined based on the obtained area where the faulty lamp is located. The deep learning target detection algorithm in the fault detection model can be a two-stage detection algorithm, such as the RCNN (Region-CNN) series of algorithms, or a one-stage detection algorithm, such as the YOLO (You Only LookOnce) series of detection algorithms. Of course, a suitable detection algorithm can also be selected according to the specific situation to construct a fault detection model. This embodiment does not specifically limit the type of deep learning target detection algorithm that is relied on when constructing the fault detection model.

[0059] Optionally, the features of the target lighting effect fusion image obtained by fusion of the attention mechanism model are relatively obvious, so a fault detection model determined by the YOLO (You Only Look Once) series detection algorithm is used as an example for detailed description. It should be noted that the attention mechanism model and the fault detection model can be two models, which are used to realize feature fusion of lighting effect images and lamp fault detection respectively, that is, the output of the attention mechanism model is used as the input of the fault detection model for lamp fault detection, and the attention mechanism model can also be introduced into the fault detection model as a complete model for lamp fault detection, which is not limited in the present disclosure.

[0060] The fault area refers to an area where the probability of a faulty lamp being present is greater than a certain threshold, and is used to determine the faulty lamp. The fault area can be determined by the area surrounding the faulty lamp predicted by the fault detection model. For example, when the target lighting effect fusion image is input into the fault detection model, the fault detection model can predict the area where the faulty lamp is located, and calculate the coordinate information of the area to obtain the corresponding position of the area in the lighting effect image to be detected, thereby determining the fault area. In another embodiment of the present disclosure, if Figure 2 As shown, when the reference lighting effect image group includes multiple reference lighting effect images, the determination of the target lighting effect fusion image in step S120 can be achieved by the following steps:

[0061] In step S210 , the lighting effect image to be detected and each reference lighting effect image are vectorized to obtain a lighting effect vector to be detected and a plurality of reference lighting effect vectors corresponding to each reference lighting effect image.

[0062] Vectorization processing refers to converting the lighting effect image to be detected and each reference lighting effect image into corresponding vectors according to the pixel information in the lighting effect image to be detected and multiple reference lighting effect images. By vectorizing the lighting effect image to be detected and multiple reference lighting effect images, the vectors corresponding to the lighting effect image to be detected and the multiple reference lighting effect images can be operated with multiple mapping matrices in the attention mechanism model, thereby fusing the information in the lighting effect image to be detected and the multiple reference lighting effect images according to the operation between the vectors.

[0063] Step S220, using multiple mapping matrices in the attention mechanism model, determine the query weight vector, key weight vector, and value weight vector corresponding to the lighting effect vector to be detected and each reference lighting effect vector.

[0064] Exemplarily, the query weight vector, key weight vector and value weight vector corresponding to the lighting effect vector to be detected and each reference lighting effect vector can be determined according to multiple mapping matrices in the attention mechanism model, such as multiplying the lighting effect vector to be detected with multiple reference lighting effect vectors and the corresponding mapping matrices to determine the weight vectors corresponding to the lighting effect vector to be referenced and the multiple reference lighting effect vectors.

[0065] In an exemplary embodiment of the present disclosure, the mapping matrix includes a first mapping matrix, a second mapping matrix, and a third mapping matrix. The following steps can be used to determine the query weight vector, the key weight vector, and the value weight vector corresponding to the lighting effect vector to be detected and each reference lighting effect vector:

[0066] The lighting effect vector to be detected and the multiple reference lighting effect vectors obtained after vectorization processing of the lighting effect image to be detected and the multiple reference lighting effect images can be expressed as p i , where i is the image index. When i=0, it represents the lighting effect vector to be detected, i.e. p0 is a vector obtained by vectorizing the lighting effect image to be detected. When i>0, it represents multiple reference lighting effect vectors obtained by vectorizing multiple reference lighting effect images, that is, {p i＞0 , i=1, 2, ..., N} is the reference lighting effect vector.

[0067] Then, the first mapping matrix can be used to perform operations with the lighting effect vector to be detected and each reference lighting effect vector to obtain the corresponding query weight vector; the second mapping matrix can be used to perform operations with the lighting effect vector to be detected and each reference lighting effect vector to obtain the corresponding key weight vector; the third mapping matrix can be used to perform operations with the lighting effect vector to be detected and each reference lighting effect vector to obtain the corresponding value weight vector.

[0068] For example, the first mapping matrix W Q , the second mapping matrix W K And the third mapping matrix W V Calculate the lighting effect vector p to be detected 0 The corresponding query weight vector q 0 , key weight vector k 0 and the value weight vector v 0 , similarly, the first mapping matrix W Q , the second mapping matrix W K And the third mapping matrix W V Calculate the reference lighting effect vector p i＞0 The corresponding query weight vector q i＞0 , key weight vector k i＞0 and the value weight vector v i＞0 ,Right now:

[0069] q i =p i W Q (2)

[0070] k i =p i W K (3)

[0071] v i =p i W V (4)

[0072] Among them, i=0, 1, 2,…,N.

[0073] It can be understood that the first mapping matrix W can be used Q , the second mapping matrix W K And the third mapping matrix W V Treat the detection lighting effect vector p in turn 0and multiple reference lighting effect vectors {p i＞0 , i = 1, 2, ..., N} for operation, and the light effect vector p to be detected can also be 0 and multiple reference lighting effect vectors {p i＞0 , i=1, 2, ..., N} are calculated simultaneously, and the present disclosure does not specifically limit the calculation method of the query weight vector, key weight vector and value weight vector corresponding to the lighting effect vector to be detected and each reference lighting effect vector.

[0074] Step S230, calculating the attention weights between the lighting effect vectors according to the query weight vectors and the key weight vectors corresponding to the lighting effect vectors, and obtaining the target fusion vector according to the value weight vectors and the attention weights corresponding to the lighting effect vectors.

[0075] For example, the target fusion vector can be calculated based on the query weight vector corresponding to the lighting effect vector to be detected, the key weight vector corresponding to each lighting effect vector, and the value weight vector corresponding to each lighting effect vector. For example, the attention weight information between the lighting effect image to be detected and each lighting effect image can be obtained by calculating the query weight vector of the lighting effect vector to be detected and the key weight vector of each lighting effect vector. Based on the multiple attention weight information, multiple reference lighting effect images that have a greater influence on the lighting effect image to be detected are selected, and the attention weights between the selected reference lighting effect image and each lighting effect image and the value weight vector corresponding to each lighting effect vector are calculated to obtain the target fusion vector.

[0076] Among them, the target fusion vector refers to the difference feature vector between the extracted lighting effect vector to be detected and multiple reference lighting effect vectors. The extracted difference feature vector can be shrunk into an image format according to the reverse process of vectorization processing, and the shrunk image is the target lighting effect fusion image.

[0077] In an exemplary embodiment of the present disclosure, the target fusion vector may be determined by the following steps:

[0078] Execute the image fusion step: Specifically, the first attention weight between the lighting effect image to be detected and each lighting effect image can be calculated based on the query weight vector corresponding to the lighting effect vector to be detected and the key weight vector corresponding to each lighting effect vector, and the initial fusion vector can be calculated based on the first attention weight and the value weight vector corresponding to each lighting effect image; then, the second attention weight between each reference lighting effect image and each lighting effect image can be calculated based on the query weight vector corresponding to each reference lighting effect vector and the key weight vector corresponding to each lighting effect vector, and multiple reference fusion vectors can be calculated based on the second attention weight and the value weight vector corresponding to each lighting effect image; finally, the query weight vector, key weight vector and value weight vector corresponding to each lighting effect vector can be updated using the initial fusion vector and the multiple reference fusion vectors, and the image fusion step is repeated based on the updated query weight vector, key weight vector and value weight vector corresponding to each lighting effect vector, so as to calculate the target attention weight between the lighting effect image to be detected and each lighting effect image according to the updated query weight vector corresponding to the lighting effect image to be detected and the updated key weight vector corresponding to each lighting effect image, and the target fusion vector can be calculated based on the target attention weight and the updated value weight vector corresponding to each lighting effect image.

[0079] First, for the lighting effect vector p to be detected 0 , according to the lighting effect vector p to be detected 0 The corresponding query weight vector q 0 The key weight vector k corresponding to each lighting effect vector j (j=0, 1, 2, ..., N) calculates the first attention weight between the lighting effect image to be detected and each lighting effect image, thereby determining the degree of influence of all lighting effect images including each reference lighting effect image and the lighting effect image to be detected on the lighting effect image to be detected according to the calculated first attention weight.

[0080] For example, the attention weight between each lighting effect image and any lighting effect image among all lighting effect images can be calculated by using formulas (5) to (7), namely:

[0081] c ij =Softmax(x ij ) (5)

[0082]

[0083] x ij =q i .k j T (7)

[0084] Among them, c ij is the lighting effect vector p i With the lighting effect vector p jThe attention weight between them represents the attention weight for the lighting effect vector p i Information analysis, lighting effect vector p j The importance of , Softmax is a normalization function, q i is the lighting effect vector p i The corresponding query weight vector, k j T is the lighting effect vector p j The transposed vector of the corresponding key weight vector, T is the transposed symbol, (q i ·k j T ) represents the lighting effect vector p i The corresponding query weight vector and lighting effect vector p j The dot product operation is performed on the transposed vector of the corresponding key weight vector, exp(x ij ) represents the ij Perform exponential operations, Represents the lighting effect vector p i The corresponding query weight vector and the key weight vectors corresponding to all lighting effect vectors are subjected to inner product and exponential operations, and the sum is calculated.

[0085] The degree of influence of each lighting effect image, including the lighting effect image to be detected, on the lighting effect image to be detected can be calculated using the attention weight formula, and the value weight vector corresponding to each lighting effect image is weightedly calculated using the attention weight values ​​of the lighting effect image to be detected and each lighting effect image, thereby determining the difference characteristics between the lighting effect image to be detected and each lighting effect image.

[0086] Then, the fusion feature formula can be used:

[0087]

[0088] Calculate the lighting effect vector p i The corresponding fusion vector, where p″ i Indicates that the lighting effect vector p i The corresponding fusion vector after fusion feature, c ij is the lighting effect vector p i With the lighting effect vector p j The attention weight between them, v is the lighting effect vector p j The corresponding value weight vector, the sum represents the sum of all lighting effect images, and N is the total number of reference lighting effect images.

[0089] It should be noted that the light effect vector p to be detected can be calculated according to formula (8): 0 And each reference lighting effect image p i＞0 The corresponding fusion vector after fusion features, that is, the initial fusion vector p′ 0 and multiple reference fusion vectors p′i＞0 Then, the initial fusion vector p′ can be used according to formula (2) to formula (4) 0 And multiple reference fusion vectors p′ i＞0 Update each lighting effect image p i The corresponding query weight vector, key weight vector and value weight vector are:

[0090] q′ i =p′ i W Q (9)

[0091] k′ i =p′ i W K (10)

[0092] v′ i =p′ i W V (11)

[0093] For example, the updated lighting effect vector p to be detected 0 The corresponding query weight vector is q 0 , the key weight vector is k 0 and the value weight vector is v 0 .

[0094] Among them, the lighting effect vector p is calculated by repeating the image fusion step 1 When the corresponding fusion vector is calculated, the lighting effect vector p 1 The corresponding query weight vector q 1 The key weight vector k corresponding to each lighting effect vector j (j=0, 1, 2, ..., N), and based on the second attention weight and the value weight vector v corresponding to each lighting effect vector j (j=0, 1, 2, ..., N) to calculate the lighting effect vector p 1 The corresponding fusion vector, i.e. the reference fusion vector, can be denoted as p″ 1 .

[0095] Similarly, multiple reference fusion vectors p′ can be calculated based on each second attention weight and each updated value weight vector i＞0 .

[0096] Furthermore, based on the initial fusion vector p′ 0 And multiple reference fusion vectors p′ i＞0 Update the query weight vector, key weight vector and value weight vector corresponding to each lighting effect vector respectively, and then update the lighting effect vector p′ based on the final updated iThe corresponding query weight vector, key weight vector and value weight vector are calculated using formula (5) to formula (8) to obtain the final target fusion vector p″ 0 .

[0097] In this example, by repeating the image fusion step multiple times, the difference features between the lighting effect images can be extracted more accurately, the image fusion effect can be improved, and the accuracy of lamp fault detection can be improved. Of course, in other examples, the initial fusion vector p′ can also be 0 As the target fusion vector p″ 0 Fault prediction is performed, which is not limited in the present disclosure.

[0098] Step S240, obtaining a target lighting effect fusion image according to the target fusion vector.

[0099] Exemplarily, the target lighting effect fusion image corresponding to the target fusion vector can be obtained by mapping the value in the target fusion vector to the corresponding color space, and then based on the creation of the initial image array, the color value mapped to the color space by the vector is filled into each pixel position in the image array. Of course, the target lighting effect fusion image corresponding to the target fusion vector can also be obtained by selecting a suitable method according to the specific situation. The present embodiment does not specifically limit the method for determining the corresponding target lighting effect vector according to the target fusion vector.

[0100] In an exemplary embodiment of the present disclosure, Figure 3 As shown, the determination of the fault area in step S130 can be achieved by the following steps:

[0101] Step S310 , dividing the target lighting effect fusion image into a plurality of grids according to preset grid division parameters, and determining a preset frame corresponding to each grid.

[0102] Among them, the preset grid division parameter refers to a value that can evenly divide the target lighting effect fusion image into a number of grids. For example, when the preset grid division parameter is S=10, the target lighting effect fusion image can be divided into an image including S*S, i.e., 10*10 grids. The size of S can be designed according to the specific situation. This embodiment does not specifically limit the specific value of S.

[0103] Optionally, different preset boxes can be distinguished according to the width and height values ​​of each preset box. For example, the width and height values ​​of the corresponding preset box in each grid can be set according to experience or clustering methods, so that the size of the object focused on by each preset box can be well distinguished, so that the fault area corresponding to the selected preset box in the target lighting effect fusion image can be determined according to the different width and height values ​​corresponding to the preset boxes.

[0104] Step S320, based on the preset frame, use the trained fault detection model to extract features from each grid to obtain a corresponding plurality of feature maps, and determine the fault area corresponding to each feature map in the target lighting effect fusion image.

[0105] Optionally, the convolution layer and downsampling layer in the fault detection model can be used to extract features of the target lighting effect fusion image to obtain a feature map corresponding to the target lighting effect fusion image, thereby predicting the feature map and determining the fault area corresponding to each feature map in the target lighting effect fusion image based on the prediction results.

[0106] It should be noted that if the feature map is processed by the fully connected layer in the fault detection model, the target lighting effect fusion image needs to be preprocessed, that is, the target lighting effect fusion image is scaled into an image of equal length and width, so that the feature map determined according to the scaled target lighting effect fusion image can be processed using the fully connected layer in the fault detection model.

[0107] In an example implementation, prediction data for each preset box in each feature map can be determined based on a fault detection model, the prediction data including the confidence of the preset box surrounding the fault area and a position prediction value; a target preset box whose confidence is greater than or equal to a preset confidence threshold is selected from each preset box, and multiple candidate bounding boxes corresponding to the target preset box in the lighting effect image to be detected are determined based on the position prediction value of the target preset box; overlapping parts of the candidate bounding boxes are de-redundanted to obtain a target bounding box, and the fault area corresponding to the target bounding box in the target lighting effect fusion image is determined.

[0108] The prediction data refers to the position information and confidence information corresponding to the preset box determined based on the fault detection model, which is used to select the target bounding box and calculate the fault area corresponding to the target bounding box in the target lighting effect fusion image. Optionally, the specific content of the prediction data can be determined according to the actual application scenario of the fault detection model. For example, in the present disclosure, the fault detection model is applied to the fault detection of lamps, so the predicted probability of each preset box surrounding the area where the faulty lamp is located can be incorporated into the confidence, thereby avoiding the prediction of the probability value of each preset box surrounding the area where the faulty lamp is located.

[0109] The position prediction value refers to the adjustment value of the width, height and plane coordinates of the preset box according to the trained fault detection model, which is used to calculate the fault area corresponding to the target bounding box in the target lighting effect fusion image. The position prediction value represents the width, height and plane coordinate values ​​of the preset box. Of course, the specific information represented by the position prediction value is related to the relevant position information of the preset box. This embodiment does not specifically limit the specific information represented by the position prediction value.

[0110] The confidence threshold refers to a reference value with a high probability that the preset frame surrounds the area where the faulty lamp is located, and is used as a basis for selecting a candidate bounding box from the preset frame. The confidence threshold can be set to 0.8, or to 0.99. Of course, it can also be set to any value between 0.8 and 0.99. The setting of the confidence threshold can refer to the confidence value predicted by the fault detection model for each preset frame. When the confidence predicted by the fault detection model for each preset frame is high, a larger value can be selected as the confidence threshold; when the confidence predicted by the fault detection model for each preset frame is low, a smaller value can be selected as the confidence threshold; of course, a confidence threshold of appropriate size can also be set according to the specific situation. This embodiment does not specifically limit the setting of the confidence threshold.

[0111] The candidate bounding box refers to a preset box whose confidence level predicted by the fault detection model is greater than the confidence threshold, and is used to determine the range of the area surrounding the faulty lamp. For example, the position of the candidate bounding box in the target lighting effect fusion image can be calculated by using the position prediction value of the candidate bounding box. The formula for calculating the position of the candidate bounding box in the target lighting effect fusion image using the position prediction value of the candidate bounding box is:

[0112]

[0113]

[0114]

[0115]

[0116] Among them, x is the horizontal coordinate information of the candidate bounding box in the target lighting effect fusion image, y is the vertical coordinate information of the candidate bounding box in the target lighting effect fusion image, w is the width of the candidate bounding box, h is the height of the candidate bounding box, and σ(x) can be a Sigmoid function, defined as S is the grid partition parameter, p x 、p y 、p w 、p h They are the horizontal coordinate values ​​c of the preset frame of the fault detection model x , vertical coordinate value c y , preset frame width c w and the high c of the preset frame h The adjustment value, c is the confidence level, exp(c w ) means the base is e and the exponent is c w The exponential function, exp(c h ) means the base is e and the exponent is c h The exponential function of .

[0117] The target bounding box refers to a candidate bounding box without overlapping parts, which is used to determine the corresponding fault area in the target lighting effect fusion image. The target bounding box can be determined by the result of the redundancy removal operation between the candidate bounding boxes, such as removing the overlapping parts in the candidate bounding boxes by a method based on NMS (Non-Maximum Suppression), that is, by calculating the intersection and union ratio between the candidate bounding box with the highest confidence and other candidate bounding boxes, removing the candidate bounding boxes with an intersection and union ratio greater than a preset threshold, and repeating the operation for all candidate bounding boxes. The candidate bounding box that is not removed is the target bounding box. The position of the target bounding box in the target lighting effect fusion image is determined according to the position information of the target bounding box, so that the fault area where the faulty lamp is located can be determined.

[0118] The intersection over union (IoU) between the candidate bounding box with the highest confidence and other candidate bounding boxes can be determined by the intersection over union formula, which is expressed as:

[0119]

[0120] Among them, S 1 S represents the area of ​​the intersection of the candidate bounding box with the highest confidence and other candidate bounding boxes. 2 Represents the area of ​​the union of the candidate bounding box with the highest confidence and other candidate bounding boxes.

[0121] In an exemplary embodiment of the present disclosure, the training of the attention mechanism model and the fault detection model can be implemented by the following steps:

[0122] A training sample set is obtained. The training data in the training sample set includes training lighting effect images, reference lighting effect images, and bounding box annotation values ​​corresponding to the training lighting effect images. The training lighting effect images and the complete lighting effect images can be first input into the attention mechanism model to obtain a fused lighting effect image. The fused lighting effect image is then input into the fault detection model to obtain a bounding box prediction value corresponding to the fused lighting effect image. Finally, the loss function is used to calculate the bounding box annotation value and the bounding box prediction value to obtain the corresponding loss value, and the model parameters of the attention mechanism model and the model parameters of the fault detection model are adjusted according to the loss value.

[0123] It is understandable that any number of training lighting effect images and reference lighting effect images can be obtained to train the attention mechanism model and the fault detection model multiple times to improve the performance of each model. Among them, the training lighting effect images may include lighting effect images of faulty lamps, and may also include lighting effect images of non-faulty lamps, and may also include the lighting effect images to be detected in step S110, and the present disclosure does not limit this. In the example implementation of the present disclosure, the training sample set can be used to train the attention mechanism model and the fault detection model separately, or the attention mechanism model and the fault detection model can be trained at the same time, and the present disclosure does not specifically limit this.

[0124] refer to Figure 4 As shown, if there is a faulty lamp in the training lighting effect image, the faulty area can be annotated in the form of an (x, y, w, h) annotation bounding box, so that the model training can be performed based on the deviation between the preset box and the bounding box annotation value.

[0125] Optionally, each fused lighting effect image corresponds to a plurality of preset frames, wherein a preset frame whose intersection-over-union ratio between the bounding box annotation values ​​corresponding to each training lighting effect image is greater than or equal to a preset threshold can be used as a positive sample frame for calculating a positive sample loss function. Correspondingly, a preset frame whose intersection-over-union ratio between the bounding box annotation values ​​corresponding to each training lighting effect image is less than a preset threshold can be used as a negative sample frame for calculating a negative sample loss function. In the example implementation of the present disclosure, the confidence of the positive sample frame in each fused lighting effect image can be set to 1, and the confidence of the negative sample frame can be set to 0.

[0126] Exemplarily, the positive sample loss function formula is expressed as:

[0127]

[0128] Among them, L 1 represents the positive sample loss function, (x i ,y i , w i ,h i ) represents the predicted positive sample box, (x i ′,y i ′,w′ i , h′ i ) represents the bounding box annotation value, c i Represents the confidence of the positive sample box, λ 1 , 2 is a regularization hyperparameter used to adjust the size of the regularization term, ∑ i∈positive Represents the weighted sum of all positive sample boxes in the positive sample set, and 1 refers to the preset confidence reference value.

[0129] The negative sample loss function formula is expressed as:

[0130] L 2 =∑ i∈negative λ 3 c i (18)

[0131] Among them, L 2 represents the negative sample loss function, c i Represents the confidence of the negative sample box, λ 3 is the regularization hyperparameter, ∑ i∈negative Represents the weighted sum of all negative sample boxes in the negative sample set.

[0132] The loss function formula can be combined with the results of the positive sample loss and negative sample loss in the training sample set, so that the attention mechanism model and the fault detection model can be judged whether the training is complete by comprehensive analysis of the loss function. The formula of the loss function is expressed as:

[0133] L 3 =L 1 +L 2 (19)

[0134] Among them, L 3 Represents the total loss function, L 1 represents the positive sample loss function, L 1 Represents the negative sample loss function.

[0135] Specifically, during the training process, the model parameters in the attention mechanism model and the fault detection model can be trained simultaneously. The model parameters can be weight parameters, bias parameters, parameter matrices, i.e., the mapping matrices W Q , W K and W V For example, L 3 As the objective function, we can first adjust the weight parameters and bias parameters in the fault detection model. Since we need to use the attention mechanism model to fuse the features of the lighting effect image before fault detection, we can further back-propagate to the attention mechanism model and adjust the mapping matrix W in the attention mechanism model. Q , W K and W V Through multiple layers of back propagation, the model parameters can eventually converge, or the training can be terminated after a certain number of iterations. Through this training method, the attention mechanism model and the fault detection model can be trained at the same time to ensure that the precision and accuracy of each model are higher, and the training efficiency can be improved. After the training is completed, the final models can be used to predict the faults of the lighting effect images to be detected.

[0136] refer to Figure 5As shown in the figure, taking the YOLO series detection algorithm as an example, a brief description is given of the prediction of the fault area in the target lighting effect fusion image by the fault detection model. When the target lighting effect fusion image is input into the fault detection model, the fault detection model can first scale the target lighting effect fusion image of size W*H to an image of equal length and width such as L*L, and divide the scaled image into virtual areas of S*S grids, and obtain the feature map corresponding to the target lighting effect fusion image through a series of convolution and downsampling operations, and then normalize the feature map through the fully connected layer to obtain a vector of length S*S*B*5, where S*S*B represents the total number of preset boxes, B represents the number of preset boxes in each grid, and 5 represents the 4 position data and confidence of the preset box. The position information of the fault area is calculated using the normalized vector information, i.e., the position data and confidence of the preset box, and the calculated position information can be corresponded to the target lighting effect fusion image. Finally, the NMS algorithm is used to remove redundant bounding boxes to obtain the final prediction result, i.e., the fault area.

[0137] Through the embodiments of the present disclosure, on the one hand, by using the image information captured by the camera and the trained fault detection model to detect faulty lamps, the difficulty of the human eye in judging the fault area can be reduced, and the time for fault detection can be shortened, thereby improving the efficiency of fault detection to a certain extent; on the other hand, the present disclosure obtains a target lighting effect fusion image by performing feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of batch lamps, and predicts the fault area in the lighting effect image to be detected by introducing the fused target lighting effect fusion image into the fault detection model, thereby improving the accuracy of the predicted fault area; on another hand, by using the fault detection model to detect faulty lamps, instead of relying on special lamps to perform self-inspection to determine whether a fault occurs, the detection cost of faulty lamps is reduced to a certain extent.

[0138] Next, refer to Figure 6 A fault detection device for a batch of lamps installed in a landscape lighting project according to an exemplary embodiment of the present disclosure is introduced.

[0139] like Figure 6 As shown, the fault detection device 600 for batch lamps installed in a landscape lighting project may include a lighting effect image acquisition module 610 , a lighting effect image fusion module 620 and a faulty lamp determination module 630 .

[0140] The lighting effect image acquisition module 610 is used to acquire lighting effect images to be detected of a batch of lamps in a landscape lighting project;

[0141] The lighting effect image fusion module 620 is used to perform feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of batch lamps to obtain a target lighting effect fusion image, where the reference lighting effect image group includes at least one reference lighting effect image;

[0142] The faulty lamp determination module 630 is used to input the target lighting effect fusion image into the trained fault detection model for prediction, obtain the fault area in the target lighting effect fusion image, and determine the faulty lamp according to the fault area.

[0143] In an example embodiment of the present disclosure, the lighting effect image fusion module 620 is configured to, when the reference lighting effect image group includes multiple reference lighting effect images, vectorize the lighting effect image to be detected and each reference lighting effect image to obtain a lighting effect vector to be detected and multiple reference lighting effect vectors corresponding to each reference lighting effect image; use multiple mapping matrices in the attention mechanism model to determine the query weight vector, key weight vector and value weight vector corresponding to the lighting effect vector to be detected and each reference lighting effect vector, respectively; calculate the attention weights between each lighting effect vector based on the query weight vector and key weight vector corresponding to each lighting effect vector, and calculate the target fusion vector based on the value weight vector and each attention weight corresponding to each lighting effect vector; obtain the target lighting effect fusion image based on the target fusion vector.

[0144] In an example embodiment of the present disclosure, multiple mapping matrices include a first mapping matrix, a second mapping matrix and a third mapping matrix; the lighting effect image fusion module 620 is configured to use the first mapping matrix to perform operations with the lighting effect vector to be detected and each reference lighting effect vector, respectively, to obtain a corresponding query weight vector; use the second mapping matrix to perform operations with the lighting effect vector to be detected and each reference lighting effect vector, respectively, to obtain a corresponding key weight vector; use the third mapping matrix to perform operations with the lighting effect vector to be detected and each reference lighting effect vector, respectively, to obtain a corresponding value weight vector.

[0145] In an exemplary embodiment of the present disclosure, the attention weight includes a first attention weight and a second attention weight, and the lighting effect image fusion module 620 is configured to perform an image fusion step: a first attention weight between the lighting effect image to be detected and each lighting effect image is calculated based on a query weight vector corresponding to the lighting effect vector to be detected and a key weight vector corresponding to each lighting effect vector, and an initial fusion vector is calculated based on the first attention weight and each value weight vector; a second attention weight between each reference lighting effect image and each lighting effect image is calculated based on a query weight vector corresponding to each reference lighting effect vector and a key weight vector corresponding to each lighting effect vector, and an initial fusion vector is calculated based on the second attention weight and each value weight vector. The value weight vector corresponding to the image is calculated to obtain multiple reference fusion vectors; the initial fusion vector and the multiple reference fusion vectors are used to update the query weight vector, key weight vector and value weight vector corresponding to each lighting effect vector; based on the updated query weight vector, key weight vector and value weight vector corresponding to each lighting effect vector, the image fusion step is repeated to calculate the target attention weights between the lighting effect image to be detected and each lighting effect image according to the updated query weight vector corresponding to the lighting effect image to be detected and the updated key weight vector corresponding to each lighting effect image; and the target fusion vector is calculated according to the target attention weight and the value weight vector corresponding to each updated lighting effect image.

[0146] In an example embodiment of the present disclosure, the faulty lamp determination module 630 is configured to divide the target lighting effect fusion image into multiple grids according to preset grid division parameters, and determine the preset frame corresponding to each grid; based on the preset frame, use the trained fault detection model to extract features of each grid to obtain corresponding multiple feature maps, and determine the fault area corresponding to each feature map in the target lighting effect fusion image.

[0147] In an example embodiment of the present disclosure, the faulty lamp determination module 630 is configured to determine prediction data for each preset box in each feature map according to a fault detection model, the prediction data including the confidence of the preset box surrounding the fault area and the position prediction value; select a target preset box whose confidence is greater than or equal to a preset confidence threshold from each preset box, and determine multiple candidate bounding boxes corresponding to the target preset box in the lighting effect image to be detected according to the position prediction value of the target preset box; perform redundancy removal on the overlapping parts of the candidate bounding boxes to obtain the target bounding box, and determine the fault area corresponding to the target bounding box in the target lighting effect fusion image.

[0148] In an example embodiment of the present disclosure, a fault detection device 600 for a batch of lamps installed in a landscape lighting project also includes a model training module, which is configured to obtain a training sample set, wherein the training data in the training sample set includes a training lighting effect image, a reference lighting effect image, and a bounding box annotation value corresponding to the training lighting effect image, the training lighting effect image and the reference lighting effect image are input into an attention mechanism model to obtain a fused lighting effect image, the fused lighting effect image is input into a fault detection model to obtain a bounding box prediction value corresponding to the fused lighting effect image, the bounding box annotation value and the bounding box prediction value are calculated using a loss function to obtain a corresponding loss value, and the model parameters of the attention mechanism model and the model parameters of the fault detection model are adjusted according to the loss value.

[0149] The specific details of each module of the fault detection device for batch lamps installed in the above-mentioned landscape lighting project have been described in detail in the corresponding fault detection method for batch lamps installed in the landscape lighting project, so they will not be repeated here.

[0150] It should be noted that although several modules or units of the fault detection device for batch lamps installed in landscape lighting projects are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units for embodiment.

[0151] In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing a method for detecting faults of a batch of lamps installed in the above-mentioned landscape lighting project is also provided.

[0152] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as devices, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware embodiments, complete software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0153] Refer to the following Figure 7 hereinafter describes an electronic device 700 according to such an embodiment of the present disclosure. Figure 5 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0154] like Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0155] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of the present disclosure. For example, the processing unit 710 can perform the following steps: Figure 1 In step S110 shown in the figure, a lighting effect image to be detected of a batch of lamps in a landscape lighting project is obtained; in step S120, feature fusion is performed on the lighting effect image to be detected and a reference lighting effect image group of a batch of lamps stored in advance to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image; in step S130, the target lighting effect fusion image is input into a trained fault detection model for prediction to obtain a fault area in the target lighting effect fusion image, and the faulty lamp is determined according to the fault area.

[0156] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .

[0157] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0158] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0159] The electronic device 700 may also communicate with one or more external devices 770 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0160] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0161] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present disclosure is stored. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present disclosure.

[0162] refer to Figure 8 As shown, a program product 800 for implementing a method for detecting a fault of a batch of lamps installed in a landscape lighting project according to an embodiment of the present disclosure is described, which can be implemented in a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus, or device.

[0163] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0164] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0165] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0166] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0167] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0168] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0169] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0170] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting faults of a batch of lamps installed in a landscape lighting project, characterized in that: include: Obtain lighting effect images of batches of lamps to be tested in landscape lighting projects; Performing feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of the batch of lamps to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image; The target lighting effect fusion image is input into a trained fault detection model for prediction, so as to obtain a fault area in the target lighting effect fusion image, and determine the faulty lamp according to the fault area.

2. The fault detection method according to claim 1, characterized in that: The step of fusing the lighting effect image to be detected with the pre-stored reference lighting effect image group of the batch of lamps to obtain a target lighting effect fusion image includes: When the reference lighting effect image group includes a plurality of reference lighting effect images, vectorizing the lighting effect image to be detected and each of the reference lighting effect images to obtain a lighting effect vector to be detected and a plurality of reference lighting effect vectors corresponding to each of the reference lighting effect images; Determine the query weight vector, the key weight vector, and the value weight vector corresponding to the lighting effect vector to be detected and each of the reference lighting effect vectors by using multiple mapping matrices in the attention mechanism model; Calculate the attention weights between the lighting effect vectors according to the query weight vectors and the key weight vectors corresponding to the lighting effect vectors, and calculate the target fusion vector according to the value weight vectors corresponding to the lighting effect vectors and the attention weights; The target lighting effect fusion image is obtained according to the target fusion vector.

3. The fault detection method according to claim 2, characterized in that: The plurality of mapping matrices include a first mapping matrix, a second mapping matrix and a third mapping matrix; The method of using a plurality of mapping matrices in the attention mechanism model to determine the query weight vector, the key weight vector, and the value weight vector respectively corresponding to the lighting effect vector to be detected and each of the reference lighting effect vectors includes: Using the first mapping matrix to perform operations on the lighting effect vector to be detected and each of the reference lighting effect vectors, respectively, to obtain a corresponding query weight vector; Using the second mapping matrix to perform operations on the lighting effect vector to be detected and each of the reference lighting effect vectors, respectively, to obtain a corresponding key weight vector; The third mapping matrix is ​​used to perform operations on the lighting effect vector to be detected and each of the reference lighting effect vectors to obtain a corresponding value weight vector.

4. The fault detection method according to claim 2, characterized in that: The attention weight includes a first attention weight and a second attention weight; The step of calculating the attention weights between the lighting effect vectors according to the query weight vectors and the key weight vectors corresponding to the lighting effect vectors, and obtaining the target fusion vector according to the value weight vectors and the attention weights corresponding to the lighting effect vectors, includes: Image fusion step: calculating a first attention weight between the lighting effect image to be detected and each of the lighting effect images according to a query weight vector corresponding to the lighting effect vector to be detected and a key weight vector corresponding to each of the lighting effect vectors, and calculating an initial fusion vector according to the first attention weight and a value weight vector corresponding to each of the lighting effect images; A second attention weight between each of the reference lighting effect images and each of the lighting effect images is calculated based on the query weight vector corresponding to each of the reference lighting effect vectors and the key weight vector corresponding to each of the lighting effect vectors, and a plurality of reference fusion vectors are calculated based on the second attention weight and the value weight vector corresponding to each of the lighting effect images; The initial fusion vector and the multiple reference fusion vectors are used to update the query weight vector, key weight vector and value weight vector corresponding to each of the lighting effect vectors. Based on the updated query weight vector, key weight vector and value weight vector corresponding to each of the lighting effect vectors, the image fusion step is repeated to calculate the target attention weights between the lighting effect image to be detected and each of the lighting effect images according to the updated query weight vector corresponding to the lighting effect image to be detected and the updated key weight vector corresponding to each of the lighting effect images, and the target fusion vector is calculated according to the target attention weight and the updated value weight vector corresponding to each of the lighting effect images.

5. The fault detection method according to claim 1, characterized in that: The step of inputting the target lighting effect fusion image into a trained fault detection model for prediction to obtain a fault area in the target lighting effect fusion image includes: Dividing the target lighting effect fusion image into a plurality of grids according to preset grid division parameters, and determining a preset frame corresponding to each of the grids; Based on the preset frame, the trained fault detection model is used to extract features from each of the grids to obtain a corresponding plurality of feature maps, and the fault area corresponding to each of the feature maps in the target lighting effect fusion image is determined.

6. The fault detection method according to claim 5, characterized in that: The determining of the fault area corresponding to each of the feature images in the target lighting effect fusion image includes: Determine prediction data of each preset box in each feature map according to the fault detection model, wherein the prediction data includes a confidence level and a position prediction value of the preset box surrounding the fault area; Selecting a target preset frame whose confidence is greater than or equal to a preset confidence threshold from each of the preset frames, and determining a plurality of candidate bounding frames corresponding to the target preset frame in the lighting effect image to be detected according to the position prediction value of the target preset frame; The overlapping parts of the candidate bounding boxes are de-redundanted to obtain a target bounding box, and the fault area corresponding to the target bounding box in the target lighting effect fusion image is determined.

7. The fault detection method according to claim 1, characterized in that: The method further comprises: Acquire a training sample set, wherein the training data in the training sample set includes a training lighting effect image, a reference lighting effect image, and a bounding box annotation value corresponding to the training lighting effect image; Inputting the training lighting effect image and the reference lighting effect image into the attention mechanism model to obtain a fused lighting effect image; Inputting the fused lighting effect image into the fault detection model to obtain a bounding box prediction value corresponding to the fused lighting effect image; The bounding box annotation value and the bounding box prediction value are calculated using a loss function to obtain a corresponding loss value, and the model parameters of the attention mechanism model and the model parameters of the fault detection model are adjusted according to the loss value.

8. A fault detection device for a batch of lamps installed in a landscape lighting project, characterized in that: include: The lighting effect image acquisition module is used to acquire lighting effect images to be tested of batches of lamps in landscape lighting projects; a lighting effect image fusion module, configured to perform feature fusion on the lighting effect image to be detected and a pre-stored reference lighting effect image group of the batch of lamps to obtain a target lighting effect fusion image, wherein the reference lighting effect image group includes at least one reference lighting effect image; The faulty lamp determination module is used to input the target lighting effect fusion image into a trained fault detection model for prediction, obtain the fault area in the target lighting effect fusion image, and determine the faulty lamp according to the fault area.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images