Substation weed detection method, device and equipment

Abstract

The invention discloses a substation weed detection method, device and equipment. The method comprises the following steps: acquiring a multispectral image and a color image corresponding to each capacitor bank in a transformer substation; performing image fusion processing on the multispectral image and the color image to obtain a to-be-detected fusion image; inputting the fusion image to be detected into the target weed detection model; to-be-detected feature information is obtained; inputting the to-be-detected feature information into a neck network to obtain local feature information; performing global feature extraction processing on the to-be-detected feature information to obtain global feature information; performing feature fusion processing on the local feature information and the global feature information to obtain to-be-detected fusion feature information; and inputting the to-be-detected fusion feature information into the single prediction head network for weed detection processing to obtain target weed detection information corresponding to each capacitor bank. According to the embodiment of the invention, the detection efficiency and accuracy of weeds in the capacitor bank can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device and equipment for detecting weeds in a substation. Background Art

[0002] As the scale of power systems continues to expand, substations, as the core components of power systems, are particularly important for safe and stable operation. In substations, weeds in the fences of capacitor banks are a common potential hazard, which not only affects the normal operation of equipment, but also may increase fire hazards, especially in dry seasons. Weeds need to be cleaned up in a timely manner. Especially in the context of large-scale power supply demand, timely handling of weed problems has become increasingly important.

[0003] At present, the weed clearing method mainly detects the weed location through manual inspection and then manually clears the weeds.

[0004] However, this method is labor-intensive and inefficient, and is unable to detect and clean weeds in a timely and comprehensive manner, which seriously threatens the safety and reliability of the substation system. Summary of the invention

[0005] The present invention provides a substation weed detection method, device, equipment and medium. The embodiments of the present invention can improve the detection efficiency and accuracy of substation weeds.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting weeds in a substation, the method comprising:

[0007] Obtain multispectral images and color images corresponding to each capacitor bank in the substation;

[0008] Perform image fusion processing on the multispectral image and the color image to obtain a fused image to be detected;

[0009] The fused image to be detected is input into the target weed detection model, which is trained based on the improved target detection model; and based on the backbone network in the target weed detection model, the fused image to be detected is subjected to deep separable convolution processing to obtain the feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network;

[0010] The feature information to be detected is input into the neck network, and based on the hybrid local channel attention mechanism, local feature extraction is performed on the feature information to be detected to obtain local feature information corresponding to the feature information to be detected; global feature extraction is performed on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected; feature fusion is performed on the local feature information and the global feature information to obtain fused feature information to be detected corresponding to the feature information to be detected;

[0011] The fused feature information to be detected is input into the single prediction head network for weed detection processing to obtain the target weed detection information corresponding to each capacitor group.

[0012] In a second aspect, an embodiment of the present invention further provides a substation weed detection device, the device comprising:

[0013] An image acquisition module, used to acquire a multispectral image and a color image corresponding to each capacitor bank in the substation;

[0014] An image fusion module is used to perform image fusion processing on the multispectral image and the color image to obtain a fused image to be detected;

[0015] The target detection model processing module is used to input the fused image to be detected into the target weed detection model, which is obtained by training based on the improved target detection model; and based on the backbone network in the target weed detection model, the fused image to be detected is subjected to deep separable convolution processing to obtain the feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network;

[0016] The feature extraction and fusion module is used to input the feature information to be detected into the neck network, perform local feature extraction processing on the feature information to be detected based on the hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction processing on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected;

[0017] The target weed detection information acquisition module is used to input the fused feature information to be detected into the single prediction head network for weed detection processing, and obtain the target weed detection information corresponding to each capacitor group.

[0018] In a third aspect, an embodiment of the present invention further provides a substation weed detection device, the substation weed detection device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program executable by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the substation weed detection method according to any embodiment of the present invention.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the substation weed detection method of any embodiment of the present invention when executed.

[0023] The technical solution of the embodiment of the present invention can provide multi-dimensional image information for subsequent weed detection by acquiring multispectral images and color images corresponding to each capacitor group; through image fusion processing, the image features of multispectral images and color images can be combined to obtain richer image features; by inputting the fused image to be detected into the target weed detection model, the advantages of the improved target detection model can be utilized to improve the weed detection effect; through deep separable convolution processing, the feature information of the fused image to be detected can be efficiently extracted, the consumption of computing resources can be reduced, and the effect of feature extraction can be improved; the neck network and the single prediction head network are introduced to further optimize the detail processing of weed detection and improve the overall detection accuracy; through local feature extraction, the fine weed features in the image can be better identified, thereby improving the detection accuracy and local detection ability; through global feature extraction, the weed distribution in the image can be more comprehensively analyzed to enhance the global recognition ability of the model; through feature fusion processing, local feature information and global feature information can be integrated to improve the overall recognition ability of the model for weeds, and enhance the accuracy and generalization ability of the model; through single prediction head network processing, efficient weed detection can be achieved to ensure that the weed situation of each capacitor group is accurately identified.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flow chart of a substation weed detection method provided by an embodiment of the present invention;

[0027] Figure 2 A flow chart of a substation weed detection method provided by an embodiment of the present invention;

[0028] Figure 3 A flow chart of a substation weed detection method provided by an embodiment of the present invention;

[0029] Figure 4 A schematic structural diagram of a substation weed detection device provided by an embodiment of the present invention;

[0030] Figure 5 A schematic structural diagram of a substation weed detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] In the technical solution of the embodiment of the present invention, the acquisition, storage and application of the substation data and weed data involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0034] Figure 1 A flowchart of a substation weed detection method provided by an embodiment of the present invention. The embodiment of the present invention is applicable to substation weed detection, and the method can be executed by a substation weed detection device, which can be implemented in the form of hardware and / or software.

[0035] See also Figure 1 The substation weed detection method shown comprises:

[0036] S101. Acquire a multispectral image and a color image corresponding to each capacitor bank in a substation.

[0037] Among them, multispectral images can refer to images captured using visible light bands and invisible light bands. Multispectral images collect more spectral information. In a specific example, in the substation weed detection scenario, the multispectral image corresponding to each capacitor bank in the substation can be a five-channel multispectral image. The five-channel multispectral image can include a red band, a green band, a blue band, a near infrared band, and a short infrared band; among them, the visible light band includes a red band, a green band, and a blue band, and the invisible band includes a near infrared band and a short-wave infrared. Multispectral images can more accurately identify the difference between weeds and capacitor banks. For example, weeds are similar to the background in the visible light band and are not easy to identify. However, in the invisible band, the reflection characteristics of the weeds are significantly different from the background, so the weeds can be accurately identified.

[0038] Among them, the color image may refer to an image captured using the visible light band. The color image expresses the color through the intensity values ​​of different lights in the red, green and blue channels of each pixel. Each pixel is synthesized through the intensity values ​​of the three channels to form a color; wherein the intensity value of each channel can be from 0-255, indicating the degree from the darkest to the brightest. The color image is used to quickly identify the weed information and capacitor bank information in the image. In a specific example, in the weed detection scenario of the substation, on a sunny day, the metal surface of the capacitor bank presents different colors and gloss in the visible light band, and the color of the weeds is different from that of the capacitor bank. The color difference between the capacitor bank and the weeds can be distinguished by the color image.

[0039] Specifically, the multispectral image and color image corresponding to each capacitor bank in the substation can be acquired through the five-channel multispectral camera and color camera carried by the drone. First, the drone data is set according to the actual on-site operation situation, where the drone data includes the drone GPS (Global Positioning System) coordinates, flight time, flight altitude, environmental parameters and sensor calibration information; a reflector is placed on the ground where the image is collected; secondly, the drone needs to hover horizontally above the capacitor bank at a vertical overlooking angle for aerial photography. The collected multispectral images and color images include the capacitor bank and weeds in the fence, and the multispectral images and color images can include images of complex environmental conditions such as different weather, lighting and angles.

[0040] Before the fusion processing of the multispectral image and the color image is carried out, the collected multispectral image needs to undergo the following preprocessing process: radiation correction and geometric correction are performed on the multispectral image; wherein, the radiation correction needs to obtain the radiation response function of the sensor, which is used to determine the relationship between the sensor output of each band and the actual radiation intensity, and the multispectral image is adjusted using the correction coefficient, and the digital value is converted into a physical radiation value, which is verified by placing a reflector to ensure that the corrected multispectral image accurately reflects the actual radiation situation; geometric correction needs to collect the spatial reference information of the multispectral image, determine the geographical location corresponding to each pixel in the multispectral image, use feature points with known positions on the ground to align the multispectral image, and use affine transformation to perform geometric correction on the image to correct the geometric distortion caused by the movement of the drone or the change of terrain.

[0041] S102: Perform image fusion processing on the multispectral image and the color image to obtain a fused image to be detected.

[0042] Image fusion refers to the image processing process of synthesizing feature information from different images. Image fusion can effectively merge the feature information in multispectral images and color images, remove redundant information, retain key information, and obtain a fused image to be detected with more comprehensive feature information.

[0043] The fused image to be detected may refer to an image that has not been detected after image fusion is completed. The fused image to be detected may be an image obtained after image fusion processing, combining spectral information, color information and different spatial features of different bands. The fused image to be detected provides information from different sources, so it contains richer features and can better support subsequent target detection tasks. Through image fusion, not only the detailed information of the image is increased, but also the missing or error that may be caused by a single image source is eliminated. In a specific example, in the substation weed detection scenario, a drone is used to collect images in the substation. The collected multispectral images and color images provide information at different levels, respectively. Among them, the color image can provide detailed features visible to the naked eye, and the multispectral image can provide the spectral features of weeds. Through image fusion processing, the images are merged into a fused image to be detected that contains information about weeds in different spectral bands. When the fused image is provided to the target weed detection model for training and prediction, it can help the model to more accurately distinguish between weed areas and non-weed areas.

[0044] S103, inputting the fused image to be detected into the target weed detection model, which is trained based on the improved target detection model; and performing deep separable convolution processing on the fused image to be detected based on the backbone network in the target weed detection model to obtain feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network.

[0045] The target weed detection model may refer to a computer vision model designed for weed detection within the fence of a capacitor bank in a substation. The target detection target can be used to identify weeds and locate the weeds, thereby improving the weed detection efficiency and reducing the difficulty of manual cleaning and the missed detection rate.

[0046] Among them, the improved target detection model may refer to a computer vision model that is optimized and improved for the traditional target detection algorithm. For example, the target detection model may refer to YOLOv10 (You Only Look Once version 10), and Yolov10 may be improved to obtain an improved target detection model, which may be improved by replacing the backbone network, optimizing the attention mechanism, and improving the loss function. Using ordinary YOLO as the target weed detection model has low detection accuracy and large positioning deviations for weed bounding boxes of different sizes. The improved target detection model can better capture small or complex weed areas.

[0047] The backbone network may refer to a basic component in an object detection model. The backbone network is used to extract image features. For example, the backbone network DarkNet (Dark Neural Network) of YOLOv10 may be replaced with MobileNet3 (Mobile Neural Network Version 3), which greatly reduces the amount of calculation and the amount of parameters.

[0048] Among them, depthwise separable convolution can refer to an efficient convolution operation, including depthwise convolution and pointwise convolution, wherein pointwise convolution can refer to the fusion of features of different channels using a 1×1 convolution kernel. Depthwise separable convolution can reduce computational complexity and the amount of parameters. In a specific example, low-dimensional features are set in a high-dimensional space through 1×1 convolution, and then depthwise separable convolution is performed. Assuming that the input feature is x, first perform 1×1 convolution to expand the number of channels, then perform depthwise separable convolution, and then restore the feature to a low dimension through 1×1 convolution. MobielNet3 introduces the Hard-Swish activation function, which retains the nonlinear characteristics of Swish and is more efficient in calculation. The definition formula of the Hard-Swish activation function is as follows:

[0049]

[0050] Where ReLU6 is a linear activation function truncated between 0 and 6. The ReLU6 linear activation function is as follows:

[0051] ReLU6(x)=min(max(x,0),6)

[0052] The feature information to be detected may refer to the feature description information related to the target extracted by the target weed detection model from the fused image to be detected. For example, when the target is a weed, the feature information to be detected may refer to the texture, shape, color distribution and other features of the weed. The fused features to be detected can help the target weed detection model distinguish between weeds and background to improve the detection accuracy.

[0053] The neck network may refer to an intermediate module connecting the backbone network and the prediction head network in the target weed detection network. The neck network is used to enhance the information transfer capability between different networks.

[0054] The single prediction head network may refer to a network structure composed of independent output modules. The single prediction head network is used to output the target category and position.

[0055] S104, input the feature information to be detected into the neck network, perform local feature extraction processing on the feature information to be detected based on the hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction processing on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected.

[0056] Among them, the mixed local channel attention mechanism can refer to an attention mechanism that combines local feature information and global feature information. The mixed local channel attention mechanism can analyze the spatial information and channel information in the image, improve the feature extraction ability of the model, and reduce the loss of key feature information. The mixed channel attention mechanism can refer to (Mixed Local Channel Attentiion, MLCA).

[0057] Feature fusion refers to the fusion of local feature information and global feature information. Local feature information is used to emphasize detail features within a small range, and is suitable for detecting small target feature details such as the edge and fineness of weeds in the capacitor bank fence; global feature information is used to emphasize the overall feature relationship in the fused image to be detected, and is suitable for identifying the overall relationship features between weeds and capacitor banks.

[0058] The fused feature information to be detected may refer to a fused feature that fuses local feature information and global feature information. The fused feature information to be detected combines local and global features, and can balance details and overall information. The fused feature to be detected takes into account both large and small targets, and can improve the detection accuracy of the target weed model.

[0059] S105 , inputting the fused feature information to be detected into the single prediction head network for weed detection processing, and obtaining target weed detection information corresponding to each capacitor group.

[0060] Among them, weed detection can refer to the process of detecting whether there are weeds in the fused image to be detected. Through steps such as image processing, feature extraction and target detection, it is analyzed whether there are weeds in the fused image to be detected of each capacitor bank, and if there are weeds, the position and range of the weeds are detected.

[0061] The target weed detection information may refer to detailed information of weeds in the capacitor bank, wherein the detailed weed information includes relevant information such as the location, size and detection confidence of the weeds, and the detailed weed information is used to accurately describe the distribution location and actual geographical location of the weeds in the fused image to be detected.

[0062] Among them, through the dual label assignment strategy, and through the consistent matching metric to optimize the matching degree of one-to-one and one-to-many branches, we can ensure that better supervision signals are obtained during the training process, and ultimately improve the reasoning performance of the model. This method achieves effective alignment of supervision information and improves the sample quality and performance of the model.

[0063] During the assignment process, both one-to-one and one-to-many methods use metrics to quantitatively evaluate the degree of consistency between predictions and instances. In order to achieve prediction-aware matching of the two branches, a unified matching metric is adopted, which is formulated as follows:

[0064]

[0065] In the formula, p represents the classification score; and b represent the bounding boxes of the prediction and instance, respectively. S represents the spatial prior, which is used to indicate whether the predicted anchor point is within the instance; α and β represent two important hyperparameters that balance the influence of the semantic prediction task and the location regression task.

[0066] In a specific example, the collected multispectral image and color image can be preprocessed to generate a fused image to be detected, and the fused image to be detected is input into a target weed detection model to obtain target weed detection information corresponding to the fused image to be detected, and the capacitor bank identified in the image and the corresponding GPS coordinates are matched, and professionals are notified to remove the weeds.

[0067] It can be seen that in the embodiment of the present application, by obtaining the multispectral image and color image corresponding to each capacitor group, multi-dimensional image information can be provided for subsequent weed detection; through image fusion processing, the image features of the multispectral image and the color image can be combined to obtain richer image features; by inputting the fused image to be detected into the target weed detection model, the advantages of the improved target detection model can be utilized to improve the detection effect of weeds; through deep separable convolution processing, the feature information of the fused image to be detected can be efficiently extracted, the consumption of computing resources can be reduced, and the effect of feature extraction can be improved; the introduction of the neck network and the single prediction head network can further optimize the detail processing of weed detection and improve the overall detection accuracy; through local feature extraction, the fine weed features in the image can be better identified, thereby improving the detection accuracy and local detection ability; through global feature extraction, the weed distribution in the image can be more comprehensively analyzed to enhance the global recognition ability of the model; through feature fusion processing, the local feature information and the global feature information can be integrated to improve the overall recognition ability of the model for weeds, and enhance the accuracy and generalization ability of the model; through single prediction head network processing, efficient weed detection can be achieved to ensure that the weed situation of each capacitor group is accurately identified.

[0068] The "image fusion processing of the multispectral image and the color image to obtain the fused image to be detected" is refined into "processing the multispectral image and the color image through wavelet transform to decompose the high-frequency components and low-frequency components of a preset number of levels; based on the high-frequency components and low-frequency components of each level, respectively determine the entropy values ​​corresponding to the high-frequency components and each low-frequency components of each level; based on the entropy values ​​corresponding to the high-frequency components and each low-frequency components of each level, determine the weights corresponding to the high-frequency components and each low-frequency components of each level; based on the weights corresponding to the high-frequency components and each low-frequency components of each level, perform weighted average fusion processing on the high-frequency components and each low-frequency components of each level to obtain component fusion data corresponding to a preset number of levels; perform inverse wavelet transform processing on the component fusion data corresponding to the preset number of levels to obtain the fused image to be detected", so as to improve the operation of weed detection in substations.

[0069] It should be noted that for the parts not described in detail in the embodiments of the present invention, reference may be made to the descriptions of other embodiments. Figure 2 The present invention provides a flowchart of a method for detecting weeds in a substation according to an embodiment of the present invention.

[0070] See also Figure 2 The substation weed detection method shown comprises:

[0071] S201. Acquire a multispectral image and a color image corresponding to each capacitor bank in a substation.

[0072] S202 , processing the multispectral image and the color image through wavelet transform, and decomposing them into high-frequency components and low-frequency components of a preset number of levels.

[0073] Among them, wavelet transform processing can refer to a processing method that decomposes a signal into different frequency components and analyzes them simultaneously in the time domain and the frequency domain. Wavelet transform processing can decompose a multispectral image into high-frequency components and low-frequency components at different levels. In a specific example, wavelet transform processing can be implemented by the following steps:

[0074] 1) By scaling and translating the wavelet function, the wavelet function can refer to the mother wavelet. The expression of the wavelet function is as follows:

[0075]

[0076] Where ψ(t) represents the wavelet function; a and b represent the constant factors of wavelet basis scaling and translation.

[0077] 2) Assuming that the wavelet function is Fourier transform, the frequency domain transform of the signal is expressed as follows:

[0078]

[0079] Where WTx(a,b) represents the wavelet transform result of the signal; X(Ω) represents the Fourier transform of the signal; Ω represents the frequency variable; ψ*(aΩ) represents the Fourier transform of the wavelet function; e jΩb Represents the phase shift of a signal in the frequency domain.

[0080] Based on the above steps, the panchromatic band data of the multispectral image can be decomposed into high-frequency components and low-frequency components at different levels through different wavelet basis functions.

[0081] The preset number of levels may refer to the number of wavelet classification levels being set according to actual experience.

[0082] The high-frequency component may refer to a high-frequency signal; and the low-frequency component may refer to a low-frequency signal.

[0083] S203: Based on the high-frequency components and low-frequency components of each level, respectively determine the entropy values ​​corresponding to the high-frequency components of each level and each low-frequency component.

[0084] In a specific example, determining the entropy values ​​corresponding to the high-frequency components of each level and each low-frequency component can be achieved by the following steps:

[0085] 1) The high-frequency components and low-frequency components of each level of wavelet decomposition are taken as separate variables, and the formula for calculating the correlation between different variables is as follows:

[0086]

[0087] In the formula, r i k is the correlation coefficient; cov represents covariance; Var represents variance; X represents the variable of different frequency components of wavelet decomposition.

[0088] 2) Based on the correlation coefficients between different components, the formula for calculating the indicators between individual variables is as follows:

[0089]

[0090] In the formula, y ij k Represents the indicator between individual variables. The above formula is a normalization formula that maps the variables within a certain interval.

[0091] S204: Determine the weights corresponding to the high-frequency components and each low-frequency component of each level based on the entropy values ​​corresponding to the high-frequency components and each low-frequency component of each level.

[0092] In a specific example, determining the weights corresponding to the high-frequency components and each low-frequency component of each level can be achieved by the following steps:

[0093] 1) The redundancy formula of different variables is as follows:

[0094]

[0095] Where, d i k represents the redundancy of the i-th sample on the k-th index; e i k It represents the actual amount of information of the i-th sample on the k-th index.

[0096] 2) The formula for calculating the weights of different indicators is as follows:

[0097]

[0098] In the formula, w i k Represents weight.

[0099] S205 , based on the weights corresponding to the high-frequency components of each level and each low-frequency component respectively, perform weighted average fusion processing on the high-frequency components of each level and each low-frequency component to obtain component fusion data corresponding to a preset number of levels.

[0100] The weighted average fusion may refer to a mathematical method for fusing different component information. The weighted average fusion calculates the importance of each component by weighted summing different components to generate an image containing all component information.

[0101] Among them, component fusion data can refer to a data set obtained by weighted fusion of components at different levels. Component fusion data is used to represent the comprehensive results of different frequency information at each level in the image. The role of component fusion data is to bridge multispectral images and color images, so that the two images can complement each other while retaining their respective advantages, forming a more expressive fusion image to be detected.

[0102] In a specific example, based on the weights corresponding to the high-frequency components and each low-frequency component of each level, the high-frequency components and each low-frequency component of each level are subjected to weighted average fusion processing to obtain component fusion data corresponding to a preset number of levels, which can be achieved by the following steps:

[0103] 3) In summary, the formula for fusing frequency information through the entropy weighting method is as follows:

[0104]

[0105] In the formula, s ij k represents the information fused by entropy weight theory, w i kis the weight of the calculation, x ij k Represents the frequency information of wavelet decomposition.

[0106] S206 , performing inverse wavelet transform processing on the component fusion data corresponding to a preset number of levels to obtain a fused image to be detected.

[0107] The inverse wavelet transform process may refer to an inverse process of the wavelet transform, which is used to reconstruct the original image or signal from the decomposed data obtained by the wavelet transform. For example, the fused image to be detected may be obtained from the component fused data through the inverse wavelet transform process.

[0108] S207, inputting the fused image to be detected into the target weed detection model, which is trained based on the improved target detection model; and performing deep separable convolution processing on the fused image to be detected based on the backbone network in the target weed detection model to obtain feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network.

[0109] S208. Input the feature information to be detected into the neck network, perform local feature extraction on the feature information to be detected based on the hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected.

[0110] S209: Input the fused feature information to be detected into the single prediction head network for weed detection processing to obtain target weed detection information corresponding to each capacitor group.

[0111] It can be seen that in this embodiment, by decomposing the multispectral image and the color image through wavelet transform, the high-frequency component and the low-frequency component of the image can be extracted in layers, so that different characteristics of the image can be processed more specifically; by calculating the entropy value of the high-frequency component and the low-frequency component at each level, the richness of the image information can be quantified, which is convenient for quantitative evaluation of the importance of the component; by determining the weight by using the entropy value, more reasonable weights can be assigned according to the information volume of the component, thereby improving the effectiveness and reliability of the fusion result; by weighted averaging the high-frequency component and the low-frequency component through weights, the rich spectral information of the multispectral image and the detailed information of the color image can be effectively combined; by inverse wavelet transform, the layered fusion result can be restored to the spatial domain to generate a fused image that retains both the multispectral information and the details of the color image, which is convenient for subsequent analysis and detection.

[0112] In some embodiments, based on the backbone network in the target weed detection model, a deep separable convolution process is performed on the fused image to be detected to obtain feature information to be detected corresponding to the fused image to be detected, including:

[0113] Based on the backbone network, the channel expansion convolution operation is performed on the fused image to be detected to expand the image feature channel to obtain a preset number of channels;

[0114] Combine a preset number of channels to perform deep convolution operation on the fused image to be detected to obtain image feature information of each channel;

[0115] A point-by-point convolution operation is performed on the image feature information corresponding to a preset number of channels to obtain the feature information to be detected.

[0116] Among them, the channel expansion convolution operation may refer to a convolution operation for expanding the number of channels of a feature map. The input feature map is processed by convolution, and then the number of channels is expanded by 1×1 convolution. The depth of the feature map is increased or decreased by adjusting the number of channels. The role of 1×1 convolution is to apply convolution to each position of the input image, but the convolution kernel size is 1, and it acts on different channels. In a specific example, suppose an input image has 3 channels (such as a color image). After channel expansion using 1×1 convolution, the number of channels of the image can be expanded from 3 to 64, thereby providing more feature information for subsequent processing steps. Since the image data of each band contains different visual information, by increasing the number of channels, the model can learn more diverse features, and ultimately improve the accuracy of weed detection.

[0117] Afterwards, the feature information to be detected is input into the partial self-attention module, and the feature information to be detected is evenly divided into the first feature information and the second feature information; the first feature information is subjected to feature extraction processing of the attention mechanism to obtain the key feature information corresponding to the first feature information; the key feature information and the second feature information are subjected to feature fusion processing to obtain the feature enhancement information to be detected corresponding to the feature information to be detected.

[0118] In a specific example, the channel dimension is adjusted by point-by-point convolution, and then spatial downsampling is achieved through depth-wise convolution, which significantly reduces the computational and parameter overhead while retaining more information. Compact inversion blocks can be used to reduce redundant computational overhead, and different stages can be sorted to determine which stages use compact designs, thereby achieving a better balance between efficiency and capacity.

[0119] It can be seen that in this embodiment, by performing a channel expansion convolution operation on the fused image to be detected, the number of feature channels of the image can be increased, thereby providing richer feature expressions for subsequent processing; by performing a deep convolution operation in combination with the expanded preset number of channels, the image feature information of each channel can be effectively extracted, thereby more finely capturing the features of the target area; through point-by-point convolution, the features extracted from each channel can be further fused and simplified, thereby generating feature information to be detected, thereby improving the feature expression efficiency and the computational performance of the detection model.

[0120] In some embodiments, the feature information to be detected is input into the neck network, and based on the hybrid local channel attention mechanism, local feature extraction processing is performed on the feature information to be detected to obtain local feature information corresponding to the feature information to be detected; global feature extraction processing is performed on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected; feature fusion processing is performed on the local feature information and the global feature information to obtain fused feature information to be detected corresponding to the feature information to be detected, including:

[0121] The feature information to be detected is input into the neck network, and local feature extraction is performed on the feature information to be detected based on the hybrid local channel attention mechanism to obtain local feature information corresponding to the feature information to be detected;

[0122] Performing feature conversion processing on the local feature information to obtain local conversion feature information;

[0123] Perform feature fusion processing on the local conversion feature information and the feature information to be detected to obtain significant feature information;

[0124] Performing global feature extraction processing on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected;

[0125] Performing feature conversion processing on the global feature information to obtain global conversion feature information;

[0126] The global conversion feature information and the significant feature information are subjected to feature fusion processing to obtain the fused feature information to be detected corresponding to the feature information to be detected.

[0127] Feature transformation refers to the operation of processing original feature information so as to change its spatial distribution, channel weight or other dimensions.

[0128] The local conversion feature information may refer to the result obtained after feature conversion processing is performed on the local feature information. The local conversion feature information includes fine-grained information captured by the local feature extraction mechanism, and further enhances the local feature information.

[0129] Among them, the salient feature information can refer to a feature representation generated by the feature fusion processing of the local conversion feature information and the feature information to be detected. The salient feature information highlights the most discernible areas and patterns in the input data for the detection task, and is an important basis for further extracting global features and final fusion.

[0130] The global conversion feature information may refer to feature information generated after specific processing of the global feature information. The global feature information mainly focuses on the spatial distribution and contextual relationship of the input data as a whole, and after the feature conversion operation, the global conversion feature information is optimized to a form more suitable for fusion with the salient feature information.

[0131] In a specific example, hybrid local channel attention can be introduced in the neck network, combining local and global features as well as channel and spatial features, which can significantly improve the detection accuracy with a small increase in parameters. The Local SE (LSE) method divides the feature map into multiple patches and applies a local attention mechanism to ensure that the important spatial features within each patch are preserved. Its workflow is as follows:

[0132] First, the input feature map (C, W, H) first undergoes local average pooling and global average pooling. The local and global pooled features are processed by one-dimensional convolution to achieve feature conversion. After one-dimensional convolution, the features are rearranged to adapt to subsequent processing steps; secondly, for the local pooled features, after one-dimensional convolution and rearrangement, they are combined with the original input features through a "multiplication" operation; then, the global pooled features are fused with the local pooled features through an "addition" operation after one-dimensional convolution and rearrangement, thereby introducing global context information; finally, the feature map processed by the local and global attention mechanisms is restored to the original spatial dimension through an unpooling operation.

[0133] It can be seen that in this embodiment, by inputting the feature information to be detected into the neck network and applying the hybrid local channel attention mechanism, it is possible to effectively extract local important information in the input features, thereby more accurately capturing fine-grained features; by performing feature conversion processing on the local feature information, the local feature information can be optimized and adjusted, and the feature conversion processing can eliminate redundant data in the local feature information; by fusing the local conversion feature information and the feature information to be detected, the coordinated representation of the local features and the original features can be achieved, thereby generating more significant feature information; by global feature extraction processing, the global contextual relationship of the feature information to be detected can be obtained, thereby achieving the grasp of the overall layout and distribution pattern of the input data; the extraction of global feature information can provide a global perspective for subsequent feature fusion, and make up for the overall information that may be missed by local features; by performing feature conversion processing on the global feature information, the global information can be optimized; by fusing the global conversion feature information with the significant feature information, the complementary representation of the global and local information can be achieved, generating a more complete and discernible fusion feature.

[0134] In some embodiments, after inputting the fused feature information to be detected into the single prediction head network for weed detection processing and obtaining the target weed detection information corresponding to each capacitor bank, the method further includes:

[0135] When the target weed detection information indicates that weeds exist in the fused image to be detected, an alarm is issued;

[0136] Perform image recognition processing on the fused image to be detected to determine the capacitor group identification information corresponding to the fused image to be detected;

[0137] Based on the capacitor bank identification information, obtaining target position information of the capacitor bank corresponding to the capacitor bank identification information;

[0138] According to the capacitor bank identification information, target location information and target weed detection information, a weed detection report is generated and sent to the client.

[0139] The alarm may refer to a warning signal that is automatically triggered when weeds are detected in the fused image to be detected. The alarm is used to prompt relevant personnel to perform subsequent processing, such as cleaning the weeds or checking the equipment.

[0140] The capacitor bank identification information may refer to a unique identifier for marking the capacitor bank. The identification information may include a combination of numbers, letters, or symbols. For example, a capacitor bank in a substation may be numbered C001 and C002 as identification information. The identification information is generated by the system when the drone collects data in combination with its GPS coordinates and device location to ensure accurate association between image acquisition and specific device location. The identification information may be extracted through image recognition technology, for example, by reading the number on the device nameplate, scanning a QR code or a barcode.

[0141] The target location information may refer to the specific location data of the capacitor bank in space. The target location information may include GPS coordinates (longitude and latitude), or the relative location of the capacitor bank in the substation, for example, the Xth station in the southeast corner. When the drone collects multispectral images and color images, it will record its own position and match the target location information of the collected image with the identification information of the capacitor bank through visual matching or geographic calibration.

[0142] The weed detection report may refer to a detection report of detailed information on the weed conditions in the capacitor bank area. The weed detection report may include capacitor bank identification information, target location information, and target weed detection information, wherein the capacitor bank identification information is used to determine that weeds appear in a specific capacitor bank area; the target location information is used to determine the geographical location of the capacitor bank; and the target weed detection information includes information such as the location and size of the weeds.

[0143] It can be seen that in this embodiment, by issuing an alarm in time when a problem is found in weed detection, timely warning can be achieved to prevent weeds from affecting the normal operation of the capacitor bank; through image recognition processing, accurate positioning and identification of the capacitor bank can be achieved to ensure that subsequent weed detection and processing operations are accurate to a specific capacitor bank; by obtaining the target position information of the capacitor bank, accurate geographic positioning can be achieved to ensure that the weed detection result is consistent with the actual position and avoid erroneous operations; by generating and sending a weed detection report, a comprehensive record of the weed situation can be achieved to help staff make decisions and handle it in a timely manner, thereby improving the maintenance efficiency and reliability of the equipment.

[0144] In some embodiments, the target weed detection model is trained in the following manner:

[0145] Acquire sample multispectral images and sample color images corresponding to the plurality of capacitor bank samples, and preset weed detection information corresponding to the plurality of capacitor bank samples;

[0146] Performing image fusion processing on sample multispectral images and sample color images corresponding to the plurality of capacitor bank samples to obtain sample fusion images corresponding to the plurality of capacitor bank samples;

[0147] The sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained for weed detection processing, so as to obtain the predicted weed detection information corresponding to each capacitor group sample; the weed detection model to be trained is a target detection model;

[0148] According to the preset weed detection information corresponding to the multiple capacitor bank samples and the predicted weed detection information corresponding to each capacitor bank sample, the weed detection model to be trained is trained to obtain a target weed detection model.

[0149] In a specific example, training a target weed detection model can be achieved by following the steps below:

[0150] 1) Due to the complex environment and changing scenes of substations, the number of weed datasets is limited. Therefore, in order to improve the generalization ability of the target weed detection model, image enhancement techniques are applied, including horizontal flipping, vertical flipping, image blurring, and random transformation of hue and saturation.

[0151] 2) The weeds in the capacitor bank fence are manually labeled using the labeling system, and the weeds in the image are labeled using the minimum enclosing rectangle. The labeled dataset is stored in the labeling format of the YOLO dataset as a txt file, which contains the category and coordinate information of the labeled box in each image, recorded as the center point coordinates (x, y) of the labeled box and the width and height (w, h) of the labeled box to determine the relative position of the weeds in the image.

[0152] 3) Divide all images into training set and validation set in a ratio of 8:2 to construct a dataset.

[0153] 4) After setting the hyperparameters, train for 300 rounds to obtain the trained target weed detection model.

[0154] It can be seen that in this embodiment, by acquiring sample images of multiple capacitor banks, multi-dimensional data collection can be achieved, providing more information sources, thereby enhancing the robustness and accuracy of the target weed detection model; through image fusion processing, the advantages of multispectral images and color images can be combined to obtain richer feature information; by inputting the fused image into the weed detection model to be trained for weed detection processing, the model can learn how to effectively identify weeds from images during the training process, thereby improving the accuracy of prediction and the effect of detection; by comparing the preset weed detection information and the prediction results, the model can be optimized through error feedback, and the parameters can be continuously adjusted, ultimately improving the accuracy and generalization ability of the target weed detection model and enhancing the adaptability of the model.

[0155] In some embodiments, the weed detection model to be trained includes a backbone network, a neck network, a multi-prediction head network, and a single prediction head network; the predicted weed detection information includes a plurality of first predicted weed detection results and a second predicted weed detection result; the sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained for weed detection processing, and the predicted weed detection information corresponding to each capacitor group sample is obtained, including:

[0156] The sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained, and based on the backbone network, the sample fusion image is subjected to deep separable convolution processing to obtain the sample feature information corresponding to the sample fusion image;

[0157] The sample feature information is input into the neck network, and based on the hybrid local channel attention mechanism, the sample feature information is subjected to local feature extraction processing to obtain sample local feature information corresponding to the sample feature information; the sample feature information is subjected to global feature extraction processing to obtain sample global feature information corresponding to the sample feature information; the sample local feature information and the sample global feature information are subjected to feature fusion processing to obtain sample fusion feature information corresponding to the sample feature information;

[0158] Inputting the sample fusion feature information into a multi-prediction head network for weed detection processing to obtain a plurality of first predicted weed detection results corresponding to each capacitor bank sample;

[0159] The sample fusion feature information is input into the single prediction head network for weed detection processing to obtain a second predicted weed detection result corresponding to each capacitor bank sample.

[0160] The first predicted weed detection result is obtained by the multi-prediction head network, so the first predicted weed detection result may include detailed information such as the presence, location, category, and size of the weeds. Exemplarily, if there are weeds of various sizes in the image, the first predicted weed detection result will give weed detection results of different scales according to different heads of the multi-prediction head network, such as small weeds and large weeds, and provide their locations and categories.

[0161] The second predicted weed detection result is obtained by a single prediction head network, and thus the second predicted weed detection result may include a single prediction result for the position of the weeds. The second weed detection result is used to provide a weed position frame, for example, marking the position of the weeds by a simple rectangular frame.

[0162] It can be seen that in this embodiment, the amount of calculation can be effectively reduced by using the depthwise separable convolution method, while retaining the key information in the image, thereby improving the efficiency and accuracy of feature extraction and helping the model to better capture the important features in the image in weed detection; by fusing local and global features, information at all levels can be integrated to improve the accuracy and robustness of weed detection; by using the multi-prediction head network, multi-dimensional weed detection can be achieved, including different scales and angles, etc., which increases the detection capability of the model and makes it more flexible in complex scenarios, and can provide multiple prediction results to adapt to different weed types and distributions; by using the single prediction head network for single-task detection, a more concise and intuitive detection result can be obtained, which is suitable for simple weed detection scenarios.

[0163] In some embodiments, according to the preset weed detection information corresponding to the plurality of capacitor bank samples and the predicted weed detection information corresponding to each capacitor bank sample, the weed detection model to be trained is trained to obtain a target weed detection model, including:

[0164] Determine the weed detection loss of the weed model to be trained according to a plurality of first predicted weed detection results corresponding to each capacitor group sample and predicted weed detection information corresponding to each capacitor group sample, and according to a second predicted weed detection result corresponding to each capacitor group sample and predicted weed detection information corresponding to each capacitor group sample, in combination with a bounding box regression loss function;

[0165] Based on the weed detection loss, the weed detection model to be trained is trained to obtain the target weed detection model.

[0166] Among them, the bounding box regression loss function can refer to a function that calculates the position error of the weeds. The bounding box is a rectangular box used to calibrate the weeds in the image. In the target weed detection task, the goal of the model is not only to identify the category of the object, but also to accurately predict the location of the object, which can be achieved by predicting the coordinates of the bounding box. The regression loss function can be used to evaluate the difference between the predicted bounding box and the actual bounding box.

[0167] Among them, weed detection loss can refer to the comprehensive loss function in the entire model training. Weed detection loss combines classification error and positioning error. The classification error measures whether the model correctly identifies the presence of weeds, while the bounding box regression loss function measures whether the model accurately predicts the location of weeds.

[0168] It can be seen that in this embodiment, by combining multiple first predicted weed detection results and second predicted weed detection results, the model can learn the weed detection task from different perspectives and enhance the model's ability to detect weeds in various situations; the bounding box regression loss function can effectively quantify the error of weed position prediction, help the model to continuously adjust, and improve the accurate recognition of weed positions; by combining multiple different prediction results and bounding box regression loss, it can ensure that the model achieves high accuracy in both recognition and location of weeds; training the model according to the weed detection loss helps to comprehensively improve the performance of the model in the weed detection task, so that it can provide accurate output in both classification and positioning; presetting the weed detection information as control data helps to guide model training, reduce unnecessary calculations, and improve the effectiveness of the training process.

[0169] Figure 3 A flow chart of a substation weed detection method provided by an embodiment of the present invention;

[0170] Figure 4 A schematic diagram of the structure of a substation weed detection device provided by an embodiment of the present invention. The embodiment of the present invention is applicable to substation weed detection, the device can execute a substation weed detection method, and the device can be implemented in the form of hardware and / or software.

[0171] See also Figure 4 The substation weed detection device shown includes: an image acquisition module 401, an image fusion module 402, a target detection model processing module 403, a feature extraction and fusion module 404 and a target weed detection information acquisition module 405, wherein:

[0172] An image acquisition module 401 is used to acquire a multispectral image and a color image corresponding to each capacitor bank in the substation;

[0173] An image fusion module 402 is used to perform image fusion processing on the multispectral image and the color image to obtain a fused image to be detected;

[0174] The target detection model processing module 403 is used to input the fused image to be detected into the target weed detection model, which is obtained by training based on the improved target detection model; and based on the backbone network in the target weed detection model, the fused image to be detected is subjected to a depthwise separable convolution process to obtain the feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network;

[0175] The feature extraction and fusion module 404 is used to input the feature information to be detected into the neck network, perform local feature extraction processing on the feature information to be detected based on the hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction processing on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected;

[0176] The target weed detection information acquisition module 405 is used to input the fused feature information to be detected into the single prediction head network for weed detection processing, and obtain the target weed detection information corresponding to each capacitor group.

[0177] The technical solution of the embodiment of the present invention can provide multi-dimensional image information for subsequent weed detection by acquiring multispectral images and color images corresponding to each capacitor group; through image fusion processing, the image features of multispectral images and color images can be combined to obtain richer image features; by inputting the fused image to be detected into the target weed detection model, the advantages of the improved target detection model can be utilized to improve the weed detection effect; through deep separable convolution processing, the feature information of the fused image to be detected can be efficiently extracted, the consumption of computing resources can be reduced, and the effect of feature extraction can be improved; the neck network and the single prediction head network are introduced to further optimize the detail processing of weed detection and improve the overall detection accuracy; through local feature extraction, the fine weed features in the image can be better identified, thereby improving the detection accuracy and local detection ability; through global feature extraction, the weed distribution in the image can be more comprehensively analyzed to enhance the global recognition ability of the model; through feature fusion processing, local feature information and global feature information can be integrated to improve the overall recognition ability of the model for weeds, and enhance the accuracy and generalization ability of the model; through single prediction head network processing, efficient weed detection can be achieved to ensure that the weed situation of each capacitor group is accurately identified.

[0178] In some embodiments, in performing image fusion processing on the multispectral image and the color image to obtain the fused image to be detected, the image fusion module 402 is specifically used to:

[0179] The multispectral image and the color image are processed by wavelet transform to decompose into high-frequency components and low-frequency components of a preset number of levels;

[0180] Based on the high-frequency components and low-frequency components of each level, respectively determine the entropy values ​​corresponding to the high-frequency components of each level and each low-frequency component;

[0181] Based on the entropy values ​​corresponding to the high-frequency components of each level and each low-frequency component, respectively, determining the weights corresponding to the high-frequency components of each level and each low-frequency component, respectively;

[0182] Based on the weights corresponding to the high-frequency components and each low-frequency component of each level, a weighted average fusion process is performed on the high-frequency components and each low-frequency component of each level to obtain component fusion data corresponding to a preset number of levels;

[0183] The component fusion data corresponding to the preset number of levels are processed by inverse wavelet transform to obtain a fused image to be detected.

[0184] In some embodiments, based on the backbone network in the target weed detection model, the target detection model processing module 403 is specifically used to perform a deep separable convolution process on the fused image to be detected to obtain the feature information to be detected corresponding to the fused image to be detected:

[0185] Based on the backbone network, the channel expansion convolution operation is performed on the fused image to be detected to expand the image feature channel to obtain a preset number of channels;

[0186] Combine a preset number of channels to perform deep convolution operation on the fused image to be detected to obtain image feature information of each channel;

[0187] A point-by-point convolution operation is performed on the image feature information corresponding to a preset number of channels to obtain the feature information to be detected.

[0188] In some embodiments, when inputting the feature information to be detected into the neck network, based on the hybrid local channel attention mechanism, performing local feature extraction processing on the feature information to be detected to obtain local feature information corresponding to the feature information to be detected; performing global feature extraction processing on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected; performing feature fusion processing on the local feature information and the global feature information to obtain fused feature information to be detected corresponding to the feature information to be detected, the feature extraction and fusion module 404 is specifically used to:

[0189] The feature information to be detected is input into the neck network, and local feature extraction is performed on the feature information to be detected based on the hybrid local channel attention mechanism to obtain local feature information corresponding to the feature information to be detected;

[0190] Performing feature conversion processing on the local feature information to obtain local conversion feature information;

[0191] Perform feature fusion processing on the local conversion feature information and the feature information to be detected to obtain significant feature information;

[0192] Performing global feature extraction processing on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected;

[0193] Performing feature conversion processing on the global feature information to obtain global conversion feature information;

[0194] The global conversion feature information and the significant feature information are subjected to feature fusion processing to obtain the fused feature information to be detected corresponding to the feature information to be detected.

[0195] In some embodiments, after inputting the fused feature information to be detected into the single prediction head network for weed detection processing to obtain the target weed detection information corresponding to each capacitor group, the target weed detection information acquisition module 405 is further used to:

[0196] In some embodiments, when the target weed detection model is trained in the following manner, the target detection model processing module 403 is specifically used to:

[0197] Acquire sample multispectral images and sample color images corresponding to the plurality of capacitor bank samples, and preset weed detection information corresponding to the plurality of capacitor bank samples;

[0198] Performing image fusion processing on sample multispectral images and sample color images corresponding to the plurality of capacitor bank samples to obtain sample fusion images corresponding to the plurality of capacitor bank samples;

[0199] The sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained for weed detection processing, so as to obtain the predicted weed detection information corresponding to each capacitor group sample; the weed detection model to be trained is a target detection model;

[0200] According to the preset weed detection information corresponding to the multiple capacitor bank samples and the predicted weed detection information corresponding to each capacitor bank sample, the weed detection model to be trained is trained to obtain a target weed detection model.

[0201] In some embodiments, in the aspect that the weed detection model to be trained includes a backbone network, a neck network, a multi-prediction head network and a single prediction head network; the predicted weed detection information includes a plurality of first predicted weed detection results and a second predicted weed detection result; the sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained for weed detection processing to obtain the predicted weed detection information corresponding to each capacitor group sample, the target detection model processing module 403 is specifically used to:

[0202] The sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained, and based on the backbone network, the sample fusion image is subjected to deep separable convolution processing to obtain the sample feature information corresponding to the sample fusion image;

[0203] The sample feature information is input into the neck network, and based on the hybrid local channel attention mechanism, the sample feature information is subjected to local feature extraction processing to obtain sample local feature information corresponding to the sample feature information; the sample feature information is subjected to global feature extraction processing to obtain sample global feature information corresponding to the sample feature information; the sample local feature information and the sample global feature information are subjected to feature fusion processing to obtain sample fusion feature information corresponding to the sample feature information;

[0204] Inputting the sample fusion feature information into a multi-prediction head network for weed detection processing to obtain a plurality of first predicted weed detection results corresponding to each capacitor bank sample;

[0205] The sample fusion feature information is input into the single prediction head network for weed detection processing to obtain a second predicted weed detection result corresponding to each capacitor bank sample.

[0206] In some embodiments, in training the weed detection model to be trained according to the preset weed detection information corresponding to the plurality of capacitor bank samples and the predicted weed detection information corresponding to each capacitor bank sample to obtain the target weed detection model, the target detection model processing module 403 is specifically used to:

[0207] Determine the weed detection loss of the weed model to be trained according to a plurality of first predicted weed detection results corresponding to each capacitor group sample and predicted weed detection information corresponding to each capacitor group sample, and according to a second predicted weed detection result corresponding to each capacitor group sample and predicted weed detection information corresponding to each capacitor group sample, in combination with a bounding box regression loss function;

[0208] Based on the weed detection loss, the weed detection model to be trained is trained to obtain the target weed detection model.

[0209] The substation weed detection device provided in the embodiment of the present invention can execute the substation weed detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the substation weed detection method.

[0210] Figure 5 A schematic structural diagram of a substation weed detection device provided by an embodiment of the present invention.

[0211] like Figure 5As shown, the substation weed detection device 500 includes at least one processor 501, and a memory connected to the at least one processor 501, such as a read-only memory (ROM) 502, a random access memory (RAM) 503, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 501 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 502 or the computer program loaded from the storage unit 508 to the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the substation weed detection device 500 can also be stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 508 is also connected to the bus 504.

[0212] Multiple components in the substation weed detection device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the substation weed detection device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0213] The processor 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 501 executes the various methods and processes described above, such as a substation weed detection method.

[0214] In some embodiments, the substation weed detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the substation weed detection device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the processor 501, one or more steps of the substation weed detection method described above may be performed. Alternatively, in other embodiments, the processor 501 may be configured to perform the substation weed detection method in any other appropriate manner (e.g., by means of firmware).

[0215] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0216] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0217] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0218] To provide interaction with a user, the systems and techniques described herein may be implemented on an operational detection device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the substation weed detection device. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0219] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0220] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS (Virtual Private Server) services.

[0221] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0222] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting weeds in a substation, characterized in that: The method comprises: Obtain multispectral images and color images corresponding to each capacitor bank in the substation; Performing image fusion processing on the multispectral image and the color image to obtain a fused image to be detected; The fused image to be detected is input into a target weed detection model, wherein the target weed detection model is obtained by training based on an improved target detection model; and based on a backbone network in the target weed detection model, a depthwise separable convolution process is performed on the fused image to be detected to obtain feature information to be detected corresponding to the fused image to be detected; the target weed detection model further includes a neck network and a single prediction head network; Input the feature information to be detected into the neck network, perform local feature extraction processing on the feature information to be detected based on the hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction processing on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected; The fused feature information to be detected is input into the single prediction head network for weed detection processing to obtain target weed detection information corresponding to each capacitor group.

2. The substation weed detection method according to claim 1, characterized in that: The performing image fusion processing on the multispectral image and the color image to obtain a fused image to be detected includes: Processing the multispectral image and the color image by wavelet transform to decompose them into high-frequency components and low-frequency components of a preset number of levels; Based on the high-frequency components and low-frequency components of each level, respectively determine the entropy values ​​corresponding to the high-frequency components of each level and each low-frequency component; Determine the weights corresponding to the high-frequency components and each low-frequency component of each level based on the entropy values ​​corresponding to the high-frequency components and each low-frequency component of each level; Based on the weights corresponding to the high-frequency components of each level and each low-frequency component, respectively, weighted average fusion processing is performed on the high-frequency components of each level and each low-frequency component, so as to obtain component fusion data corresponding to a preset number of levels; The component fusion data corresponding to the preset number of levels are subjected to inverse wavelet transformation processing to obtain a fused image to be detected.

3. The substation weed detection method according to claim 1, characterized in that: The method of performing a depth-separable convolution process on the fused image to be detected based on the backbone network in the target weed detection model to obtain feature information to be detected corresponding to the fused image to be detected includes: Based on the backbone network, performing a channel expansion convolution operation on the fused image to be detected, expanding the image feature channel, and obtaining a preset number of channels; Performing a deep convolution operation on the fused image to be detected in combination with the preset number of channels to obtain image feature information of each channel; A point-by-point convolution operation is performed on the image feature information corresponding to the preset number of channels to obtain feature information to be detected.

4. The substation weed detection method according to claim 1, characterized in that: The feature information to be detected is input into the neck network, and based on the hybrid local channel attention mechanism, local feature extraction processing is performed on the feature information to be detected to obtain local feature information corresponding to the feature information to be detected; global feature extraction processing is performed on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected; Performing feature fusion processing on the local feature information and the global feature information to obtain fused feature information to be detected corresponding to the feature information to be detected, including: Inputting the feature information to be detected into the neck network, performing local feature extraction processing on the feature information to be detected based on a hybrid local channel attention mechanism, and obtaining local feature information corresponding to the feature information to be detected; Performing feature conversion processing on the local feature information to obtain local conversion feature information; Performing feature fusion processing on the local conversion feature information and the feature information to be detected to obtain significant feature information; Performing global feature extraction processing on the feature information to be detected to obtain global feature information corresponding to the feature information to be detected; Performing feature conversion processing on the global feature information to obtain global conversion feature information; The global conversion feature information and the significant feature information are subjected to feature fusion processing to obtain fused feature information to be detected corresponding to the feature information to be detected.

5. The substation weed detection method according to claim 1, characterized in that: After inputting the fused feature information to be detected into the single prediction head network for weed detection processing to obtain target weed detection information corresponding to each capacitor group, the method further includes: When the target weed detection information indicates that weeds exist in the fused image to be detected, issuing an alarm; Performing image recognition processing on the fused image to be detected to determine the capacitor bank identification information corresponding to the fused image to be detected; Based on the capacitor bank identification information, acquiring target position information of the capacitor bank corresponding to the capacitor bank identification information; A weed detection report is generated according to the capacitor bank identification information, the target location information and the target weed detection information and sent to the client.

6. The substation weed detection method according to claim 1, characterized in that: The target weed detection model is trained in the following manner: Acquire sample multispectral images and sample color images corresponding to a plurality of capacitor bank samples, and preset weed detection information corresponding to the plurality of capacitor bank samples; Performing image fusion processing on sample multispectral images and sample color images corresponding to the plurality of capacitor bank samples to obtain sample fusion images corresponding to the plurality of capacitor bank samples; Inputting the sample fusion image corresponding to each capacitor group sample into the weed detection model to be trained for weed detection processing, and obtaining predicted weed detection information corresponding to each capacitor group sample; The weed detection model to be trained is the target detection model; The weed detection model to be trained is trained according to the preset weed detection information corresponding to the multiple capacitor group samples and the predicted weed detection information corresponding to each capacitor group sample to obtain the target weed detection model.

7. The substation weed detection method according to claim 6, characterized in that: The weed detection model to be trained includes a backbone network, a neck network, a multi-prediction head network and a single prediction head network; the predicted weed detection information includes a plurality of first predicted weed detection results and a second predicted weed detection result; the sample fusion image corresponding to each capacitor group sample is input into the weed detection model to be trained for weed detection processing, and the predicted weed detection information corresponding to each capacitor group sample is obtained, including: Inputting the sample fusion image corresponding to each capacitor group sample into the weed detection model to be trained, and performing depth-separable convolution processing on the sample fusion image based on the backbone network to obtain sample feature information corresponding to the sample fusion image; The sample feature information is input into the neck network, and based on the hybrid local channel attention mechanism, local feature extraction processing is performed on the sample feature information to obtain sample local feature information corresponding to the sample feature information; global feature extraction processing is performed on the sample feature information to obtain sample global feature information corresponding to the sample feature information; feature fusion processing is performed on the sample local feature information and the sample global feature information to obtain sample fusion feature information corresponding to the sample feature information; Inputting the sample fusion feature information into the multi-prediction head network for weed detection processing to obtain a plurality of first predicted weed detection results corresponding to each capacitor bank sample; The sample fusion feature information is input into the single prediction head network for weed detection processing to obtain a second predicted weed detection result corresponding to each capacitor group sample.

8. The substation weed detection method according to claim 7, characterized in that: The weed detection model to be trained is trained according to the preset weed detection information corresponding to the plurality of capacitor bank samples and the predicted weed detection information corresponding to each capacitor bank sample to obtain the target weed detection model, including: Determining the weed detection loss of the weed model to be trained according to the plurality of first predicted weed detection results corresponding to each capacitor group sample and the predicted weed detection information corresponding to each capacitor group sample, and according to the second predicted weed detection result corresponding to each capacitor group sample and the predicted weed detection information corresponding to each capacitor group sample in combination with a bounding box regression loss function; Based on the weed detection loss, the weed detection model to be trained is trained to obtain the target weed detection model.

9. A weed detection device for a substation, characterized in that: The device comprises: An image acquisition module, used to acquire a multispectral image and a color image corresponding to each capacitor bank in the substation; An image fusion module, used for performing image fusion processing on the multispectral image and the color image to obtain a fused image to be detected; A target detection model processing module is used to input the fused image to be detected into a target weed detection model, wherein the target weed detection model is obtained by training based on the improved target detection model; and based on the backbone network in the target weed detection model, perform a depth-separable convolution process on the fused image to be detected to obtain feature information to be detected corresponding to the fused image to be detected; the target weed detection model also includes a neck network and a single prediction head network; A feature extraction and fusion module is used to input the feature information to be detected into the neck network, perform local feature extraction processing on the feature information to be detected based on a hybrid local channel attention mechanism, and obtain local feature information corresponding to the feature information to be detected; perform global feature extraction processing on the feature information to be detected, and obtain global feature information corresponding to the feature information to be detected; perform feature fusion processing on the local feature information and the global feature information, and obtain fused feature information to be detected corresponding to the feature information to be detected; The target weed detection information acquisition module is used to input the fused feature information to be detected into the single prediction head network for weed detection processing to obtain the target weed detection information corresponding to each capacitor group.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the substation weed detection method according to any one of claims 1 to 8.

Images