Method and device for detecting weeds in transformer substation fence, computer equipment, readable storage medium and program product

Abstract

The invention relates to a method and device for detecting weeds in a transformer substation fence, computer equipment, a readable storage medium and a program product, and relates to the technical field of image detection. The method comprises the steps of obtaining a target image of a target substation in response to a weed detection instruction in a target substation fence; performing fence detection on the target substation based on the target image to obtain a fence detection frame of the target substation; the fence detection frame comprises at least one key detection point; connecting the key detection points to obtain a fence contour of the target substation; extracting a first image in the fence detection frame from the target image, and performing weed detection on the target substation based on the first image to obtain a weed detection frame containing a weed area; and when the overlapping degree between the weed detection frame and the fence contour exceeds an overlapping degree threshold value, generating a weed detection result of the target substation. The method can improve the detection accuracy of weeds in the transformer substation fence.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a method, device, computer equipment, readable storage medium and program product for detecting weeds in a substation fence. Background Art

[0002] Substations play a vital role in the power system. They are not only responsible for voltage conversion and distribution of electric energy, but also undertake many important functions such as power dispatching, system protection, power factor adjustment, and load management.

[0003] Substations are often fenced, and weeds within the fences are a hidden danger in most substations. Weeds not only cover the equipment in the substation, but also affect the heat dissipation of the equipment, which can easily cause safety accidents.

[0004] Based on this, it is necessary to detect weeds in the fence of the substation so as to clean the weeds in a timely and accurate manner. At present, manual inspection is usually used to detect weeds in the fence of the substation. However, this method is not only inefficient, but also prone to missed detection and false detection, thus affecting the accuracy of weed detection in the fence of the substation. Summary of the invention

[0005] Based on this, it is necessary to provide a weed detection method, device, computer equipment, computer-readable storage medium and computer program product inside a substation fence, which can improve the accuracy of weed detection inside the substation fence, in order to solve the above technical problems.

[0006] In a first aspect, the present application provides a method for detecting weeds within a fence of a substation, comprising: in response to a weed detection instruction for a target substation fence, obtaining a target image of the target substation; based on the target image, performing fence detection on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; connecting the key detection points to obtain a fence contour of the target substation; extracting a first image within the fence detection frame from the target image, and based on the first image, performing weed detection on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, generating a weed detection result for the target substation.

[0007] In one of the embodiments, based on the target image, fence detection is performed on the target substation to obtain a fence detection frame of the target substation, including: preprocessing the target image to obtain a preprocessed second image; inputting the second image into a fence area detection model, and performing fence detection on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0008] In one of the embodiments, the method further includes: determining a third image based on the second image and the fence detection frame; inputting the third image into a key point detection model, and performing key point detection on the third image through the key point detection model to obtain key detection points.

[0009] In one of the embodiments, based on a first image, weed detection is performed on a target substation to obtain a weed detection frame containing a weed area, including: inputting the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a re-parameterized convolutional layer; using the weed detection model, feature extraction is performed on the first image to obtain weed features in the first image; based on the weed features, a weed area is identified in the first image to obtain a weed detection frame containing the weed area.

[0010] In one of the embodiments, the method also includes: obtaining a sample image of the sample substation; the sample image includes a fence area identification and a weed identification of the sample substation; inputting the sample image into a pre-trained fence area detection model and a pre-trained weed detection model, respectively, performing model training, and obtaining a fence area detection model and a weed detection model; inputting the sample image into the fence area detection model, performing fence detection on the sample substation, and obtaining a predicted fence detection frame of the sample substation; based on the predicted fence detection frame, performing model training on the pre-trained key point detection model to obtain a key point detection model.

[0011] In one embodiment, the method further includes: determining an overlapping area between the weed detection frame and the fence outline, and a total area of ​​the weed detection frame and the fence outline; and using a ratio between the overlapping area and the total area as a degree of overlap between the weed detection frame and the fence outline.

[0012] In a second aspect, the present application also provides a weed detection device within a substation fence, comprising: an image acquisition module, used to respond to a weed detection instruction for the target substation fence and acquire a target image of the target substation; a fence area detection module, used to perform fence detection on the target substation based on the target image to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; a key point processing module, used to connect the key detection points to obtain a fence contour of the target substation; a weed detection module, used to extract a first image within the fence detection frame from the target image, and based on the first image, perform weed detection on the target substation to obtain a weed detection frame containing a weed area; a weed detection result generation module, used to generate a weed detection result of the target substation when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold.

[0013] In a third aspect, the present application also provides a computer device, including a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program: in response to a weed detection instruction for a target substation fence, obtaining a target image of the target substation; based on the target image, performing fence detection on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; connecting the key detection points to obtain a fence contour of the target substation; extracting a first image within the fence detection frame from the target image, and based on the first image, performing weed detection on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, generating a weed detection result for the target substation.

[0014] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the following steps when executed by a processor: in response to an instruction for weed detection within a fence of a target substation, obtaining a target image of the target substation; based on the target image, performing fence detection on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; connecting the key detection points to obtain a fence contour of the target substation; extracting a first image within the fence detection frame from the target image, and based on the first image, performing weed detection on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, generating a weed detection result for the target substation.

[0015] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the following steps when executed by a processor: in response to an instruction for weed detection within the fence of a target substation, obtain a target image of the target substation; based on the target image, perform fence detection on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; connect the key detection points to obtain the fence contour of the target substation; extract a first image within the fence detection frame from the target image, and based on the first image, perform weed detection on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, generate a weed detection result for the target substation.

[0016] The above-mentioned weed detection method, device, computer equipment, computer-readable storage medium and computer program product in the fence of the substation respond to the weed detection instruction for the fence of the target substation, first obtain the target image of the target substation. Based on the target image, the fence of the target substation is detected to obtain the fence detection frame of the target substation. Since the fence detection frame includes at least one key detection point, each key detection point can be further connected to obtain a more accurate fence contour of the target substation, thereby improving the accuracy of the fence contour. Then, the first image in the fence detection frame is extracted from the target image, and the weed detection of the target substation is performed based on the first image to obtain a weed detection frame containing a weed area. Finally, the overlap between the weed detection frame and the fence contour is compared to verify whether the detected weed area is within the fence. It can be understood that when the overlap exceeds the overlap threshold, it means that the weed detection frame and the fence contour are highly overlapped, and it can be considered that the weed area is within the fence of the substation. In summary, compared with the current manual inspection method, the present application can improve the accuracy of fence detection, thereby improving the accuracy of weed detection in the fence of the substation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] FIG1 is a diagram showing an application environment of a method for detecting weeds in a fence of a substation according to an embodiment;

[0019] FIG2 is a schematic diagram of a flow chart of a method for detecting weeds in a fence of a substation according to an embodiment;

[0020] FIG3 is a schematic diagram of a flow chart of fence detection in one embodiment;

[0021] FIG4 is a schematic diagram of the structure of the Backbone layer in one embodiment;

[0022] FIG5 is a schematic diagram of introducing a re-parameterized convolutional layer into a Head layer in one embodiment;

[0023] FIG6 is a schematic diagram of a key point detection process in one embodiment;

[0024] FIG7 is a schematic diagram of a weed detection process in one embodiment;

[0025] FIG8 is a schematic diagram of a process of model training in one embodiment;

[0026] FIG9 is a block diagram of a weed detection device in a substation fence according to an embodiment;

[0027] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] The weed detection method in the fence of a substation provided in an embodiment of the present application can be applied in an application environment as shown in FIG. 1 . The terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. In response to the weed detection instruction for the fence of the target substation initiated by the terminal 102, the server 104 first obtains a target image of the target substation. Then, based on the target image, the fence detection is performed on the target substation to obtain a fence detection frame of the target substation. The fence detection frame includes at least one key detection point, and each key detection point is connected to obtain the fence contour of the target substation. Then, a first image in the fence detection frame is extracted from the target image, and based on the first image, weed detection is performed on the target substation to obtain a weed detection frame containing a weed area. Finally, when the overlap between the weed detection frame and the fence contour exceeds the overlap threshold, the server 104 generates a weed detection result of the target substation. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and Internet of Things devices. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0030] In an exemplary embodiment, as shown in FIG2 , a method for detecting weeds in a fence of a substation is provided, and the method is described by taking the application of the method to the server 104 in FIG1 as an example, including:

[0031] Step S202 : in response to a weed detection instruction for a target substation fence, acquiring a target image of the target substation.

[0032] The target substation may refer to a substation that needs to be detected for weeds. The weed detection instruction may refer to an instruction for instructing to detect weeds within the fence of the target substation. It should be noted that the weed detection in this embodiment is to detect weeds within the fence of the substation, not weeds outside the fence. The target image may refer to a global image of the target substation, which may include the target substation body and the surrounding environment of the target substation, and the surrounding environment of the target substation may include the environment within the fence of the target substation and the environment outside the fence of the target substation.

[0033] For example, when the server receives an instruction to detect weeds within the fence of a target substation, it can first collect a global image of the target substation through a drone or a fixed camera for subsequent fence detection and weed detection.

[0034] Step S204: Based on the target image, fence detection is performed on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point.

[0035] Among them, fence detection can be understood as the process of detecting the fence area of ​​the target substation. The fence detection frame can refer to the detection frame of the fence area of ​​the target substation. In one example, the fence detection frame can be a rectangular frame. Key detection points can refer to key points on the fence detection frame, such as the four corners of the fence detection frame, which can be used to subsequently obtain the fence contour. Of course, in actual applications, other marking points on the fence detection frame can also be set as key detection points. In this embodiment, there is no limitation on the specific position and number of key detection points.

[0036] For example, after acquiring the global image of the target substation, the server can identify the fence area of ​​the target substation based on the global image, thereby obtaining a rectangular frame containing the fence area, namely, the fence detection frame. It can be understood that the server first draws a fence detection frame, which can greatly reduce the scope of subsequent fence contour generation and weed detection, improve the detection efficiency of the entire solution, reduce computing resource consumption, and improve the detection accuracy to a certain extent, because the fence detection frame has preliminarily located and screened the area for subsequent fence contour generation and weed detection.

[0037] In one example, the server may identify the fence area of ​​the target substation by using a pre-established fence area detection model and obtain a fence detection frame.

[0038] Step S206, connecting the key detection points to obtain the fence outline of the target substation.

[0039] The fence outline may refer to the outline of the fence of the target substation. Unlike the fence detection frame, the fence outline may be a more complex polygonal area that reflects the actual outline of the fence. This is because in actual applications, the substation fence may have different shapes, including curves, irregular polygons, etc., and using a standard rectangular frame, i.e., a fence detection frame, to approximate these shapes is not accurate enough. Therefore, key detection points are introduced in this embodiment, that is, by connecting the key detection points on the fence detection frame, a more accurate fence outline is obtained.

[0040] Exemplarily, after obtaining the fence detection frame of the target substation, the server can identify key detection points on the fence detection frame, and obtain a polygonal area, namely, the fence outline, by connecting the key detection points.

[0041] In one example, the server may identify key detection points on the fence detection frame by using a pre-established key point detection model, thereby obtaining a fence outline.

[0042] Step S208: extracting the first image within the fence detection frame from the target image, and performing weed detection on the target substation based on the first image to obtain a weed detection frame containing the weed area.

[0043] The first image may be understood as a partial image framed by the fence detection frame in the target image. Weed detection may refer to a process of detecting a weed region in the first image. The weed region may refer to a region containing weeds. The weed detection frame may refer to a detection frame containing the weed region, which may be a rectangular frame.

[0044] It should be noted that, considering the complexity of the fence outline, if weed detection is performed directly in the fence outline, the complexity of weed detection will increase, thereby affecting the detection accuracy. Therefore, in this embodiment, weed detection is performed in the first image, i.e., the image framed by the fence detection frame in the target image, to reduce the complexity of weed detection and improve the efficiency of weed detection.

[0045] For example, after obtaining the fence detection frame of the target substation, the server may capture the portion of the image framed by the fence detection frame in the target image as the first image. Based on the first image, weed detection is performed on the fence of the target substation to obtain a weed detection frame containing a weed area in the target substation.

[0046] In one example, the server may identify the weed region in the first image by using a pre-established weed detection model, thereby obtaining a weed detection frame.

[0047] Step S210: When the overlap between the weed detection frame and the fence outline exceeds an overlap threshold, a weed detection result of the target substation is generated.

[0048] It should be noted that in order to avoid detecting weeds outside the fence and further improve the accuracy of weed detection, an overlap analysis between the weed detection frame and the fence outline is performed in this embodiment. Through the overlap analysis, it can be verified whether the weed detection frame is indeed located within the fence outline. The overlap (Intersection over Union, IOU) can be understood as the degree of overlap between the weed detection frame and the fence outline. The overlap threshold can be a threshold set in advance for the overlap. If the threshold is exceeded, the higher the overlap, that is, the higher the overlap between the weed detection frame and the fence outline, the higher the accuracy of weed detection. The weed detection result can be the overlapping part between the weed detection frame and the fence outline.

[0049] In one example, the range of the overlap threshold may be set between 0 and 1, where 0 indicates no overlap and 1 indicates complete overlap.

[0050] Exemplarily, after obtaining the weed detection frame, the server may perform an overlap analysis on the weed detection frame and the fence outline, that is, analyze the overlap degree between the weed detection frame and the fence outline. When the overlap degree exceeds the overlap degree threshold, it means that the overlap degree between the weed detection frame and the fence outline is high, and it is determined that the currently detected weed area is within the fence of the target substation, and the server may output the overlapping part between the weed detection frame and the fence outline as the weed detection result. When the overlap degree between the weed detection frame and the fence outline does not exceed the overlap degree threshold, it means that the overlap degree between the weed detection frame and the fence outline is low, and it is determined that the currently detected weed area is not within the fence of the target substation, and the server may choose to perform model optimization and then re-detect, or directly output the current weed detection frame and fence outline.

[0051] In this embodiment, in response to the weed detection instruction for the target substation fence, the server first obtains the target image of the target substation. Based on the target image, the fence detection of the target substation is performed to obtain the fence detection frame of the target substation. Since the fence detection frame includes at least one key detection point, each key detection point can be further connected to obtain a more accurate fence contour of the target substation, thereby improving the accuracy of the fence contour. Then, the first image in the fence detection frame is extracted from the target image, and the weed detection of the target substation is performed based on the first image to obtain a weed detection frame containing a weed area. Finally, the overlap between the weed detection frame and the fence contour is compared to verify whether the detected weed area is within the fence. It can be understood that when the overlap exceeds the overlap threshold, it means that the weed detection frame and the fence contour are highly overlapped, and it can be considered that the weed area is within the substation fence. In summary, compared with the current manual inspection method, this embodiment can improve the accuracy of fence detection, thereby improving the accuracy of weed detection within the substation fence.

[0052] In an exemplary embodiment, as shown in FIG3 , fence detection is performed on a target substation based on a target image to obtain a fence detection frame of the target substation, including:

[0053] Step S302: preprocess the target image to obtain a preprocessed second image.

[0054] The second image may refer to an image obtained after preprocessing the target image.

[0055] For example, after acquiring the target image of the target substation, the server can first clean and remove duplicates of the target image to obtain a preliminary image in order to further improve the accuracy of subsequent detection. Then, the preliminary image is subjected to an adaptive histogram equalization operation, that is, the local histogram of the preliminary image is calculated, and then the brightness is redistributed to change the image contrast, thereby achieving the purpose of improving the image contrast and reducing the impact of overexposure and overdarkness on subsequent detection. Of course, in practical applications, other image processing methods can also be used to pre-process the target image.

[0056] Step S304, input the second image into the fence area detection model, and perform fence detection on the target substation through the fence area detection model to obtain the fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0057] Among them, the reparameterized convolution layer is a special convolution layer design, which is characterized by the use of a multi-branch structure in the model training stage. The multi-branch structure can include multiple paths or branches, and each branch can have different convolution kernel sizes, different activation functions, etc., allowing the model to learn data features from multiple angles. However, in the model application stage, this multi-branch structure will be "reparameterized" into a simpler serial structure. Reparameterization refers to replacing the existing parameterized model with a new method that is approximately equivalent. That is to say, in the model application stage, the multi-branch structure will be "folded" or "merged" into a simpler serial structure to ensure that the converted model is functionally the same as the original multi-branch model, but more computationally efficient. In this way, the advantages of the multi-branch structure can be fully utilized during model training, such as increasing model capacity, capturing more features, etc., while maintaining efficiency and simplicity when the model is applied. The fence area detection model can be a target detection model, such as the YoloV8 model.

[0058] In one example, the structure of the YoloV8 model is mainly composed of the Backbone layer, the Neck layer, and the Head layer. Figure 4 shows a schematic diagram of the structure of the Backbone layer, where the C2f module is responsible for extracting the features of the image. Since the C2f module can effectively handle feature extraction and compression, YOLOv8 can be more efficient and accurate when processing complex images. CBS is a convolution module, S=1 means that the convolution step is 1, and K=1 means that the size of the convolution kernel is 1*1. The Split operation refers to dividing the feature map into multiple parts for further processing. Multiple Bottleneck modules form a bottleneck structure. These modules are stacked and combined to form a powerful feature extraction network, which can reduce the number of channels in the middle layer, reduce the amount of calculation and the number of parameters, while maintaining the expressiveness of the model. The Neck layer is mainly responsible for multi-scale fusion in the model. By fusing the feature maps from the Backbone layer, the feature expression ability of the model is enhanced. The Neck layer includes the following components: SPPF (Spatial Pyramid Pooling Fast) module: used for pooling operations at different scales, splicing feature maps of different scales together to improve the detection ability of objects of different sizes; PAN (Path Aggregation Network) module: including two PAN modules, used for path aggregation of features at different levels, and enhancing the expression ability of feature maps through bottom-up and top-down paths. The Head layer is responsible for the final target detection and classification tasks, including the detection head, which contains a series of convolutional layers and deconvolutional layers for generating detection results, and the classification head, which uses average pooling to classify each feature map.

[0059] In one example, a re-parameterized convolution layer is introduced in the Head layer, and Figure 5 shows a schematic diagram of introducing a re-parameterized convolution layer in the Head layer. In the training phase, the re-parameterized convolution structure is composed of multiple 1×1, K×K convolutions and AVG, i.e., average pooling layers, but is equivalently replaced by a K×K convolution in the application phase. By repeatedly passing through such a re-parameterized convolution module in the Head layer, the model adopts a multi-branch training structure during training and is converted into a serial structure during application. While enhancing the feature extraction capability of the model, it also improves the running speed of the model during application.

[0060] Exemplarily, after obtaining the preprocessed second image, the server may introduce the second image input into a target detection model that has a re-parameterized convolutional layer in advance to perform fence detection, thereby outputting a fence detection frame of the target substation.

[0061] In this embodiment, a fence area detection model is used to detect the fence detection frame of the target substation, thereby improving the accuracy of fence detection. In addition, a re-parameterized convolutional layer is introduced into the fence area detection model, which can enhance the feature extraction capability of the fence area detection model while also improving the speed of the fence area detection model in detecting the fence detection frame.

[0062] In an exemplary embodiment, as shown in FIG6 , the process of acquiring key detection points further includes:

[0063] Step S602: determining a third image based on the second image and the fence detection frame.

[0064] The third image may be understood as an image including the second image and the fence detection frame, or in other words, the second image including the fence detection frame is used as the third image.

[0065] Exemplarily, after obtaining the fence detection frame of the target substation, the server may draw the fence detection frame in the second image, thereby obtaining the third image.

[0066] Step S604: input the third image into the key point detection model, and perform key point detection on the third image through the key point detection model to obtain key detection points.

[0067] The key point detection model may refer to a model for extracting key detection points on a fence detection frame. In one example, the key point detection model may be trained based on output data of a trained fence region detection model.

[0068] Exemplarily, the server inputs the obtained third image into a pre-trained key point detection model, and uses the key point detection model to extract features of the edges of the fence detection frame in the third image to obtain detection frame edge features such as edge strength and color, and based on the detection frame edge features, determine each key detection point on the fence detection frame.

[0069] In this embodiment, a key point detection model is used to detect key detection points on the fence detection frame, thereby improving the accuracy of the key detection points, thereby improving the accuracy of the fence contour, and further improving the accuracy of weed detection.

[0070] In an exemplary embodiment, as shown in FIG7 , based on the first image, weed detection is performed on the target substation to obtain a weed detection frame containing a weed area, including:

[0071] Step S702: input the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a heavily parameterized convolutional layer.

[0072] The weed detection model may also be a target detection model, such as a YoloV8 model. The meaning of the re-parameterized convolution layer and how to introduce the re-parameterized convolution layer into the weed detection model may refer to step S302, which will not be described in detail here.

[0073] Exemplarily, after extracting the image framed by the fence detection frame, that is, the first image, from the target image, the server may introduce the first image input into the target detection model in which the re-parameterized convolutional layer is pre-introduced to perform weed detection.

[0074] Step S704: extract features from the first image using a weed detection model to obtain weed features in the first image.

[0075] The weed characteristics may include but are not limited to the color, length, shape, growth position, etc. of the weeds.

[0076] Exemplarily, after the server inputs the first image into the weed detection model, the weed detection model may extract weed features from the first image to obtain weed features in the first image for subsequent weed area identification.

[0077] Step S706: based on the weed features, a weed area is identified in the first image to obtain a weed detection frame including the weed area.

[0078] Illustratively, after extracting the weed features in the first image, the weed detection model may locate a weed region containing weeds in the first image based on the weed features, and generate a weed detection frame containing the weed region.

[0079] In this embodiment, a weed detection model is used to detect weeds, which improves the accuracy of weed detection. In addition, a re-parameterized convolutional layer is introduced into the weed detection model, which can enhance the feature extraction capability of the weed detection model while also improving the speed of the weed detection model in detecting weeds.

[0080] In an exemplary embodiment, as shown in FIG8 , the model training process includes:

[0081] Step S802: Acquire a sample image of a sample substation.

[0082] The sample substation may refer to at least one substation used for model training, and the sample image may refer to a global image of the sample substation, which can be used as a training data set for model training. The sample image may include a fence area identifier and a weed identifier of the sample substation, where the fence area identifier can be used to identify the fence area of ​​the sample substation, and the weed identifier can be used to identify the weeds within the fence of the sample substation.

[0083] In one example, the server can use Labelme to annotate each sample image collected, mark the location of weeds and fences, and thus obtain fence area identification and weed identification. Labelme is an open source image annotation tool that can be used to annotate target location information in an image.

[0084] For example, when the server is training the model, it can first determine at least one sample substation for model training, thereby obtaining global images of these sample substations, i.e., sample images, as training data sets for model training. These sample images can also be collected by drones or fixed cameras.

[0085] Step S804: input the sample image into the pre-trained fence area detection model and the pre-trained weed detection model respectively, perform model training, and obtain the fence area detection model and the weed detection model.

[0086] The pre-trained fence area detection model can be understood as a model to be trained for fence detection. The pre-trained weed detection model can be understood as a model to be trained for weed detection. It can be understood that the pre-trained fence area detection model and the pre-trained weed detection model have both introduced a re-parameterized convolutional layer.

[0087] Exemplarily, after acquiring the labeled sample images, the server may input these sample images into a pre-trained fence area detection model and a pre-trained weed detection model in parallel, and perform model training until the number of training times is reached or the detection accuracy is achieved, and then stop model training to obtain a fence area detection model and a weed detection model.

[0088] Step S806: input the sample image into the fence area detection model, perform fence detection on the sample substation, and obtain a predicted fence detection frame of the sample substation.

[0089] The predicted fence detection frame can be understood as the fence detection frame for the sample substation output by the trained fence area detection model based on the sample image. The predicted fence detection frame can be used as a training data set for the key point detection model.

[0090] For example, after obtaining the trained fence area detection model, the server can further train the key point detection model. In this process, the server can input the sample image into the fence area detection model, perform fence detection on the sample substation through the fence area detection model, and output the corresponding fence detection frame, i.e., the predicted fence detection frame, as a training data set for subsequent training of the key point detection model. The fence contour identifier can also be marked in these training data sets.

[0091] Step S808: Based on the predicted fence detection frame, the pre-trained key point detection model is trained to obtain a key point detection model.

[0092] Among them, the pre-trained key point detection model can be understood as a model to be trained for key point detection.

[0093] Exemplarily, after obtaining the predicted fence detection frame output by the fence area detection model, the server can input the predicted fence detection frame into the pre-trained key point detection model and perform model training until the number of training times or the detection accuracy is reached, and then stop the model training to obtain the key point detection model.

[0094] In this embodiment, based on the labeled sample images of the sample substation, the fence area detection model and the weed detection model are trained respectively, which improves the accuracy of the fence area detection model and the weed detection model, thereby improving the accuracy of fence detection and the accuracy of weed detection in practical applications. In addition, based on the predicted fence detection frame output by the fence area detection model, the key point detection model is trained to improve the accuracy of the key point detection model, thereby improving the accuracy of key point detection in practical applications.

[0095] In an exemplary embodiment, the calculation process of the overlap degree also includes: determining the overlapping area between the weed detection frame and the fence outline, and the total area of ​​the weed detection frame and the fence outline; and using the ratio between the overlapping area and the total area as the overlap degree between the weed detection frame and the fence outline.

[0096] The overlapping area may refer to the area of ​​the overlapping portion between the weed detection frame and the fence outline, and the total area may refer to the sum of the areas of the weed detection frame and the fence outline.

[0097] For example, when calculating the overlap between the weed detection frame and the fence outline, the server may first calculate the area of ​​the overlapping portion between the weed detection frame and the fence outline and the sum of the areas of the weed detection frame and the fence outline, and then calculate the ratio of the two areas, which is the overlap between the weed detection frame and the fence outline.

[0098] In this embodiment, the overlapping area between the weed detection frame and the fence outline and the total area between the weed detection frame and the fence outline are calculated, thereby calculating the overlap degree between the weed detection frame and the fence outline, thereby improving the accuracy of the overlap degree and further improving the accuracy of weed detection.

[0099] In a specific embodiment, a method for detecting weeds in a substation fence includes:

[0100] The server pre-acquires a sample image of a sample substation, and the sample image includes a fence area mark and a weed mark of the sample substation. The sample image is respectively input into a pre-trained fence area detection model and a pre-trained weed detection model, and model training is performed to obtain a fence area detection model and a weed detection model. The sample image is input into the fence area detection model, and fence detection is performed on the sample substation to obtain a predicted fence detection frame of the sample substation. Based on the predicted fence detection frame, the pre-trained key point detection model is trained to obtain a key point detection model.

[0101] When the server receives the weed detection instruction for the fence of the target substation, it first obtains the target image of the target substation and preprocesses the target image to obtain the preprocessed second image. Secondly, the second image is input into the fence area detection model, the fence detection is performed on the target substation, and the fence detection frame of the target substation is obtained. The fence area detection model refers to a model for fence detection that introduces a re-parameterized convolution layer. Then, based on the second image and the fence detection frame, a third image is determined. The third image is input into the key point detection model, and key point detection is performed on the third image to obtain each key detection point. And each key detection point is connected to obtain the fence contour of the target substation. Then, the first image in the fence detection frame is extracted from the target image, and the first image is input into the weed detection model. The weed detection model refers to a model for weed detection that introduces a re-parameterized convolution layer. Through the weed detection model, the first image is feature extracted to obtain the weed features in the first image. Based on the weed features, the weed area is identified in the first image, and the weed detection frame containing the weed area is obtained. Finally, the overlapping area between the weed detection frame and the fence outline and the total area of ​​the weed detection frame and the fence outline are determined. The ratio between the overlapping area and the total area is used as the overlap between the weed detection frame and the fence outline. When the overlap between the weed detection frame and the fence outline exceeds the overlap threshold, the weed detection result of the target substation is generated.

[0102] In this embodiment, in response to the weed detection instruction for the target substation fence, the server first obtains the target image of the target substation. Based on the target image, the fence detection of the target substation is performed to obtain the fence detection frame of the target substation. Since the fence detection frame includes at least one key detection point, each key detection point can be further connected to obtain a more accurate fence contour of the target substation, thereby improving the accuracy of the fence contour. Then, the first image in the fence detection frame is extracted from the target image, and the weed detection of the target substation is performed based on the first image to obtain a weed detection frame containing a weed area. Finally, the overlap between the weed detection frame and the fence contour is compared to verify whether the detected weed area is within the fence. It can be understood that when the overlap exceeds the overlap threshold, it means that the weed detection frame and the fence contour are highly overlapped, and it can be considered that the weed area is within the substation fence. In summary, compared with the current manual inspection method, this embodiment can improve the accuracy of fence detection, thereby improving the accuracy of weed detection within the substation fence.

[0103] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0104] Based on the same inventive concept, the embodiment of the present application also provides a weed detection device in a substation fence for implementing the weed detection method in a substation fence mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the weed detection device in a substation fence provided below can refer to the limitations of the weed detection method in a substation fence above, and will not be repeated here.

[0105] In an exemplary embodiment, as shown in FIG9 , a weed detection device in a fence of a substation is provided, including: an image acquisition module 902, for acquiring a target image of a target substation in response to a weed detection instruction in a fence of a target substation; a fence area detection module 904, for performing fence detection on a target substation based on the target image to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; a key point processing module 906, for connecting the key detection points to obtain a fence contour of the target substation; a weed detection module 908, for extracting a first image in the fence detection frame from the target image, and performing weed detection on the target substation based on the first image to obtain a weed detection frame containing a weed area; and a weed detection result generation module 910, for generating a weed detection result of the target substation when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold.

[0106] In one of the embodiments, the fence area detection module 904 is also used to: preprocess the target image to obtain a preprocessed second image; input the second image into a fence area detection model, and perform fence detection on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0107] In one of the embodiments, the device is also used to: determine a third image based on the second image and the fence detection frame; input the third image into a key point detection model, and perform key point detection on the third image through the key point detection model to obtain each key detection point.

[0108] In one embodiment, the weed detection module 908 is further used to: input the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a re-parameterized convolutional layer; extract features of the first image through the weed detection model to obtain weed features in the first image; and identify a weed area in the first image based on the weed features to obtain a weed detection frame containing the weed area.

[0109] In one of the embodiments, the device is also used to: obtain a sample image of a sample substation; the sample image includes a fence area identification and a weed identification of the sample substation; input the sample image into a pre-trained fence area detection model and a pre-trained weed detection model, respectively, perform model training, and obtain a fence area detection model and a weed detection model; input the sample image into the fence area detection model, perform fence detection on the sample substation, and obtain a predicted fence detection frame of the sample substation; based on the predicted fence detection frame, perform model training on the pre-trained key point detection model to obtain a key point detection model.

[0110] In one of the embodiments, the device is further used to: determine the overlapping area between the weed detection frame and the fence outline, and the total area of ​​the weed detection frame and the fence outline; and use the ratio between the overlapping area and the total area as the degree of overlap between the weed detection frame and the fence outline.

[0111] Each module in the above-mentioned weed detection device in the fence of a substation can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0112] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store weed detection data within a substation fence. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting weeds within a substation fence is implemented.

[0113] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: in response to an instruction for weed detection within a fence of a target substation, a target image of the target substation is acquired; based on the target image, a fence detection is performed on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; the key detection points are connected to obtain a fence contour of the target substation; a first image within the fence detection frame is extracted from the target image, and based on the first image, a weed detection is performed on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, a weed detection result of the target substation is generated.

[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented: preprocessing the target image to obtain a preprocessed second image; inputting the second image into a fence area detection model, and performing fence detection on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0116] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the second image and the fence detection frame, a third image is determined; the third image is input into a key point detection model, and key point detection is performed on the third image through the key point detection model to obtain key detection points.

[0117] In one embodiment, when the processor executes the computer program, the following steps are further implemented: inputting the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a re-parameterized convolutional layer; extracting features of the first image through the weed detection model to obtain weed features in the first image; and identifying a weed area in the first image based on the weed features to obtain a weed detection frame containing the weed area.

[0118] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a sample image of the sample substation; the sample image includes a fence area identification and a weed identification of the sample substation; inputting the sample image into a pre-trained fence area detection model and a pre-trained weed detection model, respectively, performing model training, and obtaining a fence area detection model and a weed detection model; inputting the sample image into the fence area detection model, performing fence detection on the sample substation, and obtaining a predicted fence detection frame of the sample substation; based on the predicted fence detection frame, performing model training on the pre-trained key point detection model to obtain a key point detection model.

[0119] In one embodiment, when the processor executes the computer program, the following steps are further implemented: determining the overlapping area between the weed detection frame and the fence outline, and the total area of ​​the weed detection frame and the fence outline; and using the ratio between the overlapping area and the total area as the degree of overlap between the weed detection frame and the fence outline.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: in response to a weed detection instruction for a target substation fence, a target image of the target substation is acquired; based on the target image, a fence detection is performed on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; the key detection points are connected to obtain a fence contour of the target substation; a first image within the fence detection frame is extracted from the target image, and based on the first image, a weed detection is performed on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, a weed detection result of the target substation is generated.

[0121] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: preprocessing the target image to obtain a preprocessed second image; inputting the second image into a fence area detection model, and performing fence detection on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0122] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the second image and the fence detection frame, a third image is determined; the third image is input into a key point detection model, and key point detection is performed on the third image through the key point detection model to obtain key detection points.

[0123] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: inputting the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a re-parameterized convolutional layer; extracting features of the first image through the weed detection model to obtain weed features in the first image; based on the weed features, identifying a weed area in the first image to obtain a weed detection frame containing the weed area.

[0124] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a sample image of a sample substation; the sample image includes a fence area identification and a weed identification of the sample substation; inputting the sample image into a pre-trained fence area detection model and a pre-trained weed detection model, respectively, performing model training, and obtaining a fence area detection model and a weed detection model; inputting the sample image into the fence area detection model, performing fence detection on the sample substation, and obtaining a predicted fence detection frame of the sample substation; based on the predicted fence detection frame, performing model training on the pre-trained key point detection model to obtain a key point detection model.

[0125] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining the overlapping area between the weed detection frame and the fence outline, and the total area of ​​the weed detection frame and the fence outline; and using the ratio between the overlapping area and the total area as the degree of overlap between the weed detection frame and the fence outline.

[0126] In one embodiment, a computer program product is provided, including a computer program, which implements the following steps when executed by a processor: in response to an instruction for weed detection within a fence of a target substation, a target image of the target substation is acquired; based on the target image, a fence detection is performed on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; the key detection points are connected to obtain a fence contour of the target substation; a first image within the fence detection frame is extracted from the target image, and based on the first image, a weed detection is performed on the target substation to obtain a weed detection frame containing a weed area; when the overlap between the weed detection frame and the fence contour exceeds an overlap threshold, a weed detection result of the target substation is generated.

[0127] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: preprocessing the target image to obtain a preprocessed second image; inputting the second image into a fence area detection model, and performing fence detection on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the second image and the fence detection frame, a third image is determined; the third image is input into a key point detection model, and key point detection is performed on the third image through the key point detection model to obtain key detection points.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: inputting the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a re-parameterized convolutional layer; extracting features of the first image through the weed detection model to obtain weed features in the first image; based on the weed features, identifying a weed area in the first image to obtain a weed detection frame containing the weed area.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a sample image of a sample substation; the sample image includes a fence area identification and a weed identification of the sample substation; inputting the sample image into a pre-trained fence area detection model and a pre-trained weed detection model, respectively, performing model training, and obtaining a fence area detection model and a weed detection model; inputting the sample image into the fence area detection model, performing fence detection on the sample substation, and obtaining a predicted fence detection frame of the sample substation; based on the predicted fence detection frame, performing model training on the pre-trained key point detection model to obtain a key point detection model.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: determining the overlapping area between the weed detection frame and the fence outline, and the total area of ​​the weed detection frame and the fence outline; and using the ratio between the overlapping area and the total area as the degree of overlap between the weed detection frame and the fence outline.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0133] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0135] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting weeds in a substation fence, characterized in that: The method comprises: In response to a weed detection instruction for a target substation fence, acquiring a target image of the target substation; Based on the target image, fence detection is performed on the target substation to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; Connecting the key detection points to obtain the fence outline of the target substation; Extracting a first image within the fence detection frame from the target image, and performing weed detection on the target substation based on the first image to obtain a weed detection frame containing a weed area; When the overlap between the weed detection frame and the fence outline exceeds an overlap threshold, a weed detection result of the target substation is generated.

2. The method according to claim 1, characterized in that The performing fence detection on the target substation based on the target image to obtain a fence detection frame of the target substation includes: Preprocessing the target image to obtain a preprocessed second image; The second image is input into a fence area detection model, and fence detection is performed on the target substation through the fence area detection model to obtain a fence detection frame of the target substation; the fence area detection model refers to a model for fence detection that introduces a heavy parameterized convolutional layer.

3. The method according to claim 2, characterized in that The method further comprises: Determine a third image based on the second image and the fence detection frame; The third image is input into a key point detection model, and key point detection is performed on the third image through the key point detection model to obtain the key detection points.

4. The method according to claim 1, characterized in that: The step of performing weed detection on the target substation based on the first image to obtain a weed detection frame containing a weed area includes: Inputting the first image into a weed detection model; the weed detection model refers to a model for weed detection that introduces a heavily parameterized convolutional layer; Extracting features from the first image using the weed detection model to obtain weed features in the first image; Based on the weed features, the weed region is identified in the first image, and a weed detection frame including the weed region is obtained.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a sample image of a sample substation; the sample image includes a fence area mark and a weed mark of the sample substation; Inputting the sample image into a pre-trained fence area detection model and a pre-trained weed detection model respectively, performing model training, and obtaining the fence area detection model and the weed detection model; Inputting the sample image into the fence area detection model, performing fence detection on the sample substation, and obtaining a predicted fence detection frame of the sample substation; Based on the predicted fence detection frame, model training is performed on the pre-trained key point detection model to obtain the key point detection model.

6. The method according to claim 1, characterized in that The method further comprises: determining an overlapping area between the weed detection frame and the fence outline, and a total area of ​​the weed detection frame and the fence outline; The ratio between the overlapping area and the total area is used as the overlap degree between the weed detection frame and the fence outline.

7. A weed detection device in a substation fence, characterized in that: The device comprises: An image acquisition module, configured to acquire a target image of a target substation in response to a weed detection instruction within a fence of the target substation; A fence area detection module, configured to perform fence detection on the target substation based on the target image to obtain a fence detection frame of the target substation; the fence detection frame includes at least one key detection point; A key point processing module, used to connect the key detection points to obtain the fence outline of the target substation; a weed detection module, configured to extract a first image within the fence detection frame from the target image, and perform weed detection on the target substation based on the first image to obtain a weed detection frame containing a weed area; The weed detection result generating module is used to generate a weed detection result of the target substation when the overlap between the weed detection frame and the fence outline exceeds an overlap threshold.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images