Power distribution material target detection and instance segmentation method and device, and related equipment

Abstract

The application discloses a power distribution material target detection and instance segmentation method and device and related equipment. The method first acquires a power distribution scene image to be detected, and then inputs the power distribution scene image into a pre-trained detection segmentation model; wherein the detection segmentation model at least includes a classification head, a bounding box regression head and an instance segmentation head; finally, a prediction result output by the detection segmentation model is acquired; wherein the prediction result includes a category, a bounding box position and mask information of a power distribution material target in the power distribution scene image.

Description

Technical Field

[0001] This application belongs to the field of computer vision and deep learning technology, specifically relating to a method for target detection and instance segmentation of power distribution materials, a device for target detection and instance segmentation of power distribution materials, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Inspection of materials (such as insulators and fittings) on power distribution lines is a necessary step in material statistics and can also serve as a basis for subsequent refined analysis of materials (such as fine-grained material classification, material defect target detection, and material defect grading), which has great research significance.

[0003] Current technologies related to power distribution line material inspection mainly revolve around the location and segmentation of insulators, forming three mainstream paths: target detection technology, semantic segmentation technology, and instance segmentation schemes. These technologies provide some support for insulator inspection and subsequent material statistics and refined analysis. However, each technology still has significant limitations. Target detection technology uses bounding boxes as supervision signals, but these signals are relatively coarse, making detection accuracy susceptible to background interference. Semantic segmentation technology aims to compensate for the shortcomings of target detection; however, it operates independently of target detection, failing to achieve feature sharing, and its core application is focused on insulator defect detection, unable to effectively improve insulator location accuracy. Instance segmentation schemes rely on manually labeled Boolean masks, resulting in high labeling costs and low efficiency. Furthermore, existing technologies as a whole still revolve around the single material of insulators, and have not yet formed a targeted inspection system, thus exhibiting poor adaptability and failing to meet actual inspection needs.

[0004] In summary, the current field of power distribution line material detection suffers from problems such as insufficient detection accuracy, inadequate utilization of features, high cost of training data acquisition, and poor adaptability, which cannot meet the actual needs of accurate statistical analysis and subsequent refined analysis of power distribution materials. Therefore, developing an efficient, accurate, low-cost detection method and training data construction method that is adaptable to a variety of power distribution materials has become an urgent technical challenge. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, apparatus, and related equipment for target detection and instance segmentation of power distribution materials.

[0006] In a first aspect, embodiments of this application provide a method for target detection and instance segmentation of power distribution materials, including: Acquire images of the power distribution scene to be detected; The power distribution scene image is input into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; Obtain the prediction results output by the detection and segmentation model; the prediction results include the category, bounding box position, and mask information of the power distribution material targets in the power distribution scene image.

[0007] In some embodiments, the method further includes: Obtain the training dataset; the training dataset contains training images and corresponding synthetic mask annotations. Construct a detection and segmentation model that includes a classification head, a bounding box regression head, and an instance segmentation head; The detection and segmentation model is trained using the training dataset, where the training loss of the instance segmentation head is a hybrid loss calculated based on the predicted mask and the synthetic mask annotation map predicted by the detection and segmentation model.

[0008] In some embodiments, the method further includes: The training images are labeled to obtain the bounding boxes and category labels of all power distribution material targets in the training images; Using a segmentation model, an initial mask image is generated for each power distribution material target based on bounding boxes and category labels; All power distribution material targets are sorted in descending order based on the sum of pixel values ​​of each initial mask image; According to the sorting order, all initial mask images are combined into a single composite mask annotation image using a recursive merging algorithm.

[0009] In some embodiments, the method further includes: Calculate the mask classification loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map; Calculate the mask distillation loss between the predicted mask and the synthetic mask annotation map predicted by the detection segmentation model; The mask classification loss and mask distillation loss are weighted and summed according to a preset weighting coefficient to obtain the mixed loss.

[0010] In some embodiments, calculating the mask distillation loss between the predicted mask predicted by the detection segmentation model and the synthetic mask annotation map includes: Divide the logits value of the predicted mask predicted by the detection segmentation model by the temperature coefficient, and apply the sigmoid function to the calculation result to obtain the first probability distribution; The true value of the synthetic mask annotation map is inversely transformed by the inverse sigmoid function. The transformation result is divided by the temperature coefficient, and then the sigmoid function is applied again to obtain the second probability distribution. Calculate the KL divergence between the first probability distribution and the second probability distribution, and use it as the mask distillation loss.

[0011] In some embodiments, the method further includes: Calculate the classification loss of the classification head, the bounding box regression loss of the bounding box regression head, and the mixture loss of the instance segmentation head; The classification loss, bounding box regression loss, and mixture loss are weighted and summed according to preset weighting coefficients to obtain the total loss of the detection and segmentation model.

[0012] In some embodiments, all initial mask images are synthesized into a single composite mask annotation map using a recursive merging algorithm according to their sorting order, including: Use the initial mask image of the top-ranked power distribution material target as the current mask image; For power distribution material targets ranked second to Nth, a recursive merging operation is performed according to the ranking order. After the recursive merging of all power distribution material targets is completed, a composite mask annotation map is obtained.

[0013] Secondly, embodiments of this application provide a power distribution material target detection and instance segmentation device, comprising: The acquisition module is configured to acquire images of the power distribution scene to be detected; The segmentation module is configured to input power distribution scene images into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; The prediction module is configured to acquire the prediction results output by the detection and segmentation model; the prediction results include the category, bounding box position, and mask information of the power distribution material targets in the power distribution scene image.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the power distribution material target detection and instance segmentation method as described in the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, they implement the power distribution material target detection and instance segmentation method as described in the first aspect.

[0016] This application provides a method for power distribution material target detection and instance segmentation. First, it acquires an image of the power distribution scene to be detected. Then, it inputs the power distribution scene image into a pre-trained detection and segmentation model. The detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head. Finally, it obtains the prediction results output by the detection and segmentation model. The prediction results include the category, bounding box position, and mask information of the power distribution material targets in the power distribution scene image. This application automatically generates mask images using a pre-trained segmentation model, significantly reducing the reliance on manual pixel-level annotation and thus significantly reducing annotation costs. Secondly, the generated mask images can be efficiently compressed, thereby saving storage space. More importantly, the detection and segmentation model utilizes the high-precision boundary supervision signals provided by the mask images during training, enabling it to learn more robust feature representations.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 A flowchart illustrating a method for target detection and instance segmentation of power distribution materials provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a power distribution material target detection and instance segmentation device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0019] Explanation of reference numerals in the attached drawings: power distribution material target detection and instance segmentation device 200, acquisition module 201, segmentation module 202, prediction module 203, processor 310, memory 320, input / output interface 330, communication interface 340, and bus 350. Detailed Implementation

[0020] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0021] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0022] As described in the background section, the current field of power distribution line material detection suffers from problems such as insufficient detection accuracy, inadequate feature utilization, high training data acquisition costs, and poor adaptability. To address these technical challenges, this application proposes using instance-based segmented distillation tasks for power distribution material detection. The aim is to improve the detection accuracy of power distribution materials by integrating the advantages of positioning and segmentation through instance-based segmented distillation technology. At the same time, by automatically constructing training data, it eliminates the reliance on manual annotation, reduces training costs, and improves training efficiency, thereby achieving accurate detection of various materials in power distribution lines. This provides reliable support for subsequent refined material analysis and solves many shortcomings of existing technologies.

[0023] refer to Figure 1 This is a flowchart illustrating a method for target detection and instance segmentation of power distribution materials provided in an embodiment of this application.

[0024] like Figure 1 As shown, the method for target detection and instance segmentation of power distribution materials includes: Step S101: Obtain the image of the power distribution scene to be detected.

[0025] In the specific implementation process, users can acquire raw images of power distribution scenarios using visual acquisition devices such as cameras and webcams. These images can be transmitted to industrial control computers, edge servers, and backend processing terminals via wired communication methods such as Ethernet and industrial bus, or wireless communication methods such as 5G, Wi-Fi, and Bluetooth, where they are stored. During storage, scene-specific identifiers related to the acquisition location, acquisition time, and type of power distribution materials are added to the images to facilitate subsequent image classification in detection tasks. To adapt to the input requirements of the segmentation model, the raw power distribution scenario images need to undergo standardized preprocessing. This involves adjusting the images to the model's preset input resolution, unifying the color space and converting them to RGB format, eliminating environmental and equipment noise generated during image acquisition through Gaussian filtering and median filtering, and correcting the brightness and contrast of images with uneven lighting. After completing these preprocessing steps, the images to be labeled are obtained.

[0026] After the images are prepared, operators use annotation software (such as labelme, labelimg, etc.) to annotate each image in the dataset. The annotation targets are all power distribution material targets in the images, and the annotation content is the bounding box and category label of the corresponding power distribution material target. The segmentation model used in the data preparation stage can be of a type that supports text prompts and bounding box prompts (such as SAM, SAM2, SAM3, etc.). For each annotated image, its bounding box coordinates are used as spatial location prompts, and the category label is used as semantic text prompts. Both are fed into the encoder of the segmentation model. The encoder interacts with the image features and prompt information to generate an initial mask image for the target through pixel-by-pixel prediction. This mask is a single-channel floating-point image of the same size as the original image, and the value range of each pixel is [0,1], representing the probability or confidence that the pixel belongs to the target foreground. Finally, an initial mask image is generated independently for each target for subsequent merging and annotation. This data preparation method not only ensures the convenience and automation of image acquisition, but also allows the images to be detected to effectively reflect the actual characteristics of the power distribution materials, laying a solid foundation for the subsequent model to accurately output the type of power distribution materials, the bounding box position, and the mask information.

[0027] Step S102: Input the power distribution scene image into the pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head.

[0028] In the specific implementation process, the detection and segmentation model can be an instance segmentation model (such as Mask R-CNN, SOLO, etc.). The detection and segmentation model in this embodiment is modified from yolov8-seg. This model consists of a feature extraction backbone, a multi-scale feature fusion module (neck), and a prediction output module (head). The prediction output module contains three branches: a classification head, a bounding box regression head, and an instance segmentation head. The classification head is responsible for determining the category to which the target belongs. Relevant categories include insulators, switches, and transformers. The bounding box regression head is responsible for accurately predicting the position of the target in the image, which is presented in the form of a rectangular box. The instance segmentation head is responsible for predicting the precise contour of the target within the bounding box, generating a pixel-level mask to distinguish the foreground from the background, with the foreground being the detected target. The power distribution scene image to be detected is directly input into the pre-trained detection and segmentation model. This model performs target classification, bounding box regression, and pixel-level mask prediction simultaneously through one forward propagation, and finally outputs the prediction results corresponding to each power distribution material target in the image.

[0029] Step S103: Obtain the prediction results output by the detection and segmentation model; wherein, the prediction results include the category, bounding box position and mask information of the power distribution material targets in the power distribution scene image.

[0030] In practical implementation, the detection and segmentation model independently outputs a complete prediction result for each power distribution material target identified in the image. This result includes the category of the power distribution material target in the power distribution scene image, the bounding box position, and the mask information of each target. The bounding box position is represented by four numerically defined rectangular coordinates. This rectangle can tightly enclose the predicted target and serves as the basis for target localization and subsequent clipping operations. The category is presented in the form of integer or string labels to identify the material type corresponding to the target (e.g., insulator, circuit breaker, voltage transformer). This label is usually accompanied by a confidence score in the range of 0 to 1, which reflects the model's confidence in the category judgment. The mask information is a binary matrix of the same size as the original image or the bounding box region, consisting of 0s and 1s. The set of pixels with a value of 1 constitutes the precise outline shape of the target, while pixels with a value of 0 correspond to the image background. Mask information enables pixel-level precise segmentation, effectively distinguishes overlapping objects, and identifies non-rectangular targets. It is a key element for achieving high-precision analysis such as state recognition and size measurement.

[0031] In material statistics applications, extracting only the bounding box location and category label of the material target is sufficient. If further refined analysis based on the model output is required, the mask information output by the model can also be utilized. The specific implementation method is as follows: based on the bounding box location corresponding to the detected target, a cropping operation is performed on the original image to obtain the target's input image. Then, the target's mask information image is concatenated with the target's input image along the channel dimension to obtain new input data. This data is then fed into the subsequent network to execute the specific algorithm task.

[0032] In some embodiments, the method for detecting and segmenting power distribution material targets further includes: acquiring a training dataset; wherein the training dataset includes training images and synthetic mask annotation maps corresponding to the training images; constructing a detection and segmentation model including a classification head, a bounding box regression head, and an instance segmentation head; and training the detection and segmentation model using the training dataset, wherein the training loss of the instance segmentation head is a hybrid loss calculated based on the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map.

[0033] In the specific implementation process, in order to build a dataset that can be directly used for model training, each original training image needs to be labeled with its corresponding synthetic mask. Figure 1First, pairing is performed. After sample pairing, all samples are rationally divided into training, validation, and test sets according to a preset ratio. The training set is used for model parameter learning, the validation set for overshoot optimization, and the test set for final performance evaluation. Based on this, data augmentation operations such as rotation, scaling, and color transformation are applied to the training set samples to improve the model's robustness to different shooting conditions and environmental changes, ultimately forming a high-quality, standardized dataset that can be directly used for training detection and segmentation models.

[0034] The detection and segmentation model uses a mature instance segmentation network (such as YOLOv8-SEG) as its backbone. Its structure integrates a feature extraction backbone, a multi-scale feature fusion module, and three parallel prediction heads for classification, bounding box regression, and instance segmentation. To adapt to the needs of power distribution material detection, the classification dimension corresponding to the power distribution material category needs to be specifically configured. Simultaneously, it is ensured that the instance segmentation head can output prediction results adapted to the mixed loss. Based on this, a complete multi-task model that can be trained end-to-end is constructed. This model can ultimately output the target category, precise location, and pixel-level contour simultaneously.

[0035] After the dataset and model are built, the detection and segmentation model is trained using the training dataset. The training loss generated during the training process is a hybrid loss, which is calculated based on the mask predicted by the detection and segmentation model and the synthetic mask annotation map. Specifically, the classification head is responsible for predicting the category of the target, and its loss function is the binary cross-entropy loss (BCE loss); the bounding box regression head is responsible for predicting the bounding box of the target, and its loss function is the target detection regression loss (CIoU loss); the instance segmentation head is responsible for predicting the mask of the target, and its loss function is a weighted distillation loss and classification loss, which is different from the binary cross-entropy loss in yolov8-seg.

[0036] In some embodiments, the power distribution material target detection and instance segmentation method further includes: labeling the training image to obtain the bounding boxes and category labels of all power distribution material targets in the training image; using a segmentation model to generate an initial mask image for each power distribution material target based on the bounding boxes and category labels; sorting all power distribution material targets in descending order according to the sum of pixel values ​​of each initial mask image; and merging all initial mask images into a synthetic mask annotation image according to the sorting order using a recursive merging algorithm.

[0037] In the specific implementation process, for each power distribution material target in the training image, the image feature vector, the bounding box of the target, and the category label need to be fed into the segmentation model as spatial location cues and semantic text cues, respectively, to generate an initial mask image for the target. This initial mask image is a floating-point single-channel image with the same size as the input image, where each pixel's value ranges from [0,1]. The initial mask image is saved to disk using a lossless compression file format. During the generation and saving of the initial mask image, each power distribution material target will obtain an initial mask image. When there are many targets, the storage space occupied will be relatively large, and the initial mask image usually contains many zero values, so it is necessary and feasible to compress the initial mask image. Considering that the initial mask images of material targets rarely overlap in the power distribution drone scenario, the following method is used to merge the initial mask images of each target.

[0038] Calculate the sum of pixel values ​​of the initial mask image for each target in the image, sort all targets in descending order of pixel value sum, and obtain the index list SI of the power distribution materials sorted in descending order of mask pixel value sum. Then, synthesize the initial mask images of all targets in the image into a composite mask annotation image in the following recursive manner:

[0039] in, This is the intermediate mask image obtained after the 0th merging. The original mask for pixel values ​​and the largest electrical distribution material. For indexed list, The first merged object after sorting. For the first The intermediate mask image obtained after the second merging. For the first The intermediate mask image obtained after the second merging. The sequence number is the number of the merging step. For the sorted number Original instance segmentation mask image of a power distribution material. For the first in the sorted index list One element, For the first The upper limit of pixel values ​​for each merge.

[0040] The merging operation is performed in descending order of pixel values, first merging the mask pixel values ​​with the larger target, then merging the mask pixel values ​​with the smaller target. This merging process, ordered by pixel values ​​in descending order, is reversible, or partially reversible, and the initial mask image of the target can be recovered using the following formula:

[0041] in, Recovered from the merged total mask, sorted... Original instance segmentation mask of a power distribution material. For the final total mask obtained after merging, the first... The value of each pixel, This refers to the specific locations of pixels in the original image and the mask. Pixels representing the background or other materials.

[0042] The above mask image generation process is automatic and requires no manual interaction.

[0043] In some embodiments, the power distribution material target detection and instance segmentation method further includes: calculating the mask classification loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map; calculating the mask distillation loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map; and weighting and summing the mask classification loss and the mask distillation loss according to a preset weighting coefficient to obtain the mixed loss.

[0044] In practical implementation, mask classification loss is the fundamental loss for segmentation tasks. It is calculated by comparing the probability of each pixel predicted by the model to belong to the foreground (target) or background with the soft label (values ​​between 0 and 1) of the synthetic mask annotation map. The resulting standard loss (such as binary cross-entropy loss) is the mask classification loss. This loss drives the model to learn basic pixel-level classification capabilities. Mask distillation loss employs knowledge distillation technology, using the soft label of the synthetic mask as a teacher signal. This signal, along with the model's predicted logits, is softened by a temperature coefficient, transforming it into two smooth probability distributions. The KL divergence between these distributions is then used as the loss. This allows the model to learn richer and more robust probability distribution relationships, rather than rigid binary targets, thus achieving better generalization ability and boundary smoothness.

[0045] for and It can adopt the same form as the YOLO framework, for Then it has the following form:

[0046] in, For mixed loss, The weighting coefficients for mask classification loss are... For mask classification loss, The weighting factor for mask distillation loss is... This represents the loss during mask distillation.

[0047] In some embodiments, calculating the mask distillation loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map includes: dividing the logits value of the predicted mask predicted by the detection and segmentation model by a temperature coefficient, and applying a sigmoid function to the calculation result to obtain a first probability distribution; performing an inverse transformation on the ground truth value of the synthetic mask annotation map using the inverse sigmoid function, dividing the transformation result by a temperature coefficient, and then applying a sigmoid function to obtain a second probability distribution; and calculating the KL divergence between the first probability distribution and the second probability distribution as the mask distillation loss.

[0048] In the specific implementation process, a temperature coefficient is introduced to "soften" the original logits predicted by the segmentation model and the soft labels of the synthesized mask, transforming them into smooth, continuous probability distributions. The KL divergence between the two is then calculated as the distillation loss. For a single pixel, and It then takes the following form:

[0049] in, For temperature coefficient, The binary cross-entropy loss function is... Let KL divergence be the KL divergence. For the sigmoid function, for inverse function, For the predicted pixel mask's logits, This provides mask annotations for the corresponding pixels (i.e., the initial mask images generated during the data preparation process above). This effectively improves the robustness of the detection and segmentation model to annotation noise, achieves more natural segmentation boundaries, and enhances generalization ability.

[0050] In some embodiments, the power distribution material target detection and instance segmentation method further includes: calculating the classification loss of the classification head, the bounding box regression loss of the bounding box regression head, and the mixed loss of the instance segmentation head; and weighting and summing the classification loss, bounding box regression loss, and mixed loss according to preset weighting coefficients to obtain the total loss of the detection and segmentation model.

[0051] In the specific implementation process, the independent losses of the three task heads are first calculated separately. The classification loss is used to evaluate the accuracy of the model's prediction of the target category; the bounding box regression loss is used to measure the deviation between the predicted bounding box and the true location coordinates; and the mixture loss of the instance segmentation head is composed of the mask classification loss and the mask distillation loss, which is used to optimize pixel-level segmentation accuracy. Users can combine the importance and magnitude of each task and perform a weighted sum of these three losses according to preset weighting coefficients to form a unified total loss. For a single target, the model's final loss has a corresponding expression form, and the formula for calculating the total loss is:

[0052] in, For the total loss, The weighting coefficients for the classification loss are... For classifying losses, The weighting coefficients for the bounding box regression loss are: For bounding box regression loss, The weighting coefficients for instance segmentation loss are... The instance segmentation loss. Weighting coefficients for each loss. and temperature coefficient All parameters can be optimized using grid search. The total loss serves as the sole optimization objective during the training of the detection and segmentation model, synchronously driving the updates of all network parameters through backpropagation, ultimately promoting a collaborative improvement in the overall performance of detection and segmentation.

[0053] In some embodiments, according to the sorting order, all initial mask images are combined into a single composite mask annotation image by a recursive merging algorithm, including: using the initial mask image of the first-ranked power distribution material target as the current mask image; performing a recursive merging operation on the second to Nth ranked power distribution material targets according to the sorting order; and obtaining the composite mask annotation image after the recursive merging of all power distribution material targets is completed.

[0054] In practice, when target occlusion occurs, it is usually a smaller target occluding a larger target, and the smaller target typically has smaller pixel values. The initial mask with the largest area is set as the current mask base. Then, the remaining masks are processed sequentially in descending order of area. For each mask to be merged, pixels that do not overlap with the current base mask (i.e., background pixels in the base) are superimposed onto the base mask. This process is iterated until all masks are sequentially merged, ultimately resulting in a composite mask annotation map where each pixel belongs to only one target. This merging method can aid in the detection of small targets. If there are N targets in the image, then... This is the mask image after merging all targets, where the value range of each pixel is... Furthermore, this merging method effectively avoids label conflicts between masks, providing high-quality ground truth data for model training.

[0055] In summary, the power distribution material target detection and instance segmentation method provided in this application first acquires a power distribution scene image to be detected. Then, the power distribution scene image is input into a pre-trained detection and segmentation model. The detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head. Finally, the prediction result output by the detection and segmentation model is obtained. The prediction result includes the category, bounding box position, and mask information of the power distribution material target in the power distribution scene image. This application automatically generates mask images using a pre-trained segmentation model, significantly reducing the reliance on manual pixel-level annotation and thus significantly reducing annotation costs. Secondly, the generated mask images can be efficiently compressed, thereby saving storage space. More importantly, the detection and segmentation model utilizes the high-precision boundary supervision signal provided by the mask images during training, enabling it to learn more robust feature representations.

[0056] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described above.

[0057] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0058] Corresponding to the above embodiments, this application also proposes a power distribution material target detection and instance segmentation device.

[0059] refer to Figure 2 This is a schematic diagram of a power distribution material target detection and instance segmentation device provided in an embodiment of this application.

[0060] like Figure 2 As shown in the figure, this application embodiment provides a power distribution material target detection and instance segmentation device 200, including: The acquisition module 201 is configured to acquire images of the power distribution scene to be detected; The segmentation module 202 is configured to input the power distribution scene image into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; The prediction module 203 is configured to acquire the prediction results output by the detection and segmentation model; wherein the prediction results include the category, bounding box position and mask information of the power distribution material targets in the power distribution scene image.

[0061] Optionally, the segmentation module 202 is also configured as follows: Obtain the training dataset; the training dataset contains training images and corresponding synthetic mask annotations. Construct a detection and segmentation model that includes a classification head, a bounding box regression head, and an instance segmentation head; The detection and segmentation model is trained using the training dataset, where the training loss of the instance segmentation head is a hybrid loss calculated based on the predicted mask and the synthetic mask annotation map predicted by the detection and segmentation model.

[0062] Optionally, the acquisition module 201 is also configured as follows: The training images are labeled to obtain the bounding boxes and category labels of all power distribution material targets in the training images; Using a segmentation model, an initial mask image is generated for each power distribution material target based on bounding boxes and category labels; All power distribution material targets are sorted in descending order based on the sum of pixel values ​​of each initial mask image; According to the sorting order, all initial mask images are combined into a single composite mask annotation image using a recursive merging algorithm.

[0063] Optionally, the prediction module 203 is also configured as follows: Calculate the mask classification loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map; Calculate the mask distillation loss between the predicted mask and the synthetic mask annotation map predicted by the detection segmentation model; The mask classification loss and mask distillation loss are weighted and summed according to a preset weighting coefficient to obtain the mixed loss.

[0064] Optionally, the prediction module 203 is also configured as follows: Calculate the mask distillation loss between the predicted mask from the detection and segmentation model and the synthetic mask annotation map, including: Divide the logits value of the predicted mask predicted by the detection segmentation model by the temperature coefficient, and apply the sigmoid function to the calculation result to obtain the first probability distribution; The true value of the synthetic mask annotation map is inversely transformed by the inverse sigmoid function. The transformation result is divided by the temperature coefficient, and then the sigmoid function is applied again to obtain the second probability distribution. Calculate the KL divergence between the first probability distribution and the second probability distribution, and use it as the mask distillation loss.

[0065] Optionally, the prediction module 203 is also configured as follows: Calculate the classification loss of the classification head, the bounding box regression loss of the bounding box regression head, and the mixture loss of the instance segmentation head; The classification loss, bounding box regression loss, and mixture loss are weighted and summed according to preset weighting coefficients to obtain the total loss of the detection and segmentation model.

[0066] Optionally, the acquisition module 201 is also configured as follows: Following the sorting order, all initial mask images are synthesized into a single composite mask annotation map using a recursive merging algorithm, including: Use the initial mask image of the top-ranked power distribution material target as the current mask image; For power distribution material targets ranked second to Nth, a recursive merging operation is performed according to the ranking order. After the recursive merging of all power distribution material targets is completed, a composite mask annotation map is obtained.

[0067] In summary, the power distribution material target detection and instance segmentation device provided in this application first acquires a power distribution scene image to be detected, then inputs the power distribution scene image into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; finally, the prediction result output by the detection and segmentation model is obtained; wherein the prediction result includes the category, bounding box position, and mask information of the power distribution material target in the power distribution scene image. This application automatically generates mask images using a pre-trained segmentation model, significantly reducing the reliance on manual pixel-level annotation and substantially reducing annotation costs. Secondly, the generated mask images can be efficiently compressed, thereby saving storage space. More importantly, the detection and segmentation model utilizes the high-precision boundary supervision signal provided by the mask images during training, enabling it to learn more robust feature representations.

[0068] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0069] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0070] Corresponding to the above embodiments, this application also proposes an electronic device. (See reference...) Figure 3 The diagram below is a block diagram of an electronic device according to some embodiments of this application. It also illustrates a more specific hardware structure of an electronic device provided by an embodiment of this application. The device may include: a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. The processor 310, memory 320, input / output interface 330, and communication interface 340 are interconnected internally via the bus 350.

[0071] The processor 310 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0072] The memory 320 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 320 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 320 and is called and executed by the processor 310.

[0073] Input / output interface 330 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0074] The communication interface 340 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0075] Bus 350 includes a pathway for transmitting information between various components of the device, such as processor 310, memory 320, input / output interface 330, and communication interface 340.

[0076] It should be noted that although the above-described device only shows the processor 310, memory 320, input / output interface 330, communication interface 340, and bus 350, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0077] The electronic devices described above are used to implement the corresponding power distribution material target detection and instance segmentation methods in any of the foregoing embodiments, and have corresponding beneficial effects, which will not be elaborated here.

[0078] Based on the same concept, corresponding to the power distribution material target detection and instance segmentation method provided in any of the above embodiments, this application also provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, it implements the power distribution material target detection and instance segmentation method as described above.

[0079] The aforementioned computer-readable storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0080] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the corresponding power distribution material target detection and instance segmentation method in any of the foregoing embodiments, and have corresponding beneficial effects, which will not be elaborated here.

[0081] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0082] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0083] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. A method for target detection and instance segmentation of power distribution materials, characterized in that, include: Acquire images of the power distribution scene to be detected; The power distribution scene image is input into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; Obtain the prediction results output by the detection and segmentation model; wherein, the prediction results include the category, bounding box position, and mask information of the power distribution material targets in the power distribution scene image.

2. The method for target detection and instance segmentation of power distribution materials according to claim 1, characterized in that, The method further includes: Obtain a training dataset; wherein the training dataset includes training images and synthetic mask annotation maps corresponding to the training images; Construct a detection and segmentation model that includes a classification head, a bounding box regression head, and an instance segmentation head; The detection and segmentation model is trained using the training dataset, wherein the training loss of the instance segmentation head is a hybrid loss calculated based on the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map.

3. The method for target detection and instance segmentation of power distribution materials according to claim 2, characterized in that, The method further includes: The training images are labeled to obtain the bounding boxes and category labels of all power distribution material targets in the training images; Using a segmentation model, an initial mask image is generated for each of the power distribution material targets based on the bounding box and category label; All the power distribution material targets are sorted in descending order based on the sum of the pixel values ​​of each initial mask image; According to the sorting order, all the initial mask images are combined into a single composite mask annotation image using a recursive merging algorithm.

4. The method for target detection and instance segmentation of power distribution materials according to claim 2, characterized in that, The method further includes: Calculate the mask classification loss between the predicted mask predicted by the detection and segmentation model and the synthetic mask annotation map; Calculate the mask distillation loss between the predicted mask predicted by the detection segmentation model and the synthetic mask annotation map; The mask classification loss and the mask distillation loss are weighted and summed according to a preset weighting coefficient to obtain the mixing loss.

5. The method for target detection and instance segmentation of power distribution materials according to claim 4, characterized in that, The calculation of the mask distillation loss between the predicted mask predicted by the detection segmentation model and the synthetic mask annotation map includes: Divide the logits value of the predicted mask predicted by the detection and segmentation model by the temperature coefficient, and apply the sigmoid function to the calculation result to obtain the first probability distribution; The true value of the synthetic mask annotation map is inversely transformed by the inverse sigmoid function. The transformation result is divided by the temperature coefficient, and then the sigmoid function is applied again to obtain the second probability distribution. The KL divergence between the first probability distribution and the second probability distribution is calculated as the mask distillation loss.

6. The method for target detection and instance segmentation of power distribution materials according to claim 2, characterized in that, The method further includes: Calculate the classification loss of the classification head, the bounding box regression loss of the bounding box regression head, and the mixture loss of the instance segmentation head; The classification loss, the bounding box regression loss, and the mixture loss are weighted and summed according to preset weighting coefficients to obtain the total loss of the detection and segmentation model.

7. The method for target detection and instance segmentation of power distribution materials according to claim 3, characterized in that, The step of merging all the initial mask images into a single composite mask annotation map using a recursive merging algorithm according to the sorting order includes: Use the initial mask image of the top-ranked power distribution material target as the current mask image; For the power distribution material targets ranked second to Nth, a recursive merging operation is performed according to the ranking order. After the recursive merging of all power distribution material targets is completed, the composite mask annotation map is obtained.

8. A device for detecting and segmenting power distribution materials, characterized in that, include: The acquisition module is configured to acquire images of the power distribution scene to be detected; The segmentation module is configured to input the power distribution scene image into a pre-trained detection and segmentation model; wherein the detection and segmentation model includes at least a classification head, a bounding box regression head, and an instance segmentation head; The prediction module is configured to acquire the prediction results output by the detection and segmentation model; wherein the prediction results include the category, bounding box position, and mask information of the power distribution material targets in the power distribution scene image.

9. An electronic device, characterized in that, include: A processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions being executed by the processor to implement the steps of the power distribution material target detection and instance segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the power distribution material target detection and instance segmentation method as described in any one of claims 1 to 7.

Images