Dense cable segmentation method and system
The cable image features are extracted in complex environments through YOLACT network and ResNet, combining confidence and interchange ratio screening, and correcting the cable candidate boxes using the deviation angle coefficient, achieving accurate segmentation of dense cables, improving the accuracy and recall of inspection results Rate.
Patent Information
- Application Number
- CN202210162836.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-02-22
AI Technical Summary
The prior art is difficult to accurately divide dense cables in complex environments, resulting in inaccurate inspection results.
The YOLACT network is used to combine ResNet and FPN layers for cable image feature extraction. The cable candidate boxes are initially obtained through confidence and interleaving ratio screening, and the deviation angle coefficient is used for screening and correction, and the accurate segmentation of the cable is finally achieved.
It improves the accuracy and recall rate of cable segmentation, solves the problem of cable leakage detection in complex environments, and improves the performance of the identification algorithm.
Smart Images

Figure CN114549573B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image processing, and in particular relates to a dense cable segmentation method and system. Background Art
[0002] As the national power grid becomes larger and larger, there are more and more transmission lines. The inspection of transmission lines is now mainly completed by collecting images of transmission lines and submitting them to the system for automatic identification. However, due to the complexity and diversity of the lines, the different widths, and the different angles of image shooting, some cables that appear densely packed in the pictures cannot be accurately segmented, which has a negative impact on the inspection results. Summary of the invention
[0003] In order to solve the problems existing in the prior art, the present invention provides a dense cable segmentation method and method, which can improve the accuracy of cable segmentation.
[0004] The technical problem to be solved by the present invention is achieved through the following technical solutions:
[0005] In a first aspect, a dense cable segmentation method is provided, comprising:
[0006] Obtain a color image of dense cables, remove the image background and extract the cable candidate frame;
[0007] Conduct preliminary screening of cable candidate frames;
[0008] The cable candidate frames after preliminary screening are screened and modified to complete the cable segmentation.
[0009] In combination with the first aspect, further, extracting the cable candidate frame after removing the image background includes:
[0010] The color image of the cable is input into the YOLACT network for feature extraction. The backbone network of the YOLACT network adopts the ResNet network. The ResNet network generates five feature layers, and three to five layers are used as the input of the FPN layer. After obtaining the feature map through the FPN layer, the cable candidate box and its corresponding confidence are obtained.
[0011] In combination with the first aspect, further, the preliminary screening of the cable candidate frames includes:
[0012] Sort the cable candidate frames according to their corresponding confidence levels, from large to small, select the candidate frame with the largest confidence level, calculate the intersection-and-union ratio of the cable candidate frame and the remaining cable candidate frames, and compare the obtained intersection-and-union ratio with the threshold N. t Compare, if it is greater than the threshold N t , then delete the cable candidate frame corresponding to the intersection-union ratio, otherwise keep it, so as to achieve preliminary screening of the candidate frames.
[0013] In combination with the first aspect, further, the screening and correction of the cable candidate frame after the preliminary screening includes:
[0014] Arrange the cable candidate frames after preliminary screening from large to small according to the confidence level, and select the cable candidate frame with the highest confidence level;
[0015] Calculate the deviation angle coefficient between the cable candidate frame with the greatest confidence and the remaining cable candidate frames;
[0016] The corrected confidence between the remaining cable candidate frames and the cable candidate frame with the greatest confidence is calculated according to the confidence corresponding to the remaining cable candidate frames and the deviation angle coefficient between the remaining cable candidate frames and the cable candidate frame with the greatest confidence;
[0017] The cable candidate frames are screened and corrected according to the correction confidence.
[0018] In combination with the first aspect, further, the deviation angle coefficient is calculated by formula (1)
[0019] Rel(M,N)=cos(μ-β)×IOU(M,N) (1)
[0020] Wherein, μ and β represent the diagonal angles of the maximum cable candidate frame M and the remaining cable candidate frame N, respectively, IOU(M,N) is the intersection-over-union ratio of the maximum cable candidate frame M to the remaining cable candidate frame N, and Rel(M,N) is the deviation angle coefficient between the cable candidate frame M with the maximum confidence and the remaining cable candidate frame N.
[0021] In combination with the first aspect, further, the modified confidence is calculated by formula (2):
[0022]
[0023] Among them, s i is the confidence of the i-th candidate box, N t is the threshold value.
[0024] In a second aspect, a dense cable segmentation system is provided, comprising:
[0025] The candidate frame extraction module is used to obtain a color image of dense cables and extract the cable candidate frame after removing the image background;
[0026] A candidate frame preliminary screening module is used to perform preliminary screening of cable candidate frames;
[0027] The cable segmentation module is used to screen and modify the cable candidate frames after preliminary screening to complete cable segmentation.
[0028] Beneficial effects of the present invention: The present invention firstly collects cable images through a camera, then uses ResNet as the backbone network of YOLACT to extract features from the cable images, and uses the established cable candidate frame correction confidence calculation method to screen and correct the cable target candidate frame, and finally realizes the recognition and segmentation extraction of dense cables. This method solves the problem of missed detection of dense cables in complex environments to a certain extent, and at the same time improves the recall rate and accuracy of the recognition algorithm in cable detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION
[0030] In order to further describe the technical features and effects of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.
[0031] Example 1
[0032] like Figure 1 The present invention provides a method for dividing dense cables, comprising the following steps:
[0033] Step 1: Extract cable candidate frames
[0034] The color image of dense cables is obtained through the camera, and then the color image of the cable is input into the YOLACT network for feature extraction. The complex background in the picture can be filtered out through the YOLACT network. ResNet (residual network) is used as the backbone network to generate five different feature layers. Three to five layers are used as the input of the FPN (Feature Pyramid Networks) layer. The feature map obtained after the FPN layer is predicted (using the AnchorBoxes method) to obtain the cable candidate box and its corresponding confidence.
[0035] Step 2: Preliminary screening of candidate boxes
[0036] After the candidate boxes are extracted, the accuracy is not high enough, so they need to be initially screened. The process is mainly as follows:
[0037] First, sort the cable candidate frames according to their corresponding confidence levels from large to small. Then, we select the cable candidate frame with the largest confidence level, calculate its intersection-over-union ratio with other cable candidate frames, and compare it with the threshold N. t Compare, if it is greater than the threshold N t (given according to the empirical value), then delete it, otherwise keep it; traverse all candidate frames, retain the cable candidate frames that are greater than the threshold, and the retained cable candidate frames are the candidate frames after preliminary screening.
[0038] Step 3: Cable segmentation
[0039] After completing the preliminary screening of the cable candidate frame, in order to further improve the accuracy of cable segmentation (determining the cable candidate frame is equivalent to completing the cable segmentation), it is necessary to further correct the accuracy of the cable candidate frame. The main process is as follows:
[0040] First, the remaining cable candidate frames after preliminary screening in step 2 are arranged from large to small according to the confidence level, and the cable candidate frame with the largest confidence level is selected. Then, the deviation angle coefficient between it and the remaining cable candidate frames is calculated using formula (1):
[0041] Rel(M,N)=cos(μ-β)×IOU(M,N) (1)
[0042] Wherein, μ and β represent the diagonal angles of the maximum cable candidate box M and the remaining cable candidate box N, respectively, and IOU(M,N) is the intersection-over-union ratio of the maximum cable candidate box M and the remaining cable candidate box N.
[0043] Then, the corrected confidence between the remaining candidate boxes and the cable candidate box with the highest confidence is calculated by the confidence corresponding to the remaining candidate boxes (any cable candidate box except the cable candidate box with the highest confidence) and the deviation angle coefficient between the remaining candidate boxes and the cable candidate box with the highest confidence. The calculation method is shown in formula (2):
[0044]
[0045] Among them, s i is the confidence of the i-th candidate box, Rel(M,N) is the cable candidate box M with the highest confidence and the remaining cable candidate boxes N. This function is not a continuous function, and there will be a gradient fault. When the correlation Rel condition between the candidate box M and the candidate box N is met, the confidence s i Multiplying by 1-Rel will reduce linearly.
[0046] The cable candidate frame is screened and corrected according to the calculated correction confidence, specifically:
[0047] The corrected confidence c i and threshold N t Compare, if it is greater than the threshold N t , the cable candidate frame corresponding to the modified confidence is deleted, otherwise it is retained. The cable candidate frame finally retained is the modified cable candidate frame. Once the cable candidate frame is determined, the cable segmentation is completed.
[0048] The verification results of the method described in the present invention are shown in Table 1
[0049] Table 1 Comparison of results before and after using this method
[0050]
[0051] The present invention significantly improves the recall rate and accuracy of cable detection. The recall rate of cables is increased by 4.1% and the accuracy rate of cables is increased by 1.9% compared with the original. The average pixel accuracy rate and FPS are almost unchanged before and after optimization.
[0052] Example 2
[0053] A dense cable segmentation system is also provided, comprising:
[0054] The candidate frame extraction module is used to obtain a color image of dense cables and extract the cable candidate frame after removing the image background;
[0055] A candidate frame preliminary screening module is used to perform preliminary screening of cable candidate frames;
[0056] The cable segmentation module is used to screen and modify the cable candidate frames after preliminary screening to complete cable segmentation.
[0057] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0058] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A dense cable segmentation method, characterized in that: include: Obtain a color image of dense cables, remove the image background and extract the cable candidate frame; Conduct preliminary screening of cable candidate frames; Screen and modify the cable candidate frames after preliminary screening to complete cable segmentation; The method of extracting the cable candidate frame after removing the image background comprises: The color image of the cable is input into the YOLACT network for feature extraction. The backbone network of the YOLACT network adopts the ResNet network. The ResNet network generates five feature layers, and three to five layers are used as the input of the FPN layer. After obtaining the feature map through the FPN layer, the cable candidate box and its corresponding confidence are obtained. The preliminary screening of the cable candidate frames includes: Sort the cable candidate frames according to their corresponding confidence levels, from large to small, select the candidate frame with the largest confidence level, calculate the intersection-and-union ratio of the cable candidate frame and the remaining cable candidate frames, and compare the obtained intersection-and-union ratio with the threshold N. t Compare, if it is greater than the threshold N t , then delete the cable candidate frame corresponding to the intersection-union ratio, otherwise keep it, so as to achieve preliminary screening of the candidate frames; The screening and correction of the cable candidate frame after the preliminary screening includes: Arrange the cable candidate frames after preliminary screening from large to small according to the confidence level, and select the cable candidate frame with the highest confidence level; Calculate the deviation angle coefficient between the cable candidate frame with the greatest confidence and the remaining cable candidate frames; The corrected confidence between the remaining cable candidate frames and the cable candidate frame with the greatest confidence is calculated according to the confidence corresponding to the remaining cable candidate frames and the deviation angle coefficient between the remaining cable candidate frames and the cable candidate frame with the greatest confidence; The cable candidate frame is screened and corrected according to the correction confidence; The deviation angle coefficient is calculated by formula (1): Rel(M,N)=cos(μ-β)×IOU(M,N) (1) Wherein, μ and β represent the diagonal angles of the cable candidate frame M with the highest confidence and the remaining cable candidate frame N, respectively; IOU(M,N) is the intersection-over-union ratio of the cable candidate frame M with the highest confidence and the remaining cable candidate frame N; Rel(M,N) is the deviation angle coefficient between the cable candidate frame M with the highest confidence and the remaining cable candidate frame N; The modified confidence is calculated by formula (2): Among them, s i is the confidence of the i-th candidate box, N t is the threshold value.
2. A dense cable segmentation system, characterized in that: include: The candidate frame extraction module is used to obtain a color image of dense cables and extract the cable candidate frame after removing the image background; A candidate frame preliminary screening module is used to perform preliminary screening of cable candidate frames; The cable segmentation module is used to screen and modify the cable candidate frames after preliminary screening to complete cable segmentation; The method of extracting the cable candidate frame after removing the image background comprises: The color image of the cable is input into the YOLACT network for feature extraction. The backbone network of the YOLACT network adopts the ResNet network. The ResNet network generates five feature layers, and three to five layers are used as the input of the FPN layer. After obtaining the feature map through the FPN layer, the cable candidate box and its corresponding confidence are obtained. The preliminary screening of the cable candidate frames includes: Sort the cable candidate frames according to their corresponding confidence levels, from large to small, select the candidate frame with the largest confidence level, calculate the intersection-and-union ratio of the cable candidate frame and the remaining cable candidate frames, and compare the obtained intersection-and-union ratio with the threshold N. t Compare, if it is greater than the threshold N t , then delete the cable candidate frame corresponding to the intersection-union ratio, otherwise keep it, so as to achieve preliminary screening of the candidate frames; The screening and correction of the cable candidate frame after the preliminary screening includes: Arrange the cable candidate frames after preliminary screening from large to small according to the confidence level, and select the cable candidate frame with the highest confidence level; Calculate the deviation angle coefficient between the cable candidate frame with the greatest confidence and the remaining cable candidate frames; The corrected confidence between the remaining cable candidate frames and the cable candidate frame with the greatest confidence is calculated according to the confidence corresponding to the remaining cable candidate frames and the deviation angle coefficient between the remaining cable candidate frames and the cable candidate frame with the greatest confidence; The cable candidate frames are screened and corrected according to the correction confidence; The deviation angle coefficient is calculated by formula (1): Rel(M,N)=cos(μ-β)×IOU(M,N) (1) Wherein, μ and β represent the diagonal angles of the cable candidate frame M with the highest confidence and the remaining cable candidate frame N, respectively; IOU(M,N) is the intersection-over-union ratio of the cable candidate frame M with the highest confidence and the remaining cable candidate frame N; Rel(M,N) is the deviation angle coefficient between the cable candidate frame M with the highest confidence and the remaining cable candidate frame N; The modified confidence is calculated by formula (2): Among them, s i is the confidence of the i-th candidate box, N t is the threshold value.
