A single-terminal line defect detection method and system based on image processing

Abstract

The application provides a single-terminal wire defect detection method and system based on image processing, which comprises the following steps: extracting a standard sequence of each detection target appearing from a template image of a single-terminal wire detection frame, extracting feature information of each detection frame of the standard sequence and image feature information of the detection target in each detection frame, mapping each detection frame of a to-be-detected image after being detected by a deep learning model according to the standard sequence, marking a missing position as a position not being mapped by the standard sequence, determining a position area of the missing position according to position information of a detection frame adjacent to the missing position and position information of each detection frame in the standard sequence, extracting image feature information of the position area and performing multi-dimensional feature matching with the standard sequence to determine whether the matching is successful, extracting a standard sequence of each detection target appearing from a single-terminal wire and feature information thereof, and detecting and performing feature matching on a missing position of a to-be-detected image based on the standard sequence, so that missing and wrong detection can be effectively prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of harness detection, and in particular to a single-terminal wire defect detection method and system based on image processing. BACKGROUND

[0002] In the field of industrial visual detection, target detection models based on deep learning (such as the YOLO series) have been widely used in product defect detection, component identification, and other scenarios. However, in actual industrial environments, affected by factors such as changes in lighting, shifts in product position, fluctuations in image quality, and the like, detection models have difficulty in consistently maintaining high-precision output, and are prone to missed detection and false detection, i.e., some detection targets are not identified or have too low a confidence level and are filtered out, or non-target regions are incorrectly identified as targets or target categories are incorrectly classified.

[0003] In particular, in detection scenarios with a fixed sequence structure (such as components such as rubber feet, exposed rubber, exposed wire cores, wire core feet, and protruding heads arranged in a fixed order), when a target at a certain position in the sequence is not detected, it will lead to errors in subsequent quality judgment and affect the accuracy of product qualification determination. SUMMARY

[0004] The present application provides a single-terminal wire defect detection method and system based on image processing, a computer device, and a storage medium, which extracts a standard sequence of each detection target appearing in a single-terminal wire, detects and features matches a missing position of a to-be-detected image based on the standard sequence, and can effectively prevent missed detection and false detection.

[0005] In a first aspect, the present application provides a single-terminal wire defect detection method based on image processing, comprising:

[0006] extracting a standard sequence of each detection target appearing in a template image of a single-terminal wire detection frame, and extracting feature information of each detection frame in the standard sequence and image feature information of the detection target in each detection frame;

[0007] mapping each detection frame of a to-be-detected image after being detected by a deep learning model according to the standard sequence, and marking a position not mapped by the standard sequence as a missing position;

[0008] determining a position region of the missing position according to position information of a detection frame adjacent to the missing position and position information of each detection frame in the standard sequence;

[0009] extracting image feature information of the position region, performing multi-dimensional feature matching between the image feature information of the position region and image feature information of the detection target in the corresponding detection frame of the standard sequence, and determining whether the matching is successful.

[0010] In one of the embodiments, before the mapping of the bounding boxes of the to-be-tested image detected by the deep learning model according to the standard sequence of the appearance of each detection target, the position not mapped by the standard sequence is marked as a missing position, further comprising:

[0011] The image feature information of each detection target of a plurality of qualified template images is tested for mutual matching, and the matching threshold of each detection target of the to-be-tested image and the detection target in each bounding box of the standard sequence is set according to the matching result and the image complexity of each detection target.

[0012] In one of the embodiments, the mapping of each bounding box of the to-be-tested image detected by the deep learning model according to the standard sequence, and the position not mapped by the standard sequence is marked as a missing position, further comprising:

[0013] Each bounding box of the to-be-tested image detected by the deep learning model is filtered according to the confidence, and the bounding box with high confidence is retained;

[0014] The retained bounding boxes are sorted in the horizontal direction to form a detection sequence;

[0015] According to the greedy matching algorithm, each bounding box in the detection sequence is mapped to the corresponding bounding box of the standard sequence, and the position not mapped by the standard sequence is marked as a missing position.

[0016] In one of the embodiments, the multi-dimensional feature matching includes multi-scale matching, multi-angle matching, and multi-feature matching;

[0017] The multi-feature matching is specifically that the image feature information of the position area is matched with the template image feature information under each scale and angle combination, and the matching score of each feature is weighted and summed based on the image complexity of the corresponding detection target to obtain a feature comprehensive score; the template image is the image of the detection target in the bounding box of the missing position corresponding to the standard sequence.

[0018] In one of the embodiments, for the detection target with an image complexity lower than a preset value, further comprising color segmentation auxiliary positioning and feature matching.

[0019] In one of the embodiments, after the extraction of the image feature information of the position area, the multi-dimensional feature matching of the image feature information of the position area with the image feature information of the detection target in the corresponding bounding box of the standard sequence, and the determination of whether the matching is successful, further comprising:

[0020] The position area with successful matching is subjected to multi-dimensional feature verification with the template image, and the multi-dimensional verification includes color similarity verification, edge density verification, histogram similarity verification, and feature weighted comprehensive verification.

[0021] In one embodiment, after the image feature information of the position area is extracted, the image feature information of the position area is subjected to multi-dimensional feature matching with the image feature information of the detection target in the detection frame corresponding to the standard sequence, and it is determined whether the matching is successful or not, and the method further comprises:

[0022] A supplementary detection frame is generated in the position area where the matching is successful, and the class label, matching degree and position coordinates of the supplementary detection frame are outputted, and the supplementary detection frame is marked.

[0023] In a second aspect, the present application provides a single-terminal wire defect detection system based on image processing, comprising:

[0024] A standard sequence extraction unit is configured to extract a standard sequence of each detection target from a template image of a single-terminal wire detection frame, and extract feature information of each detection frame of the standard sequence and image feature information of the detection target in each detection frame;

[0025] A missing position detection unit is configured to map each detection frame of a to-be-detected image after deep learning model detection according to the standard sequence, and mark a position not mapped by the standard sequence as a missing position;

[0026] A position area determination unit is configured to determine a position area of the missing position according to position information of detection frames adjacent to the missing position and position information of each detection frame in the standard sequence;

[0027] A feature matching unit is configured to extract image feature information of the position area, and subject the image feature information of the position area to multi-dimensional feature matching with image feature information of the detection target in a detection frame corresponding to the standard sequence, and determine whether the matching is successful or not.

[0028] In a third aspect, the present application further provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the method of any one of the above aspects.

[0029] In a fourth aspect, the present application further provides a computer storage medium, wherein the computer storage medium stores a computer program, and the computer program is executed to implement the method of any one of the above aspects.

[0030] The application discloses a single-terminal wire defect detection method and system based on image processing, a computer device and a storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. The drawings in the following description are only corresponding to some of the embodiments of the present application, and other drawings of other embodiments can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0032] Figure 1 For one of the embodiments of the present application, a method flow chart of the single-terminal wire defect detection method based on image processing is provided.

[0033] Figure 2 For another embodiment of the present application, a method flow chart of the single-terminal wire defect detection method based on image processing is provided.

[0034] Figure 3 For one of the embodiments of the present application, a system block diagram of the single-terminal wire defect detection system based on image processing is provided. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the present application.

[0036] Unless otherwise defined, all the technical and scientific terms used in the present application have the same meanings as those commonly understood by those skilled in the technical field of the present application.

[0037] Reference Signs List Figure 1The application discloses a single-terminal line defect detection method based on image processing.

[0038] S101, a standard sequence of each detection target appearing in a template image of a single-terminal line detection frame is extracted, and feature information of each detection frame in the standard sequence and image feature information of the detection target in each detection frame are extracted.

[0039] The template image of the detection frame is a template image set obtained after detection by a deep learning model or a manually labeled template image set.

[0040] Specifically, the appearance sequence of the detection target in all template images is counted, and the sequence with the highest appearance frequency is selected as the standard sequence.

[0041] For each detection target in the standard sequence, the image region of the detection target in each detection frame is extracted from the template image conforming to the sequence mode as an appearance feature template, and multiple sample templates of each detection target are collected to cover the appearance changes of the detection target.

[0042] The size distribution of the detection frame corresponding to each detection target is counted, the average width and the average height are calculated, the interval distribution between adjacent detection frames is counted, and the average interval is calculated as the feature information of each detection frame in the standard sequence, which is used for search area positioning.

[0043] The average RGB value, the average edge density, the image complexity, the color histogram and other image feature information of each detection target in the template image are calculated as the template image feature information of each detection target in the standard sequence.

[0044] S102, according to the standard sequence, each detection frame of the to-be-detected image detected by the deep learning model is mapped, and the position not mapped by the standard sequence is marked as a missing position.

[0045] Specifically, each detection frame of the to-be-detected image detected by the deep learning model is filtered according to the confidence, and the detection frame with high confidence is reserved.

[0046] The reserved detection frame is sorted in the horizontal direction to form a detection sequence.

[0047] According to the greedy matching algorithm, each detection frame in the detection sequence is mapped to the detection frame corresponding to the standard sequence, and the position not mapped by the standard sequence is marked as a missing position.

[0048] The index and category label of the detection frame corresponding to the missing position are recorded, and potential false detection is identified. When the category label of the detection frame does not belong to any category in the standard sequence, or the number of detection frames exceeds the length of the standard sequence, it is determined that there is an extra detection; when the category of the detection frame mapped to a position is inconsistent with the standard category of the position, it is determined that there is a category error. For the detection frame suspected of false detection, the standard template corresponding to the position is used for multi-dimensional feature matching verification, and if the matching score is lower than the threshold, the detection frame is removed or the category of the detection frame is corrected.

[0049] S103, according to the position information of the detection frame adjacent to the missing position and the position information of each detection frame in the standard sequence, the position region of the missing position is determined.

[0050] Specifically, when there are detected targets on both left and right sides of the missing position, the total distance between the right boundary of the left target and the left boundary of the right target is calculated, and the missing position is divided into two parts according to the number of missing positions between them, and the corresponding search interval is assigned to the current missing position, and the search interval is appropriately expanded on the basis of the interval to increase the robustness.

[0051] When there is only a detected target on the left side of the missing position, the right boundary of the left target is taken as the starting point, and the average distance between the position and the left position in the standard sequence and the average width of the detection frame of the position are used to estimate the approximate X coordinate range of the target, and the search region is expanded to the right on the basis of the estimated range.

[0052] When there is only a detected target on the right side of the missing position, the left boundary of the right target is taken as the end point, and the average distance between the position and the right position in the standard sequence and the average width of the detection frame of the position are used to estimate the approximate X coordinate range of the target, and the search region is expanded to the right on the basis of the estimated range.

[0053] S104, the image feature information of the position region is extracted, and the image feature information of the position region is matched with the image feature information of the detected target in the detection frame corresponding to the standard sequence, and it is determined whether the matching is successful.

[0054] After determining the position region of the missing target in the to-be-tested image, the missing target is searched in the position region, and multi-dimensional feature matching is performed between the image feature information of the detected target in the detection frame corresponding to the standard sequence, and finally the matching region and the matching result of the detected target are determined. If the matching is successful, the detection is passed, otherwise the detection is failed.

[0055] The single-terminal line defect detection method based on image processing of the embodiment can further detect the to-be-detected image after being detected by the deep learning model. First, the standard sequence of each detection target of the single-terminal line is extracted from the template image. The order, category, size, and position of the detection frame in the standard sequence, and the image feature information of the detection target in the detection frame are extracted. The detection frame of the to-be-detected image after being detected by the deep learning model is mapped based on the standard sequence. The position that is not mapped by the standard sequence is marked as a missing position. Meanwhile, the misdetected and redundant detection frames are screened out. The redundant detection frames are deleted. The position region of the missing position or the misdetected detection frame is verified by multi-dimensional feature matching with the template image. The method can effectively compensate for the defects of the existing deep learning model, reduce the probability of misdetection and missed detection, and improve the detection accuracy of the single-terminal line defect based on image processing.

[0056] Referring to Figure 2 In one embodiment, the single-terminal line defect detection method based on image processing includes:

[0057] S201. A standard sequence in which each detection target appears is extracted from a template image of a single-terminal line detection frame. Feature information of each detection frame of the standard sequence and image feature information of the detection target in each detection frame are extracted.

[0058] S202. Image feature information of each detection target of a plurality of qualified template images is tested for mutual matching. Matching thresholds of the detection target of the to-be-detected image and the detection target in each detection frame of the standard sequence are set according to the matching result and the image complexity of each detection target.

[0059] Specifically, the image complexity of the detection target in each detection frame is calculated according to a plurality of template images that meet the standard sequence mode. The image complexity is comprehensively evaluated by the edge density and the gradient variance of the image. The template image is subjected to Canny edge detection. The proportion of edge pixels is calculated as the edge density. The standard deviation of the Sobel gradient is calculated as the gradient variance.

[0060] The plurality of sample templates corresponding to each detection target are tested for mutual matching. The minimum matching score, the average matching score, and the score standard deviation between the sample templates are calculated. The adaptive formula is used to calculate the matching threshold of each detection target in combination with the image complexity: the basic threshold = the minimum mutual matching score × α; the complexity adjustment = (1.0-complexity) × β; the standard deviation adjustment = the standard deviation × γ; the adaptive threshold of each detection target = the basic threshold + the complexity adjustment + the standard deviation adjustment, and is limited within a certain range. The adaptive threshold is used as the matching threshold of each detection target of the to-be-detected image and the detection target in each detection frame of the standard sequence.

[0061] S203, mapping each detection box of the to-be-tested image after detection by the deep learning model according to the standard sequence, and marking a position not mapped by the standard sequence as a missing position.

[0062] S204, determining a position region of the missing position according to position information of a detection box adjacent to the missing position and position information of each detection box in the standard sequence.

[0063] S205, extracting image feature information of the position region, performing multi-dimensional feature matching between the image feature information of the position region and image feature information of a detection target in a detection box corresponding to the standard sequence, and determining whether the matching is successful.

[0064] Specifically, the multi-dimensional feature matching includes multi-scale matching, multi-angle matching, and multi-feature matching; wherein the multi-feature matching specifically comprises: performing multi-feature matching between the image feature information of the position region and template image feature information under each scale and angle combination, and obtaining a feature comprehensive score by weighting and summing a matching score of each feature based on image complexity of the corresponding detection target; the template image is an image of a detection target in a detection box corresponding to the missing position in the standard sequence.

[0065] In a specific implementation, the multi-scale matching includes: scaling a template of each detection target at multiple scale factors such as [0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4] to generate template copies of different scales.

[0066] The multi-angle matching includes: rotating the template of each scale at multiple angles such as [-3°, 0°, 3°] to generate template copies of different angles.

[0067] The multi-feature matching includes: performing normalized cross-correlation template matching on the template of each scale-angle combination in the following four feature spaces respectively: 1. Gray feature matching: matching after converting the search region and the corresponding template into gray images; 2. RGB color feature matching: matching directly in the RGB three-channel color space; 3. HSV H channel matching: converting to the HSV color space and matching only in the H (hue) channel; 4. Color histogram matching: calculating a color histogram correlation coefficient of the search region and the corresponding template. The template is an image template of a detection target in each detection box of the standard sequence.

[0068] Feature comprehensive score: dynamically adjusting the weight of each feature according to the image complexity of the corresponding template and normalizing. For high-texture templates, the RGB and gray feature weights are higher; for low-texture templates, the color histogram feature weight is higher. The comprehensive score formula is: comprehensive score = RGB matching score × W RGB+ Gray matching score x W Gray + HSV matching score x W HSV + Histogram matching score x W Hist .

[0069] After determining the location area where the target in the image to be detected is missing, the target is searched in the location area, feature comprehensive matching is performed on the search area based on a plurality of scale-angle combined templates, template features are fully covered, detection accuracy is improved, the calculated feature comprehensive score is compared with the matching threshold of the detected target in each detection frame in the above standard sequence, if the feature comprehensive score is higher than the matching threshold, the matching is successful, indicating that the area detection is passed, otherwise the detection is not passed.

[0070] Further, for a detected target with an image complexity lower than a preset value, color segmentation assisted positioning and feature matching are further included.

[0071] In a specific implementation, for a low-texture template with a template image complexity of the detected target lower than 0.35, a color segmentation method is additionally used for assisted positioning. The average color of the template sample in the HSV space is calculated, color threshold segmentation is performed in the search area, a contour is extracted through morphological denoising, and the contour with an area and a width-height ratio meeting an expectation is selected as a candidate area. When the color segmentation positioning score is relatively high or the template matching score is relatively low, the result of the color segmentation positioning is preferentially used for positioning and feature matching of the search area.

[0072] S206, multi-dimensional feature verification is performed on the position area with a successful matching and the template image, the multi-dimensional verification includes color similarity verification, edge density verification, histogram similarity verification, and feature weighted comprehensive verification.

[0073] The matching area with a feature comprehensive score exceeding a matching threshold is further subjected to multi-dimensional verification to ensure the accuracy of the detection, and specifically:

[0074] Color similarity verification: the Euclidean distance between the average RGB color value of the matching area and the average color of the corresponding template at the position is calculated, and the smaller the distance is, the higher the similarity is. When the color distance exceeds 100, it is determined that the verification fails.

[0075] Edge density verification: Canny edge detection is performed on the matching area, and the proportion of edge pixels is calculated and compared with the average edge density of the corresponding template at the position. When the edge density difference is too large, a score is deducted.

[0076] Histogram similarity verification: the color histogram correlation coefficient of the matching area and all sample templates is calculated, and the maximum value is taken. The higher the correlation coefficient is, the higher the similarity is.

[0077] The comprehensive verification score is: color score x 0.3 + edge score x 0.2 + histogram score x 0.5. When the comprehensive verification score is lower than 0.35, it is determined that the verification fails, and the detection fails.

[0078] In S206, a supplementary detection box is generated in the position region where the matching succeeds, the class label, matching degree and position coordinates of the supplementary detection box are output, and the supplementary detection box is marked.

[0079] For the matching result passing all verifications, a supplementary detection box is generated in the matching region where the matching succeeds, the average size of the position template is used for the width and height, the Y coordinate is aligned in the center by referring to the Y coordinate of the adjacent detected target, and the right boundary does not exceed the left boundary of the right detected target to avoid overlapping. Finally, the class label, matching degree, position coordinates of the supplementary detection box are output, and the supplementary detection box is marked. The matching degree is the feature comprehensive score of the missing position.

[0080] The single-terminal line defect detection method based on image processing in the embodiment of the application searches for a missing target in a position region where a missing position is located after detecting the missing position of a to-be-detected image according to a standard sequence, performs feature comprehensive matching on an image of a search region based on a plurality of scale-angle combination templates, fully covers template features, and improves detection accuracy. For a detection target with low image complexity, color segmentation is combined for auxiliary positioning and feature matching, which can improve detection efficiency. The matching region with a feature comprehensive score exceeding a matching threshold is verified in multiple dimensions, so that the detection result is more accurate. Finally, a supplementary detection box is generated in the position region where the matching succeeds, complete information of the missing position is recorded and displayed, which is convenient for archiving and viewing.

[0081] Referring to Figure 3 The embodiment of the application also provides a single-terminal line defect detection system based on image processing. In one embodiment, the system comprises:

[0082] A standard sequence extraction unit 10 is configured to extract a standard sequence of each detection target from a template image of a single-terminal line detection box, and extract feature information of each detection box of the standard sequence and image feature information of the detection target in each detection box.

[0083] A missing position detection unit 20 is configured to map each detection box of a to-be-detected image after being detected by a deep learning model according to the standard sequence, and mark a position not mapped by the standard sequence as a missing position.

[0084] A position region determination unit 30 is configured to determine a position region of the missing position according to position information of a detection box adjacent to the missing position and position information of each detection box in the standard sequence.

[0085] The feature matching unit 40 is configured to extract image feature information of the position area, perform multi-dimensional feature matching between the image feature information of the position area and image feature information of a detection target in a detection frame corresponding to the standard sequence, and determine whether the matching is successful.

[0086] In one of the embodiments, the single-terminal line defect detection system based on image processing further comprises:

[0087] The matching threshold calculation unit is configured to perform mutual matching test on image feature information of detection targets of a plurality of qualified template images, and set matching thresholds of detection targets of the to-be-tested image and detection targets in detection frames of the standard sequence according to matching results and image complexities of the detection targets, respectively.

[0088] In one of the embodiments, the missing position detection unit 20 is specifically configured to:

[0089] Filtering detection frames of the to-be-tested image after detection by the deep learning model according to confidence levels, and retaining detection frames with high confidence levels;

[0090] Sorting the retained detection frames according to horizontal directions to form a detection sequence;

[0091] According to a greedy matching algorithm, mapping each detection frame in the detection sequence to a detection frame corresponding to the standard sequence, and marking a position that is not mapped by the standard sequence as a missing position.

[0092] In one of the embodiments, the multi-dimensional feature matching comprises multi-scale matching, multi-angle matching and multi-feature matching.

[0093] In one of the embodiments, the multi-feature matching comprises multi-scale matching, multi-angle matching and multi-feature matching.

[0094] In one of the embodiments, the single-terminal line defect detection system based on image processing further comprises:

[0095] The color segmentation auxiliary unit is configured to, for a detection target with an image complexity lower than a preset value, further include color segmentation auxiliary positioning and feature matching.

[0096] In one of the embodiments, the single-terminal line defect detection system based on image processing further comprises:

[0097] The matching verification unit is configured to perform multi-dimensional feature verification on the position area with a successful match and the template image, and the multi-dimensional verification includes color similarity verification, edge density verification, histogram similarity verification, and feature weighted comprehensive verification.

[0098] In one of the embodiments, the single-terminal line defect detection system based on image processing further comprises:

[0099] The supplementary frame generation unit is configured to generate a supplementary detection frame in the position area with a successful match, output a class label, a matching degree and position coordinates of the supplementary detection frame, and mark the supplementary detection frame.

[0100] In the above method embodiments, the specific processes of each unit performing the corresponding steps have been described in detail, and thus will not be repeated here for the sake of brevity.

[0101] The embodiments of the present application further provide a computer device, which comprises a processor and a memory, and the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the method steps of any of the method embodiments.

[0102] The embodiments of the present application further provide a computer storage medium, which stores a computer program, and the computer program is executed to implement the method steps of any of the method embodiments.

[0103] In the above embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0104] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment scheme.

[0105] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0106] Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a mobile terminal, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0107] In summary, although the present application has been disclosed with preferred embodiments as above, the protection scope of the present application is not limited thereto, and any person skilled in the art, according to the technical concept of the technical solutions of the present application, can make equivalent replacements or changes within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

[0108] The technical features of the above-described embodiments can be combined arbitrarily, and in order to make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

Claims

1. A single-terminal line defect detection method based on image processing, characterized by, The method comprises the following steps: extracting a standard sequence of each detection target appearing from a template image of a single-terminal wire belt detection frame, and extracting feature information of each detection frame of the standard sequence and image feature information of the detection target in each detection frame; performing mutual matching test on the image feature information of each detection target of a plurality of qualified template images, and setting a matching threshold of each detection target of a to-be-tested image and the detection target in each detection frame of the standard sequence according to a matching result and image complexity of each detection target; filtering each detection frame of the to-be-tested image after being detected by a deep learning model according to a confidence level, and retaining a detection frame with a high confidence level; sorting the retained detection frames in a horizontal direction to form a detection sequence; mapping each detection frame in the detection sequence to a corresponding detection frame of the standard sequence according to a greedy matching algorithm, and marking a position not mapped by the standard sequence as a missing position; determining a position region of the missing position according to position information of detection frames adjacent to the missing position and position information of each detection frame in the standard sequence; extracting image feature information of the position region, performing multi-dimensional feature matching on the image feature information of the position region and image feature information of the detection target in the corresponding detection frame of the standard sequence, and determining whether the matching is successful, wherein the multi-dimensional feature matching comprises multi-scale matching, multi-angle matching and multi-feature matching; wherein the multi-feature matching specifically comprises performing multi-feature matching on the image feature information of the position region and template image feature information under each scale and angle combination, and obtaining a feature comprehensive score by weighting and summing a matching score of each feature based on image complexity of the corresponding detection target; and the template image is an image of the detection target in the detection frame corresponding to the missing position of the standard sequence.

2. The method of claim 1, wherein, For the detection target with image complexity lower than a preset value, the method further comprises color segmentation auxiliary positioning and feature matching.

3. The method of claim 1, wherein, After the image feature information of the position region is extracted, the image feature information of the position region is matched with the image feature information of the detection target in the corresponding detection frame of the standard sequence in a multi-dimensional feature matching manner, and it is determined whether the matching is successful, the method further comprises the following steps: performing multi-dimensional feature verification on the position region with successful matching and the template image, wherein the multi-dimensional feature verification comprises color similarity verification, edge density verification, histogram similarity verification and feature weighted comprehensive verification.

4. The method of claim 1, wherein, After the image feature information of the position region is extracted, the image feature information of the position region is matched with the image feature information of the detection target in the corresponding detection frame of the standard sequence in a multi-dimensional feature matching manner, and it is determined whether the matching is successful, the method further comprises the following steps: generating a supplementary detection frame in the position region with successful matching, outputting a category label, a matching degree and a position coordinate of the supplementary detection frame, and marking the supplementary detection frame.

5. An image processing based single terminal wire defect detection system, characterized in that, The method comprises the following steps: a standard sequence extraction unit is configured to extract a standard sequence of each detection target appearing from a template image of a single-terminal wire belt detection frame, and extract feature information of each detection frame of the standard sequence and image feature information of the detection target in each detection frame; The matching threshold calculation unit is configured to perform mutual matching test on image feature information of each detection target of a plurality of qualified template images, and set a matching threshold of each detection target of the image to be detected and a detection target in each detection frame of the standard sequence according to a matching result and image complexity of each detection target. The missing position detection unit is configured to filter each detection frame of the image to be detected after being detected by the deep learning model according to a confidence level, and retain a detection frame with a high confidence level. The retained detection frame is sorted in a horizontal direction to form a detection sequence. According to a greedy matching algorithm, each detection frame in the detection sequence is mapped to a corresponding detection frame of the standard sequence, and a position not mapped by the standard sequence is marked as a missing position. The position region determination unit is configured to determine a position region of the missing position according to position information of detection frames adjacent to the missing position and position information of each detection frame of the standard sequence. The feature matching unit is configured to extract image feature information of the position region, perform multi-dimensional feature matching on the image feature information of the position region and image feature information of a detection target in a corresponding detection frame of the standard sequence, and determine whether the matching is successful, wherein the multi-dimensional feature matching includes multi-scale matching, multi-angle matching and multi-feature matching. The multi-feature matching specifically includes multi-feature matching of the image feature information of the position region and template image feature information under each scale and angle combination, and obtaining a feature comprehensive score by weighting and summing a matching score of each feature based on image complexity of a corresponding detection target; and the template image is an image of a detection target in a detection frame corresponding to the missing position of the standard sequence.

6. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores a computer program, which, when loaded and executed by the processor, implements the method of any one of claims 1 to 4.

7. A computer storage medium, characterized in that The computer storage medium stores a computer program, which, when executed, implements the method of any one of claims 1 to 4.

Images