Anti-vibration hammer bolt detection method, device and electronic equipment

Abstract

The present invention provides a method, device and electronic device for detecting anti-vibration hammer bolts, the method comprising: extracting a straight-line fitting area of ​​an anti-vibration hammer wire from an image to be detected, and determining the anti-vibration hammer contour based on the straight-line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; determining an anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and performing target detection on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt. The present invention determines the anti-vibration hammer contour based on the straight-line fitting area of ​​the anti-vibration hammer wire, and determines the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, so that the position detection result of the anti-vibration hammer bolt can be accurately obtained based on the anti-vibration hammer bolt area map, avoiding the problem in the traditional method that the bolt position cannot be accurately detected due to the unclear geometric features of the anti-vibration hammer bolt.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method, a device and an electronic device for detecting anti-vibration hammer bolts. Background Art

[0002] After a long period of operation, the anti-vibration hammer in the overhead transmission line will be offset due to the vibration of the conductor, which will make the anti-vibration hammer function ineffective. Therefore, the anti-vibration hammer needs to be reset. The robot needs to loosen and tighten the anti-vibration hammer bolts during the anti-vibration hammer reset task, so the anti-vibration hammer bolt detection is required.

[0003] Since the geometric features of anti-vibration hammer bolts are not obvious and the edge features are not highly distinguishable, the traditional detection methods are not very robust. At present, deep learning algorithms are also used in anti-vibration hammer bolt detection, but in the transmission line scenario, anti-vibration hammer bolts are small targets, and deep learning algorithms have low accuracy for small target detection. Summary of the invention

[0004] The invention provides a method, a device and an electronic device for detecting an anti-vibration hammer bolt, which are used to solve the defect of low detection accuracy of the anti-vibration hammer bolt in an overhead power transmission line environment in the prior art.

[0005] The present invention provides a method for detecting a vibration-proof hammer bolt, comprising:

[0006] Extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire;

[0007] Based on the anti-vibration hammer profile and the structural dimensions of the anti-vibration hammer, an anti-vibration hammer bolt area map is determined, and target detection is performed on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt.

[0008] According to a vibration-proof hammer bolt detection method provided by the present invention, the straight line fitting area of ​​the vibration-proof hammer wire is extracted from the image to be detected, and the contour of the vibration-proof hammer is determined based on the straight line fitting area and the positional relationship between the vibration-proof hammer and the vibration-proof hammer wire, including:

[0009] Extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine the straight line fitting area;

[0010] Based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, the straight line fitting area is expanded to determine a second region of interest map of the anti-vibration hammer;

[0011] Perform edge detection on the second region of interest image to determine the contour of the anti-vibration hammer.

[0012] According to a vibration-proof hammer bolt detection method provided by the present invention, extracting a first region of interest map of the vibration-proof hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine the straight line fitting area includes:

[0013] Performing image preprocessing on the first region of interest image, and performing edge point detection on the preprocessed first region of interest image to obtain a wire edge feature set; the wire edge feature set includes a plurality of wire edge points;

[0014] The target straight line endpoints are determined from the plurality of wire edge points, and straight line fitting is performed on each target straight line endpoint to determine the straight line fitting area.

[0015] According to a vibration-proof hammer bolt detection method provided by the present invention, the image preprocessing of the first region of interest map includes:

[0016] The first image of interest is subjected to grayscale processing, noise reduction processing, equalization processing and de-texturing processing in sequence.

[0017] According to a vibration-proof hammer bolt detection method provided by the present invention, the target straight line endpoints are determined from the multiple wire edge points, and straight line fitting is performed on each target straight line endpoint to determine the straight line fitting area, including:

[0018] Performing Hough transformation on each wire edge point, and using the wire edge points that meet preset conditions after Hough transformation as straight line search points, wherein the preset condition is that the number of intersecting Hough space curves is greater than a threshold;

[0019] Determine the direction of the line in Cartesian space based on the line search point, and determine the candidate line endpoints from each wire edge point based on the line direction;

[0020] Determine the straight line length based on the candidate straight line endpoints, if the straight line length is greater than a preset length, use the corresponding candidate straight line endpoints as the target straight line endpoints, and perform straight line fitting on each target straight line endpoint to obtain a wire edge straight line set;

[0021] Based on the slope of each straight line in the wire edge straight line set, each straight line is classified to obtain a left edge straight line set and a right edge straight line set;

[0022] Based on the left edge straight line set and the right edge straight line set, least square fitting is performed on the first region of interest map to determine the straight line fitting region.

[0023] According to a vibration-proof hammer bolt detection method provided by the present invention, the edge detection is performed on the second region of interest image to determine the contour of the vibration-proof hammer, including:

[0024] Performing edge detection on the second region of interest map to obtain an anti-vibration hammer edge map, and determining the circumferential gradient of each non-zero point in the anti-vibration hammer edge map;

[0025] Determine a circular gradient straight line based on the circumferential gradient of each non-zero point, and determine an accumulation matrix based on the circular gradient straight line;

[0026] If the difference between any point in the accumulation matrix and the threshold is greater than zero, the corresponding point in the accumulation matrix is ​​used as a candidate circle center point, and the candidate circle center point is compared with the neighborhood to determine the target circle center point;

[0027] Based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map, a candidate anti-vibration hammer profile is determined, and the candidate anti-vibration hammer profile that meets the preset constraint condition is used as the anti-vibration hammer profile.

[0028] According to a vibration-proof hammer bolt detection method provided by the present invention, determining a candidate vibration-proof hammer profile based on the distance between the target circle center point and each vibration-proof hammer edge point in the vibration-proof hammer edge map includes:

[0029] Determine the candidate circle radius value based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map;

[0030] Count the radius values ​​of each candidate circle, and take the candidate circle radius value whose count value is greater than the preset value as the valid circle radius value;

[0031] The effective circle radius value is used as the radius, and the corresponding target circle center point is used as the circle center to obtain an effective circular contour, and the effective circular contour is used as the candidate anti-vibration hammer contour.

[0032] According to a vibration-proof hammer bolt detection method provided by the present invention, the target detection is performed on the vibration-proof hammer bolt area map to obtain the position detection result of the vibration-proof hammer bolt, including:

[0033] Inputting the anti-vibration hammer bolt area map into a target detection model to obtain the position detection result output by the target detection model;

[0034] Among them, the target detection model is obtained by training based on a sample anti-vibration hammer bolt area map and a sample position detection result; the target detection model is a lightweight model obtained by sparsely training the YOLOv3 model.

[0035] The present invention also provides an anti-vibration hammer bolt detection device, comprising:

[0036] A contour determination unit, used for extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire;

[0037] The bolt detection unit is used to determine the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and perform target detection on the anti-vibration hammer bolt area map to obtain the position detection result of the anti-vibration hammer bolt.

[0038] The present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned anti-vibration hammer bolt detection methods are implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned anti-vibration hammer bolt detection methods are implemented.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned anti-vibration hammer bolt detection methods are implemented.

[0041] The anti-vibration hammer bolt detection method, device and electronic equipment provided by the present invention determine the anti-vibration hammer contour based on the straight line fitting area of ​​the anti-vibration hammer wire, and determine the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, so that the position detection result of the anti-vibration hammer bolt can be accurately obtained based on the anti-vibration hammer bolt area map, avoiding the problem in traditional methods that the bolt position cannot be accurately detected due to the unclear geometric features of the anti-vibration hammer bolt. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is one of the flow charts of the anti-vibration hammer bolt detection method provided by the present invention;

[0044] Figure 2 It is a schematic flow chart of a first region of interest image preprocessing method provided by the present invention;

[0045] Figure 3It is a schematic diagram of the flow of the method for obtaining the straight line fitting area provided by the present invention;

[0046] Figure 4 It is a schematic flow chart of the method for obtaining the profile of the anti-vibration hammer provided by the present invention;

[0047] Figure 5 It is a flow chart of the lightweight model acquisition method provided by the present invention;

[0048] Figure 6 This is the second flow chart of the anti-vibration hammer bolt detection method provided by the present invention;

[0049] Figure 7 It is a structural schematic diagram of the anti-vibration hammer bolt detection device provided by the present invention;

[0050] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Since the geometric features of anti-vibration hammer bolts are not obvious and the edge features are not highly distinguishable, the traditional detection methods are not very robust. At present, deep learning algorithms are also used in anti-vibration hammer bolt detection, but in the transmission line scenario, anti-vibration hammer bolts are small targets, and deep learning algorithms have low accuracy for small target detection. At the same time, deep learning algorithms deployed on embedded devices have problems such as low operating efficiency and severe memory consumption.

[0053] To this end, the present invention provides a method for detecting anti-vibration hammer bolts. Figure 1 : is a flow chart of the anti-vibration hammer bolt detection method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0054] Step 110: extract the straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determine the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire.

[0055] Specifically, the image to be detected refers to an image containing information about the anti-vibration hammer, the anti-vibration hammer wire and the anti-vibration hammer bolt, wherein the image to be detected can be a color image obtained by a camera, or a video frame image captured from a video obtained by a camera.

[0056] The straight-line fitting area of ​​the anti-vibration hammer wire refers to a closed area determined by the contour of the anti-vibration hammer wire, and both ends of the anti-vibration hammer wire are usually connected to the anti-vibration hammer. Therefore, the contour of the anti-vibration hammer can be obtained based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire (such as the installation position relationship between the anti-vibration hammer and the anti-vibration hammer wire) and the straight-line fitting area of ​​the anti-vibration hammer wire.

[0057] For example, based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, a region of interest map of the anti-vibration hammer can be obtained, and the contour of the anti-vibration hammer can be extracted from the region of interest map of the anti-vibration hammer.

[0058] Step 120: Based on the anti-vibration hammer profile and the structural dimensions of the anti-vibration hammer, determine the anti-vibration hammer bolt area map, perform target detection on the anti-vibration hammer bolt area map, and obtain the anti-vibration hammer bolt position detection result.

[0059] Specifically, since the structural dimensions of the anti-vibration hammer include the position information of the anti-vibration hammer bolt, the anti-vibration hammer bolt area map can be determined based on the contour of the anti-vibration hammer and the structural dimensions of the anti-vibration hammer, so that target detection can be performed on the anti-vibration hammer bolt area map. For example, the anti-vibration hammer bolt can be input into the target detection model to obtain the position detection result of the anti-vibration hammer bolt.

[0060] It can be seen that the embodiment of the present invention determines the contour of the anti-vibration hammer based on the straight-line fitting area of ​​the anti-vibration hammer wire, and determines the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, thereby avoiding the problem of the inability to accurately detect the bolt position due to the unclear geometric features of the anti-vibration hammer bolt in the traditional method. The embodiment of the present invention first extracts the straight-line fitting area of ​​the anti-vibration hammer wire that is easy to identify, and then accurately determines the contour of the anti-vibration hammer based on the straight-line fitting area, and finally locates the anti-vibration hammer bolt according to the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, thereby improving the detection accuracy of the anti-vibration hammer bolt.

[0061] The anti-vibration hammer bolt detection method provided in an embodiment of the present invention determines the anti-vibration hammer contour based on the straight line fitting area of ​​the anti-vibration hammer wire, and determines the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer. Therefore, the position detection result of the anti-vibration hammer bolt can be accurately obtained based on the anti-vibration hammer bolt area map, avoiding the problem in traditional methods that the bolt position cannot be accurately detected due to the unclear geometric features of the anti-vibration hammer bolt.

[0062] Based on the above embodiment, extracting the straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, including:

[0063] Extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine a straight line fitting region;

[0064] Based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, the straight line fitting area is expanded to determine a second region of interest map of the anti-vibration hammer;

[0065] Perform edge detection on the second region of interest image to determine the contour of the anti-vibration hammer.

[0066] Specifically, since the anti-vibration hammer is fixed under the transmission line, in order to obtain the accurate outline of the anti-vibration hammer, the first region of interest map of the anti-vibration hammer wire can be determined by detecting the transmission line. Specifically, a histogram similarity matching search is performed on the central area of ​​the detected image in a sliding window manner to obtain an aluminum stranded wire region map, and the aluminum stranded wire region map is expanded with a fixed pixel area to obtain a first region of interest map.

[0067] Among them, the histogram similarity is measured by Bhattacharyya distance, and its calculation method is shown in formula (1) and formula (2):

[0068]

[0069]

[0070] Among them, H dest The histogram representing the target image block can be obtained by calculating the histogram of the fixed aluminum strand texture map. search Represents the histogram of the search area block, H dest and H search The image blocks of the histograms represented are of equal size, I and J represent the grayscale value ranges of the target image block histogram and the search area block histogram, respectively, and N represents the number of histogram bars. Represents the normalization coefficient.

[0071] After obtaining the first area of ​​interest map, based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, quantitative expansion can be performed in the x-axis direction of the image to be detected to determine the second area of ​​interest map of the anti-vibration hammer, and edge detection can be performed on the second area of ​​interest map to determine the contour of the anti-vibration hammer.

[0072] Based on any of the above embodiments, extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine a straight line fitting region includes:

[0073] Performing image preprocessing on the first region of interest image, and performing edge point detection on the preprocessed first region of interest image to obtain a wire edge feature set; the wire edge feature set includes a plurality of wire edge points;

[0074] The target straight line endpoints are determined from a plurality of wire edge points, and a straight line fitting is performed on each target straight line endpoint to determine a straight line fitting area.

[0075] Specifically, after obtaining the first area of ​​interest map, image preprocessing (such as grayscale processing, noise reduction processing, etc.) can be performed on it, so that edge point detection can be accurately performed based on the preprocessed first area of ​​interest map, and the edge features of the anti-vibration hammer wire can be obtained to form a wire edge feature set.

[0076] The wire edge feature set includes the features of the wire and the objects around the wire. To obtain the wire, the interfering features need to be excluded, the target straight line endpoints of the wire need to be extracted, and straight line fitting is performed on each target straight line endpoint to determine the straight line fitting area.

[0077] Based on any of the above embodiments, performing image preprocessing on the first region of interest map includes:

[0078] The first image of interest is subjected to grayscale processing, noise reduction processing, equalization processing and de-texturing processing in sequence.

[0079] Specifically, the noise around the wire and the wire texture will interfere with the extraction of the straight line features of the wire edge. Therefore, image preprocessing is needed to improve this problem. Image preprocessing mainly uses five methods. The first region of interest image is grayed and Gaussian filtered to suppress image noise, then histogram equalization is performed to enhance the image, morphological closing operation is used to eliminate the wire surface texture, and finally the Canny edge detection algorithm is used to detect edge points.

[0080] like Figure 2 As shown, the first region of interest image is grayed to obtain the first region of interest grayscale image. The first region of interest grayscale image is Gaussian filtered to obtain a noise reduction image.

[0081] The grayscale image of the first region of interest is denoised by two-dimensional Gaussian filtering, and the method is shown in formula (3):

[0082]

[0083] Where A represents the amplitude, (x0, y0) represents the center coordinate of the Gaussian filter, and σ x and σ y Represent the variance in the x-axis and y-axis directions respectively. Optionally, σ x and σ y Set to 2.

[0084] After obtaining the denoised image, the denoised image is subjected to histogram equalization to obtain an equalized image, and then a morphological closing operation is performed on the equalized image to obtain a wire detexturized image. Canny edge detection is performed on the wire detexturized image to obtain a wire edge feature set.

[0085] It can be seen that the embodiment of the present invention eliminates the interference of the surrounding environment of the wire and the surface texture of the wire through grayscale, Gaussian filtering noise reduction, histogram equalization and morphological closing operation, which is conducive to extracting an accurate set of wire edge features and further improves the accuracy of the obtained straight line fitting area.

[0086] Based on any of the above embodiments, determining target straight line endpoints from a plurality of wire edge points, and performing straight line fitting on each target straight line endpoint to determine a straight line fitting area includes:

[0087] Perform Hough transform on each wire edge point, and use the wire edge points that meet preset conditions after Hough transform as straight line search points. The preset condition is that the number of intersecting Hough space curves is greater than a threshold value.

[0088] Determine the direction of the line in Cartesian space based on the line search point, and determine the candidate line endpoints from each wire edge point based on the line direction;

[0089] Determine the straight line length based on the candidate straight line endpoints. If the straight line length is greater than the preset length, use the corresponding candidate straight line endpoints as target straight line endpoints, and perform straight line fitting on each target straight line endpoint to obtain a wire edge straight line set.

[0090] Based on the slope of each straight line in the wire edge straight line set, each straight line is classified to obtain a left edge straight line set and a right edge straight line set;

[0091] Based on the left edge straight line set and the right edge straight line set, a least squares fitting is performed on the first region of interest map to determine a straight line fitting region.

[0092] Specifically, the wire edge straight line set is determined based on the following steps:

[0093] Step a1: mark and detect the edge points in the wire edge feature set. If all edge points have been marked, proceed to step e1; otherwise, obtain the extraction points.

[0094] Step b1, perform Hough transform on the extracted points, and find the maximum number of Hough space curves intersecting at the point (ρ0, θ0). If the number of intersecting curves at the point (ρ0, θ0) is less than the threshold T, return to step a; otherwise, obtain the extracted point set and proceed to step c1.

[0095] The Hough transform uses polar coordinates to express the linear expression, and the method is shown in formula (4):

[0096] ρ=x0cosθ+y0sinθ (4)

[0097] Wherein, ρ represents the polar diameter of the polar coordinates, θ represents the polar angle of the polar coordinates, and x0 and y0 represent the values ​​of the extracted point in the x-axis direction and the y-axis direction in the Cartesian coordinate system, respectively.

[0098] Step c1, based on the point (ρ0, θ0), obtain the direction of the line in Cartesian space, and find the middle point of the line and the candidate line endpoints in the extracted point set based on the direction of the line.

[0099] Step d1, based on the candidate straight line endpoints, calculate the straight line length. If the straight line length is greater than the minimum straight line length L, the corresponding candidate straight line endpoint is used as the target straight line endpoint, and the points in the extracted point set are marked; otherwise, return to step a1.

[0100] Step e1: sum up the target straight line endpoints to obtain a wire edge straight line set.

[0101] Since the distribution of the straight lines in the wire edge straight line set on the image is discontinuous, it is impossible to directly determine the wire area. This problem needs to be solved by distinguishing the straight lines as the left and right edges of the wire and fitting them, such as Figure 3 As shown in the figure, after obtaining the wire edge straight line set, the straight line fitting area is obtained based on the following steps:

[0102] Step a2, calculating the slope of each straight line in the wire edge straight line set to obtain a straight line slope set.

[0103] Step b2, performing bi-means clustering on the straight line slope set to obtain a left edge straight line slope set and a right edge straight line slope set.

[0104] Step c2, dividing the wire edge straight line set based on the left edge straight line slope set and the right edge straight line slope set to obtain the left edge straight line set and the right edge straight line set.

[0105] Step d2, performing least square fitting on the wires in the first region of interest map based on the left edge straight line set and the right edge straight line set to obtain a straight line fitting region.

[0106] Based on any of the above embodiments, edge detection is performed on the second region of interest map to determine the contour of the anti-vibration hammer, including:

[0107] Performing edge detection on the second region of interest map to obtain an anti-vibration hammer edge map, and determining the circumferential gradient of each non-zero point in the anti-vibration hammer edge map;

[0108] Determine a circular gradient straight line based on the circular gradient of each non-zero point, and determine a cumulative matrix based on the circular gradient straight line;

[0109] If the difference between any point in the accumulation matrix and the threshold is greater than zero, the corresponding point in the accumulation matrix is ​​used as a candidate circle center point, and the candidate circle center point is compared with the neighborhood to determine the target circle center point;

[0110] Based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map, a candidate anti-vibration hammer profile is determined, and the candidate anti-vibration hammer profile that meets the preset constraint conditions is used as the anti-vibration hammer profile.

[0111] Specifically, after obtaining the straight line fitting area, the second region of interest of the anti-vibration hammer can be expanded according to the straight line fitting area and the contour of the anti-vibration hammer can be extracted, such as Figure 4 As shown, the vibration damper profile is determined based on the following steps:

[0112] Step a3: quantitatively expand the straight line fitting area along the x-axis direction of the camera color image to obtain a second region of interest image of the anti-vibration hammer.

[0113] Step b3: performing edge detection on the second region of interest map to obtain an edge map, and performing gradient calculation based on each non-zero point in the edge map to obtain a circular gradient.

[0114] Step c3, obtaining a graphic gradient straight line based on the circular gradient and the non-zero point, and accumulating each point on the graphic gradient straight line to obtain an accumulation matrix.

[0115] Step d3, summing the number of points in the matrix and the threshold T c The points whose difference is greater than zero are taken as candidate center points, and the candidate center points are compared with the 4 neighborhoods. If the candidate center point is greater than the value in any of the 4 neighborhoods, the candidate center point is determined to be the target center point; otherwise, the invalid candidate center points are eliminated.

[0116] Step e3, traverse the distance values ​​from the edge points of each anti-vibration hammer to the center point of the target circle, and calculate the minimum radius R according to the set value. min and the maximum radius R max Filter and sort the distance values ​​to obtain the circle radius value.

[0117] Step f3, count the circle radius values, and when the count value is greater than the threshold value T r The radius value is used as the effective circle radius.

[0118] Step g3, repeat steps e3 to g3 until all target circle center points and effective circle radii are calculated to obtain the circle to be screened.

[0119] Step h3, performing regional condition constraints on the circle to be screened to obtain the contour of the anti-vibration hammer.

[0120] Among them, the regional condition constraint is applied to the circle to be screened, and the method is shown in equations (5), (6) and (7):

[0121] r1<r<r2 (5)

[0122] x α -d1<x<x β +d1 (6)

[0123] d1=k|x β -x α | (7)

[0124] Among them, r is the radius of the circle to be screened, r1 and r2 are the minimum radius and maximum radius of the circle to be screened, respectively, and x is the x-axis coordinate value of the circle to be screened. α 、x β are the minimum and maximum x-axis coordinates of the fitted wire, d1 is the area range value, k is the area range weight, and optionally, k is set to 3.

[0125] It can be seen that the embodiment of the present invention obtains the second region of interest of the vibration-proof hammer by expanding the straight-line fitting area and setting regional constraints, so that the range of vibration-proof hammer contour detection is more concentrated, eliminating the interference of some circular objects in the transmission line scenario, and the accuracy of vibration-proof hammer detection is significantly higher.

[0126] Based on any of the above embodiments, determining a candidate anti-vibration hammer profile based on a distance between a target circle center point and each anti-vibration hammer edge point in an anti-vibration hammer edge map includes:

[0127] Determine the candidate circle radius value based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map;

[0128] Count the radius values ​​of each candidate circle, and take the candidate circle radius value whose count value is greater than the preset value as the valid circle radius value;

[0129] The effective circle radius value is used as the radius, and the corresponding target circle center point is used as the circle center to obtain the effective circle contour, and the effective circle contour is used as the candidate anti-vibration hammer contour.

[0130] Specifically, the distance values ​​from each anti-vibration hammer edge point to the target circle center point in the anti-vibration hammer edge map are traversed, and the minimum radius R is set. min and the maximum radius R max The distance values ​​are screened and sorted to obtain candidate circle radius values.

[0131] Next, the candidate circle radius values ​​are counted, and the value greater than the threshold T rThe radius value of the effective circle is taken as the radius value of the effective circle. The effective circle radius value is taken as the radius, and the corresponding target circle center point is taken as the circle center to obtain the effective circle contour, and the effective circle contour is taken as the candidate anti-vibration hammer contour.

[0132] Based on any of the above embodiments, target detection is performed on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt, including:

[0133] The anti-vibration hammer bolt area map is input into the target detection model to obtain the position detection result output by the target detection model;

[0134] Among them, the target detection model is trained based on the sample anti-vibration hammer bolt area map and the sample position detection results; the target detection model is a lightweight model obtained by sparse training of the YOLOv3 model.

[0135] Specifically, based on the circular contour and structure of the anti-vibration hammer, the approximate position of the bolt can be estimated, and the anti-vibration hammer bolt area map is obtained. The YOLOv3 model after model compression is used to perform target detection on the anti-vibration hammer bolt area map to obtain the bolt position detection result, which specifically includes:

[0136] Step a4, dividing the regions of fixed size based on the circular contour of the anti-vibration hammer and the structure of the anti-vibration hammer to obtain an anti-vibration hammer bolt region map.

[0137] Step b4: perform bolt detection on the anti-vibration hammer bolt area map based on the model compressed YOLOv3 model to obtain a position detection result.

[0138] It should be noted that the YOLOv3 model without model compression has a large parameter capacity and a slow inference speed in embedded devices. Therefore, the YOLOv3 model needs to be compressed. Figure 5 As shown, the process of compressing the YOLOv3 model is as follows:

[0139] Step a5: Perform YOLOv3 model training based on the sample anti-vibration hammer bolt area map and the sample position detection results to obtain a basic model.

[0140] Step b5: perform sparse training on the basic model to obtain a sparse model.

[0141] The basic model is sparsely trained as shown in formula (8):

[0142]

[0143] Among them, x and y are the input and target of the training respectively, W is the weight of the training model, ∑ (x，y) l(f(x, W), y) is the training loss function of the convolutional layer, γ is the scaling factor of the batch normalization layer, ∑γ∈Γ g(γ) is the penalty term of the scaling factor, and λ is the balancing factor.

[0144] Step c5, sorting the scaling factor of each channel in the sparse model to obtain a scaling factor sorting table.

[0145] Step d5, based on the scaling factor ranking table, the scaling factor channels whose quantity approaches zero are trimmed to obtain a trimmed model.

[0146] Step e5: fine-tune the cropped model to obtain a lightweight model.

[0147] It can be seen that the embodiment of the present invention obtains the bolt area map through the anti-vibration hammer contour and the anti-vibration hammer structure size, and then performs detection by YOLOv3 based on model compression, which improves the loading rate and detection frame rate of the YOLOv3 model while ensuring the anti-vibration hammer bolt detection accuracy, and avoids the problems of low operating efficiency and serious memory consumption in the traditional method of deploying deep learning algorithms on embedded devices.

[0148] Based on any of the above embodiments, the present invention also provides a method for detecting anti-vibration hammer bolts, such as Figure 6 As shown, the method includes:

[0149] First, a camera color image is obtained, and the wire region of interest in the color image is extracted to obtain a wire region of interest map; the wire region of interest map is image preprocessed to obtain a wire edge feature set;

[0150] Next, the wire edge straight line features in the wire edge feature set are extracted to obtain a wire edge straight line feature set, and based on the wire edge straight line feature set, straight line fitting is performed on the wires in the wire region of interest map to obtain a wire fitting region;

[0151] Then, the wire fitting area is expanded to obtain an area of ​​interest map of the anti-vibration hammer, and the anti-vibration hammer is extracted from the area of ​​interest map of the anti-vibration hammer to obtain a circular contour of the anti-vibration hammer.

[0152] Finally, the bolt region of interest map is obtained based on the circular contour and structure of the anti-vibration hammer, and the bolt region of interest map is detected based on the YOLOv3 model after model compression to obtain the bolt position.

[0153] The anti-vibration hammer bolt detection device provided by the present invention is described below. The anti-vibration hammer bolt detection device described below and the anti-vibration hammer bolt detection method described above can be referred to each other.

[0154] Based on any of the above embodiments, the present invention provides a vibration-proof hammer bolt detection device, such as Figure 7 As shown, the device comprises:

[0155] The contour determining unit 710 is used to extract the straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determine the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire;

[0156] The bolt detection unit 720 is used to determine the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and perform target detection on the anti-vibration hammer bolt area map to obtain the anti-vibration hammer bolt position detection result.

[0157] Based on any of the above embodiments, the contour determining unit 710 includes:

[0158] A straight line fitting region determining unit, used for extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine the straight line fitting region;

[0159] An anti-vibration hammer region of interest determination unit, configured to expand the straight line fitting region based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, and determine a second region of interest map of the anti-vibration hammer;

[0160] The anti-vibration hammer profile determining unit is used to perform edge detection on the second region of interest image to determine the anti-vibration hammer profile.

[0161] Based on any of the above embodiments, the straight line fitting area determination unit includes:

[0162] An edge feature determination unit is used to perform image preprocessing on the first region of interest image, and perform edge point detection on the preprocessed first region of interest image to obtain a wire edge feature set; the wire edge feature set includes a plurality of wire edge points;

[0163] The fitting unit is used to determine the target straight line endpoints from the multiple wire edge points, and perform straight line fitting on each target straight line endpoint to determine the straight line fitting area.

[0164] Based on any of the above embodiments, the edge feature determination unit is used to:

[0165] The first image of interest is subjected to grayscale processing, noise reduction processing, equalization processing and de-texturing processing in sequence.

[0166] Based on any of the above embodiments, the fitting unit includes:

[0167] A line search unit, used to perform Hough transform on each wire edge point, and use the wire edge points that meet preset conditions after Hough transform as line search points, wherein the preset condition is that the number of intersecting Hough space curves is greater than a threshold value;

[0168] A straight line endpoint determination unit, used to determine the straight line direction in Cartesian space based on the straight line search point, and determine candidate straight line endpoints from each wire edge point based on the straight line direction;

[0169] A fitting subunit is used to determine the straight line length based on the candidate straight line endpoints. If the straight line length is greater than a preset length, the corresponding candidate straight line endpoint is used as the target straight line endpoint, and each target straight line endpoint is fitted to obtain a wire edge straight line set;

[0170] A straight line classification unit, used for classifying each straight line in the wire edge straight line set based on the slope of each straight line, to obtain a left edge straight line set and a right edge straight line set;

[0171] A straight line fitting unit is used to perform least square fitting on the first region of interest map based on the left edge straight line set and the right edge straight line set to determine the straight line fitting region.

[0172] Based on any of the above embodiments, the anti-vibration hammer profile determination unit includes:

[0173] an anti-vibration hammer edge map determining unit, configured to perform edge detection on the second region of interest map to obtain an anti-vibration hammer edge map, and determine a circumferential gradient of each non-zero point in the anti-vibration hammer edge map;

[0174] An accumulation matrix determination unit, configured to determine a circular gradient straight line based on the circular gradient of each non-zero point, and determine an accumulation matrix based on the circular gradient straight line;

[0175] A neighborhood comparison unit, configured to, if the difference between any point in the accumulation matrix and the threshold is greater than zero, use the corresponding point in the accumulation matrix as a candidate circle center point, and perform neighborhood comparison on the candidate circle center point to determine a target circle center point;

[0176] The candidate anti-vibration hammer profile determining unit is used to determine the candidate anti-vibration hammer profile based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map, and use the candidate anti-vibration hammer profile that meets the preset constraint condition as the anti-vibration hammer profile.

[0177] Based on any of the above embodiments, the candidate anti-vibration hammer profile determining unit includes:

[0178] a candidate circle radius determination unit, configured to determine a candidate circle radius value based on a distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map;

[0179] An effective circle radius determination unit is used to count the values ​​of the candidate circle radius, and take the candidate circle radius value corresponding to the count value being greater than the preset value as the effective circle radius value;

[0180] The effective contour determining unit is used to obtain an effective circular contour by taking the effective circle radius value as the radius and the corresponding target circle center point as the circle center, and taking the effective circular contour as the candidate anti-vibration hammer contour.

[0181] Based on any of the above embodiments, the bolt detection unit 720 is used to:

[0182] Inputting the anti-vibration hammer bolt area map into a target detection model to obtain the position detection result output by the target detection model;

[0183] Among them, the target detection model is obtained by training based on a sample anti-vibration hammer bolt area map and a sample position detection result; the target detection model is a lightweight model obtained by sparsely training the YOLOv3 model.

[0184] Figure 8 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 810, a memory 820, a communications interface 830 and a communication bus 840, wherein the processor 810, the memory 820 and the communications interface 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 820 to execute the anti-vibration hammer bolt detection method, which includes: extracting the straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; determining the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural size of the anti-vibration hammer, and performing target detection on the anti-vibration hammer bolt area map to obtain the position detection result of the anti-vibration hammer bolt.

[0185] In addition, the logic instructions in the above-mentioned memory 820 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0186] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the anti-vibration hammer bolt detection method provided by the above-mentioned methods, and the method includes: extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; determining the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and performing target detection on the anti-vibration hammer bolt area map to obtain the position detection result of the anti-vibration hammer bolt.

[0187] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned anti-vibration hammer bolt detection method, the method comprising: extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; determining an anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and performing target detection on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt.

[0188] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0189] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting anti-vibration hammer bolts, characterized in that: include: Extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; Based on the anti-vibration hammer profile and the structural dimensions of the anti-vibration hammer, an anti-vibration hammer bolt area map is determined, and target detection is performed on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt.

2. The anti-vibration hammer bolt detection method according to claim 1, characterized in that: The step of extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, comprises: Extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine the straight line fitting area; Based on the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire, the straight line fitting area is expanded to determine a second region of interest map of the anti-vibration hammer; Perform edge detection on the second region of interest image to determine the contour of the anti-vibration hammer.

3. The anti-vibration hammer bolt detection method according to claim 2, characterized in that: The step of extracting a first region of interest map of the anti-vibration hammer wire from the image to be detected, and performing edge detection on the first region of interest map to determine the straight line fitting region includes: Performing image preprocessing on the first region of interest image, and performing edge point detection on the preprocessed first region of interest image to obtain a wire edge feature set; the wire edge feature set includes a plurality of wire edge points; The target straight line endpoints are determined from the plurality of wire edge points, and straight line fitting is performed on each target straight line endpoint to determine the straight line fitting area.

4. The anti-vibration hammer bolt detection method according to claim 3, characterized in that: The performing image preprocessing on the first region of interest map includes: The first region of interest image is subjected to grayscale processing, noise reduction processing, equalization processing and de-texturing processing in sequence.

5. The anti-vibration hammer bolt detection method according to claim 3, characterized in that: The step of determining target straight line endpoints from the plurality of wire edge points, and performing straight line fitting on each target straight line endpoint to determine the straight line fitting area includes: Performing Hough transformation on each wire edge point, and using the wire edge points that meet preset conditions after Hough transformation as straight line search points, wherein the preset condition is that the number of intersecting Hough space curves is greater than a threshold; Determine the direction of the line in Cartesian space based on the line search point, and determine the candidate line endpoints from each wire edge point based on the line direction; Determine the straight line length based on the candidate straight line endpoints, if the straight line length is greater than a preset length, use the corresponding candidate straight line endpoints as the target straight line endpoints, and perform straight line fitting on each target straight line endpoint to obtain a wire edge straight line set; Based on the slope of each straight line in the wire edge straight line set, each straight line is classified to obtain a left edge straight line set and a right edge straight line set; Based on the left edge straight line set and the right edge straight line set, least square fitting is performed on the first region of interest map to determine the straight line fitting region.

6. The anti-vibration hammer bolt detection method according to claim 2, characterized in that: The performing edge detection on the second region of interest image to determine the contour of the anti-vibration hammer includes: Performing edge detection on the second region of interest map to obtain an anti-vibration hammer edge map, and determining the circumferential gradient of each non-zero point in the anti-vibration hammer edge map; Determine a circular gradient straight line based on the circumferential gradient of each non-zero point, and determine an accumulation matrix based on the circular gradient straight line; If the difference between any point in the accumulation matrix and the threshold is greater than zero, the corresponding point in the accumulation matrix is ​​used as a candidate circle center point, and the candidate circle center point is compared with the neighborhood to determine the target circle center point; Based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map, a candidate anti-vibration hammer profile is determined, and the candidate anti-vibration hammer profile that meets the preset constraint condition is used as the anti-vibration hammer profile.

7. The anti-vibration hammer bolt detection method according to claim 6, characterized in that: The determining of the candidate anti-vibration hammer profile based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map comprises: Determine the candidate circle radius value based on the distance between the target circle center point and each anti-vibration hammer edge point in the anti-vibration hammer edge map; Count the radius values ​​of each candidate circle, and take the candidate circle radius value whose count value is greater than the preset value as the valid circle radius value; The effective circle radius value is used as the radius, and the corresponding target circle center point is used as the circle center to obtain an effective circular contour, and the effective circular contour is used as the candidate anti-vibration hammer contour.

8. The anti-vibration hammer bolt detection method according to any one of claims 1 to 7, characterized in that: The performing target detection on the anti-vibration hammer bolt area map to obtain a position detection result of the anti-vibration hammer bolt includes: Inputting the anti-vibration hammer bolt area map into a target detection model to obtain the position detection result output by the target detection model; Among them, the target detection model is obtained by training based on a sample anti-vibration hammer bolt area map and a sample position detection result; the target detection model is a lightweight model obtained by sparsely training the YOLOv3 model.

9. An anti-vibration hammer bolt detection device, characterized in that: include: A contour determination unit, used for extracting a straight line fitting area of ​​the anti-vibration hammer wire from the image to be detected, and determining the anti-vibration hammer contour based on the straight line fitting area and the positional relationship between the anti-vibration hammer and the anti-vibration hammer wire; The bolt detection unit is used to determine the anti-vibration hammer bolt area map based on the anti-vibration hammer contour and the structural dimensions of the anti-vibration hammer, and perform target detection on the anti-vibration hammer bolt area map to obtain the position detection result of the anti-vibration hammer bolt.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the anti-vibration hammer bolt detection method according to any one of claims 1 to 8 are implemented.

Images