A machine vision cable defect recognition method and system

Abstract

The present invention provides a machine vision cable defect recognition method and system, comprising: acquiring an original cable image; determining the diameter of the cable in the cable image by using a non-edge detection method for the original cable image; correcting the cable image by establishing a cable shaking model and compensating the deformed image by using a reverse compensation method; and splicing the compensated cable image by using an image splicing algorithm to obtain a processed image. The solution provided by the present invention takes the motion state of the cable into consideration in cable defect recognition, analyzes the motion state of the optical cable, takes the row coordinates of the upper left corner point of the cable image as a reference line, establishes a model, and reversely compensates the cable image, thereby achieving correction of the deformed cable image while retaining the edge features of the cable, thereby improving the accuracy of cable defect recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a machine vision cable defect recognition method and system. Background Art

[0002] The cable surface mainly plays the role of insulation, waterproofing, and moisture-proofing, so cable surface inspection is particularly important. At present, the main method for detecting cable surface defects is manual inspection, which is easily affected by subjective consciousness, has low efficiency, poor accuracy, and high false detection rate, and cannot meet the requirements of some special industries. Moreover, if the cables are in the air, manual inspection will have safety risks. Summary of the invention

[0003] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a cable defect recognition method based on machine vision. By using the scheme proposed by the present invention, a large number of image data sets can be grouped, and the allocation is accurate, simple and fast.

[0004] The present invention adopts the following technical solution:

[0005] A cable defect recognition method based on machine vision comprises the following steps:

[0006] Get the original cable image;

[0007] Determine the diameter of the cable in the original cable image by using a non-edge detection method;

[0008] Correct the cable image by building a cable shaking model and using the reverse compensation method to compensate for the deformed image;

[0009] Using the image stitching algorithm to stitch the cable image on the compensated image to obtain the processed image;

[0010] For the processed images, the improved algorithm based on Mask RCNN is used to identify cable defects.

[0011] Specifically, the diameter of the cable in the cable image is determined using a non-edge detection method, specifically:

[0012] The original cable image is segmented by threshold to obtain a binary image. The number of pixels in each column of the white optical cable area is recorded and averaged to calculate the cable diameter.

[0013] Specifically, the sloshing model is:

[0014] Among them, (x1, y1) are the coordinates corresponding to the pixels of the original image, (x2, y2) are the coordinates corresponding to the pixels of the image after shaking, which respectively represent the displacement vectors of the image after shaking relative to the original image in the horizontal and vertical directions, and k and α respectively represent the zoom factor and rotation angle of the image after shaking relative to the original image.

[0015] Specifically, the deformed image is compensated by the reverse compensation method, which is as follows:

[0016] The optical cable area and the background area are separated by threshold segmentation;

[0017] Use the traversal method to obtain the edge coordinates of the optical cable;

[0018] Use xUP to represent the row coordinates of the upper edge point of the optical cable, and let d = xA-xUP. If d is greater than zero, the optical cable area column where xUP is located moves down by d; if d is less than zero, the optical cable area column where xUP is located moves up by d.

[0019] Specifically, the image stitching algorithm is used to stitch the cable images, specifically:

[0020] Construct the Hessian matrix;

[0021] Among them, Lxx, Lxy, Lyy are the second-order derivatives of the image in each direction after Gaussian filtering, and σ represents the scale.

[0022] Construct a Gaussian pyramid; that is, a combination of images of different sizes;

[0023] Locate feature points; compare each pixel point calculated by the Hessian matrix with its neighboring points. If the pixel point is an extreme point, take it as the primary feature point, use linear interpolation to interpolate the three-dimensional image to obtain sub-pixel feature points, set a fixed threshold, remove all points less than the threshold, and obtain feature points;

[0024] Determine the main direction of the feature point; realize the rotation invariance of the image by counting the Haar wavelet features in the neighborhood of the feature point, that is, draw a circle with the located feature point as the center and the set multiple N of the scale of the feature point as the radius, divide the circular area into N parts, count the Haar wavelet responses of each sector area respectively, accumulate them into a new vector, and select the longest direction of the vector from the N vectors as the main direction of the feature point.

[0025] Specifically, the splicing of cable images using the image splicing algorithm also includes eliminating the splicing seams, specifically:

[0026] Scan the pixels in the image to obtain the edge features and texture features of the image;

[0027] After being processed by pixel-level fusion method, an image with no stitching seams is obtained.

[0028] Specifically, the processed image is used based on the improved Mask RCNN algorithm to identify cable defects, specifically:

[0029] The improved Mask RCNN algorithm is as follows: Based on the ROI Align layer of the Mask RCNN algorithm, a deformable sampling grid is added to adapt to ROI areas of different shapes and sizes by locally deforming the feature map.

[0030] Another aspect of the present invention provides a machine vision cable defect recognition system, including:

[0031] Image acquisition unit: acquires original cable image;

[0032] Detection unit: Determine the diameter of the cable in the original cable image using a non-edge detection method;

[0033] Image compensation unit: corrects the cable image by establishing a cable shaking model and using the reverse compensation method to compensate for the deformed image to obtain a compensated image;

[0034] Image stitching unit: stitching the cable image with the compensated image using the image stitching algorithm to obtain the processed image;

[0035] Defect recognition unit: The processed images are used to identify cable defects based on the improved Mask RCNN algorithm.

[0036] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a machine vision cable defect identification method when executing the computer program.

[0037] Yet another aspect of the present invention provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, a method for identifying cable defects using machine vision is implemented.

[0038] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention proposes a machine vision cable defect recognition method, which specifically comprises the following steps: obtaining an original cable image; determining the diameter of the cable in the cable image by using a non-edge detection method for the original cable image; correcting the cable image by establishing a cable shaking model and compensating the deformed image by using a reverse compensation method; and splicing the compensated cable image by using an image splicing algorithm to obtain a processed image. The solution provided by the present invention takes the motion state of the cable into consideration in cable defect recognition, analyzes the motion state of the optical cable, takes the row coordinates of the upper left corner point of the cable image as a reference line, establishes a model, and reversely compensates the cable image, thereby achieving correction of the deformed cable image while retaining the edge features of the cable, thereby improving the accuracy of cable defect recognition.

[0040] 2) Using the improved Mask RCNN algorithm, not only can subtle defects in the image be detected, but the detection rate is also greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a cable defect recognition method using machine vision provided by an embodiment of the present invention;

[0042] Figure 2 A structural diagram of a cable defect recognition method using machine vision provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described below through specific implementation modes.

[0046] The present invention proposes a machine vision cable defect recognition method, which can realize the grouping of a large number of image data sets, and the allocation is accurate, simple and fast.

[0047] like Figure 1 The present invention provides a method for identifying cable defects using machine vision, which specifically includes the following steps:

[0048] S101; obtaining an original cable image;

[0049] The image acquisition unit is mainly composed of a camera. The lighting unit and the image acquisition unit complete the acquisition of the optical cable image through reasonable design. The control unit includes a microcontroller and an industrial computer. The microcontroller is mainly used to control the opening and closing of the light source and the camera to realize intelligent production. The industrial computer is mainly used for background image processing operations.

[0050] S102: Determine the diameter of the cable in the original cable image using a non-edge detection method;

[0051] The cable image is subjected to non-edge detection method to determine the diameter of the cable in the cable image, specifically:

[0052] The original cable image is segmented by threshold to obtain a binary image. The number of pixels in each column of the white optical cable area is recorded and averaged to calculate the cable diameter.

[0053] The error between the detected value and the true value of the cable diameter is large, which will lead to a high false alarm rate of defects. Therefore, it is necessary to accurately detect the cable diameter. The cable diameter detection includes edge detection and non-edge detection. Common image edge detection operators include Sobel, Laplacian, Canny, and LoG. Edge detection usually requires the cable to be relatively still, but there are a large number of images with large cable edge fluctuations in the collected images, so the edge detection method cannot detect the edge of the optical cable;

[0054] The present invention firstly divides the original image by threshold value to obtain a binary image, and then counts the number of white pixel points in each column of the white area of ​​the optical cable, so as to avoid mis-finding the optical cable boundary due to background interference.

[0055] Since there are many statistical data on optical cable diameters, it is necessary to select one value to represent the overall level. Therefore, the average of the data is selected as the diameter of the optical cable. When detecting images with obvious image edge jitter, the method of the present invention can greatly improve the detection accuracy.

[0056] S103: Correcting the cable image by establishing a cable shaking model and compensating the deformed image using a reverse compensation method;

[0057] The sloshing model is specifically:

[0058]

[0059] Among them, (x1, y1) are the coordinates corresponding to the pixels of the original image, (x2, y2) are the coordinates corresponding to the pixels of the image after shaking, which respectively represent the displacement vectors of the image after shaking relative to the original image in the horizontal and vertical directions, and k and α respectively represent the zoom factor and rotation angle of the image after shaking relative to the original image.

[0060] The space where the image acquisition device and the object under test are located is a three-dimensional space. The image acquisition device and the support body are fixed to the ground and are stationary relative to the ground. Objects may have different forms of movement in the three-dimensional space, such as translation, rotation, jumping and wave-like movements. The image we observe exists in a two-dimensional plane, that is, the information of the three-dimensional object can be reflected by the number of pixels and the gray value of the pixels. Based on this, a shaking model is established.

[0061] The deformed image is compensated by the reverse compensation method, specifically:

[0062] The optical cable area and the background area are separated by threshold segmentation;

[0063] Use the traversal method to obtain the edge coordinates of the optical cable;

[0064] Use xUP to represent the row coordinates of the upper edge point of the optical cable, and let d = xA-xUP. If d is greater than zero, the optical cable area column where xUP is located moves down by d; if d is less than zero, the optical cable area column where xUP is located moves up by d.

[0065] When the camera captures the image of the optical cable surface, its shooting direction is perpendicular to the optical cable axis, which is equivalent to the human eye looking down at an object. The image should be a rectangular image, but the edge of the image fluctuates, indicating that the optical cable moves horizontally during operation.

[0066] Since the purpose of de-jittering is to better detect defects in the optical cable area and the production efficiency is high, the translation model is selected to describe the shaking by comprehensively considering the detection needs and production efficiency. Through the analysis of the deformed image, the fluctuation of the edge of the optical cable is relative, that is, the fluctuation of the upper edge of the optical cable is the same as the fluctuation of the lower edge.

[0067] S104: Using an image stitching algorithm to stitch the compensated image to obtain a processed image;

[0068] Then use the image stitching algorithm to stitch the cable images, specifically:

[0069] Construct the Hessian matrix;

[0070] Among them, Lxx, Lxy, Lyy are the second-order derivatives of the image in each direction after Gaussian filtering, and σ represents the scale.

[0071] The Hessian matrix is ​​used to detect feature points, which can describe the curvature of the function and is mainly composed of second-order derivatives.

[0072] The elements in the matrix represent the amount of change around the pixel. When the second-order derivative of the determinant of a point is zero, the point is a feature point.

[0073] Construct a Gaussian pyramid; that is, a combination of images of different sizes;

[0074] Locate feature points; Each pixel in the image has a neighborhood point. Compare each pixel calculated by the Hessian matrix with its neighborhood point. If the pixel is an extreme point, take the point as the primary feature point and use linear interpolation to interpolate the three-dimensional image to obtain sub-pixel feature points. Set a fixed threshold and remove all points less than the threshold to obtain feature points.

[0075] Determine the main direction of the feature point; realize the rotation invariance of the image by counting the Haar wavelet features in the neighborhood of the feature point, that is, draw a circle with the located feature point as the center and the set multiple N of the scale of the feature point as the radius, divide the circular area into N parts, count the Haar wavelet responses of each sector area respectively, accumulate them into a new vector, and select the longest direction of the vector from the N vectors as the main direction of the feature point.

[0076] Generate feature descriptors.

[0077] After the feature points are determined, first draw a square frame with a side length of 20 times the scale of the feature points. The direction of the frame is the same as the main direction of the feature points. Then divide the square frame into 16 small squares on average, and count the horizontal and vertical Haar wavelet features of the pixels in the small squares. Each small square corresponds to 4 Haar wavelet eigenvalues. Therefore, each feature vector is 64-dimensional, and the vector dimension is reduced by half, which improves the matching efficiency of the algorithm.

[0078] After stitching together multiple images, it can be clearly seen that there is a thin gap between the two adjacent images, and the grayscale of the pixels next to them is obviously different.

[0079] Since the image of the optical cable surface is collected by multiple cameras, the reflected light entering the camera may be different. The light source will have lower and lower luminous efficiency if it is operated for a long time, or the light source at a certain angle is affected by the ambient light, which may cause the stitched image to have stitching seams.

[0080] The appearance of splicing seams makes the image look unnatural and has a certain deviation from the actual optical cable expansion diagram. In addition, it also has a certain impact on the detection of later defects. Therefore, it is necessary to eliminate the splicing seams to make the image transition natural.

[0081] This method scans every pixel in the image, so it can obtain many detailed data of the image, such as edge features and texture features, which paves the way for subsequent analysis and detection of target features. The image processed by the pixel-level fusion method not only becomes richer in content and details, but also retains the original information of the image as much as possible.

[0082] After the cable images are stitched together using the image stitching algorithm, the stitching seam elimination is also included, specifically:

[0083] Scan the pixels in the image to obtain the edge features and texture features of the image.

[0084] After being processed by pixel-level fusion method, an image with no stitching seams is obtained.

[0085] S105: For the processed image, the cable defects are identified based on the improved algorithm of Mask RCNN.

[0086] For the processed images, the cable defects are identified based on the improved Mask RCNN algorithm, specifically:

[0087] The improved Mask RCNN algorithm is as follows: Based on the ROI Align layer of the Mask RCNN algorithm, a deformable sampling grid is added to adapt to ROI areas of different shapes and sizes by locally deforming the feature map.

[0088] The improved Mask RCNN algorithm proposed in the present invention is an improved algorithm of the ROI Align layer. The main idea is to add a deformable sampling grid on the basis of the ROI Align layer. The role of ROI Align in the Mask RCNN algorithm is to map ROI areas of different sizes to the feature map and perform corresponding sampling operations. ROI Align is relatively fixed in the position division inside the ROI, so it may happen that the ROI does not completely match the object contour, affecting the accuracy of object detection and segmentation. Therefore, improving the ROI Align layer is an important means to improve the performance of the Mask RCNN algorithm.

[0089] The traditional ROI pooling method uses bilinear interpolation to calculate the pixel value of the sampling point, but this method has a certain accuracy error. Especially when the ROI boundary alignment is inaccurate, the error will be more significant. To solve this problem, Deformable ROI pooling adopts the idea of ​​deformable convolution and introduces some learnable deformation parameters.

[0090] For the input feature map and ROI area, the ROI Align layer is used to map the ROI area to the feature map and perform the corresponding sampling operation to obtain the ROI feature map. On the ROI feature map, a displacement vector is calculated for each sampling point. This displacement vector is equivalent to the position offset calculated in the ROI Align layer. This displacement vector is learnable, so it can be automatically adjusted according to the data. For each sampling point, bilinear interpolation is used to collect pixel values ​​from the corresponding position on the feature map. Then, the sampling point is offset according to the displacement vector, and then interpolation is performed on the feature map. In this way, a new sampling point is obtained. All new sampling points are converged according to the standard ROI pooling method to obtain the final ROI feature vector.

[0091] like Figure 2 The embodiment of the present invention further provides a machine vision cable defect recognition system, comprising:

[0092] Image acquisition unit 201: acquires original cable image;

[0093] The image acquisition unit is mainly composed of a camera. The lighting unit and the image acquisition unit complete the acquisition of the optical cable image through reasonable design. The control unit includes a microcontroller and an industrial computer. The microcontroller is mainly used to control the opening and closing of the light source and the camera to realize intelligent production. The industrial computer is mainly used for background image processing operations.

[0094] Detection unit 202: Determine the diameter of the cable in the original cable image using a non-edge detection method;

[0095] The cable image is subjected to non-edge detection method to determine the diameter of the cable in the cable image, specifically:

[0096] The original cable image is segmented by threshold to obtain a binary image. The number of pixels in each column of the white optical cable area is recorded and averaged to calculate the cable diameter.

[0097] There is a large error between the detected cable diameter and the true value.

[0098] This will lead to a high false alarm rate of defects, so it is necessary to accurately detect the diameter of the cable. The diameter detection of the cable includes edge detection and non-edge detection. Common image edge detection operators include Sobel, Laplacian, Canny, and LoG. Edge detection usually requires the cable to be relatively still, but there will be a large number of images with large cable edge fluctuations in the collected images.

[0099] Therefore, it is impossible to detect the edge of the optical cable using the edge detection method;

[0100] The present invention firstly divides the original image by threshold value to obtain a binary image, and then counts the number of white pixel points in each column of the white area of ​​the optical cable, so as to avoid mis-finding the optical cable boundary due to background interference.

[0101] Since there are many statistical data on optical cable diameters, it is necessary to select one value to represent the overall level. Therefore, the average of the data is selected as the diameter of the optical cable. When detecting images with obvious image edge jitter, the method of the present invention can greatly improve the detection accuracy.

[0102] Image compensation unit 203: corrects the cable image by establishing a cable shaking model and compensating the deformed image using a reverse compensation method to obtain a compensated image;

[0103] The sloshing model is specifically:

[0104]

[0105] Among them, (x1, y1) are the coordinates corresponding to the pixels of the original image, (x2, y2) are the coordinates corresponding to the pixels of the image after shaking, which respectively represent the displacement vectors of the image after shaking relative to the original image in the horizontal and vertical directions, and k and α respectively represent the zoom factor and rotation angle of the image after shaking relative to the original image.

[0106] The space where the image acquisition device and the object under test are located is a three-dimensional space. The image acquisition device and the support body are fixed to the ground and are stationary relative to the ground. Objects may have different forms of movement in the three-dimensional space, such as translation, rotation, jumping and wave-like movements. The image we observe exists in a two-dimensional plane, that is, the information of the three-dimensional object can be reflected by the number of pixels and the gray value of the pixels. Based on this, a shaking model is established.

[0107] The deformed image is compensated by the reverse compensation method, specifically:

[0108] The optical cable area and the background area are separated by threshold segmentation;

[0109] Use the traversal method to obtain the edge coordinates of the optical cable;

[0110] Use xUP to represent the row coordinates of the upper edge point of the optical cable, and let d = xA-xUP. If d is greater than zero, the optical cable area column where xUP is located moves down by d; if d is less than zero, the optical cable area column where xUP is located moves up by d.

[0111] When the camera captures the image of the optical cable surface, its shooting direction is perpendicular to the optical cable axis, which is equivalent to the human eye looking down at an object. The image should be a rectangular image, but the edge of the image fluctuates, indicating that the optical cable moves horizontally during operation.

[0112] Since the purpose of de-jittering is to better detect defects in the optical cable area and the production efficiency is high, the translation model is selected to describe the shaking by comprehensively considering the detection needs and production efficiency. Through the analysis of the deformed image, the fluctuation of the edge of the optical cable is relative, that is, the fluctuation of the upper edge of the optical cable is the same as the fluctuation of the lower edge.

[0113] Image stitching unit 204: stitching the cable image on the compensated image using an image stitching algorithm to obtain a processed image;

[0114] Then use the image stitching algorithm to stitch the cable images, specifically:

[0115] Construct the Hessian matrix;

[0116]

[0117] Among them, Lxx, Lxy, Lyy are the second-order derivatives of the image in each direction after Gaussian filtering, and σ represents the scale.

[0118] The Hessian matrix is ​​used to detect feature points, which can describe the curvature of the function. It is mainly composed of second-order derivatives. The elements in the matrix represent the amount of change around the pixel point. When the second-order derivative of the determinant of a point is zero, the point is a feature point.

[0119] Construct a Gaussian pyramid; that is, a combination of images of different sizes;

[0120] Locate feature points; Each pixel in the image has a neighborhood point. Compare each pixel calculated by the Hessian matrix with its neighborhood point. If the pixel is an extreme point, take the point as the primary feature point and use linear interpolation to interpolate the three-dimensional image to obtain sub-pixel feature points. Set a fixed threshold and remove all points less than the threshold to obtain feature points.

[0121] Determine the main direction of the feature point; realize the rotation invariance of the image by counting the Haar wavelet features in the neighborhood of the feature point, that is, draw a circle with the located feature point as the center and the set multiple N of the scale of the feature point as the radius, divide the circular area into N parts, count the Haar wavelet responses of each sector area respectively, accumulate them into a new vector, and select the longest direction of the vector from the N vectors as the main direction of the feature point.

[0122] Generate feature descriptors.

[0123] After the feature points are determined, a square frame is first drawn with a side length of 20 times the size of the feature point. The direction of the frame is the same as the main direction of the feature point. Then the square frame is evenly divided into 16 small squares, and the horizontal and vertical Haar wavelet features of the pixels in the small squares are counted; each small square corresponds to 4 Haar wavelet eigenvalues; therefore, each feature vector is 64-dimensional, and the vector dimension is reduced by half, which improves the matching efficiency of the algorithm. After splicing multiple images, it can be clearly seen that there is a thin gap between the two adjacent images, and the grayscale of the pixels next to it is obviously different.

[0124] Since the image of the optical cable surface is collected by multiple cameras, the reflected light entering the camera may be different. The light source will have lower and lower luminous efficiency if it is operated for a long time, or the light source at a certain angle is affected by the ambient light, which may cause the stitched image to have stitching seams.

[0125] The appearance of splicing seams makes the image look unnatural and has a certain deviation from the actual optical cable expansion diagram. In addition, it also has a certain impact on the detection of later defects. Therefore, it is necessary to eliminate the splicing seams to make the image transition natural.

[0126] This method scans every pixel in the image, so it can obtain many detailed data of the image, such as edge features and texture features, which paves the way for subsequent analysis and detection of target features. The image processed by the pixel-level fusion method not only becomes richer in content and details, but also retains the original information of the image as much as possible.

[0127] After the cable images are stitched together using the image stitching algorithm, the stitching seam elimination is also included, specifically:

[0128] Scan the pixels in the image to obtain the edge features and texture features of the image.

[0129] After being processed by pixel-level fusion method, an image with no stitching seams is obtained.

[0130] Defect recognition unit 205: identifies cable defects based on the processed image based on the improved Mask RCNN algorithm.

[0131] For the processed images, the cable defects are identified based on the improved Mask RCNN algorithm, specifically:

[0132] The improved Mask RCNN algorithm is as follows: Based on the ROI Align layer of the Mask RCNN algorithm, a deformable sampling grid is added to adapt to ROI areas of different shapes and sizes by locally deforming the feature map.

[0133] The improved Mask RCNN algorithm proposed in the present invention is an improved algorithm of the ROI Align layer. The main idea is to add a deformable sampling grid on the basis of the ROI Align layer. The role of ROI Align in the Mask RCNN algorithm is to map ROI areas of different sizes to the feature map and perform corresponding sampling operations. ROI Align is relatively fixed in the position division inside the ROI, so it may happen that the ROI does not completely match the object contour, affecting the accuracy of object detection and segmentation. Therefore, improving the ROI Align layer is an important means to improve the performance of the Mask RCNN algorithm.

[0134] The traditional ROI pooling method uses bilinear interpolation to calculate the pixel value of the sampling point, but this method has a certain accuracy error. Especially when the ROI boundary alignment is inaccurate, the error will be more significant. To solve this problem, Deformable ROIpooling adopts the idea of ​​deformable convolution and introduces some learnable deformation parameters.

[0135] For the input feature map and ROI area, the ROI Align layer is used to map the ROI area to the feature map and perform the corresponding sampling operation to obtain the ROI feature map. On the ROI feature map, a displacement vector is calculated for each sampling point. This displacement vector is equivalent to the position offset calculated in the ROI Align layer. This displacement vector is learnable, so it can be automatically adjusted according to the data. For each sampling point, bilinear interpolation is used to collect pixel values ​​from the corresponding position on the feature map. Then, the sampling point is offset according to the displacement vector, and then interpolation is performed on the feature map. In this way, a new sampling point is obtained. All new sampling points are converged according to the standard ROI pooling method to obtain the final ROI feature vector.

[0136] Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer code 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer code 311, a method for tracing and identifying groundwater pollution sources provided by an embodiment of the present invention is implemented.

[0137] Since the electronic device introduced in this embodiment is the device used to implement the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, the technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present invention falls within the scope of protection of the present invention.

[0138] Please refer to FIG. 4 , which is a schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0139] like Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 411 is stored. When the computer program 411 is executed by a processor, a method for tracing and identifying a groundwater pollution source provided by an embodiment of the present invention is implemented.

[0140] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] 1) The present invention proposes a machine vision cable defect recognition method, which specifically comprises the following steps: obtaining an original cable image; determining the diameter of the cable in the cable image using a non-edge detection method for the original cable image; correcting the cable image by establishing a cable shaking model and compensating the deformed image using a reverse compensation method; and splicing the compensated cable image using an image stitching algorithm to obtain a processed image. The solution provided by the present invention takes the motion state of the cable into consideration in cable defect recognition, analyzes the motion state of the optical cable, takes the row coordinates of the upper left corner of the cable image as the reference line, establishes a model, and reversely compensates the cable image, thereby achieving correction of the deformed cable image while retaining the edge features of the cable, thereby improving the accuracy of cable defect recognition.

[0143] 2) Using the improved Mask RCNN algorithm, not only can subtle defects in the image be detected, but the detection rate is also greatly improved.

[0144] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

[0145] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. The various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

[0146] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A cable defect recognition method based on machine vision, characterized in that: The steps include: Get the original cable image; Determine the diameter of the cable in the original cable image by using a non-edge detection method; Correct the cable image by building a cable shaking model and using the reverse compensation method to compensate for the deformed image; Using the image stitching algorithm to stitch the cable image on the compensated image to obtain the processed image; For the processed images, the improved algorithm based on Mask-RCNN is used to identify cable defects; The sloshing model is specifically: ； Where (x1, y1) is the coordinate of the original image pixel, (x2, y2) is the coordinate of the image pixel after shaking, which respectively represent the horizontal and vertical displacement vectors of the image after shaking relative to the original image, and k and α represent the zoom factor and rotation angle of the image after shaking relative to the original image. The deformed image is compensated by the reverse compensation method, specifically: The cable area and the background area are separated by threshold segmentation; Use the traversal method to obtain the edge coordinates of the cable; Let xUP represent the row coordinate of the upper edge point of the cable, let d = xA-xUP, if d is greater than zero, the cable area column where xUP is located moves down by d; if d is less than zero, the cable area column where xUP is located moves up by d; Then use the image stitching algorithm to stitch the cable images, specifically: Construct the Hessian matrix; ; Among them, Lxx, Lxy, Lyy are the second-order derivatives of the image in each direction after Gaussian filtering, and σ represents the scale; Construct a Gaussian pyramid; that is, a combination of images of different sizes; Locate feature points; compare each pixel point calculated by the Hessian matrix with its neighboring points. If the pixel point is an extreme point, take it as the primary feature point, use linear interpolation to interpolate the three-dimensional image to obtain sub-pixel feature points, set a fixed threshold, remove all points less than the threshold, and obtain feature points; Determine the main direction of the feature point; realize the rotation invariance of the image by counting the Haar wavelet features in the neighborhood of the feature point, that is, draw a circle with the located feature point as the center and the set multiple N of the scale of the feature point as the radius, divide the circular area into N parts, count the Haar wavelet response of each sector area respectively, accumulate them into a new vector, and select the longest direction of the vector from the N vectors as the main direction of the feature point; Generate descriptors; For the processed images, the cable defects are identified based on the improved Mask RCNN algorithm, specifically: The improved Mask RCNN algorithm is as follows: Based on the ROI Align layer of the Mask RCNN algorithm, a deformable sampling grid is added to adapt to ROI areas of different shapes and sizes by locally deforming the feature map.

2. The cable defect recognition method based on machine vision according to claim 1, characterized in that: The cable image is subjected to non-edge detection method to determine the diameter of the cable in the cable image, specifically: The original cable image is segmented by threshold to obtain a binary image. The number of pixels in each column of the white cable area is recorded and averaged to calculate the cable diameter.

3. The cable defect recognition method based on machine vision according to claim 2, characterized in that: After the cable images are stitched together using the image stitching algorithm, the stitching seam elimination is also included, specifically: Scan the pixels in the image to obtain the edge features and texture features of the image; After being processed by pixel-level fusion method, an image with no stitching seams is obtained.

4. A machine vision cable defect recognition system, characterized in that: include: Image acquisition unit: acquires original cable image; Detection unit: Determine the diameter of the cable in the original cable image using a non-edge detection method; Image compensation unit: corrects the cable image by establishing a cable shaking model and using the reverse compensation method to compensate for the deformed image to obtain a compensated image; Image stitching unit: stitching the cable image with the compensated image using the image stitching algorithm to obtain the processed image; Defect recognition unit: for processed images, based on the improved Mask RCNN algorithm, cable defects are identified; The sloshing model is specifically: ； Where (x1, y1) is the coordinate of the original image pixel, (x2, y2) is the coordinate of the image pixel after shaking, which respectively represent the horizontal and vertical displacement vectors of the image after shaking relative to the original image, and k and α represent the zoom factor and rotation angle of the image after shaking relative to the original image. The deformed image is compensated by the reverse compensation method, specifically: The cable area and the background area are separated by threshold segmentation; Use the traversal method to obtain the edge coordinates of the cable; Let xUP represent the row coordinate of the upper edge point of the cable, let d = xA-xUP, if d is greater than zero, the cable area column where xUP is located moves down by d; if d is less than zero, the cable area column where xUP is located moves up by d; Then use the image stitching algorithm to stitch the cable images, specifically: Construct the Hessian matrix; ; Among them, Lxx, Lxy, Lyy are the second-order derivatives of the image in each direction after Gaussian filtering, and σ represents the scale; Construct a Gaussian pyramid; that is, a combination of images of different sizes; Locate feature points; compare each pixel point calculated by the Hessian matrix with its neighboring points. If the pixel point is an extreme point, take it as the primary feature point, use linear interpolation to interpolate the three-dimensional image to obtain sub-pixel feature points, set a fixed threshold, remove all points less than the threshold, and obtain feature points; Determine the main direction of the feature point; realize the rotation invariance of the image by counting the Haar wavelet features in the neighborhood of the feature point, that is, draw a circle with the located feature point as the center and the set multiple N of the scale of the feature point as the radius, divide the circular area into N parts, count the Haar wavelet response of each sector area respectively, accumulate them into a new vector, and select the longest direction of the vector from the N vectors as the main direction of the feature point; Generate descriptors; For the processed images, the cable defects are identified based on the improved Mask RCNN algorithm, specifically: The improved Mask RCNN algorithm is as follows: Based on the ROI Align layer of the Mask RCNN algorithm, a deformable sampling grid is added to adapt to ROI areas of different shapes and sizes by locally deforming the feature map.

5. An electronic device, characterized in that: include: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method steps of any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 3 are implemented.

Images