A cable deformation defect recognition method and device based on binocular vision

Through binocular vision, the deformation defects of the cable are identified, and the three-dimensional coordinates are constructed using binocular cameras and cable segmentation models, which solves the accuracy problem of cable bending judgment and ensures the safe operation of the cable.

CN114882004BActive Publication Date: 2024-11-08STATE GRID JIANGSU ELECTRIC POWER CO LTD RESEARCH INSTITUTE +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210619525.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-02
Publication Date
2024-11-08
Estimated Expiration
2042-06-02

AI Technical Summary

Technical Problem

The prior art is difficult to effectively identify the deformation defects of the cable, which leads to the inability to detect hidden dangers of safe operation of the cable in time, especially the monocular camera cannot judge the physical changes caused by the bending of the cable.

Method used

Using binocular vision method, images are obtained through binocular cameras, image segmentation and three-dimensional coordinate point set construction based on the trained cable segmentation model, segmented straight line fitting to obtain slope change rate. If the change rate is greater than the preset threshold, the cable is judged. There are deformation defects.

Benefits of technology

By constructing the three-dimensional spatial coordinates of the cable, the accuracy of judging the cable bending degree is improved, ensuring the safe operation of the cable, and overcoming the shortcomings of insufficient recognition of monocular cameras.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114882004B_ABST
    Figure CN114882004B_ABST
Patent Text Reader

Abstract

The invention discloses a cable deformation defect recognition method and device based on binocular vision, comprising: acquiring a captured image of a binocular camera; segmenting a left-view image and a right-view image based on a cable segmentation model to obtain a left-view edge image and a right-view edge image; extracting coordinate points of a lower edge of a cable from the left-view edge image and the right-view edge image to generate a left-view lower edge point set and a right-view lower edge point set; acquiring a lower edge three-dimensional coordinate point set according to the left-view lower edge point set, the right-view lower edge point set and the parameters of a binocular camera; performing piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment; acquiring a slope change rate according to the slope of the straight line segment, and if the slope change rate is greater than a preset threshold value, the cable has a deformation defect; the invention can use a binocular vision method to recognize the deformation of the cable, thereby timely discovering hidden dangers to the safe operation of the cable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a cable deformation defect recognition method and device based on binocular vision, belonging to the technical field of power transmission cables. Background Art

[0002] If the cable is deformed to the point where the internal insulation is damaged, problems will arise. Each voltage level has a corresponding maximum bending radius that cannot exceed the applicable range. Excessive bending will cause physical changes in the internal insulation and other structures, directly affecting the safe operation of the cable. Generally, when the metal armor is bent, it will squeeze the internal semi-conductive layer and the insulation layer. When it reaches a certain degree, it will cause dents on the insulation surface. The formation of dents is the biggest hidden danger to the safe operation of the cable. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method and device for identifying cable deformation defects based on binocular vision, which uses a binocular vision method to identify cable deformation, thereby promptly discovering hidden dangers to the safe operation of the cable.

[0004] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0005] In a first aspect, the present invention provides a cable deformation defect recognition method based on binocular vision, comprising:

[0006] Acquire a captured image of a binocular camera, wherein the captured image includes a left-view image and a right-view image;

[0007] Based on the trained cable segmentation model, the left view image and the right view image are segmented to obtain the left view edge image and the right view edge image respectively;

[0008] Extracting the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image to generate a left-view edge point set and a right-view edge point set;

[0009] Obtain the lower edge three-dimensional coordinate point set according to the edge point set under the left view, the edge point set under the right view and the parameters of the binocular camera;

[0010] Performing piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment;

[0011] The rate of change of the slope is obtained according to the slope of the straight line segment. If the rate of change of the slope is greater than a preset threshold, the cable has a deformation defect.

[0012] Optionally, the cable segmentation model is constructed based on Mask-RCNN; the training of the cable segmentation model includes:

[0013] Obtain multiple cable images and add edge labels to each cable image to generate sample images;

[0014] Divide sample images into training sets and validation sets;

[0015] Use each sample image in the training set to train the cable segmentation model; after each round of training with each sample image, use the sample image in the validation set to verify the training, and obtain the evaluation index of the cable segmentation model after this round of training;

[0016] The optimal model is selected according to the evaluation indicators of each round as the trained cable segmentation model.

[0017] Optionally, obtaining the three-dimensional coordinate point set of the lower edge includes:

[0018] Get the parameters of the left and right view cameras of the binocular camera;

[0019] A left-view matrix model and a right-view matrix model are constructed based on the parameters of the left-view camera and the right-view camera respectively;

[0020] The left viewing angle matrix model is:

[0021]

[0022] The right viewing matrix model is:

[0023]

[0024] Among them, (u 1 ,v 1 )、(u 2 ,v 2 ) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, (X w ,Y w ,Z w ) is (u 1 ,v 1 )、(u 2 ,v 2 ) corresponds to the three-dimensional coordinate point; Z c1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras, respectively; P 1 and P 2 The left and right view rotation matrices are obtained according to the parameters of the left view camera and the right view camera respectively;

[0025] The left view matrix model and the right view matrix model are fitted by the least square method to obtain three-dimensional coordinate points, and a three-dimensional coordinate point set is generated according to the three-dimensional coordinate points.

[0026] Optionally, the rate of change d of the slope is:

[0027]

[0028] Where n is the number of straight line segments, k i is the slope of the i-th straight line segment.

[0029] In a second aspect, the present invention provides a cable deformation defect recognition device based on binocular vision, the device comprising:

[0030] An image acquisition module is used to acquire an image acquired by a binocular camera, wherein the acquired image includes a left-view image and a right-view image;

[0031] An image segmentation module is used to segment the left-view image and the right-view image based on the trained cable segmentation model to obtain a left-view edge image and a right-view edge image;

[0032] An edge point set module is used to extract the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image to generate a left-view edge point set and a right-view edge point set;

[0033] A three-dimensional coordinate point set module is used to obtain a lower edge three-dimensional coordinate point set according to the lower edge point set of the left view, the lower edge point set of the right view and the parameters of the binocular camera;

[0034] A slope acquisition module is used to perform piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment;

[0035] The defect judgment module is used to obtain the rate of change of the slope according to the slope of the straight line segment. If the rate of change of the slope is greater than a preset threshold, the cable has a deformation defect.

[0036] Optionally, the cable segmentation model is constructed based on Mask-RCNN; the training of the cable segmentation model includes:

[0037] Obtain multiple cable images and add edge labels to each cable image to generate sample images;

[0038] Divide sample images into training sets and validation sets;

[0039] Use each sample image in the training set to train the cable segmentation model; after each round of training with each sample image, use the sample image in the validation set to verify the training, and obtain the evaluation index of the cable segmentation model after this round of training;

[0040] The optimal model is selected according to the evaluation indicators of each round as the trained cable segmentation model.

[0041] Optionally, obtaining the three-dimensional coordinate point set of the lower edge includes:

[0042] Get the parameters of the left and right view cameras of the binocular camera;

[0043] A left-view matrix model and a right-view matrix model are constructed based on the parameters of the left-view camera and the right-view camera respectively;

[0044] The left viewing angle matrix model is:

[0045]

[0046] The right viewing matrix model is:

[0047]

[0048] Among them, (u 1 ,v 1 )、(u 2 ,v 2 ) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, (X w ,Y w ,Z w ) is (u 1 ,v 1 )、(u 2 ,v 2 ) corresponds to the three-dimensional coordinate point; Z c1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras, respectively; P 1 and P 2 The left and right view rotation matrices are obtained according to the parameters of the left view camera and the right view camera respectively;

[0049] The left view matrix model and the right view matrix model are fitted by the least square method to obtain three-dimensional coordinate points, and a three-dimensional coordinate point set is generated according to the three-dimensional coordinate points.

[0050] Optionally, the rate of change d of the slope is:

[0051]

[0052] Where n is the number of straight line segments, k i is the slope of the i-th straight line segment.

[0053] In a third aspect, a cable deformation defect recognition device based on binocular vision includes a processor and a storage medium;

[0054] The storage medium is used to store instructions;

[0055] The processor is used to operate according to the instructions to execute the steps according to the above method.

[0056] In a fourth aspect, a computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] The present invention provides a cable deformation defect identification method and device based on binocular vision, which overcomes the shortcoming that a monocular camera cannot determine cable deformation defects. The three-dimensional spatial coordinates of the cable are constructed through binocular vision to obtain the bending degree of the cable and determine whether it has a deformation defect. It has high recognition accuracy and provides guarantee for the safe operation of the cable. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flow chart of a cable deformation defect identification method based on binocular vision provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0061] Embodiment 1:

[0062] like Figure 1 As shown, an embodiment of the present invention provides a cable deformation defect recognition method based on binocular vision, comprising the following steps:

[0063] 1. Obtain the captured images of the binocular camera, which include a left-view image and a right-view image.

[0064] 2. Based on the trained cable segmentation model, the left view image and the right view image are segmented to obtain the left view edge image and the right view edge image respectively;

[0065] The cable segmentation model is built based on Mask-RCNN. The training of the cable segmentation model includes:

[0066] S21, acquiring multiple cable images and adding edge labels to each cable image to generate a sample image;

[0067] S22, dividing the sample images into a training set and a validation set;

[0068] S23, using each sample image in the training set to train the cable segmentation model; wherein, after each round of training using each sample image, the training is verified using the sample image in the verification set to obtain the evaluation index of the cable segmentation model after this round of training;

[0069] S24. Select the optimal model as the trained cable segmentation model according to the evaluation index of each round.

[0070] 3. Extract the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image respectively to generate the left-view edge point set and the right-view edge point set.

[0071] 4. Obtain the lower edge three-dimensional coordinate point set according to the edge point set under the left view, the edge point set under the right view and the parameters of the binocular camera; specifically:

[0072] Obtaining the three-dimensional coordinate point set of the lower edge includes:

[0073] S41, obtaining parameters of the left-view camera and the right-view camera of the binocular camera;

[0074] S42, constructing a left-view matrix model and a right-view matrix model based on parameters of the left-view camera and the right-view camera respectively;

[0075] The left view matrix model is:

[0076]

[0077] The right viewing matrix model is:

[0078]

[0079] Among them, (u 1 ,v 1 )、(u 2 ,v 2 ) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, (X w ,Y w ,Z w ) is (u 1 ,v 1 )、(u 2 ,v 2 ) corresponds to the three-dimensional coordinate point; Z c1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras, respectively; P 1 and P 2They are the left and right view rotation matrices obtained according to the parameters of the left view camera and the right view camera respectively; the camera parameters are composed of the product of the intrinsic parameter matrix and the extrinsic parameter matrix. The intrinsic parameter matrix is ​​determined by the factory, and the extrinsic parameter matrix describes a series of rotation and translation operations.

[0080] S44. Fitting the left-view matrix model and the right-view matrix model by the least square method to obtain three-dimensional coordinate points, and generating a three-dimensional coordinate point set according to the three-dimensional coordinate points.

[0081] 5. Perform piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments {l 1 ,l 2 ,…,l n} and the slope of each straight line segment {k 1 ,k 2 ,…,k n};

[0082] 6. Obtain the rate of change of the slope according to the slope of the straight line segment. The rate of change of the slope d is:

[0083]

[0084] Where n is the number of straight line segments, k i is the slope of the i-th straight line segment;

[0085] If the rate of change of the slope is greater than a preset threshold, the cable has a deformation defect.

[0086] Embodiment 2:

[0087] The embodiment of the present invention provides a cable deformation defect recognition device based on binocular vision, the device comprising:

[0088] An image acquisition module is used to obtain images acquired by a binocular camera, wherein the acquired images include a left-view image and a right-view image;

[0089] An image segmentation module is used to segment the left-view image and the right-view image based on the trained cable segmentation model to obtain a left-view edge image and a right-view edge image;

[0090] An edge point set module is used to extract the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image to generate a left-view edge point set and a right-view edge point set;

[0091] A three-dimensional coordinate point set module is used to obtain a lower edge three-dimensional coordinate point set according to the lower edge point set of the left view, the lower edge point set of the right view and the parameters of the binocular camera;

[0092] A slope acquisition module is used to perform piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment;

[0093] The defect judgment module is used to obtain the rate of change of the slope according to the slope of the straight line segment. If the rate of change of the slope is greater than a preset threshold, the cable has a deformation defect.

[0094] Specifically, the cable segmentation model is built based on Mask-RCNN; the training of the cable segmentation model includes:

[0095] Obtain multiple cable images and add edge labels to each cable image to generate sample images;

[0096] Divide sample images into training sets and validation sets;

[0097] Use each sample image in the training set to train the cable segmentation model; after each round of training with each sample image, use the sample image in the validation set to verify the training, and obtain the evaluation index of the cable segmentation model after this round of training;

[0098] The optimal model is selected according to the evaluation indicators of each round as the trained cable segmentation model.

[0099] Specifically, obtaining the three-dimensional coordinate point set of the lower edge includes:

[0100] Get the parameters of the left and right view cameras of the binocular camera;

[0101] A left-view matrix model and a right-view matrix model are constructed based on the parameters of the left-view camera and the right-view camera respectively;

[0102] The left view matrix model is:

[0103]

[0104] The right viewing matrix model is:

[0105]

[0106] Among them, (u 1 ,v 1 )、(u 2 ,v 2 ) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, (X w ,Y w ,Z w ) is (u 1 ,v 1 )、(u 2 ,v 2 ) corresponds to the three-dimensional coordinate point; Zc1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras, respectively; P 1 and P 2 The left and right view rotation matrices are obtained according to the parameters of the left view camera and the right view camera respectively;

[0107] The left view matrix model and the right view matrix model are fitted by the least square method to obtain three-dimensional coordinate points, and a three-dimensional coordinate point set is generated according to the three-dimensional coordinate points.

[0108] Specifically, the rate of change of the slope d is:

[0109]

[0110] Where n is the number of straight line segments, k i is the slope of the i-th straight line segment.

[0111] Embodiment three:

[0112] Based on the first embodiment, the embodiment of the present invention provides a cable deformation defect recognition device based on binocular vision, including a processor and a storage medium;

[0113] The storage medium is used to store instructions;

[0114] The processor is used to operate according to the instructions to execute the steps according to the above method.

[0115] Embodiment 4:

[0116] Based on the first embodiment, the embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A cable deformation defect recognition method based on binocular vision, characterized in that: include: Acquire a captured image from a binocular camera, wherein the captured image includes a left-view image and a right-view image; Based on the trained cable segmentation model, the left view image and the right view image are segmented to obtain the left view edge image and the right view edge image respectively; Extracting the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image to generate a left-view lower edge point set and a right-view lower edge point set; Obtain the lower edge three-dimensional coordinate point set according to the lower edge point set of the left view, the lower edge point set of the right view and the parameters of the binocular camera; Performing piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment; The rate of change of the slope is obtained according to the slope of the straight line segment. If the rate of change of the slope is greater than a preset threshold, the cable has a deformation defect.

2. A cable deformation defect identification method based on binocular vision according to claim 1, characterized in that: The cable segmentation model is built based on Mask-RCNN; The training of the cable segmentation model includes: Acquire multiple cable images and add edge labels to each cable image to generate a sample image; Divide the sample images into training set and validation set; Use each sample image in the training set to train the cable segmentation model. After each round of training with each sample image, use the sample images in the validation set to verify the training and obtain the evaluation index of the cable segmentation model after this round of training. The optimal model is selected according to the evaluation indicators of each round as the trained cable segmentation model.

3. The cable deformation defect identification method based on binocular vision according to claim 1 is characterized in that: The obtaining of the three-dimensional coordinate point set of the lower edge comprises: Get the parameters of the left and right view cameras of the binocular camera; Construct a left-view matrix model and a right-view matrix model based on the parameters of the left-view camera and the right-view camera respectively; The left viewing matrix model is: The right viewing matrix model is: Among them, (u1, v1) and (u2, v2) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, respectively. w ,Y w ,Z w ) is the three-dimensional coordinate point corresponding to (u1, v1) and (u2, v2); Z c1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras respectively; P 1 and P 2 The left and right view rotation matrices are obtained according to the parameters of the left view camera and the right view camera respectively; The left view matrix model and the right view matrix model are fitted by the least square method to obtain three-dimensional coordinate points, and a three-dimensional coordinate point set is generated according to the three-dimensional coordinate points.

4. The cable deformation defect identification method based on binocular vision according to claim 1 is characterized in that: The rate of change of the slope d is: Where n is the number of straight line segments, k i is the slope of the i-th straight line segment.

5. A cable deformation defect recognition device based on binocular vision, characterized in that: The device comprises: An image acquisition module is used to obtain an image captured by a binocular camera, wherein the image captured includes a left-view image and a right-view image; An image segmentation module is used to segment the left-view image and the right-view image based on the trained cable segmentation model to obtain a left-view edge image and a right-view edge image; An edge point set module is used to extract the coordinate points of the lower edge of the cable from the left-view edge image and the right-view edge image to generate a left-view lower edge point set and a right-view lower edge point set; A three-dimensional coordinate point set module is used to obtain the lower edge three-dimensional coordinate point set based on the left-view lower edge point set, the right-view lower edge point set, and the parameters of the binocular camera; A slope acquisition module is used to perform piecewise straight line fitting on the three-dimensional coordinate points in the three-dimensional coordinate point set to obtain continuous straight line segments and the slope of each straight line segment; The defect judgment module is used to obtain the slope change rate based on the slope of the straight line segment. If the slope change rate is greater than a preset threshold, the cable has a deformation defect.

6. The cable deformation defect identification device based on binocular vision according to claim 5 is characterized in that: The cable segmentation model is built based on Mask-RCNN; The training of the cable segmentation model includes: Acquire multiple cable images and add edge labels to each cable image to generate a sample image; Divide the sample images into training set and validation set; Use each sample image in the training set to train the cable segmentation model. After each round of training with each sample image, use the sample images in the validation set to verify the training and obtain the evaluation index of the cable segmentation model after this round of training. The optimal model is selected according to the evaluation indicators of each round as the trained cable segmentation model.

7. The cable deformation defect identification device based on binocular vision according to claim 5 is characterized in that: The obtaining of the three-dimensional coordinate point set of the lower edge comprises: Get the parameters of the left and right view cameras of the binocular camera; Construct a left-view matrix model and a right-view matrix model based on the parameters of the left-view camera and the right-view camera respectively; The left viewing matrix model is: The right viewing matrix model is: Among them, (u1, v1) and (u2, v2) are the corresponding coordinate points of the edge point set under the left view and the edge point set under the right view, respectively. w ,Y w ,Z w ) is the three-dimensional coordinate point corresponding to (u1, v1) and (u2, v2); Z c1 and Z c2 are the left and right perspective translation vectors obtained according to the parameters of the left and right perspective cameras respectively; P 1 and P 2 The left and right view rotation matrices are obtained according to the parameters of the left view camera and the right view camera respectively; The left view matrix model and the right view matrix model are fitted by the least square method to obtain three-dimensional coordinate points, and a three-dimensional coordinate point set is generated according to the three-dimensional coordinate points.

8. The cable deformation defect identification device based on binocular vision according to claim 5 is characterized in that: The rate of change of the slope d is: Where n is the number of straight line segments, k i is the slope of the i-th straight line segment.

9. A cable deformation defect recognition device based on binocular vision, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.