A method and device for detecting internal defects of pipelines using stereoscopic visual imaging

Abstract

The present invention relates to the technical field of engineering monitoring, and in particular to a method and device for detecting internal defects of a pipeline by stereoscopic visual imaging. The method comprises the following steps: obtaining a depth image of the interior of a pipeline to be detected; determining an initial frame image in the depth image, and selecting a key frame image located after a time series of the initial frame image according to a preset angle condition and / or mileage condition; constructing a three-dimensional model of the interior of the pipeline to be detected; extracting point cloud data from all the depth images, performing cylinder fitting and segmentation processing on all the point cloud data, and determining a cylinder segment of a non-cylindrical surface according to a preset cylinder determination threshold; bringing the position information and image information of the defect target into the three-dimensional model for reconstruction to obtain a three-dimensional model with defect information. The method can quickly construct an internal model of a pipeline with defect information.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering monitoring, and in particular to a method and a device for detecting internal defects of a pipeline using stereoscopic visual imaging. Background Art

[0002] As an important infrastructure, underground pipelines have played a significant role in the rapid development of social economy. With the improvement of underground pipelines, problems such as pipeline aging and corrosion defects have gradually emerged with the passage of time, which not only restricts the development of social economy, but also has a huge impact on people's life safety. Therefore, it is urgent to carry out scientific and effective detection of underground pipelines and timely repair pipelines with defects.

[0003] Traditional pipeline detection methods such as laser detection and magnetic leakage detection can no longer meet the current requirements of underground pipeline detection due to high equipment costs, inconvenience in carrying and poor analysis capabilities. In the era of the Internet of Things, there is an urgent need for a faster, more efficient and accurate detection method for underground pipelines. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method and device for detecting internal defects of a pipeline by stereoscopic visual imaging, which overcomes the above problems or at least partially solves the above problems.

[0005] The present invention provides a method for detecting internal defects of a pipeline by stereoscopic visual imaging, comprising:

[0006] Obtaining a depth image of the interior of the pipeline to be tested;

[0007] Determine an initial frame image in the depth image, and select a key frame image located after the initial frame image time sequence according to a preset angle condition and / or mileage condition;

[0008] Acquire first point cloud data in the current key frame image, generate a first grid model corresponding to the current key frame image according to the first point cloud data, generate a second grid model according to the second point cloud data in the previous key frame image, fuse the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and construct a three-dimensional model of the interior of the pipeline to be tested;

[0009] Extracting point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold;

[0010] Determine a target depth image corresponding to the cylindrical segment of the non-cylindrical surface, perform defect target recognition on the target depth image, locate the position information of the defect target, bring the position information and image information of the defect target into the three-dimensional model for reconstruction, and obtain a three-dimensional model with defect information.

[0011] Preferably, before acquiring the depth image inside the pipeline to be tested, the method includes:

[0012] Acquire a depth image acquired in real time by an acquisition device;

[0013] After the front-end device acquires the depth image in real time, it includes:

[0014] Identify key areas in the depth image collected in real time, and compress non-key areas;

[0015] Record the original image information of the non-key area and send the compressed depth image.

[0016] Preferably, the determining of the initial frame image in the depth image, selecting a key frame image located after the time sequence of the initial frame image according to a preset angle condition and / or mileage condition, comprises:

[0017] Acquire a time series of depth images arranged after the initial frame image;

[0018] Acquire a first time node when a preset angle condition is triggered in the gyroscope data, and / or a second time node when a preset mileage condition is triggered in the odometer data;

[0019] According to the first time node and / or the second time node, a corresponding depth image is searched in the time sequence and used as the key frame image.

[0020] Preferably, the step of acquiring first point cloud data in the current key frame image, generating a first mesh model corresponding to the current key frame image according to the first point cloud data, and generating a second mesh model according to second point cloud data in the previous key frame image comprises:

[0021] Performing noise reduction processing on the point cloud data of all key frame images to obtain noise-reduced point cloud data;

[0022] The Poisson algorithm is used to process the noise-reduced point cloud data to obtain a grid model.

[0023] Preferably, the calculated rotation and translation matrix between the current key frame image and the previous key frame image is integrated into the first grid model and the second grid model, including:

[0024] Calibrate the intrinsic parameters and extrinsic parameters of the acquisition device, obtain the pixel coordinate values ​​and physical coordinate values ​​of the previous key frame image and the current key frame image, and calculate the rotation and translation matrix between the previous key frame image and the current key frame image;

[0025] The angle change data and mileage change data of the acquisition device between the time nodes of the previous key frame image and the current key frame image are obtained, the rotation and translation matrix is ​​corrected according to the angle change data and the mileage change data, and the corrected rotation and translation matrix is ​​used in the fusion grid model.

[0026] Preferably, extracting the point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold comprises:

[0027] Based on the result of the segmentation processing, the central axis point of each segmented cylinder is calculated to determine the radius of each segmented cylinder;

[0028] A cylinder radius threshold is preset corresponding to each segmented cylinder, and whether the corresponding segmented cylinder is a cylinder is determined according to the cylinder radius threshold.

[0029] Preferably, after obtaining the three-dimensional model with defect information, the method further comprises:

[0030] Obtain the video stream data collected by the front-end acquisition device;

[0031] The three-dimensional model with defect information and the video stream data are sent.

[0032] A device for detecting internal defects of a pipeline by stereoscopic visual imaging is also provided, comprising:

[0033] An image acquisition module is used to acquire a depth image of the interior of the pipeline to be tested;

[0034] A key frame determination module, used to determine an initial frame image in the depth image, and select a key frame image located after the time sequence of the initial frame image according to a preset angle condition and / or mileage condition;

[0035] A model building module is used to obtain first point cloud data in the current key frame image, generate a first grid model corresponding to the current key frame image according to the first point cloud data, generate a second grid model according to the second point cloud data in the previous key frame image, fuse the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and build a three-dimensional model of the interior of the pipeline to be tested;

[0036] A point cloud fitting module, used to extract point cloud data from all the depth images, perform cylindrical fitting on all the point cloud data and process them in segments, and determine the cylindrical segments of non-cylindrical surfaces according to a preset cylindrical determination threshold;

[0037] A model reconstruction module is used to determine a target depth image corresponding to the cylindrical segment of the non-cylindrical surface, identify defective targets in the target depth image, locate the position information of the defective targets, bring the position information and image information of the defective targets into the three-dimensional model for reconstruction, and obtain a three-dimensional model with defect information.

[0038] An electronic device is also provided, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the method for detecting internal defects of a pipeline by stereoscopic visual imaging as described above is implemented.

[0039] A computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting internal defects of a pipeline by stereoscopic visual imaging as described above is implemented.

[0040] The embodiments of the present invention include the following advantages:

[0041] The method provided by the present invention selects key frame images from the depth image collected by the binocular camera to initially build a three-dimensional model. The modeling speed is fast and is conducive to real-time transmission. The approximate defect position in the pipeline is found by fitting the point cloud data in the image, and then the depth image of the approximate time node is located. After the defect is identified in the depth image, the depth image with defect characteristics is brought into the three-dimensional model for reconstruction, and a three-dimensional model that can display the defect feature information inside the pipeline is obtained, so that the defect of the pipeline can be quickly found when provided to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the steps of a method for detecting internal defects of a pipeline by stereoscopic visual imaging according to the present invention;

[0043] Figure 2 It is a structural block diagram of a device for detecting internal defects of a pipeline using stereoscopic visual imaging according to the present invention.

[0044] Figure 3 It is a structural block diagram of a computer device for detecting internal defects of a pipeline by stereoscopic visual imaging according to the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] The execution subject of the method for detecting internal defects of a pipeline by stereoscopic visual imaging provided in the embodiment of the present application is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user device, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital processing, a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the user classification model training method and the user classification method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0047] The following describes a method for detecting internal defects of a pipeline using stereoscopic visual imaging disclosed in the present application, taking a computer device as an example of an execution subject.

[0048] like Figure 1 , shows a flow chart of the steps of a method for detecting internal defects of a pipeline by stereoscopic visual imaging according to the present invention, the method steps comprising:

[0049] S100, obtaining a depth image of the interior of the pipeline to be tested;

[0050] Specifically, the device for collecting the depth image inside the pipeline uses a miniaturized binocular camera, which can collect depth images in the inner wall of the pipeline of 300 to 500 mm. The collected depth images are transmitted to the backend server via optical fiber, and the backend server splices the acquired depth images to form a three-dimensional model of the pipeline for users to use.

[0051] Since the internal environment of the pipeline is dark, fill lights are set around the binocular camera used in this method to compensate for the optical path inside the pipeline so that the collected depth image meets the requirements of clarity and frame rate.

[0052] S200, determining an initial frame image in the depth image, and selecting a key frame image located after the initial frame image time sequence according to a preset angle condition and / or mileage condition;

[0053] It should be noted that the above initial frame image is the first key frame image collected by the binocular camera after it stabilizes in the pipeline. In addition to the above-mentioned fill light, the binocular camera is also equipped with a walking mechanism to walk along the pipeline for shooting. At the same time, it is also equipped with a gyroscope to record the angle change of the binocular camera and an odometer to measure the walking distance.

[0054] It is understandable that when the binocular camera moves inside the pipeline, the gyroscope can record the data of the angle deflection of the binocular camera in the pipeline, and the odometer can record the distance data of the binocular camera in the pipeline, and transmit the recorded data together with the collected image data to the background server. In the background server, the corresponding angle conditions and / or mileage conditions are pre-set, and when a certain condition is triggered, the corresponding image in the acquired image data is extracted as a key frame image to improve the data processing speed.

[0055] S300, acquiring first point cloud data in the current key frame image, generating a first grid model corresponding to the current key frame image according to the first point cloud data, generating a second grid model according to the second point cloud data in the previous key frame image, fusing the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and constructing a three-dimensional model of the interior of the pipeline to be tested;

[0056] Specifically, based on the depth image captured by the binocular camera, the point cloud data in the image can be quickly extracted, a grid model can be constructed through the point cloud data, and then the rotation and translation matrices between adjacent images can be fused to construct a three-dimensional model between adjacent images. The above technical solution extracts key frame images, reduces the amount of data calculation, and splices the three-dimensional models constructed by multiple adjacent key frame images to form a three-dimensional model inside the pipeline based on the initial key frame image to the current key frame image.

[0057] S400, extracting point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold;

[0058] Specifically, by performing cylinder fitting on the point clouds in all depth images, the non-cylindrical part of the fitted cylinder can be quickly determined according to the preset cylinder determination threshold. Of course, for underground pipelines, the entire pipeline is not of uniform size, and the cylinder determination threshold should also be set accordingly according to the actual different pipeline segments. It can be understood that the non-cylindrical cylindrical segment is a defective segment, which can be a pipeline rupture, a pipeline deformation due to extrusion, a foreign object penetration, or a residual obstacle. The information of these defective segments can be presented as point cloud data in the depth image captured by the binocular camera.

[0059] S500, determining a target depth image corresponding to the cylindrical segment of the non-cylindrical surface, identifying a defect target on the target depth image, locating position information of the defect target, bringing the position information and image information of the defect target into the three-dimensional model for reconstruction, and obtaining a three-dimensional model with defect information.

[0060] Specifically, the corresponding target depth image is found through the information of the cylindrical segment of the non-cylindrical surface, and the defective target is identified using image recognition technology. Since the gyroscope and odometer carried by the binocular camera can locate the position information of the defective target photographed by the binocular camera in the pipeline, the position information of the defective target in the three-dimensional pipeline can also be determined. Finally, the originally constructed three-dimensional model is reconstructed through the obtained position information and image information, that is, the final three-dimensional model with defect information is obtained. The user can operate the final three-dimensional model to intuitively and quickly view the information about the position, shape and size of the defective part in the pipeline model.

[0061] The above embodiment will be further described below.

[0062] In one embodiment, before acquiring the depth image inside the pipeline to be tested, the process includes:

[0063] Acquire a depth image acquired in real time by an acquisition device;

[0064] After the front-end device acquires the depth image in real time, it includes:

[0065] Identify key areas in the depth image collected in real time, and compress non-key areas;

[0066] Record the original image information of the non-key area and send the compressed depth image.

[0067] The front-end acquisition device of the binocular camera used in this embodiment is embedded with an image preprocessing element, which can automatically identify and compress the collected complete depth image. Specifically, after identifying the non-key areas in the image, the non-key areas are pre-compressed, and the depth image of the key areas is retained after transmission compression. It can be understood that through the recognition training of the internal features of the pipeline and the defective targets, the key areas in the image can be quickly identified, and the compression of the non-key areas will not affect the image quality after transmission. The original complete image can be restored by viewing the compressed record.

[0068] In one embodiment, the determining of the initial frame image in the depth image, and selecting a key frame image located after the time sequence of the initial frame image according to a preset angle condition and / or mileage condition, includes:

[0069] Acquire a time series of depth images arranged after the initial frame image;

[0070] Acquire a first time node when a preset angle condition is triggered in the gyroscope data, and / or a second time node when a preset mileage condition is triggered in the odometer data;

[0071] According to the first time node and / or the second time node, a corresponding depth image is searched in the time sequence and used as the key frame image.

[0072] It can be understood that the first frame image taken by the binocular camera after it is stabilized in the pipeline can be used as the initial starting point for mileage recording or angle monitoring, and the image collected at the same time node when the binocular camera travels a certain mileage and / or the binocular camera is deflected by a certain angle can be set as the selected depth image.

[0073] In one embodiment, the step of acquiring first point cloud data in a current key frame image, generating a first mesh model corresponding to the current key frame image according to the first point cloud data, and generating a second mesh model according to second point cloud data in a previous key frame image includes:

[0074] Performing noise reduction processing on the point cloud data of all key frame images to obtain noise-reduced point cloud data;

[0075] The Poisson algorithm is used to process the noise-reduced point cloud data to obtain a grid model.

[0076] It can be understood that the mesh model can be a triangular mesh.

[0077] In one embodiment, the calculated rotation and translation matrix between the current key frame image and the previous key frame image is merged into the first mesh model and the second mesh model, including:

[0078] Calibrate the intrinsic parameters and extrinsic parameters of the acquisition device, obtain the pixel coordinate values ​​and physical coordinate values ​​of the previous key frame image and the current key frame image, and calculate the rotation and translation matrix between the previous key frame image and the current key frame image;

[0079] The angle change data and mileage change data of the acquisition device between the time nodes of the previous key frame image and the current key frame image are obtained, the rotation and translation matrix is ​​corrected according to the angle change data and the mileage change data, and the corrected rotation and translation matrix is ​​used in the fusion grid model.

[0080] In one embodiment, extracting point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold includes:

[0081] Based on the result after the segmentation processing, the central axis point of each segmented cylinder is calculated to determine the radius of each segmented cylinder;

[0082] A cylinder radius threshold is preset corresponding to each segmented cylinder, and whether the corresponding segmented cylinder is a cylinder is determined according to the cylinder radius threshold.

[0083] It can be understood that each segmented cylinder corresponds to a judgment condition of a cylinder radius threshold. After the measured radius is determined, if the condition does not meet the preset judgment condition, the segmented cylinder is judged as a non-cylinder and this signal is transmitted.

[0084] In one embodiment, after obtaining the three-dimensional model with defect information, the method includes:

[0085] Obtain the video stream data collected by the front-end acquisition device;

[0086] The three-dimensional model with defect information and the video stream data are sent.

[0087] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0088] Reference Figure 2 , shows a device for detecting internal defects of a pipeline by stereoscopic visual imaging, comprising:

[0089] An image acquisition module 100 is used to acquire a depth image of the interior of the pipeline to be tested;

[0090] A key frame determination module 200 is used to determine an initial frame image in the depth image, and select a key frame image located after the initial frame image time sequence according to a preset angle condition and / or mileage condition;

[0091] The model building module 300 is used to obtain the first point cloud data in the current key frame image, generate a first grid model corresponding to the current key frame image according to the first point cloud data, generate a second grid model according to the second point cloud data in the previous key frame image, fuse the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and build a three-dimensional model of the interior of the pipeline to be tested;

[0092] The point cloud fitting module 400 is used to extract the point cloud data in all the depth images, perform cylindrical fitting on all the point cloud data and process them in segments, and determine the cylindrical segments of the non-cylindrical surface according to a preset cylinder determination threshold;

[0093] The model reconstruction module 500 is used to determine the target depth image corresponding to the cylindrical segment of the non-cylindrical surface, identify the defect target on the target depth image, locate the position information of the defect target, bring the position information and image information of the defect target into the three-dimensional model for reconstruction, and obtain a three-dimensional model with defect information.

[0094] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0095] like Figure 3 , shows a computer device for detecting internal defects of a pipeline by stereoscopic visual imaging of the present invention, which may specifically include the following:

[0096] In an embodiment of the present invention, the present invention also provides a computer device. The above-mentioned computer device 12 is manifested in the form of a general computing device. The components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0097] The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.

[0098] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0099] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 31 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). Although Figure 3Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, which are configured to perform the functions of various embodiments of the present invention.

[0100] A program / utility 41 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0101] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), may communicate with one or more devices that enable a user to interact with the computer device 12, and / or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and / or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 21 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

[0102] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a method for detecting internal defects of a pipeline by stereoscopic visual imaging provided by an embodiment of the present invention.

[0103] In an embodiment of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for detecting internal defects of a pipeline by stereoscopic visual imaging as provided in all embodiments of the present application.

[0104] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPOM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device.

[0105] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0106] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0107] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0108] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0109] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0113] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0114] The above is a detailed introduction to the method and device for detecting internal defects of pipelines by stereoscopic visual imaging provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting internal defects of a pipeline by stereoscopic visual imaging, characterized in that: include: Obtaining a depth image of the interior of the pipeline to be tested; Determine an initial frame image in the depth image, and select a key frame image located after the initial frame image time sequence according to a preset angle condition and / or mileage condition; Acquire first point cloud data in the current key frame image, generate a first grid model corresponding to the current key frame image according to the first point cloud data, generate a second grid model according to the second point cloud data in the previous key frame image, fuse the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and construct a three-dimensional model of the interior of the pipeline to be tested; Extracting point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold; Determine a target depth image corresponding to the cylindrical segment of the non-cylindrical surface, perform defect target recognition on the target depth image, locate the position information of the defect target, bring the position information and image information of the defect target into the three-dimensional model for reconstruction, and obtain a three-dimensional model with defect information.

2. The method according to claim 1, characterized in that Before obtaining the depth image inside the pipeline to be tested, the method includes: Acquire a depth image acquired in real time by an acquisition device; After the acquisition device acquires the depth image in real time, the method includes: Identify key areas in the depth image collected in real time, and compress non-key areas; Record the original image information of the non-key area and send the compressed depth image.

3. The method according to claim 1, characterized in that The determining of the initial frame image in the depth image and selecting a key frame image located after the time sequence of the initial frame image according to a preset angle condition and / or mileage condition include: Acquire a time series of depth images arranged after the initial frame image; Acquire a first time node when a preset angle condition is triggered in the gyroscope data, and / or a second time node when a preset mileage condition is triggered in the odometer data; According to the first time node and / or the second time node, a corresponding depth image is searched in the time sequence and used as the key frame image.

4. The method according to claim 1, characterized in that The step of acquiring first point cloud data in the current key frame image, generating a first mesh model corresponding to the current key frame image according to the first point cloud data, and generating a second mesh model according to second point cloud data in the previous key frame image includes: Performing noise reduction processing on the point cloud data of all key frame images to obtain noise-reduced point cloud data; The Poisson algorithm is used to process the noise-reduced point cloud data to obtain a grid model.

5. The method according to claim 1, characterized in that The calculated rotation and translation matrix between the current key frame image and the previous key frame image is integrated into the first mesh model and the second mesh model, including: Calibrate the intrinsic parameters and extrinsic parameters of the acquisition device, obtain the pixel coordinate values ​​and physical coordinate values ​​of the previous key frame image and the current key frame image, and calculate the rotation and translation matrix between the previous key frame image and the current key frame image; The angle change data and mileage change data of the acquisition device between the time nodes of the previous key frame image and the current key frame image are obtained, the rotation and translation matrix is ​​corrected according to the angle change data and the mileage change data, and the corrected rotation and translation matrix is ​​used in the fusion grid model.

6. The method according to claim 1, characterized in that The step of extracting the point cloud data from all the depth images, performing cylinder fitting and segment processing on all the point cloud data, and determining the cylinder segment of the non-cylindrical surface according to a preset cylinder determination threshold comprises: Based on the result after the segmentation processing, the central axis point of each segmented cylinder is calculated to determine the radius of each segmented cylinder; A cylinder radius threshold is preset corresponding to each segmented cylinder, and whether the corresponding segmented cylinder is a cylinder is determined according to the cylinder radius threshold.

7. The method according to claim 1, characterized in that After the three-dimensional model with defect information is obtained, the method includes: Obtain the video stream data collected by the front-end acquisition device; The three-dimensional model with defect information and the video stream data are sent.

8. A device for detecting internal defects of a pipeline by stereoscopic visual imaging, characterized in that: include: An image acquisition module is used to acquire a depth image of the interior of the pipeline to be tested; A key frame determination module, used to determine an initial frame image in the depth image, and select a key frame image located after the time sequence of the initial frame image according to a preset angle condition and / or mileage condition; A model building module is used to obtain first point cloud data in the current key frame image, generate a first grid model corresponding to the current key frame image according to the first point cloud data, generate a second grid model according to the second point cloud data in the previous key frame image, fuse the calculated rotation and translation matrix between the current key frame image and the previous key frame image into the first grid model and the second grid model, and build a three-dimensional model of the interior of the pipeline to be tested; A point cloud fitting module, used to extract point cloud data from all the depth images, perform cylindrical fitting on all the point cloud data and process them in segments, and determine the cylindrical segments of non-cylindrical surfaces according to a preset cylindrical determination threshold; A model reconstruction module is used to determine a target depth image corresponding to the cylindrical segment of the non-cylindrical surface, identify defective targets in the target depth image, locate the position information of the defective targets, bring the position information and image information of the defective targets into the three-dimensional model for reconstruction, and obtain a three-dimensional model with defect information.

9. An electronic device, characterized in that The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images