A pipeline self-calibration twin modeling method and related equipment

Abstract

The present application belongs to the technical field of pipeline detection, and discloses a pipeline self-calibration twin modeling method and related equipment. A positioning feature pattern is projected on the inner wall of the pipeline to be detected, and a pan-tilt camera is used to identify the positioning feature pattern to generate key feature points that are added to local point cloud data. The key feature points are used as reference points when splicing local point cloud data, thereby reducing splicing errors and effectively improving the accuracy of modeling results.

Description

Technical Field

[0001] The present application relates to the technical field of pipeline detection, and in particular to a pipeline self-calibration twin modeling method and related equipment. Background Art

[0002] When inspecting underground pipelines, pipeline robots are often used to perform twin modeling on the inner surface of the pipeline, and the results of the twin modeling are used to identify whether the pipeline is damaged. When using a pipeline robot to perform twin modeling on the inner surface of the pipeline, the inner surface of the pipeline is generally scanned using a laser scanning device carried by the pipeline robot to obtain multiple frames of local point cloud data, which are then spliced ​​to obtain overall point cloud data, and finally the overall point cloud data is used for twin modeling.

[0003] However, the inner surface of a general pipeline does not have obvious local features that are different from other locations. When stitching local point cloud data, it is impossible to extract key points from the local feature positions as stitching reference points, which makes the stitching of local point cloud data difficult and easily produces large stitching errors, resulting in poor authenticity of the final modeling results. Summary of the invention

[0004] The purpose of this application is to provide a pipeline self-calibration twin modeling method and related equipment, which can effectively improve the accuracy of modeling results.

[0005] In a first aspect, the present application provides a pipeline self-calibration twin modeling method, which performs twin modeling of a pipeline under test based on a pipeline robot carrying a laser scanning device, a pan-tilt camera, and a positioning laser light; the method comprises the following steps:

[0006] A1. Projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; a set of positioning feature patterns includes a plurality of positioning feature patterns spaced apart along the axial straight line of the measured pipe;

[0007] A2. Scan the measured pipeline using the laser scanning device to obtain multiple frames of local point cloud data;

[0008] A3. Using the pan-tilt camera to identify the positioning feature pattern, generate key feature points of each frame of the local point cloud data and add them to the corresponding local point cloud data;

[0009] A4. splicing the local point cloud data based on the key feature points to obtain the overall point cloud data;

[0010] A5. Generate a twin model of the measured pipeline based on the overall point cloud data.

[0011] By projecting a positioning feature pattern on the inner wall of the measured pipe, and using a pan-tilt camera to identify the positioning feature pattern to generate key feature points and add them to the local point cloud data, the key feature points are used as reference points when splicing the local point cloud data, thereby reducing the splicing error and effectively improving the accuracy of the modeling results.

[0012] Preferably, a group of the positioning characteristic patterns includes a first two-dimensional code located at the head end, a second two-dimensional code located at the end, and a plurality of different first characteristic patterns located between the first two-dimensional code and the second two-dimensional code;

[0013] After step A3 and before step A4, the method further includes the following steps:

[0014] A6. If the pan-tilt camera detects the second two-dimensional code, the projection position of the positioning feature pattern is updated, and the process returns to step A1 until the entire scan of the pipeline under test is completed.

[0015] Since the projection range of the positioning laser light is limited, each time the pipeline robot advances a certain distance, the projection position is re-determined and projected, so that the projection position moves forward step by step with the pipeline robot. The second QR code is used as the identifier of the end position of the same group of positioning feature patterns to trigger the pipeline robot to update the projection position, thereby realizing automatic update of the projection position.

[0016] Preferably, step A1 comprises:

[0017] During the movement of the pipeline robot, the projection direction of the positioning laser light is adjusted according to the real-time posture data of the pipeline robot to keep the position of the positioning feature pattern fixed.

[0018] Preferably, the real-time posture data is determined based on the measured posture data measured by the sensor carried by the pipeline robot and the travel distance measured by a meter on the wire take-up vehicle of the pipeline robot.

[0019] Based on the measured posture data measured by the sensors carried by the pipeline robot, the real-time posture data of the pipeline robot is determined in combination with the travel distance measured by the meter on the wire-winding vehicle. This can improve the accuracy of the real-time posture data and more effectively ensure the fixation of the position of the positioning feature pattern.

[0020] Preferably, step A2 comprises:

[0021] During the movement of the pipeline robot, the laser scanning device is used to scan the pipeline to be measured to obtain point cloud data in the coordinate system of the laser scanning device, which is recorded as preliminary point cloud data;

[0022] The coordinate data of the preliminary point cloud data is converted into coordinate data in a reference coordinate system to obtain the local point cloud data.

[0023] Preferably, step A3 comprises:

[0024] During the movement of the pipeline robot, the pan-tilt camera is used to continuously collect images of the inner wall of the pipeline under test in real time;

[0025] Identify the positioning feature pattern from each frame of the image, and obtain the coordinates of the positioning feature pattern in the gimbal camera coordinate system;

[0026] Convert the coordinates of the positioning feature pattern in the gimbal camera coordinate system into coordinates in the reference coordinate system, which are recorded as feature coordinates;

[0027] According to the feature coordinates, key feature points of the local point cloud data of each frame are generated and added to the corresponding local point cloud data.

[0028] In the second aspect, the present application provides a pipeline self-calibration twin modeling device, which performs twin modeling of the pipeline under test based on a pipeline robot carrying a laser scanning device, a pan-tilt camera, and a positioning laser light; including:

[0029] A projection module, used for projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; the set of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipe;

[0030] A scanning module, used for scanning the pipeline under test using the laser scanning device to obtain multiple frames of local point cloud data;

[0031] A recognition module, used to use the pan / tilt camera to recognize the positioning feature pattern, generate key feature points of the local point cloud data of each frame and add them to the corresponding local point cloud data;

[0032] A splicing module, used for splicing the local point cloud data based on the key feature points to obtain overall point cloud data;

[0033] A model generation module is used to generate a twin model of the measured pipeline according to the overall point cloud data.

[0034] In a third aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, it runs the steps in the pipeline self-calibration twin modeling method described above.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, runs the steps in the pipeline self-calibration twin modeling method described above.

[0036] In a fifth aspect, the present application provides a pipeline self-calibration twin modeling system, comprising a ground terminal, a wire take-up vehicle and a pipeline robot, wherein the ground terminal is communicatively connected to the wire take-up vehicle, and the wire take-up vehicle is connected to the pipeline robot via a cable; the wire take-up vehicle is used to retract and release the cable, and to supply power to the pipeline robot via the cable and communicate with the pipeline robot; the ground terminal monitors the pipeline robot via the wire take-up vehicle; the pipeline robot is provided with a laser scanning device, a pan / tilt camera and a positioning laser light;

[0037] The ground terminal is used to perform:

[0038] B1. Projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; a set of positioning feature patterns includes a plurality of positioning feature patterns spaced apart along the axial straight line of the measured pipe;

[0039] B2. Scan the measured pipeline using the laser scanning device to obtain multiple frames of local point cloud data;

[0040] B3. Using the PTZ camera to identify the positioning feature pattern, generating key feature points of each frame of the local point cloud data and adding them to the corresponding local point cloud data;

[0041] B4. splicing the local point cloud data based on the key feature points to obtain the overall point cloud data;

[0042] B5. Generate a twin model of the measured pipeline based on the overall point cloud data.

[0043] Beneficial effects: The pipeline self-calibration twin modeling method and related equipment provided in the present application project a positioning feature pattern on the inner wall of the measured pipeline, and use a pan-tilt camera to identify the positioning feature pattern to generate key feature points and add them to the local point cloud data. The key feature points are used as reference points when splicing the local point cloud data, thereby reducing splicing errors and effectively improving the accuracy of the modeling results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a pipeline self-calibration twin modeling method provided in an embodiment of the present application.

[0045] Figure 2 A schematic diagram of the structure of a pipeline self-calibration twin modeling device provided in an embodiment of the present application.

[0046] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0047] Figure 4 A schematic diagram of the structure of the pipeline self-calibration twin modeling system provided in an embodiment of the present application.

[0048] Figure 5 This is a schematic diagram of the working status of the pipeline robot.

[0049] Explanation of reference numerals: 1. Projection module; 2. Scanning module; 3. Recognition module; 4. Stitching module; 5. Model generation module; 100. Ground terminal; 200. Wire-reeling vehicle; 300. Pipeline robot; 400. Laser scanning device; 500. PTZ camera; 600. Positioning laser light; 301. Processor; 302. Memory; 303. Communication bus. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0051] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0052] Please refer to Figure 1 , Figure 1 A pipeline self-calibration twin modeling method in some embodiments of the present application is based on a pipeline robot carrying a laser scanning device, a pan-tilt camera and a positioning laser light to perform twin modeling on the pipeline under test; the method comprises the following steps:

[0053] A1. Use a positioning laser lamp to project a set of fixed positioning feature patterns on the inner wall of the measured pipe; a set of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipe;

[0054] A2. Use a laser scanning device to scan the pipeline to obtain multiple frames of local point cloud data;

[0055] A3. Use the PTZ camera to identify and locate feature patterns, generate key feature points of each frame of local point cloud data and add them to the corresponding local point cloud data;

[0056] A4. Based on the key feature points, local point cloud data are spliced ​​to obtain the overall point cloud data;

[0057] A5. Generate a twin model of the tested pipeline based on the overall point cloud data.

[0058] By projecting a positioning feature pattern on the inner wall of the measured pipe, and using a pan-tilt camera to identify the positioning feature pattern to generate key feature points and add them to the local point cloud data, the key feature points are used as reference points when splicing the local point cloud data, thereby reducing the splicing error and effectively improving the accuracy of the modeling results.

[0059] Among them, the pipeline self-calibration twin modeling method can be applied to Figure 4 The surface terminal 100 in the pipeline self-calibration twin modeling system is shown, but is not limited thereto.

[0060] The positioning feature patterns in a group of positioning feature patterns may be arranged at equal intervals or at unequal intervals.

[0061] Preferably, each positioning feature pattern in a group of positioning feature patterns is different from each other, so that it is easier to distinguish different key feature points.

[0062] In some preferred embodiments, a set of positioning feature patterns includes a first two-dimensional code at the head end, a second two-dimensional code at the end, and a plurality of different first feature patterns between the first two-dimensional code and the second two-dimensional code. Figure 5 In the figure, a is the first two-dimensional code, b is the second two-dimensional code, c is the first characteristic image, and d is the pipeline under test.

[0063] The first two-dimensional code and the second two-dimensional code may record different two-dimensional code information. For example, the two-dimensional code information of the first two-dimensional code includes a starting symbol (representing the starting point of a set of positioning feature patterns currently projected) and a group number of a set of positioning feature patterns currently projected; the second two-dimensional code includes an ending symbol (representing the ending point of a set of positioning feature patterns currently projected) and a group number of a set of positioning feature patterns currently projected; wherein the starting symbol is an identification symbol indicating that the pipeline robot has started the recognition process of the set of positioning feature patterns currently projected, the ending symbol is an identification symbol indicating that the pipeline robot has ended the recognition process of the set of positioning feature patterns currently projected, and the group number in the two-dimensional code information is an identity identification symbol of a set of positioning feature patterns currently projected; the group numbers in the two-dimensional code information of different groups of positioning feature patterns are different.

[0064] In this embodiment, after step A3 and before step A4, the following steps are further included:

[0065] A6. If the PTZ camera detects the second QR code, the projection position of the positioning feature pattern is updated and the process returns to step A1 until the entire scan of the pipeline under test is completed.

[0066] Since the projection range of the positioning laser light is limited, each time the pipeline robot advances a certain distance, the projection position is re-determined and projected, so that the projection position moves forward step by step with the pipeline robot. The second QR code is used as the identifier of the end position of the same group of positioning feature patterns to trigger the pipeline robot to update the projection position, thereby realizing automatic update of the projection position.

[0067] Specifically, when the pan-tilt camera detects the second QR code and identifies the termination symbol therefrom, it determines that the pipeline robot has completed the recognition process of the currently projected set of positioning feature patterns, thereby updating the projection position of the positioning feature pattern, and returning to execute step A1, so that the positioning laser light projects the next set of positioning feature patterns according to the updated projection position.

[0068] Among them, when updating the projection position of the positioning feature pattern, the projection position of the second two-dimensional code in the currently projected set of positioning feature patterns can be used as the projection position of the first two-dimensional code in the next set of positioning feature patterns. The other positioning feature patterns are calculated based on their relative position relationship with the first two-dimensional code (the relative position relationship is preset according to actual needs, for example, each positioning feature pattern maintains a preset distance from the first two-dimensional code) to ensure that the local point cloud data obtained before and after the projection position is updated have key feature points with the same position as reference points for the splicing process.

[0069] Specifically, step A1 includes:

[0070] During the movement of the pipeline robot, the projection direction of the positioning laser light is adjusted according to the real-time posture data of the pipeline robot to keep the position of the positioning feature pattern fixed.

[0071] In fact, the positioning laser light can be a mechanical laser light or a phased laser light (phased array laser light).

[0072] Mechanical laser lights are driven to rotate by mechanical means to adjust the projection direction. When the positioning laser light is a mechanical laser light, if multiple positioning feature patterns are projected at the same time, since the inner wall of the pipe is not an arc with the positioning laser light as the center point, as the mechanical laser light rotates, the relative positions of the positioning feature patterns projected onto the inner wall of the pipe will change, making it difficult to ensure that the positions of each positioning feature pattern are fixed; and, since the rotation speed of the mechanical laser light is limited, if each positioning feature pattern is projected in a sequential circular projection manner, then within the exposure time of each frame of the pan-tilt camera, it is impossible to ensure that all positioning feature patterns are projected, so that the image captured by the pan-tilt camera can generally only show one positioning feature pattern projected within the shooting range, so the sequential circular projection method has no practical significance. For this reason, the positioning feature patterns can be projected one by one, and each positioning feature pattern is projected for a preset duration to ensure that there is a same key feature point between two adjacent frames of local point cloud data. This method makes it easy to ensure that the projection position of each positioning feature pattern remains fixed.

[0073] Phased laser light adjusts the projection direction by adjusting the phase of each light source in the laser light, and can project each positioning feature pattern in a cycle at a very high speed. When the positioning laser light is a phased laser light, each positioning feature pattern can be projected in a cycle. Since the projection direction and projection pattern can be quickly switched, multiple cycle projections of all positioning feature patterns can be completed within the exposure time of each frame of the gimbal camera, so that the image captured by the gimbal camera can display all positioning feature patterns projected within the shooting range (equivalent to realizing the simultaneous projection of all positioning feature patterns in disguise), and since the projection position of each positioning feature pattern can be adjusted separately, it can better ensure that the projection position of each positioning feature pattern is fixed. In addition, two adjacent frames of local point cloud data can have multiple identical key feature points, which is conducive to further improving the stitching accuracy and the accuracy of the modeling results.

[0074] Among them, the pipeline robot is provided with sensors for measuring posture data, such as a gyroscope and an odometer. The gyroscope is used to measure the posture angle of the pipeline robot, and the odometer is used to measure the forward distance of the pipeline robot. The posture data of the pipeline robot in the reference coordinate system (the reference coordinate system is a fixed coordinate system, for example, the coordinate system of the pipeline robot at the initial moment of the twin modeling process of the measured pipeline can be used as the reference coordinate system, so that at the initial moment, the coordinate system of the pipeline robot coincides with the reference coordinate system) is calculated according to the posture angle and the forward distance (the specific calculation method is the existing technology and is not described in detail here). The posture data of the pipeline robot in the reference coordinate system measured by the sensor carried by the pipeline robot is referred to as the measured posture data. The measured posture data can be directly used as the real-time posture data to adjust the projection direction of the positioning laser light.

[0075] However, more preferably, when the pipeline robot is connected to the wire take-up vehicle via a cable and a meter is provided on the wire take-up vehicle (for example Figure 4 In the pipeline self-calibration twin modeling system shown, the pipeline robot 300 and the wire take-up vehicle 200 are determined in real time according to the measured posture data measured by the sensor carried by the pipeline robot and the travel distance measured by the meter on the wire take-up vehicle based on the pipeline robot. For example, the posture angle of the pipeline robot measured by the sensor carried by the pipeline robot is used as the effective posture angle. When the deviation between the travel distance measured by the meter and the forward distance measured by the sensor carried by the pipeline robot is within the allowable range (which can be set according to actual needs), the average value of the travel distance measured by the meter and the forward distance measured by the sensor carried by the pipeline robot is calculated as the effective forward distance. When the deviation between the travel distance measured by the meter and the forward distance measured by the sensor carried by the pipeline robot is not within the allowable range, the travel distance measured by the meter is used as the effective forward distance. Finally, the posture data of the pipeline robot in the reference coordinate system is calculated according to the effective posture angle and the effective forward distance as the real-time posture data.

[0076] Sometimes, due to problems such as slipping and obstacles, there will be a large error between the distance measured by the odometer and the actual distance traveled. Based on the measurement posture data measured by the sensor carried by the pipeline robot and the travel distance measured by the meter on the wire-winding vehicle, the real-time posture data of the pipeline robot is determined, which can improve the accuracy of the real-time posture data and more effectively ensure the fixation of the position of the positioning feature pattern.

[0077] Among them, adjusting the projection direction of the positioning laser light according to the real-time position data of the pipeline robot to keep the position of the positioning feature pattern fixed, specifically includes:

[0078] If the current moment is the starting moment of the first projection of the positioning feature pattern, each positioning feature pattern is projected in a default direction (which can be set according to actual needs), and the initial position of each positioning feature pattern is identified by the gimbal camera (the initial position is the position in the reference coordinate system, and the position in the gimbal camera coordinate system is identified by the image captured by the gimbal camera, and is converted to the position in the reference coordinate system through coordinate transformation; the coordinate conversion method is a prior art and is not described in detail here); wherein the gimbal camera is a 3D camera, which can identify the three-dimensional position coordinates of each positioning feature pattern;

[0079] If the current moment is after the start moment, the projection angle of each positioning feature pattern at the current moment is calculated based on the position of the positioning feature pattern recognized by the gimbal camera at the previous moment (the position in the reference coordinate system) and the real-time posture data at the current moment;

[0080] Each positioning feature pattern is projected according to the projection angle of each positioning feature pattern at the current moment.

[0081] Furthermore, according to the position of the positioning feature pattern recognized by the PTZ camera at the last moment and the real-time posture data at the current moment, the projection angle of each positioning feature pattern at the current moment is calculated, including:

[0082] According to the position of the positioning feature pattern identified at the previous moment and the real-time posture data at the current moment, the coordinates of the positioning feature pattern in the pipeline robot coordinate system at the current moment are calculated and recorded as the first coordinates;

[0083] According to the coordinate conversion relationship between the positioning laser light coordinate system and the pipeline robot coordinate system (obtained in advance by calibration), the first coordinate is converted into the coordinate of the positioning feature pattern in the positioning laser light coordinate system at the current moment, which is recorded as the second coordinate;

[0084] Calculate the vector pointing from the origin of the positioning laser light coordinate system at the current moment to the positioning feature pattern according to the second coordinate;

[0085] The direction angle of the vector is calculated as the projection angle of the corresponding positioning feature pattern at the current moment.

[0086] Specifically, step A2 includes:

[0087] When the pipeline robot is moving, the laser scanning device is used to scan the pipeline to be measured to obtain point cloud data in the coordinate system of the laser scanning device, which is recorded as preliminary point cloud data;

[0088] The coordinate data of the preliminary point cloud data is converted into coordinate data in the reference coordinate system (specifically, the coordinate data of the preliminary point cloud data is first converted into coordinate data in the pipeline robot coordinate system according to the coordinate conversion relationship between the laser scanning device coordinate system and the pipeline robot coordinate system, and then the coordinate data of the preliminary point cloud data in the pipeline robot coordinate system is converted into coordinate data in the reference coordinate system according to the real-time posture data of the pipeline robot) to obtain local point cloud data.

[0089] Specifically, step A3 includes:

[0090] A301. During the movement of the pipeline robot, the pan-tilt camera is used to continuously collect images of the inner wall of the pipeline being tested in real time;

[0091] A302. Identify the positioning feature pattern from each frame image, and obtain the coordinates of the positioning feature pattern in the PTZ camera coordinate system;

[0092] A303. Convert the coordinates of the positioning feature pattern in the PTZ camera coordinate system into coordinates in the reference coordinate system, recorded as feature coordinates;

[0093] A304. Generate key feature points of each frame of local point cloud data according to the feature coordinates and add them to the corresponding local point cloud data.

[0094] Among them, in step A301, the pan-tilt camera is instructed to capture images of the inner wall of the measured pipe at a preset shooting frequency (which can be set according to actual needs).

[0095] Among them, in step A302, a pre-trained image recognition model or other image recognition algorithm can be used to identify the positioning feature pattern (since each positioning feature pattern in a group of positioning feature patterns is different, identity recognition can be used to determine which feature pattern each positioning feature pattern is), and extract the coordinates of the positioning feature pattern in the gimbal camera coordinate system.

[0096] Among them, in step A303, the coordinates of the positioning feature pattern in the gimbal camera coordinate system are first converted into the coordinates of the positioning feature pattern in the pipeline robot coordinate system according to the coordinate conversion relationship between the gimbal camera coordinate system and the pipeline robot coordinate system, and then the coordinates of the positioning feature pattern in the pipeline robot coordinate system are converted into the coordinates of the positioning feature pattern in the reference coordinate system according to the real-time posture data, which are recorded as feature coordinates.

[0097] Among them, in step A304, the image corresponding to each frame of local point cloud data is determined according to the shooting time of each frame image of the inner wall of the measured pipeline and the acquisition time of each frame of local point cloud data, and the corresponding key feature points are generated and added to the corresponding local point cloud data according to the feature coordinates of the positioning feature pattern in the image corresponding to each frame of local point cloud data.

[0098] In order to facilitate the matching of the same key feature points during subsequent splicing, each key feature point may be marked. For example, in some embodiments, the order of the positioning feature patterns of each shape in a set of positioning feature patterns remains fixed (for example, Figure 5 The sorting of each positioning feature pattern in the projection is consistent before and after the projection position is updated), step A3 also includes: when the first two-dimensional code of the current projection is first identified, the group number is extracted, and the extracted group number and the sorting sequence number of each positioning feature pattern are used to form the number of each positioning feature pattern (for example, the group number of a group of positioning feature patterns of the current projection is A, and the sorting sequence number of the first two-dimensional code is 1, then the number of the first two-dimensional code of the current projection is A1, but the combination of numbers is not limited to this), after step A304, the number of the positioning feature pattern corresponding to the key feature point in the local point cloud data is used as annotation information, and the corresponding key feature point is annotated.

[0099] Thus, in step A4, when splicing, the key feature points in each local point cloud data are matched according to the annotation information of the key feature points. Specifically, the key feature points with the same annotation information are matched, and in addition, the key feature points corresponding to the first two-dimensional code are matched with the key feature points corresponding to the second two-dimensional code of the previous group (for example, the group numbers are sorted in a preset order, assuming that a group of positioning feature patterns includes 11 positioning feature patterns, and the group numbers are sorted in the order of 26 English letters. If one of the two local point cloud data to be spliced ​​contains a key feature point numbered B1, and the other contains a key feature point numbered A11, the B1 key feature point is the key feature point corresponding to the first two-dimensional code of the B group, and the A11 key feature point is the key feature point corresponding to the second two-dimensional code of the A group. Since the projection position is updated to make the projection position of the first two-dimensional code of the next group the same as the projection position of the second two-dimensional code of the previous group, the A11 key feature point is matched with the B1 key feature point).

[0100] Among them, in step A4, the method of splicing local point cloud data based on key feature points is a prior art and will not be described in detail here.

[0101] Among them, in step A5, the method of generating a twin model of the measured pipeline based on the overall point cloud data is a prior art and will not be described in detail here.

[0102] From the above, it can be seen that the pipeline self-calibration twin modeling method uses a positioning laser light to project a set of fixed-position positioning feature patterns on the inner wall of the measured pipeline; a set of positioning feature patterns includes multiple positioning feature patterns arranged at intervals along the axial straight line of the measured pipeline; a laser scanning device is used to scan the measured pipeline to obtain multiple frames of local point cloud data; a pan-tilt camera is used to identify the positioning feature patterns, and key feature points of each frame of local point cloud data are generated and added to the corresponding local point cloud data; the local point cloud data are spliced ​​based on the key feature points to obtain the overall point cloud data; a twin model of the measured pipeline is generated based on the overall point cloud data; thereby, the splicing error can be reduced and the accuracy of the modeling results can be effectively improved.

[0103] refer to Figure 2 The present application provides a pipeline self-calibration twin modeling device, which performs twin modeling of the pipeline under test based on a pipeline robot carrying a laser scanning device, a pan-tilt camera and a positioning laser light; including:

[0104] Projection module 1 is used to project a set of fixed positioning feature patterns on the inner wall of the measured pipe using a positioning laser lamp; the set of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipe (for the specific process, refer to step A1 above);

[0105] Scanning module 2, used to scan the pipeline under test using a laser scanning device to obtain multiple frames of local point cloud data (for the specific process, refer to step A2 above);

[0106] Recognition module 3, used to use the PTZ camera to identify and locate feature patterns, generate key feature points of each frame of local point cloud data and add them to the corresponding local point cloud data (for the specific process, refer to step A3 above);

[0107] A stitching module 4 is used to stitch the local point cloud data based on the key feature points to obtain the overall point cloud data (for the specific process, refer to step A4 above);

[0108] The model generation module 5 is used to generate a twin model of the measured pipeline according to the overall point cloud data (for the specific process, refer to step A5 above).

[0109] In some embodiments, the pipeline self-calibration twin modeling device further includes:

[0110] The updating module is used to update the projection position of the positioning feature pattern when the pan-tilt camera detects the second QR code, and then make the projection module 1, the scanning module 2 and the recognition module 3 perform their respective functional operations again until the entire scan of the tested pipeline is completed (for the specific process, refer to step A6 above).

[0111] Please refer to Figure 3 , Figure 3A structural schematic diagram of an electronic device provided in an embodiment of the present application, the present application provides an electronic device, including: a processor 301 and a memory 302, the processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not marked), the memory 302 stores a computer program executable by the processor 301, when the electronic device is running, the processor 301 executes the computer program to execute the pipeline self-calibration twin modeling method in any optional implementation of the above embodiment, to achieve the following functions: using a positioning laser light to project a group of fixed-position positioning feature patterns on the inner wall of the measured pipeline; a group of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipeline; using a laser scanning device to scan the measured pipeline to obtain multiple frames of local point cloud data; using a pan-tilt camera to identify the positioning feature patterns, generating key feature points of each frame of local point cloud data and adding them to the corresponding local point cloud data; splicing the local point cloud data based on the key feature points to obtain overall point cloud data; generating a twin model of the measured pipeline according to the overall point cloud data.

[0112] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the pipeline self-calibration twin modeling method in any optional implementation of the above embodiments is executed to achieve the following functions: a group of fixed-position positioning feature patterns are projected on the inner wall of the measured pipeline using a positioning laser light; a group of positioning feature patterns include multiple positioning feature patterns arranged at intervals along the axial straight line of the measured pipeline; a laser scanning device is used to scan the measured pipeline to obtain multiple frames of local point cloud data; a pan-tilt camera is used to identify the positioning feature patterns, and key feature points of each frame of local point cloud data are generated and added to the corresponding local point cloud data; the local point cloud data are spliced ​​based on the key feature points to obtain overall point cloud data; and a twin model of the measured pipeline is generated based on the overall point cloud data. Among them, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0113] refer to Figure 4 , Figure 5 The present application provides a pipeline self-calibration twin modeling system, including a ground terminal 100, a wire take-up vehicle 200 and a pipeline robot 300. The ground terminal 100 is connected to the wire take-up vehicle 200 in communication, and the wire take-up vehicle 200 is connected to the pipeline robot 300 through a cable; the wire take-up vehicle 200 is used to retract and release the cable, and to supply power to the pipeline robot 300 through the cable and communicate with the pipeline robot 300; the ground terminal 100 monitors the pipeline robot 300 through the wire take-up vehicle 200; the pipeline robot 300 is provided with a laser scanning device 400, a pan-tilt camera 500 and a positioning laser light 600;

[0114] The ground terminal 100 is used to perform:

[0115] B1. Projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light 600; the set of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipe (for the specific process, refer to step A1 above);

[0116] B2. Scan the pipeline under test using the laser scanning device 400 to obtain multiple frames of local point cloud data (refer to step A2 above for the specific process);

[0117] B3. Using the PTZ camera 500 to identify and locate the characteristic pattern, the key characteristic points of each frame of local point cloud data are generated and added to the corresponding local point cloud data (for the specific process, refer to step A3 above);

[0118] B4. Based on the key feature points, local point cloud data are spliced ​​to obtain the overall point cloud data (refer to step A4 above for the specific process);

[0119] B5. Generate a twin model of the tested pipeline based on the overall point cloud data (refer to step A5 above for the specific process).

[0120] Among them, after step B3 and before step B4, the following steps are also included:

[0121] B6. If the pan-tilt camera 500 detects the second two-dimensional code, it updates the projection position of the positioning feature pattern and returns to step B1 until the entire scan of the detected pipeline is completed (for the specific process, refer to step A6 above).

[0122] The positioning laser light 600 may be a mechanical laser light or a phase-controlled laser light, and preferably a phase-controlled laser light.

[0123] The pipeline robot 300 is provided with sensors for measuring posture data, such as a gyroscope and an odometer. The gyroscope is used to measure the posture angle of the pipeline robot, and the odometer is used to measure the forward distance of the pipeline robot.

[0124] Preferably, the cable take-up vehicle 200 is provided with a meter counter for measuring the pay-out length of the cable to estimate the travel distance of the pipeline robot (for example, the travel distance of the pipeline robot is obtained by subtracting the pay-out length at the initial moment from the real-time pay-out length, but not limited to this).

[0125] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0126] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0128] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0129] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A pipeline self-calibration twin modeling method, characterized in that: Based on the pipeline robot equipped with laser scanning device, pan-tilt camera and positioning laser light, twin modeling of the pipeline under test is carried out; Includes steps: A1. Projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; a set of positioning feature patterns includes a plurality of positioning feature patterns spaced apart along the axial straight line of the measured pipe; A2. Scan the measured pipeline using the laser scanning device to obtain multiple frames of local point cloud data; A3. Using the pan-tilt camera to identify the positioning feature pattern, generate key feature points of each frame of the local point cloud data and add them to the corresponding local point cloud data; A4. splicing the local point cloud data based on the key feature points to obtain the overall point cloud data; A5. Generate a twin model of the measured pipeline according to the overall point cloud data; The positioning laser light is a phased laser light. When a group of fixed-position positioning feature patterns are projected on the inner wall of the measured pipe by using the positioning laser light, each positioning feature pattern is projected in a sequential circular projection manner. Within the exposure time of each frame of the image taken by the pan-tilt camera, multiple circular projections of all positioning feature patterns are completed. A group of positioning feature patterns includes a first two-dimensional code located at the head end, a second two-dimensional code located at the end, and a plurality of different first feature graphs located between the first two-dimensional code and the second two-dimensional code; the two-dimensional code information of the first two-dimensional code includes a starting symbol and a group number of a group of positioning feature patterns currently projected; The second two-dimensional code includes a termination symbol and a group number of a set of positioning feature patterns currently projected; wherein the start symbol is an identification symbol indicating that the pipeline robot has started the identification process of the set of positioning feature patterns currently projected, the termination symbol is an identification symbol indicating that the pipeline robot has ended the identification process of the set of positioning feature patterns currently projected, and the group number in the two-dimensional code information is the identity identification symbol of the set of positioning feature patterns currently projected; the group numbers in the two-dimensional code information of different groups of positioning feature patterns are different; After step A3 and before step A4, the method further includes the following steps: A6. If the pan-tilt camera detects the second two-dimensional code, the projection position of the positioning feature pattern is updated, and the process returns to step A1 until the entire scan of the measured pipeline is completed; specifically, when the pan-tilt camera detects the second two-dimensional code and identifies a termination symbol therefrom, it is determined that the pipeline robot has completed the recognition process of the currently projected set of positioning feature patterns, thereby updating the projection position of the positioning feature pattern, and returning to step A1 to make the positioning laser light project the next set of positioning feature patterns according to the updated projection position; Among them, when updating the projection position of the positioning feature pattern, the projection position of the second two-dimensional code in the currently projected set of positioning feature patterns is used as the projection position of the first two-dimensional code in the next set of positioning feature patterns. Other positioning feature patterns are calculated according to their respective preset relative position relationships with the first two-dimensional code to ensure that there are key feature points in the same position between the local point cloud data obtained before and after the projection position is updated as reference points for the splicing process.

2. The pipeline self-calibration twin modeling method according to claim 1, characterized in that: Step A1 includes: During the movement of the pipeline robot, the projection direction of the positioning laser light is adjusted according to the real-time posture data of the pipeline robot to keep the position of the positioning feature pattern fixed.

3. The pipeline self-calibration twin modeling method according to claim 2 is characterized in that: The real-time posture data is determined based on the measured posture data measured by the sensor carried by the pipeline robot and the travel distance measured by the meter on the wire-reeling vehicle of the pipeline robot.

4. The pipeline self-calibration twin modeling method according to claim 1, characterized in that: Step A2 includes: During the movement of the pipeline robot, the laser scanning device is used to scan the pipeline to be measured to obtain point cloud data in the coordinate system of the laser scanning device, which is recorded as preliminary point cloud data; The coordinate data of the preliminary point cloud data is converted into coordinate data in a reference coordinate system to obtain the local point cloud data.

5. The pipeline self-calibration twin modeling method according to claim 1, characterized in that: Step A3 includes: During the movement of the pipeline robot, the pan-tilt camera is used to continuously collect images of the inner wall of the pipeline under test in real time; Identify the positioning feature pattern from each frame of the image, and obtain the coordinates of the positioning feature pattern in the gimbal camera coordinate system; Convert the coordinates of the positioning feature pattern in the gimbal camera coordinate system into coordinates in the reference coordinate system, which are recorded as feature coordinates; According to the feature coordinates, key feature points of the local point cloud data of each frame are generated and added to the corresponding local point cloud data.

6. A pipeline self-calibration twin modeling device, characterized in that: Based on the pipeline robot equipped with laser scanning device, pan-tilt camera and positioning laser light, twin modeling of the pipeline under test is carried out; including: A projection module, used for projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; the set of positioning feature patterns includes a plurality of positioning feature patterns arranged at intervals along the axial straight line of the measured pipe; A scanning module, used for scanning the pipeline under test using the laser scanning device to obtain multiple frames of local point cloud data; A recognition module, used to use the pan / tilt camera to recognize the positioning feature pattern, generate key feature points of the local point cloud data of each frame and add them to the corresponding local point cloud data; A splicing module, used for splicing the local point cloud data based on the key feature points to obtain overall point cloud data; A model generation module, used to generate a twin model of the measured pipeline according to the overall point cloud data; The positioning laser light is a phased laser light. When a group of fixed-position positioning feature patterns are projected on the inner wall of the measured pipe by using the positioning laser light, each positioning feature pattern is projected in a sequential circular projection manner. Within the exposure time of each frame of the image taken by the pan-tilt camera, multiple circular projections of all positioning feature patterns are completed. A group of positioning feature patterns includes a first two-dimensional code located at the head end, a second two-dimensional code located at the end end, and a plurality of different first feature maps located between the first two-dimensional code and the second two-dimensional code; the two-dimensional code information of the first two-dimensional code includes a start symbol and a group number of a group of positioning feature patterns currently projected; the second two-dimensional code includes an end symbol and a group number of a group of positioning feature patterns currently projected; wherein the start symbol is an identification symbol indicating that the pipeline robot has started the identification process of the group of positioning feature patterns currently projected, the end symbol is an identification symbol indicating that the pipeline robot has ended the identification process of the group of positioning feature patterns currently projected, and the group number in the two-dimensional code information is an identity identification symbol of a group of positioning feature patterns currently projected; the group numbers in the two-dimensional code information of different groups of positioning feature patterns are different; The pipeline self-calibration twin modeling device also includes: An updating module is used to update the projection position of the positioning feature pattern when the pan-tilt camera detects the second two-dimensional code, and then make the projection module, the scanning module and the recognition module perform their respective functional operations again until the entire scan of the measured pipeline is completed; specifically, when the pan-tilt camera detects the second two-dimensional code and identifies a termination symbol therefrom, it is determined that the pipeline robot has ended the recognition process of a set of positioning feature patterns currently projected, thereby updating the projection position of the positioning feature pattern, and making the projection module, the scanning module and the recognition module perform their respective functional operations again, so that the positioning laser light projects the next set of positioning feature patterns according to the updated projection position; Among them, when updating the projection position of the positioning feature pattern, the projection position of the second two-dimensional code in the currently projected set of positioning feature patterns is used as the projection position of the first two-dimensional code in the next set of positioning feature patterns. Other positioning feature patterns are calculated according to their respective preset relative position relationships with the first two-dimensional code to ensure that there are key feature points in the same position between the local point cloud data obtained before and after the projection position is updated as reference points for the splicing process.

7. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, it runs the steps in the pipeline self-calibration twin modeling method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the pipeline self-calibration twin modeling method as described in any one of claims 1 to 5 are performed.

9. A pipeline self-calibration twin modeling system, characterized in that: It includes a ground terminal, a wire-taking vehicle and a pipeline robot. The ground terminal is connected to the wire-taking vehicle in communication, and the wire-taking vehicle is connected to the pipeline robot through a cable. The wire-taking vehicle is used to retract and release the cable, and to supply power to the pipeline robot through the cable and communicate with the pipeline robot. The ground terminal monitors the pipeline robot through the wire-taking vehicle. The pipeline robot is provided with a laser scanning device, a pan-tilt camera and a positioning laser light. The ground terminal is used to perform: B1. Projecting a set of fixed positioning feature patterns on the inner wall of the measured pipe using the positioning laser light; a set of positioning feature patterns includes a plurality of positioning feature patterns spaced apart along the axial straight line of the measured pipe; B2. Scan the measured pipeline using the laser scanning device to obtain multiple frames of local point cloud data; B3. Using the PTZ camera to identify the positioning feature pattern, generating key feature points of each frame of the local point cloud data and adding them to the corresponding local point cloud data; B4. splicing the local point cloud data based on the key feature points to obtain the overall point cloud data; B5. Generate a twin model of the measured pipeline according to the overall point cloud data; The positioning laser light is a phased laser light. When a group of fixed-position positioning feature patterns are projected on the inner wall of the measured pipe by using the positioning laser light, each positioning feature pattern is projected in a sequential circular projection manner. Within the exposure time of each frame of the image taken by the pan-tilt camera, multiple circular projections of all positioning feature patterns are completed. A group of positioning feature patterns includes a first two-dimensional code located at the head end, a second two-dimensional code located at the end, and a plurality of different first feature graphs located between the first two-dimensional code and the second two-dimensional code; the two-dimensional code information of the first two-dimensional code includes a starting symbol and a group number of a group of positioning feature patterns currently projected; The second two-dimensional code includes a termination symbol and a group number of a set of positioning feature patterns currently projected; wherein the start symbol is an identification symbol indicating that the pipeline robot has started the identification process of the set of positioning feature patterns currently projected, the termination symbol is an identification symbol indicating that the pipeline robot has ended the identification process of the set of positioning feature patterns currently projected, and the group number in the two-dimensional code information is the identity identification symbol of the set of positioning feature patterns currently projected; the group numbers in the two-dimensional code information of different groups of positioning feature patterns are different; After step B3 and before step B4, the method further includes the following steps: B6. If the pan-tilt camera detects the second two-dimensional code, the projection position of the positioning feature pattern is updated, and the process returns to step B1 until the entire scan of the measured pipeline is completed; specifically, when the pan-tilt camera detects the second two-dimensional code and identifies the termination symbol therefrom, it is determined that the pipeline robot has completed the recognition process of the currently projected set of positioning feature patterns, thereby updating the projection position of the positioning feature pattern, and returning to step B1 to make the positioning laser light project the next set of positioning feature patterns according to the updated projection position; Among them, when updating the projection position of the positioning feature pattern, the projection position of the second two-dimensional code in the currently projected set of positioning feature patterns is used as the projection position of the first two-dimensional code in the next set of positioning feature patterns. Other positioning feature patterns are calculated according to their respective preset relative position relationships with the first two-dimensional code to ensure that there are key feature points in the same position between the local point cloud data obtained before and after the projection position is updated as reference points for the splicing process.

Images