Pipe jacking construction guiding system and equipment based on computer vision
By using computer vision technology, combined with image acquisition, feature extraction, and 3D modeling, precise guidance and monitoring of the pipe jacking construction process have been achieved, solving the problems of complex operation and low efficiency in existing technologies, and improving the accuracy and efficiency of construction.
Patent Information
- Application Number
- CN202410990566.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-08
AI Technical Summary
Existing pipe jacking construction guidance technology is complex to operate, inefficient, and has limited modular functions, failing to effectively improve the accuracy and efficiency of construction guidance.
A computer vision-based pipe jacking construction guidance system is adopted. Through image acquisition, feature extraction, target monitoring and data feedback components, it realizes real-time monitoring and guidance of underground pipelines and construction equipment, including image processing, edge detection, feature extraction, target recognition, position and attitude estimation and 3D modeling.
It improves the accuracy and efficiency of pipe jacking construction, reduces operational complexity, enables accurate positioning and real-time monitoring of underground pipelines and construction equipment, and optimizes the construction process.
Smart Images

Figure CN118918294A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision, and in particular to a pipe jacking construction guiding system and equipment based on computer vision. Background Art
[0002] Pipe jacking construction is a method of underground pipeline construction, which is usually used for laying underground pipelines in cities. In pipe jacking construction, engineers dig one or more working wells on the surface, and then use pipe jacking machinery and equipment to push the pipeline from the working well to the underground section by section, and finally complete the installation and laying of the pipeline. This construction method is suitable for pipeline construction in long distances, large scale, deep pipelines, complex terrain and other situations. In order to further improve the safety and construction efficiency of pipe jacking construction, guidance is required. The existing technology generally uses underground radars, electromagnetic detectors and other equipment to scan underground to obtain the location and situation of underground pipelines and obstacles, and provide reference and guidance for pipe jacking construction. The use of equipment scanning has high requirements for the equipment itself, otherwise it is difficult to obtain accurate scanning results. Secondly, professional technicians are required to operate, which undoubtedly increases the complexity and cost of the operation.
[0003] Prior art 1, application number: CN 202410025993.1 discloses a pipe jacking construction method, including: installing a connecting pipe on the tunneling head, filling the connecting pipe with water, installing a horizontal ruler at the front end of the tunneling head, and setting up a three-dimensional coordinate control point; the pipe jacking device pushes the jacking section and the tunneling head forward, and forms a jacking channel by the tunneling head from the starting well, and observes the liquid level difference at the front and rear ends of the connecting pipe and measures the reading of the three-dimensional control point on the horizontal ruler every time the tunneling head advances forward a preset distance, and corrects the posture of the tunneling head according to the liquid level difference and the reading. Although the posture change of the tunneling head can be monitored in time, and the posture of the tunneling head can be corrected in time to correct the route of the jacking section, so that the jacking section can reach the preset position more accurately, the accuracy of the pipe jacking construction is improved. However, the horizontal ruler can only adjust the structural posture, and cannot achieve accurate positioning of pipe jacking construction and equipment. It also requires manual participation, which reduces the accuracy of posture adjustment to a certain extent.
[0004] Prior art 2, application number: CN 202410450376.6 discloses a method for controlling settlement of rectangular pipe jacking construction, including: analyzing and judging the geological conditions of rectangular pipe jacking construction, and setting rectangular pipe jacking construction parameters according to the analysis and judgment results; setting monitoring positions on the rectangular pipe jacking construction line, and judging whether the construction line has settled during the rectangular pipe jacking construction through the monitoring positions; for the area where settlement has occurred, grouting is injected into the settlement area through the grouting holes arranged on the rectangular pipe jacking to achieve grouting compensation and lifting. Although it can ensure that during the construction of shallow buried large-section rectangular pipe jacking, the settlement of each area and each point of the construction line can be accurately monitored, and then grouting and lifting compensation can be quickly made according to the settlement area, ensuring timely repair of geological structure changes caused by rectangular pipe jacking construction, and effectively avoiding related damage caused by settlement. However, it only involves the detection of settlement and lacks the optimization of the construction guidance plan, resulting in a low degree of intelligence.
[0005] Prior art three, application number: CN 202410160318.X discloses a method for monitoring and controlling pipe jacking construction, including: setting up a reference axis across the starting well on the ground, the axis direction of the reference axis is parallel to the established axis of the pipe jacking machine excavation, and slidingly setting a reference block on the reference axis; setting a mounting platform on the machine head guide rail, slidingly setting the laser theodolite on the mounting platform, and the sliding direction of the laser theodolite is parallel to the established axis of the pipe jacking machine excavation; an offset monitoring component is set between the reference block and the laser theodolite; in the front section of the equalizing ring reset, the laser theodolite and the reference block are controlled to slide forward synchronously, and it is judged whether the laser theodolite is offset and corrected; in the rear section of the equalizing ring reset, the laser theodolite and the reference block are controlled to slide backward synchronously, and it is judged whether the laser theodolite is corrected accurately. Although the offset of the laser theodolite in the starting well can be discovered in time, it is ensured that the pipe jacking machine can be pushed forward according to the established axis. However, the monitoring is realized by relying on the mechanical structure, and its degree of automation is low, and accurate guidance of pipe jacking construction cannot be achieved.
[0006] At present, the existing technologies 1, 2 and 3 have the problems of complex operation, low efficiency and limited module functions, and cannot effectively improve the efficiency of pipe jacking construction guidance. Therefore, the present invention provides a pipe jacking construction guidance system and equipment based on computer vision, which can realize accurate guidance and monitoring of underground pipeline construction process by integrating and optimizing functional modules. Summary of the invention
[0007] The main purpose of the present invention is to provide a computer vision-based pipe jacking construction guidance system and equipment to solve the problems in the prior art of complex operation, low efficiency, limited module functions, and failure to more effectively improve the efficiency of pipe jacking construction guidance.
[0008] To achieve the above object, the present invention provides the following technical solutions:
[0009] A pipe jacking construction guidance system based on computer vision, the pipe jacking construction guidance system based on computer vision comprising:
[0010] Image acquisition component, used to collect images of underground pipeline construction sites using cameras, laser scanners and other equipment, and to extract feature information of underground pipelines and construction equipment by processing the collected images through denoising, enhancement, filtering and edge detection;
[0011] The target monitoring component is used to detect and identify targets through feature extraction algorithms, determine the location and posture of underground pipelines and construction equipment, and perform three-dimensional modeling and positioning of the detected location and posture of underground pipelines and construction equipment;
[0012] The data feedback component is used to monitor the underground pipeline construction process in real time, feedback the location, status and construction progress of the underground pipeline to the operator, display the underground pipeline image, construction scene information and operation instructions, support the interaction and feedback between the operator and the system; record the data during the underground pipeline construction process, including construction trajectory, depth and speed information, conduct data analysis and statistics, and provide reference for the optimization and improvement of the construction process.
[0013] As a further improvement of the present invention, the image acquisition component includes:
[0014] An image processing module is used to collect images of underground pipeline construction sites using cameras or laser scanners to obtain real-time images of underground pipelines and construction equipment; and to perform denoising, enhancement and filtering on the collected images;
[0015] The edge detection module is used to convert the collected image into a grayscale image, smooth the grayscale image using Gaussian filtering, perform convolution operations on the smoothed grayscale image using the Sobel operator, calculate the horizontal and vertical gradients of each pixel, and obtain the gradient amplitude and gradient direction of each pixel; combine the horizontal and vertical gradients to calculate the gradient amplitude of each pixel to indicate the intensity of the grayscale change of the point; according to the horizontal and vertical gradients, use the arctan function to calculate the gradient direction of each pixel; set the gradient amplitude threshold, set the lower gradient amplitude to 0, and the higher gradient amplitude to 255 to extract the edge; refine the extracted edge and retain the main edge line; make the edge line thinner through non-maximum suppression processing, and only retain the pixels with the largest local gradient; set double thresholds, divide the gradient amplitude into three categories: strong edge, weak edge and non-edge, and obtain a clear edge image; connect the strong edge with its adjacent weak edge through the edge connection algorithm to form a complete edge line;
[0016] The feature extraction module is used to detect key points in the image using the SURF algorithm. The key points represent the local features of shape, texture and color in the image of underground pipelines and construction equipment. The box filter is used to calculate the image gradient to obtain the local direction information near the key points. The local features extracted from each key point are described using descriptors. The descriptors are used to describe the features of the area around the key points, including the position, scale and direction of the shape, texture and color. The Hellenger feature matching is used to match the feature points extracted from the image, and the similarity of the descriptors is compared to determine whether the two feature points match, and establish the corresponding relationship between the feature points of the two images. The feature information of the underground pipelines and construction equipment is extracted from the matched feature points to describe the characteristics and features of the underground pipelines and construction equipment in the image.
[0017] As a further improvement of the present invention, the target monitoring component includes:
[0018] The target detection and recognition module is used to use the feature extraction algorithm to perform key point detection and feature extraction on the collected images of underground pipelines and construction equipment, and obtain key points and local feature descriptors; the extracted feature descriptors are matched with the constructed feature library, and the position and direction of the underground pipelines and construction equipment in the image are determined by Hellenger distance matching;
[0019] The target position and posture estimation module is used to estimate the precise position and posture information of underground pipelines and construction equipment in the scene through the matched feature points and their positions with the help of triangulation;
[0020] The 3D modeling and positioning module is used to perform 3D modeling and positioning based on the target position and posture information, and to generate models of underground pipelines and construction equipment in underground pipeline and construction equipment scenes for guidance and monitoring.
[0021] As a further improvement of the present invention, the target detection and recognition module includes:
[0022] The key point search submodule is responsible for detecting the brightness values of surrounding pixels to quickly determine whether they are key points and determine the locations with significant features in the image, which are the key points.
[0023] The local feature description submodule is responsible for assigning a main direction to each key point around the detected key point. The main direction is calculated based on the gradient information of the pixels around the key point. After obtaining the key point position and main direction, the BRIEF algorithm is used to generate a local feature descriptor to describe the local features around the key point.
[0024] The descriptor output submodule is responsible for outputting the local feature descriptor around each key point position. The descriptor includes the gradient, color or texture information around the key point.
[0025] As a further improvement of the present invention, the target detection and recognition module includes:
[0026] The feature library construction submodule is used to build a feature library to store the feature descriptors of known underground pipelines and construction equipment; the extracted feature descriptors of the target objects are matched with the descriptors in the constructed feature library. In the matching process, the Hellenger distance is used as a metric to measure the similarity between the two descriptors;
[0027] The nearest neighbor matching submodule is used to compare each extracted feature descriptor with all descriptors in the feature library using Hellenger distance matching and find the descriptor with the smallest distance;
[0028] The position and direction determination submodule is used to find the position information of the feature points in the image through the matching results obtained by nearest neighbor matching, and use triangulation to obtain the position and direction of the underground pipeline and construction equipment in the image in combination with the position and correspondence of the feature points, thereby realizing the determination of the position and direction.
[0029] As a further improvement of the present invention, the target position and posture estimation module includes:
[0030] The feature point matching submodule is used to match the feature descriptors obtained by the target detection and recognition module in the target image to find the corresponding feature points, including the position and local feature descriptors;
[0031] The triangulation submodule is used to perform triangulation using the position information of the matched feature points. By observing the same world point in the camera field of view, the distance and angle between the device and the target are calculated to determine the precise position and posture of the target in the scene.
[0032] The position and attitude estimation submodule is used to estimate the position and attitude of underground pipelines and construction equipment in the scene with the help of the distance and angle measurement results of triangulation, combined with equipment parameters and scene information. The position includes the three-dimensional coordinate position of the target in the scene, and the attitude includes the rotation angle and direction information of the target.
[0033] As a further improvement of the present invention, the position and posture estimation submodule includes:
[0034] The parameter acquisition unit is used to collect the focal length of the camera or sensor, the internal parameters of the principal point coordinates, and the external parameters of the relative position and direction relationship between the camera and the scene; at the same time, it obtains the lighting conditions and terrain structure;
[0035] The triangulation unit is used to find the corresponding feature points through the feature point matching submodule, obtain the pixel coordinates in the image, and use the triangulation principle to calculate the distance and angle between the target object and the observation device according to the pixel coordinates, intrinsic parameters and extrinsic parameters of the feature points; the measured distance and angle information is converted from the camera coordinate system to the scene coordinate system to obtain the position of the target object in the three-dimensional space;
[0036] The position and attitude estimation unit is used to combine device parameters and scene information to calculate the three-dimensional coordinate position of the target object in the scene, including cropping, filtering or fusing the position information of multiple feature points; calculate the rotation angle and direction attitude information of the target object, and estimate the direction and angle of the target object through the position changes of feature points at different viewing angles.
[0037] As a further improvement of the present invention, the three-dimensional modeling and positioning module includes:
[0038] The 3D modeling submodule is used to perform 3D modeling of underground pipelines and construction equipment based on the target position and posture information using 3D modeling software; the shape, size and orientation of the target object are accurately presented in 3D space based on the estimated position and posture information;
[0039] A positioning submodule is used to position the generated three-dimensional model with the underground pipeline and construction equipment scene, align it with the underground pipeline and construction equipment scene, and reflect the position and posture of the target object in the underground pipeline and construction equipment scene;
[0040] The guidance and monitoring submodule is used to use the generated three-dimensional model to guide the operator to locate and monitor the target object; by superimposing the generated model into the real-time image, the operator can view the position and posture of the target object in the underground pipeline and construction equipment scene in real time.
[0041] As a further improvement of the present invention, the data feedback component includes:
[0042] The information feedback module is used to obtain the location and posture information of underground pipelines and construction equipment from the target monitoring components, and update the location, status and construction progress of underground pipelines in real time by combining the data collected by the system in real time, the construction trajectory, depth and speed; and feed back the information to the operator;
[0043] The information display module is used to obtain the image information of underground pipelines and construction scenes using the image acquisition component, and effectively display the image information on the operator's monitoring interface; the operator can view the image of the underground pipeline and the scene information of the construction site in real time;
[0044] The operation feedback module is used to support the interaction between operators and the system, including operation input and command transmission functions; at the same time, it records various data during the underground pipeline construction process, construction trajectory, depth and speed, and conducts data analysis and statistics.
[0045] To achieve the above object, the present invention also provides the following technical solutions:
[0046] An electronic device, characterized in that it includes a processor and a memory coupled to the processor, wherein the memory stores program instructions that can be executed by the processor; when the processor executes the program instructions stored in the memory, the computer vision-based pipe jacking construction guidance system is implemented.
[0047] The image acquisition component of the present invention collects image data of the underground pipeline construction site through equipment such as cameras and laser scanners, and preprocesses and extracts features from the images; provides real-time underground pipeline information and construction scene images, provides necessary data support for subsequent target monitoring and data feedback, and helps to achieve real-time monitoring and guidance of the construction process. The target monitoring component performs target detection and identification through feature extraction algorithms, determines the position and posture of underground pipelines and construction equipment, and performs three-dimensional modeling and positioning; accurately monitors and identifies underground pipelines and construction equipment, provides a basis for the guidance system to achieve accurate positioning and modeling of underground pipelines, and improves construction accuracy and precision. The data feedback component monitors the underground pipeline construction process in real time, feeds back the position, status and construction progress of the underground pipeline to the operator, supports human-computer interaction and data recording and analysis; provides operators with timely and clear underground pipeline information and construction scene feedback, facilitates operators to make real-time adjustments and decisions, optimizes the construction process, and records and analyzes construction data at the same time, providing reference for future construction improvements. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 It is a functional module schematic diagram of an embodiment of a pipe jacking construction guidance system based on computer vision of the present invention;
[0049] Figure 2 This is a schematic diagram of the functional modules of an image acquisition component of an embodiment of a pipe jacking construction guidance system based on computer vision of the present invention;
[0050] Figure 3 This is a functional module diagram of a target monitoring component of an embodiment of a pipe jacking construction guidance system based on computer vision of the present invention;
[0051] Figure 4 This is a functional module diagram of a data feedback component of an embodiment of a pipe jacking construction guidance system based on computer vision of the present invention;
[0052] Figure 5It is a structural schematic diagram of an embodiment of an electronic device of the present invention;
[0053] Figure 6 It is a schematic structural diagram of an embodiment of the storage medium of the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] The terms "first", "second" and "third" in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" and "third" can explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.
[0056] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0057] like Figure 1 As shown, this embodiment provides an embodiment of a pipe jacking construction guidance system based on computer vision. In this embodiment, the pipe jacking construction guidance system based on computer vision includes an image acquisition component 1, a target monitoring component 2, and a data feedback component 3 which are electrically connected in sequence;
[0058] Among them, the image acquisition component 1 is used to use cameras, laser scanners and other equipment to collect images of underground pipeline construction sites, and process the collected images through denoising, enhancement, filtering and edge detection to extract feature information of underground pipelines and construction equipment; the target monitoring component 2 is used to detect and identify targets through feature extraction algorithms to determine the position and posture of underground pipelines and construction equipment; the detected position and posture of underground pipelines and construction equipment are three-dimensionally modeled and positioned; the data feedback component 3 is used to monitor the underground pipeline construction process in real time, feedback the position, status and construction progress of the underground pipeline to the operator, display underground pipeline images, construction scene information and operation instructions, etc., to support interaction and feedback between operators and the system; record data during the underground pipeline construction process, including construction trajectory, depth and speed information, conduct data analysis and statistics, and provide reference for optimization and improvement of the construction process.
[0059] Preferably, the image acquisition component of this embodiment collects image data of the underground pipeline construction site through devices such as cameras and laser scanners, and preprocesses and extracts features from the images; provides real-time underground pipeline information and construction scene images, provides necessary data support for subsequent target monitoring and data feedback, and helps to achieve real-time monitoring and guidance of the construction process. The target monitoring component performs target detection and recognition through feature extraction algorithms, determines the position and posture of underground pipelines and construction equipment, and performs three-dimensional modeling and positioning; accurately monitors and identifies underground pipelines and construction equipment, provides a basis for the guidance system to achieve accurate positioning and modeling of underground pipelines, and improves construction accuracy and precision. The data feedback component monitors the underground pipeline construction process in real time, feeds back the position, status and construction progress of the underground pipeline to the operator, supports human-computer interaction and data recording and analysis; provides operators with timely and clear underground pipeline information and construction scene feedback, facilitates operators to make real-time adjustments and decisions, optimizes the construction process, and records and analyzes construction data at the same time, providing reference for future construction improvements.
[0060] In summary, these components of this embodiment cooperate with each other to form a complete pipe jacking construction guidance system based on computer vision; the image acquisition component provides a data source, the target monitoring component performs target detection and positioning, and the data feedback component feeds back key information to the operator in real time, so as to realize the system's monitoring, guidance and optimization of the underground pipeline construction process; the technical effects of each component complement each other, and together provide solid support for the efficient operation and construction quality of the system.
[0061] Furthermore, if Figure 2 As shown, the image acquisition component specifically includes:
[0062] The image processing module is used to collect images of the underground pipeline construction site using a camera or laser scanner and other equipment to obtain real-time images of the underground pipeline and construction equipment; and to perform denoising, enhancement and filtering on the collected images;
[0063] The edge detection module is used to convert the collected image into a grayscale image, smooth the grayscale image using Gaussian filtering, perform convolution operations on the smoothed grayscale image using the Sobel operator, calculate the horizontal and vertical gradients of each pixel, and obtain the gradient amplitude and gradient direction of each pixel; combine the horizontal and vertical gradients to calculate the gradient amplitude of each pixel to indicate the intensity of the grayscale change of the point; use the arctan function to calculate the gradient direction of each pixel based on the horizontal and vertical gradients; set the gradient amplitude threshold, set the lower gradient amplitude to 0, and the higher gradient amplitude to 255 to extract the edge; refine the extracted edge to eliminate unnecessary small edge fragments and retain the main edge line; make the edge line thinner through non-maximum suppression processing, and only retain the pixels with the largest local gradient to further improve the quality of the edge image; set double thresholds to divide the gradient amplitude into three categories: strong edge, weak edge and non-edge, and obtain a clear edge image; connect the strong edge with its adjacent weak edge through the edge connection algorithm to form a complete edge line, and further improve the contour edge of the underground pipeline and construction equipment;
[0064] The feature extraction module is used to detect key points in the image using the SURF algorithm. The key points represent local features such as shape, texture and color in the image of underground pipelines and construction equipment; the box filter is used to calculate the image gradient to obtain local directional information near the key points; the local features extracted from each key point are described using descriptors, which are used to describe the features of the area around the key point, including the position, scale and direction of the shape, texture and color; the Hellenger feature matching is used to match the feature points extracted from the image, and the similarity of the descriptors is compared to determine whether the two feature points match, and establish the corresponding relationship between the feature points of the two images; the feature information of the underground pipelines and construction equipment, such as shape, texture and color, is extracted from the matched feature points, which is used to describe the characteristics and features of the underground pipelines and construction equipment in the image.
[0065] Preferably, the image processing module of this embodiment performs denoising, enhancement and filtering on the collected images to improve the image quality and clarity, which is beneficial to the subsequent feature extraction and target analysis; it helps the system to obtain clear and accurate images of underground pipelines and construction equipment, and provides basic data for subsequent processing. The edge detection module extracts edge information in the image through algorithms such as the Sobel operator, accurately displays the outline of underground pipelines and construction equipment, and is used for subsequent feature extraction and matching; it can accurately locate the position of underground pipelines and construction equipment, and provide important shape information for subsequent feature processing. The feature extraction module uses the SURF algorithm to extract key points and local feature descriptors in the image, accurately describing the shape, texture, color and other features of underground pipelines and construction equipment; it can extract key features from the image, help the system to realize the recognition, matching and positioning of underground pipelines and construction equipment, and improve the accuracy and efficiency of the system.
[0066] In summary, this embodiment improves image quality through the image processing module, accurately extracts contour information through the edge detection module, and extracts key features through the feature extraction module. These modules jointly complete the processing and analysis of underground pipelines and construction equipment images, providing key technical support for the top pipe construction guidance system based on computer vision. The accuracy, real-time and stability of the system are improved, thereby realizing accurate monitoring and guidance of the underground engineering construction process, and ensuring the safety and efficiency of the construction.
[0067] Furthermore, if Figure 3 As shown, the target monitoring component 2 specifically includes:
[0068] The target detection and recognition module is used to use the feature extraction algorithm to perform key point detection and feature extraction on the collected images of underground pipelines and construction equipment, and obtain key points and local feature descriptors; the extracted feature descriptors are matched with the constructed feature library, and the position and direction of the underground pipelines and construction equipment in the image are determined by Hellenger distance matching;
[0069] The target position and posture estimation module is used to estimate the precise position and posture information of underground pipelines and construction equipment in the scene through the matched feature points and their positions with the help of triangulation;
[0070] The 3D modeling and positioning module is used to perform 3D modeling and positioning based on the target position and posture information, and to generate models of underground pipelines and construction equipment in underground pipeline and construction equipment scenes for guidance and monitoring.
[0071] Preferably, the target detection and recognition module of this embodiment processes the image of underground pipelines and construction equipment by using a feature extraction algorithm to obtain key points and local feature descriptors, thereby realizing target detection and recognition of underground pipelines and construction equipment; it can accurately extract the key features of the target object from the image, providing a basis for subsequent position and posture estimation; it determines the position and direction of the object by matching, provides the system with target position information, and provides a basis for guidance and monitoring. The target position and posture estimation module estimates the accurate position and posture information of the underground pipeline and construction equipment in the scene based on the matched feature points and their positions, using methods such as triangulation; it realizes the precise positioning and posture estimation of the target object, provides accurate position and direction information for subsequent guidance and three-dimensional modeling, and improves the positioning accuracy of the system. The three-dimensional modeling and positioning module performs three-dimensional modeling and positioning based on the acquired target position and posture information, generates a model of the underground pipeline and construction equipment in the underground pipeline and construction equipment scene, and is used for guidance and monitoring; through three-dimensional modeling, the position and direction information of the target object in the scene is converted into a real three-dimensional model, which is convenient for operators to monitor and guide the engineering scene in real time, and realizes fine management and optimization of the construction process.
[0072] To sum up, the above modules of this embodiment can realize accurate detection, positioning and modeling of underground pipelines and construction equipment, provide real-time and accurate monitoring and guidance information for operators, help improve construction efficiency, reduce risks, and ensure project quality and safety; it is of great significance to the computer vision-based pipe jacking construction guidance system, which helps the system to be intelligent and automated, and improves the overall project management level.
[0073] Furthermore, the target detection and recognition module specifically includes:
[0074] The key point search submodule is responsible for detecting the brightness values of surrounding pixels to quickly determine whether they are key points and determine the locations with significant features in the image, which are the key points.
[0075] The local feature description submodule is responsible for assigning a main direction to each key point around the detected key point. The main direction is calculated based on the gradient information of the pixels around the key point. After obtaining the key point position and main direction, the BRIEF algorithm is used to generate a local feature descriptor to describe the local features around the key point.
[0076] The descriptor output submodule is responsible for outputting the local feature descriptor around each key point position. The descriptor includes information such as gradient, color or texture around the key point.
[0077] Preferably, the key point search submodule of this embodiment determines the position of the key point by quickly detecting the brightness value of the surrounding pixels, judging the position of the image with significant features, and determining the position of the key point; the submodule can quickly and accurately locate the key points in the image, providing a basis for further feature extraction and description; by detecting the key points, the system can focus on the most significant feature points in the image, thereby realizing the recognition and positioning of the target object. The local feature description submodule assigns a main direction to each detected key point, calculates the main direction based on the pixel gradient information around the key point, and generates a local feature descriptor using the BRIEF algorithm; the assignment of the main direction can make the generated feature descriptor have rotation invariance, enhancing the system's recognition ability of the target object; through the generation of local feature descriptors, the system can more effectively describe the local features around each key point, and improve the uniqueness and recognition accuracy of the target object. The descriptor output submodule outputs the local feature descriptors around each key point, including information such as gradient, color or texture; the role of this submodule is to output the processed local feature descriptors to provide a data basis for subsequent feature matching and recognition. These descriptors are an important basis for the system to identify the target object, and can help the system accurately identify underground pipelines and construction equipment.
[0078] To sum up, the three sub-modules of this embodiment cooperate with each other to jointly realize the detection, positioning and identification of the target object; the rapid detection of key points, the assignment of main directions and the generation of local feature descriptors all provide the system with important feature information, which is of key significance for the identification and positioning of the target object; through the collaborative work of these sub-modules, the system can effectively process image data, realize the accurate detection and identification of targets such as underground pipelines and construction equipment, and thus provide reliable support for the guidance and monitoring of the system.
[0079] Furthermore, the target detection and recognition module also includes:
[0080] The feature library construction submodule is used to build a feature library to store the feature descriptors of known underground pipelines and construction equipment; the extracted feature descriptors of the target objects are matched with the descriptors in the constructed feature library. In the matching process, the Hellenger distance is used as a metric to measure the similarity between the two descriptors;
[0081] The nearest neighbor matching submodule is used to compare each extracted feature descriptor with all descriptors in the feature library using Hellenger distance matching and find the descriptor with the smallest distance;
[0082] The position and direction determination submodule is used to find the position information of the feature points in the image through the matching results obtained by nearest neighbor matching, and use triangulation to obtain the position and direction of the underground pipeline and construction equipment in the image in combination with the position and correspondence of the feature points, thereby realizing the determination of the position and direction.
[0083] Preferably, the feature library construction submodule of this embodiment establishes a feature library, which stores the feature descriptors of known underground pipelines and construction equipment; by constructing the feature library, the system can quickly compare with known objects during the matching process, thereby improving the recognition accuracy and efficiency of the target object. The nearest neighbor matching submodule uses Hellenger distance matching to compare the extracted feature descriptors with all descriptors in the feature library, and find the descriptor with the smallest distance; the nearest neighbor matching can find the feature descriptor that is most similar to the target object, thereby determining the relevant position of the target object in the image, providing a basis for subsequent position and direction determination. The position and direction determination submodule determines the position information of the feature point in the image through the matching results obtained by the nearest neighbor matching, and infers the position and direction of the underground pipeline and construction equipment in the image by combining the position and correspondence of the feature points using triangulation and other methods; the position and direction determination module can accurately measure the position and orientation of the target object in the image, providing key support for the system to achieve accurate positioning and posture estimation of underground pipelines and construction equipment.
[0084] In summary, the three submodules of this embodiment work together to identify and locate underground pipelines and construction equipment; the feature library construction submodule provides storage and management of known feature descriptors, the nearest neighbor matching submodule finds the most similar feature descriptors during the matching process, and the position and direction determination submodule determines the position and direction of the target object in the image by combining the matching results and feature point position information. Together, they enable the system to achieve efficient and accurate detection and identification of underground pipelines and construction equipment.
[0085] Furthermore, the target position and posture estimation module specifically includes:
[0086] The feature point matching submodule is used to match the feature descriptors obtained by the target detection and recognition module in the target image to find the corresponding feature points, including the position and local feature descriptors;
[0087] The triangulation submodule is used to perform triangulation using the position information of the matched feature points. By observing the same world point in the camera field of view, the distance and angle between the device and the target are calculated to determine the precise position and posture of the target in the scene.
[0088] The position and attitude estimation submodule is used to estimate the position and attitude of underground pipelines and construction equipment in the scene with the help of the distance and angle measurement results of triangulation, combined with equipment parameters and scene information. The position includes the three-dimensional coordinate position of the target in the scene, and the attitude includes information such as the rotation angle and direction of the target.
[0089] Preferably, the feature point matching submodule of this embodiment finds the corresponding feature points in the target image by matching the extracted feature descriptors, including position and local feature descriptors; the effectiveness of feature point matching directly affects the accuracy of subsequent posture estimation, and can help the system accurately track the position and direction changes of the target object in different image frames. The triangulation submodule uses the position information of the matched feature points to calculate the distance and angle between the device and the target through triangulation to determine the precise position and posture of the target in the scene; triangulation is a commonly used technical means, and through accurate distance measurement and angle measurement, the three-dimensional coordinate position and posture information of the target in the scene can be obtained, providing a basis for subsequent modeling and monitoring. The position and posture estimation submodule uses the results of triangulation, combined with equipment parameters and scene information, to accurately estimate the position and posture of underground pipelines and construction equipment in the scene, including three-dimensional coordinate position and rotation angle information; the position and posture estimation submodule can provide accurate target position and direction information, provide an important data basis for the guidance and monitoring functions of the system, and help achieve accurate operation and monitoring requirements.
[0090] In summary, the three submodules of this embodiment work together in the target position and attitude estimation module. Feature point matching ensures that the feature points of the target object are accurately located in the image, triangulation achieves accurate distance and angle measurement, and position and attitude estimation effectively combines these data to give the target's precise position and attitude information in the scene. The various submodules jointly enable the system to accurately locate the position and attitude of underground pipelines and construction equipment, providing key support for the guidance and monitoring of the system.
[0091] Furthermore, the position and attitude estimation submodule specifically includes:
[0092] The parameter acquisition unit is used to collect internal parameters such as the focal length and principal point coordinates of the camera or sensor and external parameters such as the relative position and direction relationship between the camera and the scene; and to obtain lighting conditions and terrain structure at the same time;
[0093] The triangulation unit is used to find the corresponding feature points through the feature point matching submodule, obtain the pixel coordinates in the image, and use the triangulation principle to calculate the distance and angle between the target object and the observation device according to the pixel coordinates, intrinsic parameters and extrinsic parameters of the feature points; the measured distance and angle information is converted from the camera coordinate system to the scene coordinate system to obtain the position of the target object in the three-dimensional space;
[0094] The position and attitude estimation unit is used to combine device parameters and scene information to calculate the three-dimensional coordinate position of the target object in the scene, including cropping, filtering or fusing the position information of multiple feature points; calculate the rotation angle and direction and other attitude information of the target object, and estimate the direction and angle of the target object through the position changes of feature points under different viewing angles.
[0095] Preferably, the parameter acquisition unit of this embodiment provides necessary basic data for subsequent triangulation and attitude estimation by collecting internal and external parameters of the camera or sensor, as well as information such as lighting conditions and scene structure; accurate parameter information can improve the accuracy and stability of position and attitude estimation, while effectively reducing errors in the data processing process, and providing accurate reference standards for subsequent steps. The triangulation unit obtains pixel coordinates through feature point matching, calculates the distance and angle between the target object and the observation device using the triangulation principle, and performs coordinate system conversion to obtain the position of the target object in three-dimensional space; the triangulation unit is the core part of position and attitude estimation, and can accurately estimate the position and attitude of the target object in the scene through precise distance measurement and angle measurement calculation. The position and attitude estimation unit combines device parameters and scene information to calculate the three-dimensional coordinate position and attitude information of the target object, including clipping, filtering or fusing the position information of multiple feature points, and calculating the rotation angle and direction information of the target object; the position and attitude estimation unit can provide more accurate and comprehensive target position and attitude information by integrating multiple data sources and information, providing important support for the guidance, monitoring and operation of the system.
[0096] In summary, the parameter acquisition unit of this embodiment provides basic data, the triangulation unit implements the key step of position estimation, and the position and attitude estimation unit effectively integrates multiple information sources to provide accurate target position and attitude information. All units together promote the system's accurate estimation of the position and attitude of underground pipelines and construction equipment, providing key support for the realization of the system's functions.
[0097] Furthermore, the 3D modeling and positioning module specifically includes:
[0098] The 3D modeling submodule is used to perform 3D modeling of underground pipelines and construction equipment based on the target position and posture information using 3D modeling software; the shape, size and orientation of the target object are accurately presented in 3D space based on the estimated position and posture information;
[0099] A positioning submodule is used to position the generated three-dimensional model with the underground pipeline and construction equipment scene, align it with the underground pipeline and construction equipment scene, and reflect the position and posture of the target object in the underground pipeline and construction equipment scene;
[0100] The guidance and monitoring submodule is used to use the generated three-dimensional model to guide the operator to locate and monitor the target object; by superimposing the generated model into the real-time image, the operator can view the position and posture of the target object in the underground pipeline and construction equipment scene in real time.
[0101] Preferably, the 3D modeling submodule of this embodiment accurately performs 3D modeling of underground pipelines and construction equipment based on target position and posture information using professional 3D modeling software; accurately presents the shape, size and orientation of the target object in 3D space according to the estimated position and posture information. The 3D modeling submodule converts the position and posture information obtained from the actual scene into an accurate 3D model, presents the real form of the underground pipeline and construction equipment in a digital environment, helps the operator to better understand the geometric structure, position and direction of the target object, and provides visualization support for subsequent positioning and guidance. The positioning submodule positions the generated 3D model with the underground pipeline and construction equipment scene, aligns the model with the actual scene, and reflects the accurate position and posture of the target object in the scene; the positioning submodule aligns the generated 3D model with the actual scene to ensure that the model perfectly matches the real scene, which helps the operator to accurately observe, locate and monitor the target object and improve the accuracy and efficiency of the operation. The guidance and monitoring submodule uses the generated three-dimensional model to guide the operator to locate and monitor the target object, superimposes the model into the real-time image, and displays the position and posture of the target object in the scene in real time; the guidance and monitoring submodule enables the operator to understand the position and status of the target object in real time through the real-time feedback of the three-dimensional model, providing strong support for operational decisions. The operator can accurately locate and monitor based on this information to achieve more precise operation and management.
[0102] To sum up, the 3D modeling submodule of this embodiment is responsible for establishing a 3D model of the target object in the real environment, the positioning submodule ensures that the model is aligned with the actual scene, and the guidance and monitoring submodule uses these models for real-time operation and monitoring; each submodule complements each other and together provides reliable technical support for the positioning and operation of underground pipelines and construction equipment.
[0103] Furthermore, the 3D modeling submodule specifically includes:
[0104] A feature extraction unit is used to extract key feature points, edge information or surface features of underground pipelines and construction equipment; a 3D modeling software is used to perform modeling, and based on the acquired position and posture information, the extracted feature points or data points are used to perform surface fitting to generate a 3D model of the underground pipeline and construction equipment;
[0105] The information import unit is used to obtain the target position and posture information of underground pipelines and construction equipment, including the three-dimensional coordinates, rotation angle and direction of the target object; the obtained target position and posture information is imported into the three-dimensional modeling software to locate and rotate the model;
[0106] The model adjustment unit is used to create a basic structural model of underground pipelines and construction equipment in the three-dimensional modeling software, and use basic geometric bodies such as cylinders and boxes to create the initial shape of the target object; according to the imported target position information, the created basic structural model is accurately placed at the corresponding position; according to the imported target posture information, the model is rotated, tilted or scaled accordingly to make it completely match the posture of the target object in the actual scene.
[0107] Preferably, the feature extraction unit of this embodiment is used to extract key feature points, edge information or surface features from the images or data of underground pipelines and construction equipment. These feature points can help establish the geometric shape and structure of the target object; extracting key feature points and edge information helps capture the important features of the target object and provides a basis for subsequent modeling. Accurate feature extraction can help accurately represent the shape and structure of the target object in three-dimensional space. The information import unit obtains the position and posture information of the target object, including three-dimensional coordinates, rotation angle, etc., and imports this information into the three-dimensional modeling software to accurately locate and adjust the model; importing the position and posture information can ensure that the three-dimensional model is accurately aligned with the actual scene. This helps to ensure that the model is consistent with the position and direction of the actual object and improve the accuracy of modeling. The model adjustment unit creates a basic structural model of the target object in the three-dimensional modeling software, and accurately places the model according to the imported position and posture information, and rotates, tilts or scales it according to the posture information to make it completely match the actual scene; through the model adjustment unit, the accurate position positioning and posture adjustment of the model are realized, so that it accurately represents the shape, size and orientation of the actual object, which helps to generate a three-dimensional model that highly matches the actual scene and provides a reliable basis for further application.
[0108] In summary, through the above units, the 3D modeling submodule of this embodiment can effectively realize the 3D modeling of the target object, ensuring a high degree of consistency between the model and the actual scene; and provides accurate and reliable digital model support for guidance, monitoring and operation processes.
[0109] Furthermore, the feature extraction unit specifically includes:
[0110] The curve selection subunit is used to obtain the feature point data of underground pipelines and construction equipment, perform Bezier curve fitting on the feature points, select the Bezier curve degree 3 and the number of control points 6 according to the position and shape of the feature points, and fit the path of the feature points;
[0111] The surface fitting subunit is used to fit the surface of the entire underground pipeline or construction equipment by combining multiple Bezier curves; multiple Bezier curves are connected and combined to achieve fitting and reconstruction of the overall surface;
[0112] The adjustment and optimization subunit is used to adjust and optimize the fitted surface, including fine-tuning the control points of the Bezier curve.
[0113] Preferably, the curve selection subunit of this embodiment obtains the feature point data of the underground pipeline and construction equipment, performs Bezier curve fitting on it, selects the appropriate Bezier curve order and the number of control points, and fits the path of the feature points; through the curve selection subunit, the scattered feature point data can be converted into a continuous curve, which better expresses the actual form of the underground pipeline and construction equipment, and provides a basis for subsequent surface fitting and model reconstruction. The surface fitting subunit fits the surface of the entire underground pipeline or construction equipment by adopting a combination of multiple Bezier curves to achieve the fitting and reconstruction of the overall surface; the surface fitting subunit is responsible for combining the curve fitting results to form a complete surface model, reconstructing the surface shape of the underground pipeline and construction equipment, and can more accurately and comprehensively reconstruct the geometric shape of the actual object. The adjustment and optimization subunit adjusts and optimizes the fitted surface, including fine-tuning the control points of the Bezier curve to improve the fitting accuracy and model quality; the role of the adjustment and optimization subunit is to further optimize the fitting results, ensure that the surface fits the feature points more accurately, increase the accuracy and authenticity of the model, and provide a more accurate data basis for subsequent applications.
[0114] In summary, the surface reconstruction process of underground pipelines and construction equipment in this embodiment is more perfect, which can realize accurate restoration of the object shape and provide a reliable data basis for subsequent modeling, analysis and visualization. The synergy of each sub-unit can realize the smooth conversion and reconstruction from feature points to curves and then to surfaces, providing important support for the morphological recognition and reconstruction of underground pipelines and construction equipment in engineering applications.
[0115] Furthermore, if Figure 4 As shown, the data feedback component 3 specifically includes:
[0116] The information feedback module is used to obtain the location and posture information of underground pipelines and construction equipment from the target monitoring components, and update the location, status and construction progress of underground pipelines in real time in combination with the data collected by the system in real time, such as construction trajectory, depth and speed, and feed back the information to the operator;
[0117] The information display module is used to obtain the image information of underground pipelines and construction scenes using the image acquisition component, and effectively display the image information on the operator's monitoring interface; the operator can view the image of the underground pipeline and the scene information of the construction site in real time;
[0118] The operation feedback module is used to support the interaction between operators and the system, including operation input, command transmission and other functions; at the same time, it records various data during the underground pipeline construction process, such as construction trajectory, depth and speed, for data analysis and statistics.
[0119] Preferably, the information feedback module of this embodiment obtains the position and posture information of underground pipelines and construction equipment from the target monitoring component, and combines the real-time collected data, such as construction trajectory, depth and speed, to update the position, status and construction progress of the underground pipeline in real time and feedback to the operator; the information feedback module can help the operator understand the position, status and progress of the underground pipeline construction process in real time, so that it can correct and adjust the construction plan in time, ensure that the construction is carried out as planned, and improve work efficiency and accuracy. The information display module uses the image information of the underground pipeline and the construction scene obtained by the image acquisition component, and effectively displays it on the operator's monitoring interface, so that the operator can view the image of the underground pipeline and the scene information of the construction site in real time; the information display module provides a visual display method to help the operator intuitively understand the actual situation of the underground pipeline and the construction scene, provide important support for making correct decisions, and enhance the real-time monitoring and management capabilities of the construction site. The operation feedback module supports the interaction between operators and the system, including operation input, command transmission and other functions; at the same time, it records various data during the underground pipeline construction process, such as construction trajectory, depth and speed, and conducts data analysis and statistics; the operation feedback module realizes two-way communication between operators and the system, allowing operators to perform corresponding operations based on the information fed back by the system and achieve instant response; by recording and analyzing data, it provides a basis for the optimization of the construction process and promotes the improvement of work efficiency and quality.
[0120] In summary, the information feedback module, information display module and operation feedback module in the data feedback component of this embodiment work together to enable operators to timely understand the situation of underground pipeline construction, realize long-term monitoring and real-time adjustment of the construction process, thereby improving construction efficiency, accuracy and safety. The comprehensive effect of these modules helps to ensure the smooth progress of the jacking construction process and optimize the construction process.
[0121] like Figure 5 As shown, this embodiment provides an embodiment of an electronic device. In this embodiment, the electronic device 4 includes a processor 41 and a memory 42 coupled to the processor 41.
[0122] The memory 42 stores program instructions for implementing the computer vision-based pipe jacking construction guidance system of any of the above-mentioned embodiments.
[0123] The processor 41 is used to execute program instructions stored in the memory 42 to perform pipe jacking construction guidance based on computer vision.
[0124] The processor 41 may also be referred to as a CPU (Central Processing Unit). The processor 41 may be an integrated circuit chip having signal processing capabilities. The processor 41 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0125] Further, Figure 6 The schematic diagram of the structure of the storage medium of an embodiment of the present application is that the storage medium 5 of the embodiment of the present application stores program instructions 51 that can implement all the above methods, wherein the program instructions 51 can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.
[0126] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0127] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units. The above is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present invention.
[0128] The specific implementation methods of the invention are described in detail above, but they are only examples, and the invention is not limited to the specific implementation methods described above. For those skilled in the art, any equivalent modification or substitution of the invention is also within the scope of the invention, therefore, the equalization, modification, improvement, etc. made without departing from the spirit and principle of the invention should be included in the scope of the invention.
Claims
1. A pipe jacking construction guidance system based on computer vision, characterized in that: The pipe jacking construction guidance system based on computer vision includes: Image acquisition component, used to use cameras and laser scanners to collect images of underground pipeline construction sites, and to extract feature information of underground pipelines and construction equipment by processing the collected images through denoising, enhancement, filtering and edge detection; The target monitoring component is used to detect and identify targets through feature extraction algorithms, determine the location and posture of underground pipelines and construction equipment, and perform three-dimensional modeling and positioning of the detected location and posture of underground pipelines and construction equipment; The data feedback component is used to monitor the underground pipeline construction process in real time, feedback the location, status and construction progress of the underground pipeline to the operator, display the underground pipeline image, construction scene information and operation instructions, support the interaction and feedback between the operator and the system; record the data during the underground pipeline construction process, including construction trajectory, depth and speed information, conduct data analysis and statistics, and provide reference for the optimization and improvement of the construction process.
2. The computer vision-based pipe jacking construction guidance system according to claim 1, characterized in that: Image acquisition components, including: An image processing module is used to collect images of underground pipeline construction sites using cameras or laser scanners to obtain real-time images of underground pipelines and construction equipment; and to perform denoising, enhancement and filtering on the collected images; The edge detection module is used to convert the collected image into a grayscale image, smooth the grayscale image using Gaussian filtering, perform convolution operations on the smoothed grayscale image using the Sobel operator, calculate the horizontal and vertical gradients of each pixel, and obtain the gradient amplitude and gradient direction of each pixel; combine the horizontal and vertical gradients to calculate the gradient amplitude of each pixel to indicate the intensity of the grayscale change of the point; according to the horizontal and vertical gradients, use the arctan function to calculate the gradient direction of each pixel; set the gradient amplitude threshold, set the lower gradient amplitude to 0, and the higher gradient amplitude to 255 to extract the edge; refine the extracted edge and retain the main edge line; make the edge line thinner through non-maximum suppression processing, and only retain the pixels with the largest local gradient; set double thresholds, divide the gradient amplitude into three categories: strong edge, weak edge and non-edge, and obtain a clear edge image; connect the strong edge with its adjacent weak edge through the edge connection algorithm to form a complete edge line; The feature extraction module is used to detect key points in the image using the SURF algorithm. The key points represent the local features of shape, texture and color in the image of underground pipelines and construction equipment; the image gradient is calculated using a box filter to obtain local directional information near the key points; the local features extracted from each key point are described using descriptor expressions. The descriptors are used to describe the features of the area around the key points, including the position, scale and directional information of the shape, texture and color; the feature points extracted from the image are matched using Hellenger feature matching, and whether two feature points match is determined by comparing the similarity of the descriptors, and the corresponding relationship between the feature points of the two images is established; the feature information of the underground pipelines and construction equipment is extracted from the matched feature points to describe the characteristics and features of the underground pipelines and construction equipment in the image.
3. The computer vision-based pipe jacking construction guidance system according to claim 1, characterized in that: Target monitoring components, including: The target detection and recognition module is used to use the feature extraction algorithm to perform key point detection and feature extraction on the collected images of underground pipelines and construction equipment, and obtain key points and local feature descriptors; the extracted feature descriptors are matched with the constructed feature library, and the position and direction of the underground pipelines and construction equipment in the image are determined by Hellenger distance matching; The target position and posture estimation module is used to estimate the precise position and posture information of underground pipelines and construction equipment in the scene through the matched feature points and their positions with the help of triangulation; The 3D modeling and positioning module is used to perform 3D modeling and positioning based on the target position and posture information, and to generate models of underground pipelines and construction equipment in underground pipeline and construction equipment scenes for guidance and monitoring.
4. The computer vision-based pipe jacking construction guidance system according to claim 3, characterized in that: Target detection and recognition module, including: The key point search submodule is responsible for detecting the brightness values of surrounding pixels to quickly determine whether they are key points and determine the locations with significant features in the image, which are the key points. The local feature description submodule is responsible for assigning a main direction to each key point around the detected key point. The main direction is calculated based on the gradient information of the pixels around the key point. After obtaining the key point position and main direction, the BRIEF algorithm is used to generate a local feature descriptor to describe the local features around the key point. The descriptor output submodule is responsible for outputting the local feature descriptor around each key point position. The descriptor includes the gradient, color or texture information around the key point.
5. The computer vision-based pipe jacking construction guidance system according to claim 3, characterized in that: Target detection and recognition module, including: The feature library construction submodule is used to build a feature library to store the feature descriptors of known underground pipelines and construction equipment; the extracted feature descriptors of the target objects are matched with the descriptors in the constructed feature library. In the matching process, the Hellenger distance is used as a metric to measure the similarity between the two descriptors; The nearest neighbor matching submodule is used to compare each extracted feature descriptor with all descriptors in the feature library using Hellenger distance matching and find the descriptor with the smallest distance; The position and direction determination submodule is used to find the position information of the feature points in the image through the matching results obtained by nearest neighbor matching, and use triangulation to obtain the position and direction of the underground pipeline and construction equipment in the image in combination with the position and correspondence of the feature points, thereby realizing the determination of the position and direction.
6. The computer vision-based pipe jacking construction guidance system according to claim 3, characterized in that: Target position and attitude estimation module, including: The feature point matching submodule is used to match the feature descriptors obtained by the target detection and recognition module in the target image to find the corresponding feature points, including the position and local feature descriptors; The triangulation submodule is used to perform triangulation using the position information of the matched feature points. By observing the same world point in the camera field of view, the distance and angle between the device and the target are calculated to determine the precise position and posture of the target in the scene. The position and attitude estimation submodule is used to estimate the position and attitude of underground pipelines and construction equipment in the scene with the help of the distance and angle measurement results of triangulation, combined with equipment parameters and scene information. The position includes the three-dimensional coordinate position of the target in the scene, and the attitude includes the rotation angle and direction information of the target.
7. The computer vision-based pipe jacking construction guidance system according to claim 6, characterized in that: Position and attitude estimation submodule, including: The parameter acquisition unit is used to collect the focal length of the camera or sensor, the internal parameters of the principal point coordinates, and the external parameters of the relative position and direction relationship between the camera and the scene; at the same time, it obtains the lighting conditions and terrain structure; The triangulation unit is used to find the corresponding feature points through the feature point matching submodule, obtain the pixel coordinates in the image, and use the triangulation principle to calculate the distance and angle between the target object and the observation device according to the pixel coordinates, intrinsic parameters and extrinsic parameters of the feature points; the measured distance and angle information is converted from the camera coordinate system to the scene coordinate system to obtain the position of the target object in the three-dimensional space; The position and attitude estimation unit is used to combine device parameters and scene information to calculate the three-dimensional coordinate position of the target object in the scene, including cropping, filtering or fusing the position information of multiple feature points; calculate the rotation angle and direction attitude information of the target object, and estimate the direction and angle of the target object through the position changes of feature points at different viewing angles.
8. The computer vision-based pipe jacking construction guidance system according to claim 3, characterized in that: 3D modeling and positioning module, including: The 3D modeling submodule is used to perform 3D modeling of underground pipelines and construction equipment based on the target position and posture information using 3D modeling software; the shape, size and orientation of the target object are accurately presented in 3D space based on the estimated position and posture information; A positioning submodule is used to position the generated three-dimensional model with the underground pipeline and construction equipment scene, align it with the underground pipeline and construction equipment scene, and reflect the position and posture of the target object in the underground pipeline and construction equipment scene; The guidance and monitoring submodule is used to use the generated three-dimensional model to guide the operator to locate and monitor the target object; by superimposing the generated model into the real-time image, the operator can view the position and posture of the target object in the underground pipeline and construction equipment scene in real time.
9. The computer vision-based pipe jacking construction guidance system according to claim 1, characterized in that: Data feedback components, including: The information feedback module is used to obtain the location and posture information of underground pipelines and construction equipment from the target monitoring components, and update the location, status and construction progress of underground pipelines in real time by combining the data collected by the system in real time, the construction trajectory, depth and speed; and feed back the information to the operator; The information display module is used to obtain the image information of underground pipelines and construction scenes using the image acquisition component, and effectively display the image information on the operator's monitoring interface; the operator can view the image of the underground pipeline and the scene information of the construction site in real time; The operation feedback module is used to support the interaction between operators and the system, including operation input and command transmission functions; at the same time, it records various data during the underground pipeline construction process, construction trajectory, depth and speed, and conducts data analysis and statistics.
10. An electronic device, characterized in that: It includes a processor and a memory coupled to the processor, wherein the memory stores program instructions that can be executed by the processor; when the processor executes the program instructions stored in the memory, the computer vision-based pipe jacking construction guidance system as described in any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
Pipe jacking construction monitoring control method
CN117704151A
Pipe jacking construction method
CN117846616A
Rectangular pipe jacking construction settlement control method
CN118208242A
Cited By
Concrete tower drum ring piece positioning and splicing method and system based on visual model
CN121437623A
