Machine vision-driven pipeline robot intelligent operation platform

Abstract

The present invention discloses a pipeline robot intelligent operation platform driven by machine vision, which relates to the field of image data processing technology. The platform includes: a focus establishment module for performing initial data acquisition and pipeline fitting modeling, and configuring pipeline focus coordinates; an image acquisition module for performing image acquisition in the pipeline; a focus recognition module for performing image focus recognition; a position update module for performing position update optimization based on the focus position point; an additional acquisition module for establishing additional frame images; a defect positioning module for generating defect locations after identifying and verifying multiple focus recognition results. The present invention solves the technical problem that the existing pipeline inspection technology lacks efficient defect detection and positioning capabilities in complex environments, resulting in insufficient detection accuracy and coverage, and achieves the technical effect of high-precision detection of defects and full coverage inspection in complex environments through multiple identification verifications, trust authentication and dynamic path optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a pipeline robot intelligent operation platform driven by machine vision. Background Art

[0002] With the acceleration of industrialization, pipeline systems, as important infrastructure for transporting energy, liquids or gases, are widely used in petrochemical, energy, electricity and other fields. However, since pipeline systems are often widely distributed and in complex environments, internal defects (such as corrosion, cracks, and blockages) may seriously threaten operational safety and efficiency. Traditional manual inspection methods are limited in coverage, low in efficiency, and lack of accuracy, especially in environments with many bends, complex branches, and poor lighting conditions, and cannot achieve efficient defect identification and positioning. Summary of the invention

[0003] This application provides a machine vision-driven pipeline robot intelligent operation platform to solve the technical problem that existing pipeline inspection technologies lack efficient defect detection and positioning capabilities in complex environments, resulting in insufficient detection accuracy and coverage.

[0004] The present application provides a machine vision-driven pipeline robot intelligent operation platform, which includes: a focus establishment module, which is used to perform pipeline fitting modeling in a preset three-dimensional coordinate system and configure pipeline focus coordinates after initial data acquisition of the pipeline; an image acquisition module, which is used to control the pipeline robot to enter the pipeline, generate pipeline robot coordinates according to the real-time response position of the pipeline robot and the preset three-dimensional coordinate system, and activate the image acquisition device of the pipeline robot to perform image acquisition in the pipeline and establish a real-time frame image; an attention recognition module, which is used to determine whether the image position of the real-time frame image is within the range of the pipeline focus coordinates. If the image position of the real-time frame image is within the range of the pipeline focus coordinates, the real-time frame image is image focus recognized and established. A first focus recognition result; a position update module, used to extract the coordinates of the focus position point of the first focus recognition result, and control the pipeline robot to perform position update optimization according to the coordinates of the focus position point to establish an optimization result; an additional acquisition module, used to control the pipeline robot to move to the position point of the optimization result, and establish a cleaning response strategy according to the first focus recognition result, after cleaning the focus position point according to the cleaning response strategy, control the image acquisition device to perform additional image acquisition to establish an additional frame image; a defect localization module, used to enhance the recognition of the additional frame image, establish a second focus recognition result, and generate defect localization through the coordinates of the focus position point after recognition and verification according to the first focus recognition result and the second focus recognition result.

[0005] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0006] The machine vision-driven pipeline robot intelligent operation platform provided in this application relates to the field of image data processing technology. Through initial data collection and three-dimensional modeling, the pipeline focus coordinates are determined; the robot is controlled to collect images in real time, identify the focus area and generate the first recognition result; the robot position is optimized according to the recognition result, the focus point is cleaned and additional collection is performed to generate high-precision additional images; after multiple recognition result verifications, the defect is accurately located. It solves the technical problem that the existing pipeline inspection technology lacks efficient defect detection and positioning capabilities in complex environments, resulting in insufficient detection accuracy and coverage, and realizes the technical effect of high-precision detection of defects and full coverage inspection in complex environments through multiple recognition verifications, trust authentication and dynamic path optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0008] Figure 1 A schematic diagram of the structure of a machine vision-driven pipeline robot intelligent operation platform provided in an embodiment of the present application;

[0009] Figure 2 A schematic diagram of the process of performing position update optimization by the position update module of the machine vision-driven pipeline robot intelligent operation platform provided in an embodiment of the present application.

[0010] Explanation of the reference numerals: attention establishment module 11 , image acquisition module 12 , attention identification module 13 , position update module 14 , additional acquisition module 15 , defect location module 16 . DETAILED DESCRIPTION

[0011] This application provides a machine vision-driven pipeline robot intelligent operation platform to solve the technical problem that existing pipeline inspection technologies lack efficient defect detection and positioning capabilities in complex environments, resulting in insufficient detection accuracy and coverage.

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0013] It should be noted that the terms "first", "second", etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices.

[0014] Embodiment 1, as Figure 1 As shown, the present application provides a machine vision-driven pipeline robot intelligent operation platform, the platform comprising:

[0015] The focus establishment module 11 is used to perform pipeline fitting modeling in a preset three-dimensional coordinate system and configure pipeline focus coordinates after initial data collection on the pipeline.

[0016] Specifically, the focus establishment module 11 of the present application is the core startup module of the pipeline robot intelligent operation platform. Its main task is to complete the digital reconstruction of the pipeline structure in a preset three-dimensional coordinate system through initial data collection and modeling, and determine the coordinates of key focus areas, providing an accurate basis for subsequent inspections and operations.

[0017] First, before entering the pipeline, the pipeline robot uses the installed high-definition camera and laser scanning equipment to perform initial data collection to capture the morphological information inside the pipeline. This includes key geometric features such as the diameter, length, curvature, and branch structure of the pipeline. The goal of this process is to obtain high-quality raw data to ensure the accuracy of subsequent modeling. During the collection process, depth data is obtained through laser radar (LiDAR) technology, and high-resolution internal images are generated with the camera to form complete spatial information.

[0018] After completing the data collection, a three-dimensional coordinate system needs to be constructed as a reference frame. The three-dimensional coordinate system takes the pipeline entrance as the origin and defines the position of all points in the pipeline through the x, y, and z axes. This coordinate system provides a unified spatial reference for the pipeline robot, ensuring that the robot can accurately position itself when moving and operating. Using the SLAM (Simultaneous Localization and Mapping) algorithm, the robot can dynamically update its position information inside the pipeline to be consistent with the three-dimensional coordinate system.

[0019] Next, the pipeline fitting modeling is performed based on the collected data. The discrete point cloud data is converted into a continuous geometric model through fitting algorithms (such as least squares method or B-spline curve fitting), thereby restoring the real structure of the pipeline. In order to improve the accuracy of the model, the RANSAC (random sampling consensus) algorithm can be used to filter noise points to ensure the reliability of the fitting results. In addition, the model is optimized and smoothed through 3D modeling tools (such as CAD software) to generate a high-precision digital model.

[0020] After generating the 3D model, the pipeline’s geometric features and historical data can be combined to determine the pipeline’s focus coordinates. These focus coordinates usually correspond to areas where there may be defects, blockages, or corrosion, such as pipe bends, branch interfaces, and vulnerable locations. By analyzing the pipeline’s geometric features and combining the fault information in the historical database, the module automatically marks these key areas and generates the corresponding 3D coordinates.

[0021] Finally, a data package containing a 3D pipeline model and key coordinates of interest is output. These data not only include the overall structure and texture information of the pipeline, but also clearly mark the coordinates of the areas that need to be inspected, which can provide an accurate input basis for the subsequent image acquisition module and the attention recognition module.

[0022] The image acquisition module 12 is used to control the pipeline robot to enter the pipeline, generate the pipeline robot coordinates according to the real-time response position of the pipeline robot and the preset three-dimensional coordinate system, and activate the image acquisition device of the pipeline robot to perform image acquisition in the pipeline and establish a real-time frame image.

[0023] Optionally, the image acquisition module 12 of the present application is a key link in the intelligent operation of the pipeline robot. Its main function is to control the movement of the robot inside the pipeline and collect image data in real time to ensure that the images of key positions during the inspection process are clear and efficient and accurate.

[0024] When the pipeline robot enters the pipeline, the robot's real-time response position is first matched with the preset three-dimensional coordinate system to generate the current coordinates of the pipeline robot. The real-time response position refers to the dynamic position information obtained by the robot through built-in sensors (such as inertial measurement unit IMU, odometer or ultrasonic positioning module). After this information is processed by the SLAM (Simultaneous Localization and Mapping) algorithm, it is associated with the reference position in the preset three-dimensional coordinate system to generate the precise coordinates of the current robot.

[0025] After obtaining the robot's coordinates, the module activates the image acquisition device for real-time image acquisition. The image acquisition device is usually composed of a high-definition camera and an auxiliary light source, which can adapt to the environment of insufficient light and narrow space inside the pipeline. The camera will adjust the focal length and angle according to the robot's current coordinates and posture to ensure that the acquired image covers the target area of ​​the inner wall of the pipeline. During this process, the robot will dynamically adjust the movement speed and direction according to the acquisition requirements to avoid image blur due to excessive movement.

[0026] In order to enhance the clarity of the captured images, the module uses image stabilization and correction technology. Since the pipeline robot may be affected by vibration or environmental interference during movement, the captured images may be offset or blurred. The built-in gyroscope and accelerometer can sense the robot's movement state in real time and correct the captured images to ensure the clarity of each frame.

[0027] In addition, module 12 is also responsible for establishing real-time frame images, that is, during the image acquisition process, the continuously acquired image frames are stored and managed in chronological order. Real-time frame images are the core data of the inspection task, and these frame images can be used to restore the dynamic picture inside the pipeline. In order to optimize data transmission and storage efficiency, JPEG or H.264 encoding technology can be used to compress image data while retaining the necessary resolution information to meet the needs of subsequent analysis.

[0028] The entire image acquisition process is closely integrated with the pipeline's 3D model and robot coordinates to ensure that the scope and accuracy of image acquisition are consistent with the preset task requirements. Through this module, the pipeline robot can provide a clear internal view in real time, providing a reliable visual basis for subsequent attention identification and defect analysis.

[0029] The focus recognition module 13 is used to determine whether the image position of the real-time frame image is within the pipeline focus coordinate range. If the image position of the real-time frame image is within the pipeline focus coordinate range, the real-time frame image is subjected to image focus recognition to establish a first focus recognition result.

[0030] Furthermore, the attention identification module 13 is also used to perform the following steps:

[0031] P31: After image preprocessing of the real-time frame image, a dual image channel is constructed, and the dual image channel includes an original image channel and a grayscale channel; P32: After edge detection of the dual image channels respectively, edge sharing of the dual image channels is performed, and a region of interest is configured according to the edge sharing result; P33: Multi-scale sliding window attention recognition of the dual image channels is performed using the region of interest, a multi-scale fusion recognition result is established, and the multi-scale fusion recognition result is output as the first attention recognition result.

[0032] It should be understood that the focus identification module 13 of the present application is the core link for intelligent analysis of real-time frame images during the pipeline robot inspection process. Its main task is to determine whether the image contains a focus area, and conduct in-depth analysis of the included area to generate a first focus identification result, providing reliable support for subsequent optimization and defect location.

[0033] Specifically, when the real-time frame image is received, it is first preprocessed. This process includes operations such as denoising, contrast enhancement, and brightness equalization, which aims to improve image quality and ensure the reliability and accuracy of subsequent processing. After the preprocessing is completed, a dual image channel is constructed, including an original image channel and a grayscale channel. Among them, the original image channel can retain the texture and color information in the color image, which is suitable for analyzing the detailed features of the target. The grayscale channel can simplify the calculation complexity by extracting the brightness information of the image, and provide a higher contrast basis for edge detection. The combination of these two channels enables subsequent processing to focus on both global information and highlight key details.

[0034] Next, edge detection is performed on the dual image channels respectively to extract the areas with large changes in pixel values ​​in each channel, which usually correspond to the contours of the object. The edge detection process can use algorithms such as Canny or Sobel to extract clear edges in a noisy environment. Subsequently, the module performs an edge sharing operation to compare and fuse the detection results of the original image channel and the grayscale channel, retaining the common edge parts in the two channels. Edge sharing helps to eliminate pseudo-edges and noise areas, and only retains contour information with high confidence in the target object. Based on the shared edge results, the region of interest is further defined, that is, the area that may contain target defects or anomalies. The precise delineation of the region of interest provides scope optimization for subsequent calculations.

[0035] Furthermore, the module uses multi-scale sliding window technology to perform detailed analysis of the dual image channels based on the region of interest. Multi-scale sliding windows can adapt to the diversity of target objects by scanning images at different scales. For example, a small-scale window is suitable for detecting small defects (such as cracks or corrosion spots), while a large-scale window is suitable for identifying larger targets (such as blockages or structural deformations).

[0036] At each scale, the features in the window are identified, and the recognition results of the two channels are fused to generate a multi-scale fusion recognition result. This result integrates feature information at different scales to ensure the comprehensiveness and accuracy of recognition. Finally, the fusion result is output as the first focus recognition result, which contains detailed information such as the location, type, and confidence of the target feature, providing reliable support for subsequent defect location and response strategies.

[0037] Through the above steps, the pipeline robot can intelligently screen key areas from complex real-time images and complete efficient analysis, laying the foundation for the accuracy and efficiency of inspection tasks.

[0038] Furthermore, step P33 of the embodiment of the present application also includes:

[0039] P33-1: Based on the region of interest, feature extraction is performed on dual image channels respectively to establish a fusion feature set, which includes texture features, color features, and shape features; P33-2: A multi-scale window is configured according to the fusion feature set, and a mapping window sliding step is established according to the multi-scale window and feature density; P33-3: The multi-scale window and the mapping window sliding step are used to perform traversal identification of the region of interest, and traversal trust is established; P33-4: Joint authentication analysis is performed based on the traversal trust to generate a multi-scale fusion identification result.

[0040] Optionally, the in-depth analysis and feature recognition process of the region of interest can be further refined to improve the accuracy and reliability of multi-scale fusion recognition through more comprehensive feature extraction and intelligent window sliding strategies.

[0041] First, within the region of interest, the module extracts features from the original image channel and the grayscale channel respectively to construct a complete fused feature set. The fused feature set includes three main features: texture features, color features, and shape features. For example, in the original image channel, color features are extracted to analyze the color distribution of the inner wall of the pipe and identify areas of corrosion or stains; at the same time, texture features and shape features are extracted through the grayscale channel to describe the detail pattern and geometric contour of the surface, respectively. For example, the gray-level co-occurrence matrix (GLCM) is used to analyze textures, the color histogram is used for color distribution, and the Hough transform is used to identify shape features. All extracted features will be fused between the two channels to form a comprehensive feature set, which provides a data basis for subsequent multi-scale window configuration.

[0042] Next, based on the statistical analysis of the fused feature set, the module dynamically configures a multi-scale window. A multi-scale window refers to a sliding window covering different sizes to accommodate the diversity of feature sizes. For example, a small window is used to detect tiny targets such as cracks, and a large window is used to identify blockages or abnormal shapes in larger areas. At the same time, in order to improve traversal efficiency, the module dynamically adjusts the step size of the sliding window according to the density and distribution of features in the fused feature set. Areas with high feature density use a smaller step size for fine scanning, while areas with lower density use a larger step size to increase speed. This adaptive strategy not only optimizes computational efficiency, but also avoids missing key features.

[0043] Next, the region of interest is scanned pixel by pixel or region by region using multi-scale windows and mapping sliding steps to perform traversal recognition. During the traversal process, the fusion features extracted in each window are analyzed to determine the degree of match with the target feature pattern, and a traversal confidence is assigned to each window. The traversal confidence reflects the reliability of the recognition result in the window. The higher the confidence, the closer the feature is to the preset target, such as a corroded area or a blockage. Low-confidence areas will be marked as secondary areas, and high-confidence areas will be processed first.

[0044] After completing the traversal identification of the area of ​​interest, the module conducts a joint authentication analysis on the traversal confidence of all windows. This process can integrate the recognition results of multi-scale windows through weighted average or maximum likelihood estimation methods, eliminate isolated or low-confidence areas, and integrate high-confidence feature information to generate the final multi-scale fusion recognition result. This result not only includes the type and location of the target, but also comes with a confidence score, providing a high-precision input basis for subsequent defect location and response strategies.

[0045] The position updating module 14 is used to extract the coordinates of the focus position point of the first focus recognition result, control the pipeline robot to perform position updating optimization according to the coordinates of the focus position point, and establish an optimization result.

[0046] Further, such as Figure 2 As shown, the location updating module 14 is also used to perform the following steps:

[0047] P41: Configure the stay fitness of the location point based on the pipeline fitting modeling result; P42: Establish a focus angle evaluation based on the first focus identification result, and generate a focus angle evaluation result; P43: Use the focus angle evaluation result and the coordinates of the focus location point to perform adaptive screening of the observation location point, and establish a screened location point set; P44: Take the real-time response location as the starting point and the screened location point set as the observation end point, perform path optimization, and establish path fitness; P45: Perform fitness analysis based on the screened location point set to generate observation fitness; P46: Use the stay fitness, the path fitness, and the observation fitness to perform position update optimization and establish an optimization result.

[0048] It should be understood that the position update module 14 of the present application is the core part of the pipeline robot's dynamic adjustment of the operating position and path optimization. Its main function is to generate the optimal motion strategy through comprehensive analysis of the focus position points, pipeline structure and robot status to achieve efficient inspection and precise operation.

[0049] First, based on the results of pipeline fitting modeling, a stay fitness is configured for each location of interest. Stay fitness is an indicator for evaluating the robot's stay at a specific location, which comprehensively considers the structural characteristics of the pipeline (such as curvature and stenosis) and detection requirements (such as the complexity of the feature area). Locations with high fitness indicate that the robot can stay and operate safely and efficiently; low fitness indicates that there may be operational risks or inefficiency. The stay fitness can be calculated by combining the pipeline geometry and historical data to ensure that the stay strategy is reasonable.

[0050] Next, based on the first focus identification result, the angle of each focus position point is analyzed to generate a focus angle evaluation result. The focus angle refers to the angle offset required to observe the target point from the current robot perspective. By analyzing the range and complexity of the focus angle, the convenience and field of view coverage of the robot when observing the position point can be evaluated. The evaluation result is used to determine whether the robot posture or position needs to be adjusted to optimize the observation conditions.

[0051] Furthermore, the observation point adaptation screening is performed by combining the evaluation results of the angle of interest with the coordinates of the position points of interest. This process screens out key points suitable for robot observation by comprehensively analyzing the angle convenience and coordinate position of each position point, forming a set of screening position points. The screened set not only ensures the visibility of the target, but also eliminates redundant points that are not conducive to detection or unnecessary, thereby improving the inspection efficiency.

[0052] Next, the path optimization is performed with the current real-time response position of the robot as the starting point and the set of filtered position points as the observation end point. Path optimization can use Dijkstra algorithm, etc. to calculate the optimal path from the current position of the robot to the target point, taking into account the geometric characteristics of the pipeline (such as bends, narrow areas) and the robot's motion restrictions (such as turning radius). After the path optimization is completed, the path fitness is configured for each path. A path with a high fitness is shorter, safer and more efficient.

[0053] Furthermore, based on the set of filtered location points, fitness analysis is performed on each location point to generate observation fitness. Observation fitness combines the stay fitness, the evaluation results of the focus angle, and the path fitness to comprehensively evaluate the actual convenience and effectiveness of the robot reaching the target point and performing operations. Exemplarily, a weighted multi-objective optimization model can be used to quantitatively integrate the stay, path, and observation fitness. This analysis ensures that the location point finally selected can meet the inspection needs to the greatest extent.

[0054] Finally, the stay fitness, path fitness and observation fitness are jointly optimized, and the genetic algorithm or particle swarm optimization algorithm is used to calculate the best position update optimization result. The optimization result includes the target point that the robot needs to move to and its path information, guiding the robot to complete the inspection task with the highest efficiency.

[0055] Furthermore, step P43 of the embodiment of the present application also includes:

[0056] P43-1: Configure the bend features and branch features according to the pipeline fitting modeling results; P43-2: Perform additional impact analysis of the focus angle based on the bend features and the branch features, and establish additional impact evaluation results; P43-3: Perform adaptive screening of observation position points based on the additional impact evaluation results and the focus angle evaluation results, and establish a set of screening position points.

[0057] In a possible embodiment of the present application, through an in-depth analysis of the pipeline structure characteristics, the interference problem of bends and branch areas on the selection of observation positions is focused on solving.

[0058] First, based on the pipeline fitting modeling results, the bend and branch features in the pipeline are extracted. The bend features include the curvature radius, bend angle, and directionality, which describe the degree and complexity of the bending of the pipeline in space; the branch features include the branch position, opening angle, and number of branches, marking the multi-path distribution area in the pipeline. These features reflect the possible limitations of the pipeline structure on the movement and observation of the robot. By establishing a digital model of these features, an accurate structural information foundation is provided for subsequent analysis.

[0059] Next, the extracted curve features and branch features are combined to perform additional impact analysis on the angle of interest to evaluate the potential interference of these structural complexities on the robot's observation task. Curves may limit the viewing angle and reduce the observation coverage; branches may introduce target occlusion or multipath interference, affecting the integrity of the observation. By quantifying these additional impacts, a weighted model can be used to evaluate the degree of interference of curve and branch features, and additional impact evaluation results can be generated to ensure that subsequent screening can fully consider the impact of structural complexity.

[0060] Furthermore, the results of additional impact evaluation and attention angle evaluation are used to comprehensively screen the adaptability of observation points. Through multi-objective optimization algorithms (such as hierarchical analysis method), the adaptability of each position point is comprehensively scored in combination with the characteristics of curves and branches, and points with poor viewing angles or large structural interference are eliminated, while giving priority to the position points with the best observation conditions. The final output of the set of screening position points can not only ensure the accuracy of the observation task, but also minimize the interference of the pipeline structure on the operation and operation of the robot, providing efficient input for subsequent path optimization.

[0061] Through the above steps, the module can effectively screen out the most adaptable set of position points in a complex pipeline environment, providing a high-quality input basis for robot path optimization and observation tasks.

[0062] The additional acquisition module 15 is used to control the pipeline robot to move to the position point of the optimization result, and establish a cleaning response strategy according to the first focus identification result. After cleaning the focus position point according to the cleaning response strategy, the image acquisition device is controlled to perform additional image acquisition to establish an additional frame image.

[0063] Furthermore, the additional acquisition module 15 is further configured to perform the following steps:

[0064] P51: configuring acquisition parameters according to the coordinates of the focus point and the position point of the optimization result, wherein the acquisition parameters include focusing distance, auxiliary light parameters, and aperture parameters; P52: after adjusting the parameters of the image acquisition device according to the acquisition parameters, performing additional image acquisition to establish the additional frame image.

[0065] Specifically, the additional acquisition module 15 of the present application is used to perform in-depth processing and high-quality data acquisition on the position points of interest after the pipeline robot is initially identified. By controlling the robot to move to the position point in the optimization result, a cleaning response strategy is generated according to the first focus identification result, and the cleaning operation of the target area is performed to remove stains, sediments or obstacles to ensure the clarity of subsequent image acquisition. After the cleaning is completed, the module controls the image acquisition device to perform additional image acquisition to generate more accurate additional frame images to meet the needs of high-precision analysis.

[0066] Specifically, first, according to the coordinates of the target focus point and the position point in the optimization result, the module dynamically configures the acquisition parameters to ensure that the image acquisition device can adapt to the environmental conditions of the target area. The acquisition parameters may include focus distance, auxiliary light parameters and aperture parameters. Among them, the focus distance can be used to calculate the distance between the target point and the device in real time through laser ranging or visual feedback technology, adjust the focal length, and ensure the clarity of the shot. The auxiliary light parameters can be used to dynamically adjust the brightness, angle and color temperature of the auxiliary light source in combination with the light conditions inside the pipeline to avoid interference with the image quality caused by shadows and reflections. The aperture parameters can be used to control the aperture size according to the brightness level of the target area, optimize the amount of light entering, and prevent overexposure or underexposure. Through the comprehensive regulation of these parameters, the best equipment configuration environment is created for additional image acquisition.

[0067] After completing the parameter configuration, the image acquisition device is controlled to perform additional image acquisition, and detailed shots are taken of the target area after cleaning to capture high-resolution detail information. To improve the stability and clarity of the shooting, the module integrates optical image stabilization and electronic image stabilization technologies to compensate for possible micro-movements of the robot. In addition, image enhancement technologies such as super-resolution reconstruction can be used to post-process the acquired images to further improve the resolution and contrast of the images. The additional frame images generated in the end are stored and annotated, and compared and analyzed with the real-time frame images to provide high-quality data support for subsequent identification verification and defect location.

[0068] Through the additional acquisition module, the pipeline robot achieves precise target cleaning and acquisition of high-quality image data, providing comprehensive support for detailed inspection and defect analysis in complex pipeline environments.

[0069] The defect location module 16 is used to enhance the recognition of the additional frame image, establish a second focus recognition result, perform recognition verification based on the first focus recognition result and the second focus recognition result, and generate the defect location through the coordinates of the focus position point.

[0070] Furthermore, the defect location module 16 is further configured to perform the following steps:

[0071] P61: Perform trust identification on the first focus identification result and the second focus identification result respectively, and establish a trust coefficient; P62: Use the trust coefficient to perform trust authentication on the first focus identification result and the second focus identification result, and establish a trust authentication result; P63: Use the trust authentication result and the coordinates of the focus position point to complete defect positioning.

[0072] It should be understood that the defect location module 16 of the present application completes the final location and feature annotation of the target defect by performing a refined analysis of the additional frame image and comprehensive verification of multiple recognition results. This module plays a role in converting the recognition results into actionable data at the end of the inspection task, ensuring the accuracy and reliability of the defect information.

[0073] First, the trust level of the first and second recognition results is identified, and a trust coefficient is established for each result. Trust level identification is mainly based on a comprehensive evaluation of multi-dimensional indicators such as the model confidence of the recognition result, the feature matching rate, and the clarity of the attached image. For example, after the attached frame image is extracted through high-resolution features, if the identified defect features are highly consistent with the model expectations, its trust level will be assigned a higher score. The trust coefficient is a reliability parameter of the quantitative result, usually ranging from 0 to 1, which is used to indicate the credibility of the recognition result and provide support for subsequent trust certification.

[0074] Next, based on the calculated trust coefficient, the first and second identification results of interest are trusted and authenticated. This process verifies the overall reliability of the identification results by comprehensively analyzing the degree of agreement between the two identification results and their respective trust coefficients. For example, if the two results are highly consistent in defect location, type, and severity, and the trust coefficients are both high, the trust authentication result is marked as reliable; if there is a significant difference or the trust coefficient is low, the authentication level is lowered. The module calculates the final trust authentication result through a weighted model or probabilistic inference, providing high-quality input for defect location.

[0075] Furthermore, based on the trust authentication results, the module combines the coordinates of the focus point to complete the defect location. This process matches the trust authentication results with the specific location in the three-dimensional coordinate system to generate the precise spatial coordinates of the defect. At the same time, the module classifies the defects by type and severity level, such as corrosion, cracks or blockage, and adds confidence scores to support subsequent decision-making. Finally, the defect location results output by the module include the three-dimensional coordinates, type information, severity and authentication confidence of the target, providing intuitive and actionable data for subsequent maintenance or repair tasks.

[0076] Through the defect location module, the pipeline robot completes the last step from data collection to result analysis, achieves high-precision marking and location of defects, and provides comprehensive data support for pipeline inspection and repair work.

[0077] Furthermore, the embodiment of the present application also includes:

[0078] The transmission module is used to synchronize the located defects to the inspection system; the report generation module is used to receive the located defect data, update the pipeline fitting modeling results through the defect data, and generate an inspection report.

[0079] Optionally, after the defect location module completes the accurate marking of the target defect, the transmission module and the report generation module can be responsible for linking the location results with the inspection system to ensure the synchronization of defect information and the summary of the inspection process. The collaborative work of these two modules provides closed-loop support for data management and feedback of inspection tasks.

[0080] Specifically, after positioning is completed, the module synchronizes the complete data of the defect, such as spatial coordinates, type information, severity level and certification confidence, to the inspection system in real time through the transmission module. The transmission module uses low-latency and high-reliability wireless communication technology (such as 5G or LoRa) to ensure that data can be quickly and stably transmitted to the central control platform in a complex pipeline environment.

[0081] During the synchronization process, the module performs necessary compression and encryption on the defect data to improve transmission efficiency and ensure data security. For example, the lightweight JSON format is used for data encapsulation, and the AES encryption algorithm is applied to prevent sensitive information leakage. The real-time transmission of defect data enables the inspection system to receive and process the test results in a timely manner, providing a basis for subsequent decision-making.

[0082] After the transmission is completed, the report generation module receives and parses the located defect data, compares and updates it with the initial pipeline fitting modeling results. The pipeline fitting modeling result update refers to the real-time correction of the geometric model and feature distribution of the pipeline based on the newly added defect information. For example, the crack location or blockage area is marked in the model to reflect the latest status of the pipeline. Subsequently, the module automatically generates a detailed inspection report based on the updated model and defect data. The inspection report may include the following: defect distribution map, which intuitively displays the location and type of defects based on the three-dimensional model. Data statistics, including the number, classification and severity distribution of defects. Maintenance recommendations, providing targeted repair or optimization solutions based on the defect type and level. During the report generation process, data visualization tools (such as Matplotlib or Plotly) can be used to generate charts, and natural language generation technology (NLG) can be combined to automatically write report content to ensure the accuracy and readability of the information.

[0083] Through the collaborative work of the transmission module and the report generation module, the inspection system can achieve a closed-loop operation from defect data synchronization to task summary. This design not only improves the efficiency and transparency of the inspection work, but also provides decision support for subsequent pipeline maintenance and optimization.

[0084] In summary, the embodiments of the present application have at least the following technical effects:

[0085] This application determines the coordinates of interest in the pipeline through initial data collection and three-dimensional modeling; controls the robot to collect images in real time, identifies the area of ​​interest and generates a first recognition result; optimizes the robot position based on the recognition result, cleans the point of interest and performs additional collection to generate high-precision additional images; and after combining multiple recognition result verifications, completes accurate defect positioning and achieves efficient inspection and defect detection in complex pipeline environments.

[0086] The technical effect of high-precision detection of defects and full-coverage inspection in complex environments has been achieved through multiple identification verifications, trust authentication and dynamic path optimization.

[0087] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0088] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0089] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. The machine vision-driven pipeline robot intelligent operation platform is characterized by: The platform includes: The focus establishment module is used to perform pipeline fitting modeling in a preset three-dimensional coordinate system and configure pipeline focus coordinates after initial data collection of the pipeline; An image acquisition module is used to control the pipeline robot to enter the pipeline, generate the pipeline robot coordinates according to the real-time response position of the pipeline robot and the preset three-dimensional coordinate system, and activate the image acquisition device of the pipeline robot to perform image acquisition in the pipeline and establish a real-time frame image; A focus recognition module, used for judging whether the image position of the real-time frame image is within the pipeline focus coordinate range, and if the image position of the real-time frame image is within the pipeline focus coordinate range, performing image focus recognition on the real-time frame image to establish a first focus recognition result; A position updating module, used for extracting the coordinates of the focus position point of the first focus recognition result, controlling the pipeline robot to perform position updating optimization according to the coordinates of the focus position point, and establishing an optimization result; An additional acquisition module is used to control the pipeline robot to move to the position point of the optimization result, and establish a cleaning response strategy according to the first focus recognition result, and after cleaning the focus position point according to the cleaning response strategy, control the image acquisition device to perform additional image acquisition to establish an additional frame image; A defect location module, used for performing enhanced recognition on the additional frame image, establishing a second focus recognition result, performing recognition verification based on the first focus recognition result and the second focus recognition result, and generating a defect location through the coordinates of a focus position point; The attention identification module is used for: After image preprocessing is performed on the real-time frame image, a dual image channel is constructed, wherein the dual image channel includes an original image channel and a grayscale channel; After edge detection is performed on the dual image channels respectively, edge sharing of the dual image channels is performed, and a region of interest is configured according to the edge sharing result; The interest region is used to perform multi-scale sliding window attention recognition of dual image channels, a multi-scale fusion recognition result is established, and the multi-scale fusion recognition result is output as a first attention recognition result.

2. The machine vision-driven pipeline robot intelligent operation platform according to claim 1, characterized in that: When the attention recognition module performs multi-scale sliding window attention recognition of dual image channels using the interest region to establish a multi-scale fusion recognition result, it also includes: Based on the region of interest, feature extraction of dual image channels is performed respectively to establish a fusion feature set, wherein the fusion feature set includes texture features, color features, and shape features; configuring a multi-scale window according to the fusion feature set, and establishing a mapping window sliding step according to the multi-scale window and feature density; Using the multi-scale window and the mapping window sliding step to perform traversal identification of the interest area, and establish traversal confidence; A joint authentication analysis is performed based on the traversal trust level to generate a multi-scale fusion recognition result.

3. The machine vision driven pipeline robot intelligent operation platform according to claim 1, characterized in that: The location update module is used for: Configure the stay fitness of the location points based on the pipeline fitting modeling results; Establishing an attention angle evaluation according to the first attention recognition result, and generating an attention angle evaluation result; Adaptively screen the observation points using the focus angle evaluation results and the coordinates of the focus position points, and establish a set of screened position points; Taking the real-time response position as a starting point and the screening position point set as an observation end point, performing path optimization and establishing path fitness; Performing fitness analysis based on the set of screening position points to generate observed fitness; The stay fitness, the path fitness, and the observation fitness are used to perform position update optimization and establish an optimization result.

4. The machine vision driven pipeline robot intelligent operation platform according to claim 3, characterized in that: The location update module is also used for: Configure bend features and branch features according to pipeline fitting modeling results; Perform additional impact analysis of the angle of interest based on the curve characteristics and the branch characteristics, and establish additional impact evaluation results; Adaptive screening of observation location points is performed according to the additional impact evaluation result and the focus angle evaluation result, and a screening location point set is established.

5. The machine vision driven pipeline robot intelligent operation platform according to claim 1, characterized in that: The additional acquisition module is used for: According to the coordinates of the focus point and the position point of the optimization result, the acquisition parameters are configured, and the acquisition parameters include focusing distance, auxiliary light parameters, and aperture parameters; After adjusting the parameters of the image acquisition device according to the acquisition parameters, additional image acquisition is performed to establish the additional frame image.

6. The machine vision driven pipeline robot intelligent operation platform according to claim 1, characterized in that: The defect location module is also used for: Performing trustworthiness identification on the first attention recognition result and the second attention recognition result respectively to establish a trust coefficient; Using the trust coefficient to perform trust authentication on the first attention recognition result and the second attention recognition result, and establish a trust authentication result; The trust authentication result and the coordinates of the focus point are used to complete the defect location.

7. The machine vision driven pipeline robot intelligent operation platform according to claim 1, characterized in that: The platform also includes: Transmission module, used to synchronize the located defects to the inspection system; The report generation module is used to receive the located defect data, update the pipeline fitting modeling results through the defect data, and generate an inspection report.

Images