A method and apparatus for pipeline inspection defect identification based on visual images
By performing real-time visual acquisition, geometric posture compensation, and image optimization on the mobile inspection device, combined with texture feature analysis and defect database matching, the problem of interference from posture changes and brightness differences in pipeline inspection was solved, achieving efficient and accurate pipeline defect identification and assessment.
Patent Information
- Application Number
- CN202611620175.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-20
- Publication Date
- 2026-11-17
AI Technical Summary
Existing visual image-based pipeline inspection methods are prone to posture changes, speed fluctuations, and motion jitter during movement inside the pipeline, resulting in image acquisition position shifts and motion blur. Furthermore, differences in brightness and texture inside the pipeline interfere with the extraction of visual features of defects, making it difficult to quickly identify pipeline defects.
The mobile inspection device performs real-time visual acquisition, geometric posture compensation, and reverse deblurring to output the first inspection image; the image is then enhanced for texture details and contrast to output the second inspection image; frame-by-frame visual feature recognition and texture feature analysis are performed to construct a pipe wall texture feature set; similarity matching and confidence comparison are performed based on the defect database to mark the defect points to be tested; secondary image acquisition and defect region separation are performed on the defect points to be tested to generate defect evaluation results.
It improves the image clarity and defect identifiability of pipeline inspection, reduces the impact of light reflection and texture noise, enhances the accuracy of defect identification and positioning precision, and realizes the quantifiable assessment and traceability of defects.
Smart Images

Figure CN123118643A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and in particular to a method and apparatus for identifying pipeline inspection defects based on visual images. Background Technology
[0002] With the widespread application of pipeline systems in fields such as oil transportation, chemical production, energy transportation, and industrial manufacturing, pipelines are susceptible to damage during long-term operation due to factors such as fluid erosion, corrosive media, mechanical vibration, and temperature changes. This can lead to defects such as cracks, corrosion, deformation, leakage, and breakage on the inner wall of the pipeline. Because some pipelines have long transport distances and relatively narrow internal spaces, manual inspection of the pipeline interior is inefficient. Therefore, mobile inspection devices equipped with visual acquisition components are increasingly being used to continuously acquire images of the pipeline interior and detect defects through image processing and visual recognition.
[0003] Existing vision-based pipeline inspection methods typically utilize mobile inspection devices to acquire images of the pipeline's inner wall, then identify pipeline defects through image enhancement, texture analysis, edge detection, and defect classification. However, the mobile inspection device is prone to posture changes, speed fluctuations, and motion jitter during its movement within the pipeline, causing positional shifts and motion blur in the acquired images. Simultaneously, the pipeline interior relies on spotlights for visual illumination, which can lead to localized reflections on the metal pipe walls. Furthermore, differences in brightness and texture exist between different pipe sections, interfering with the extraction of visual defect features. When the inspection device moves continuously and acquires a large number of images, frame-by-frame analysis is required. Existing methods struggle to quickly filter out target areas with actual defect features from continuous images, resulting in low pipeline defect identification efficiency. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention proposes a method and apparatus for identifying pipeline inspection defects based on visual images, thereby resolving at least one of the aforementioned technical problems.
[0005] To achieve the above objectives, this invention provides a pipeline inspection defect identification method based on visual images, comprising the following steps: Drive the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then perform geometric posture compensation and reverse deblurring processing to output the first inspection image. The first inspection image is enhanced in texture detail and dynamically optimized in contrast, and the second inspection image is output. The second inspection image is subjected to frame-by-frame image visual feature recognition and texture feature analysis to construct a pipe wall texture feature set for different frames; Based on the pipeline internal defect database, the pipe wall texture feature set is subjected to defect visual feature similarity matching, and confidence comparison analysis is performed to mark the defect points to be tested. Secondary image acquisition is performed on the defect point to be tested to divide the defect area image and background image; The defect area image is classified and its morphology is calculated to generate a defect assessment result. The defect assessment result is then encapsulated and a pipeline inspection defect report is output.
[0006] This specification provides a pipeline inspection defect identification device based on visual images, used to perform the pipeline inspection defect identification method based on visual images as described above, including: The first processing module drives the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then performs geometric posture compensation and reverse deblurring processing, and outputs the first inspection image. The second processing module is used to enhance the texture details and dynamically optimize the contrast of the first inspection image, and output the second inspection image. The visual analysis module is used to perform frame-by-frame image visual feature recognition and texture feature analysis on the second inspection image, and to construct a pipe wall texture feature set for different frames. The confidence calculation module is used to perform visual feature similarity matching of defects in the pipe wall texture feature set based on the internal defect database of the pipe, and to perform confidence comparison analysis to mark the defect points to be tested. The secondary acquisition module is used to acquire secondary images of the defect points to be tested, and to divide the defect area image and the background image. The defect assessment module is used to classify defects and calculate defect morphology in the defect area image, generate defect assessment results, encapsulate the defect assessment results into data, and output a pipeline inspection defect report.
[0007] The beneficial effects of this invention are specifically as follows: By fusing the movement posture, speed, and image displacement information of the inspection device, geometric posture compensation and reverse deblurring are performed on the real-time acquired images, reducing the impact of robot motion jitter, posture deviation, and motion blur on the pipe wall imaging quality, and improving the structural clarity and defect identifiability of the original inspection images. By performing texture detail enhancement and local contrast dynamic optimization on the first inspection image, the brightness difference in different areas of the pipe wall is improved, enhancing the visual difference between abnormal areas such as cracks, corrosion, and damage and normal pipe walls, and reducing the missed detection of defects caused by strong light, dark areas, and uneven pipe wall texture. By performing frame-by-frame visual feature recognition on continuous inspection images, and combining brightness, color, texture, and edge features for cross-frame analysis, a pipe wall texture feature set with temporal continuity and spatial correspondence is formed, which can enhance the continuous feature expression of real defects and reduce the impact of single-frame image noise on the recognition results.
[0008] By performing similarity matching between multi-frame pipe wall texture features and a defect database, and combining this with continuous cross-frame comparisons based on defect confidence levels, abnormal features at the same pipe wall location are continuously confirmed. This suppresses false defects caused by light reflection, particle texture, and transient noise, improving the accuracy of defect point identification. Secondary image acquisition is performed by scheduling the inspection device based on the spatial location of the defect point, and region segmentation is implemented using the defect edge contour. This effectively separates the defect area from the normal background, improving the positioning accuracy of defect boundaries and providing clearer, more independent visual data for defect size measurement and type identification. By classifying and identifying defect areas and calculating changes in length, width, area, and shape, visual defect features are transformed into quantifiable defect assessment results. These results are then encapsulated with spatial coordinates and defect images, achieving integrated recording of pipeline defect type, location, shape, and severity. This enhances the traceability of inspection results and their engineering application value. Attached Figure Description
[0009] Figure 1 This is a schematic diagram of the steps of a pipeline inspection defect identification method based on visual images according to the present invention. Figure 2 A detailed flowchart illustrating the implementation steps for driving a mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, followed by geometric attitude compensation and reverse deblurring to output the first inspection image. Figure 3 To enhance texture details and dynamically optimize contrast in the first inspection image, a detailed flowchart of the implementation steps for outputting the second inspection image is provided. Figure 4 A schematic diagram of real-time inspection images within the pipeline; Figure 5 This is a schematic diagram of the first inspection image; Figure 6 This is a schematic diagram of the second inspection image. Detailed Implementation
[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0011] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. It should also be understood that, as used in this specification and the appended claims, the term "and / or" refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0012] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0013] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0014] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0015] To illustrate the technical solution of the present invention, specific embodiments are described below.
[0016] In the embodiments of the present invention, see Figure 1 The diagram below illustrates the steps of a pipeline inspection defect identification method based on visual images according to the present invention. In this example, the steps of the pipeline inspection defect identification method based on visual images include: Drive the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then perform geometric posture compensation and reverse deblurring processing to output the first inspection image. The first inspection image is enhanced in texture detail and dynamically optimized in contrast, and the second inspection image is output. The second inspection image is subjected to frame-by-frame image visual feature recognition and texture feature analysis to construct a pipe wall texture feature set for different frames; Based on the pipeline internal defect database, the pipe wall texture feature set is subjected to defect visual feature similarity matching, and confidence comparison analysis is performed to mark the defect points to be tested. Secondary image acquisition is performed on the defect point to be tested to divide the defect area image and background image; The defect area image is classified and its morphology is calculated to generate a defect assessment result. The defect assessment result is then encapsulated and a pipeline inspection defect report is output.
[0017] In this embodiment, a metal conveying pipeline with an inner diameter of 600 mm and a length of 180 m is used as the inspection target. The mobile inspection device is equipped with a 1920×1080 pixel industrial camera, an inertial measurement unit, and an odometer. The searchlight illuminance is set to 800 lx, the industrial camera sampling frequency is set to 10 frames / s, the inspection device's axial movement speed along the pipeline is set to 0.15 m / s, the inertial measurement unit sampling frequency is 100 Hz, and the odometer sampling frequency is 50 Hz. After the inspection begins, 20 inspection images are continuously acquired within a pipe section approximately 12.0 m to 12.3 m from the pipeline inlet. Since the inspection speed is 0.15 m / s, the theoretical axial displacement between two adjacent frames is 0.15 ÷ 10 = 0.015 m. Therefore, the theoretical inspection distance corresponding to 20 frames is approximately 0.30 m. Frames 8 to 14 were selected as the key analysis frames. During frame 8, the odometer recorded an inspection mileage of 12.105 m, in frame 9 it was 12.120 m, and in frame 10 it was 12.135 m. The inertial measurement unit recorded a yaw angle of 0.8°, a pitch angle of 1.2°, and a roll angle of 2.1° at the corresponding times. The roll angle varied by 1.5° within adjacent sampling periods, exceeding the preset jitter judgment condition. Based on the image structure reference point, the average pixel displacement of adjacent frames was calculated to be 28 pixels. A correspondence between image displacement and pipeline spatial displacement was established using the actual axial displacement of 0.015 m. The image rotation compensation angle was calculated based on the roll angle change, and the translation compensation amount was calculated based on the reference point displacement. A geometric inverse transformation was performed on images with attitude offset. The degree of motion blur was determined based on the continuous frame displacement change rate, and the blurred edges in the motion direction were deblurred in reverse to obtain the first inspection image after unified compensation for attitude and motion state.
[0018] Taking the 10th frame as an example, the average brightness of the effective pipe wall area was calculated to be 142, and the standard deviation of brightness was 31. The local average brightness of the area directly illuminated by the searchlight reached 226, while the average brightness of the normal pipe wall area was approximately 138. Therefore, directly enhancing the overall contrast could easily cause overexposure in the bright areas. The image was divided into local brightness analysis regions. For areas with an average brightness below 120, local contrast was increased; for areas between 120 and 200, the original brightness ratio was maintained; and for bright areas exceeding 220, the enhancement was limited. For the pipe wall texture area, local binary texture enhancement was used to amplify the grayscale gradient between the edges of small cracks and the normal metal texture. After dynamic optimization, the local brightness of the suspected defect area in the 10th frame was adjusted from 128 to 136, and the normal pipe wall background was adjusted from 142 to 145. Simultaneously, the average gradient of the defect edge increased from 18 to 31, while the bright areas of the searchlight remained below 245, avoiding the generation of new overexposed textures. By processing 20 original images in the same way, a second consecutive inspection image sequence is obtained, providing a unified image basis for subsequent frame-by-frame visual feature calculations.
[0019] The effective pipe wall region is extracted in each frame of the image, and brightness, color, texture, and edge features are calculated respectively. For the 12.105 m to 12.195 m pipe segment corresponding to frames 8 to 14, spatial correspondence is performed on the images with an inter-frame displacement of 0.015 m, and features at the same pipe wall location are associated. Taking the suspected defect area as an example, the detected texture change degree is 0.71, edge continuity is 0.78, and color anomaly degree is 0.62 in frame 8; 0.76, 0.84, and 0.65 in frame 9; 0.81, 0.88, and 0.68 in frame 10; and 0.83, 0.91, and 0.70 in frame 11. The texture change degree of normal pipe wall regions is usually below 0.35, and the color anomaly degree is below 0.30. Therefore, the above continuously increasing features are identified as abnormal texture changes. The fine texture structure of the crack region is described by local binary mode, and the texture contrast and directionality are calculated by gray-level co-occurrence relationship. Then, continuous edges are obtained by gradient edge detection, thus forming a pipe wall texture feature set that includes the degree of texture change, edge continuity, color anomaly degree and corresponding pipe mileage of each frame.
[0020] The defect database pre-stores standard visual features for five types of defects: cracks, corrosion, deformation, leakage, and breakage. Taking the suspected abnormal area at position 12.135 m in frame 10 as an example, its texture structure, edge direction, color variation, and morphological features are matched with the templates for the five defect types, respectively. The similarity scores are: crack 0.88, corrosion 0.42, deformation 0.31, leakage 0.36, and breakage 0.55. Therefore, crack is selected as the current suspected defect type. Further, the defect confidence is calculated based on visual feature similarity, texture variation, edge continuity, and color anomaly. For example, by combining the four features with weights of 0.40, 0.30, 0.20, and 0.10, the defect confidence for frame 10 can be calculated as 0.88×0.40+0.81×0.30+0.88×0.20+0.68×0.10=0.83. Using the same method, the confidence levels for frames 8, 9, and 10 were calculated to be 0.76, 0.81, and 0.83, respectively, all of which are not lower than the confidence threshold of 0.75. Furthermore, the pipe position in the three frames changed continuously by 0.015 m. Therefore, it was determined that the suspected area in the three frames belonged to the same defect, and the position was marked as the defect point to be tested.
[0021] The inspection mileages corresponding to frames 8, 9, and 10 are 12.105 m, 12.120 m, and 12.135 m, respectively. The average of these three frames is 12.120 m. Considering the fixed axial offset between the industrial camera's installation position and the inspection device's reference point, a correction of 0.030 m is used. Therefore, the defect's axial position is calculated to be 12.120 + 0.030 = 12.150 m. Thus, the distance between the defect and the pipe inlet is determined to be approximately 12.15 m. Then, based on the lateral pixel position of the defect area center relative to the pipe wall's unfolded center in the 1920×1080 pixel image, a circumferential conversion is performed. Combined with the pipe's inner diameter of 600 mm, the corresponding pipe inner wall circumference is 600π ≈ 1885 mm. If the ratio of the defect center to the pipe wall's unfolded direction is approximately 0.32, the circumferential position is approximately 1885 × 0.32 ≈ 603 mm. After attitude correction by combining yaw, pitch, and roll angles, the internal spatial coordinates of the defect point to be measured in the pipeline are formed. The inspection device returns to the target area based on these coordinates, reduces its moving speed within 0.30 m of the target position, and makes fine adjustments within 0.15 m before and after the target position, maintaining the 800 lx illumination condition of the searchlight, and re-acquires no less than 10 frames of retest images. The frame with the clearest defect outline is selected as the high-definition retest image.
[0022] For the 1920×1080 pixel retest image, a local analysis region of approximately 300×300 pixels was established with the center of the defect point as the reference. Gradient edge detection was used to extract the defect edges, and isolated edges with a length of less than 10 pixels were filtered out. The test revealed a continuous crack profile at a position of 12.15 m, with a profile length of approximately 86 pixels. Approximately 30 pixels of normal pipe wall were retained on each side of the profile as background references. The area inside the profile was divided into the defect region image, and the adjacent area outside the profile was divided into the background image. Using the spatial scale calibration relationship of the pipe wall, the pixel size was converted to the actual size, and the actual crack length was calculated to be approximately 43 mm, the maximum width approximately 0.8 mm, and the defect region area approximately 18 mm². 2 The profile change along the crack length direction was calculated, yielding a maximum directional change of approximately 12°. This was combined with the crack type, 43 mm length, 0.8 mm maximum width, and 18 mm... 2 Based on the changes in area and shape, the severity of the defect was assessed, classifying it as a level two defect. The final determination included the defect number, crack type, level two defect status, defect length of 43 mm, maximum width of 0.8 mm, and defect area of 18 mm². 2 Images of the axial position (approximately 12.15m), the circumferential position (approximately 603 mm), the defect area, and the background image are uniformly encapsulated and associated with the original inspection image, high-definition retest image, and acquisition time to form a complete pipeline inspection defect report.
[0023] In one embodiment, see Figure 2 , Figure 2 A detailed flowchart illustrating the implementation steps for driving a mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, followed by geometric attitude compensation and reverse deblurring to output the first inspection image. Figure 4 A schematic diagram of real-time inspection images within the pipeline; Figure 5 The diagram shows the first inspection image. The specific steps for driving the mobile inspection device to enter the target pipeline to perform real-time visual acquisition of the pipeline interior, then performing geometric posture compensation and reverse deblurring processing, and outputting the first inspection image are as follows: Drive the mobile inspection device into the target pipeline, turn on the front-end searchlight, and use the onboard industrial camera to perform real-time visual acquisition of the pipeline interior at a preset sampling frequency to obtain real-time inspection images of the pipeline interior. The mobile inspection device acquires its own movement parameters based on the mobile self-test unit to obtain a mobile inspection parameter set; the mobile self-test unit includes an inertial measurement component, a mileage detection component and a motion control component, and the mobile inspection parameter set includes yaw angle, pitch angle, roll angle, movement speed, movement direction and movement distance.
[0024] On the local side of the mobile inspection device, geometric posture compensation and reverse deblurring are performed on the real-time inspection image inside the pipe based on the mobile inspection parameter set, and the first inspection image is output.
[0025] In this embodiment, a mobile inspection device is used to visually inspect the interior of a metal conveying pipeline with an inner diameter of 600 mm and a length of 180 m. The mobile inspection device is equipped with an industrial camera and a ring-shaped searchlight at its front end. The industrial camera has a resolution of 1920×1080 pixels and a preset sampling frequency of 10 frames / s. The initial illuminance of the searchlight is set to 800 lx. The mobile inspection device is driven from the pipeline inlet into the pipeline and moves along the pipeline axis at an initial speed of 0.15 m / s, allowing the industrial camera to continuously acquire real-time visual images of the pipeline's inner wall. Each acquired real-time inspection image is simultaneously timestamped, establishing a time correspondence between the image frame and the actual movement position of the inspection device. For example, when the device moves to a position 12.0 m from the pipeline inlet, the industrial camera acquires an image of the current pipe wall and marks this image as the real-time inspection image corresponding to the 12.0 m pipeline position, providing a basis for subsequent defect location calculation.
[0026] The mobile self-testing unit continuously collects the motion status of the inspection device. The inertial measurement unit acquires yaw, pitch, and roll angles at a sampling frequency of 100 Hz, the mileage detection unit acquires the moving distance and direction at a sampling frequency of 50 Hz, and the motion control unit synchronously outputs the moving speed. The acquisition times of the three components are uniformly calibrated, and the motion parameters are time-aligned according to the image timestamps of the industrial camera, forming a set of motion inspection parameters corresponding to each frame of real-time inspection image. For example, when the device is located 12.0 m from the pipe inlet, the detected yaw angle is 0.8°, pitch angle is 1.2°, roll angle is 2.1°, moving speed is 0.15 m / s, and the moving direction is towards the pipe. Continuous parameter changes are used to further determine whether the device has continuous deflection or momentary jitter. When the roll angle change reaches 1.5° or more or the moving speed change exceeds 0.03 m / s within adjacent sampling periods, the corresponding image is marked as a motion disturbance image, providing a basis for subsequent compensation processing.
[0027] On the local side of the mobile inspection device, attitude compensation parameters and motion blur compensation parameters for the real-time inspection image are established based on the mobile inspection parameter set. For the imaging plane offset caused by yaw, pitch, and roll angles, a reverse geometric transformation is performed on the real-time inspection image according to the corresponding angles to restore the inner wall edge of the pipe to a unified reference direction. For motion blur caused by moving speed and displacement of adjacent image frames, an axial movement distance of approximately 0.015 m between adjacent image frames is determined based on a moving speed of 0.15 m / s and a sampling frequency of 10 frames / s. A reverse blur kernel is established using the movement direction to perform reverse deblurring processing on the image affected by motion trailing. Taking an image acquired at 12.0 m as an example, after attitude compensation, the roll direction offset is corrected, and the trailing image generated along the pipe axis is restored, resulting in a first inspection image with improved edge continuity and texture clarity. Simultaneously, the corresponding 12.0 m spatial position and mobile inspection parameters are retained.
[0028] In one embodiment, the specific steps for performing geometric pose compensation and reverse deblurring on the real-time inspection image within the pipe based on the mobile inspection parameter set on the local side of the mobile inspection device, and outputting the first inspection image are as follows: The mobile inspection device extracts real-time inspection images of the pipe on the local side. Multiple reference points are obtained by marking the pipe inner wall structure reference points on the real-time inspection images inside the pipe. By matching adjacent image frames for multiple reference points and calculating the positional changes of subsequent image frames, the displacement, displacement direction, and displacement rate between image frames are obtained. The attitude change of the inspection device is analyzed based on the mobile inspection parameter set to obtain the attitude change amount. Motion state correlation analysis is performed based on attitude change and image frame displacement, displacement direction and displacement change rate to determine the degree of motion jitter of the mobile inspection device. Based on the degree of motion jitter, the jitter state is classified and identified to obtain the motion jitter type, which includes translational jitter, vertical posture jitter, and compound jitter. Based on the motion jitter type, jitter posture compensation calculation is performed to obtain a motion compensation parameter set, which includes image translation compensation parameters and image rotation compensation parameters. Motion blur calculation is performed based on the inter-frame displacement and displacement change rate of the image, and blur compensation operation is performed to output motion blur compensation parameters. Geometric posture compensation is performed on the real-time inspection image inside the pipe based on the motion compensation parameter set, and reverse deblurring is performed according to the motion blur compensation parameters to output the first inspection image.
[0029] In this embodiment, after the mobile inspection device enters the target pipeline, the industrial camera continuously acquires real-time inspection images of the pipeline at a sampling frequency of 10 frames / s, each with a resolution of 1920×1080 pixels. Since the relative displacement of the image itself needs to be determined based on the fixed structure of the pipeline wall, reference points of the pipeline wall structure are extracted in each frame. Specifically, the intersection of pipe wall welds, pipe wall edges, intersections of fixed textures, and obvious structural turning points can be selected as reference features, and the coordinates of the reference points are obtained through corner detection and local texture feature extraction. Taking continuous images of the device located 12.0 m from the pipeline inlet as an example, reference points of the pipeline wall are extracted in the current frame and the next frame, and matched using local feature descriptions. Incorrect matching points with excessively large matching distances and insufficient texture similarity are eliminated, retaining reference point pairs with stable correspondences. Based on the effective reference point matching results in adjacent image frames, the inter-frame displacement of each reference point is calculated according to the difference in pixel coordinates between the corresponding reference points in the preceding and following images, and the corresponding displacement direction is determined based on the direction of change of the reference point coordinates. Displacement statistics are performed on multiple reference points within the same frame, and the median displacement is used as the overall displacement of the current image frame relative to the previous image frame to reduce the impact of local abnormal textures on the calculation results. Since the industrial camera sampling frequency is 10 frames / s, the theoretical axial movement distance between adjacent images at a moving speed of 0.15 m / s is approximately 0.015 m. Therefore, the displacement generated by normal continuous movement is used as the benchmark. The rate of change of displacement between adjacent frames over time is further calculated. When the displacement of consecutive frames suddenly increases or the direction reverses, it is recorded as a change in motion state characteristics, forming image motion parameters that include inter-frame displacement, displacement direction, and rate of change of displacement.
[0030] During the movement of the mobile inspection device, the inertial measurement unit collects yaw, pitch, and roll angles at 100 Hz, while the odometer collects movement speed, direction, and distance at 50 Hz. The motion data is time-aligned according to the image acquisition timestamps, ensuring each frame corresponds to the device's attitude state. Taking a set of continuous images from 12.0 m from the pipe inlet as an example, if a yaw angle of 0.8°, a pitch angle of 1.2°, and a roll angle of 2.1° are detected, and the roll angle changes significantly in the next frame, the displacement direction of the reference point in the adjacent frame is used for correlation judgment. When the roll angle change reaches 1.5° or more within adjacent sampling periods, or the movement speed change exceeds 0.03 m / s, the corresponding attitude change is correlated with the image displacement change. By comparing the consistency between the attitude change direction and the reference point displacement direction, the image offset caused by the device's own attitude change and the image displacement caused by normal axial movement are distinguished, thereby determining the current degree of motion jitter.
[0031] Based on the determined degree of motion jitter, the current motion state of the inspection device is classified. When multiple pipe wall reference points mainly produce consistent displacement along the horizontal direction of the image, and the corresponding yaw angle changes are significant, it is identified as translational jitter. When the pipe wall structure exhibits overall rotation and vertical deflection in the image, and maintains a corresponding relationship with changes in pitch or roll angles, it is identified as vertical attitude jitter. When the image simultaneously exhibits overall displacement and rotational changes, and both types of changes exceed the corresponding change thresholds, it is identified as composite jitter. Taking the case where the roll angle changes from 2.1° to 3.8°, and the reference points simultaneously produce horizontal and vertical displacements, the current state is determined to be composite jitter. The image translation compensation is calculated based on the average displacement of the reference points, and the image rotation compensation is calculated based on the changes in yaw, pitch, and roll angles, forming a motion compensation parameter set including horizontal translation, vertical translation, and image rotation angles.
[0032] After obtaining the inter-frame displacement and displacement change rate, the degree of image blurring generated during the movement of the inspection device is further determined. Since the industrial camera's sampling frequency is 10 frames / s, the theoretical axial movement distance between adjacent frames is approximately 0.015 m at a moving speed of 0.15 m / s. When the moving speed changes or short-term jitter occurs, the image displacement corresponding to a single frame exposure will change, resulting in blurring along the pipe wall edge in the direction of movement. Motion blurring parameters are established based on the displacement, displacement direction, and displacement change rate of adjacent frames. The displacement direction is used as the blur kernel direction, and the blur kernel length is determined by the displacement change amplitude. For image frames with a large displacement change rate, the inverse deblurring compensation intensity is increased; for image frames with stable motion, a lower compensation intensity is used to avoid over-sharpening the normal texture of the pipe wall, ultimately generating motion blur compensation parameters corresponding to the current image.
[0033] Based on the aforementioned motion compensation parameter set, geometric attitude correction was performed on the real-time inspection image inside the pipe acquired at a distance of 12.0 m from the pipe inlet. First, the overall position of the pipe wall was translated in the reverse direction according to the image translation compensation parameters, and then the image was rotated in the reverse direction according to the image rotation compensation parameters, so that the reference points of the pipe wall structure were restored to a unified spatial reference position. Subsequently, inverse deblurring was performed along the actual motion direction according to the motion blur compensation parameters to restore the clarity of the pipe wall welds, edges, and fine textures. After processing, the reference point positions before and after compensation were matched again. If the residual position deviation of the reference point after compensation was within the preset allowable range, the processing result was retained; otherwise, the corresponding compensation parameters were readjusted. Finally, a first inspection image with unified motion attitude and suppressed motion trailing was obtained, and it was associated and saved with the 12.0 m pipe position and corresponding motion inspection parameters.
[0034] In one embodiment, see Figure 3 , Figure 3To enhance texture details and dynamically optimize contrast in the first inspection image, a detailed flowchart of the implementation steps for outputting the second inspection image is provided. Figure 6 This is a schematic diagram of the second inspection image; the specific steps for enhancing texture details and dynamically optimizing contrast of the first inspection image to output the second inspection image are as follows: The first inspection image is uploaded to the cloud image processing platform for global brightness distribution analysis to generate global brightness distribution features. Based on the global brightness distribution characteristics, the first inspection image is optimized by adaptive histogram equalization to obtain a globally optimized brightness image. The global brightness optimization image is divided into regions, and local brightness is calculated to obtain the local brightness of the image at multiple locations; Based on the local brightness of the image, regional brightness difference analysis is performed to obtain local brightness abnormal regions, which include local overexposure, underexposure and uneven lighting regions. Local brightness optimization is performed on the aforementioned areas of abnormal brightness to obtain a brightness-optimized standard image; The brightness-optimized standard image is enhanced with texture detail and dynamically optimized for contrast, and a second inspection image is output.
[0035] In this embodiment, the first inspection image acquired by the mobile inspection device at a location 12.0 m from the pipeline inlet is selected as an example. The image resolution is 1920×1080 pixels, corresponding to a front-end searchlight illuminance of 800 lx. Grayscale statistics are performed on the 1920×1080 pixels of the first inspection image to calculate the average brightness, standard deviation of brightness, and pixel proportion in different brightness ranges, forming a global brightness distribution characteristic. The statistics show an average image brightness of approximately 142 and a standard deviation of approximately 31. The average brightness of the area directly illuminated by the searchlight is approximately 226, and the average brightness of the normal pipe wall area is approximately 138, indicating that the overall brightness of the pipe wall is in a moderate range, but the searchlight reflection creates a clearly bright area. Pixels with brightness below 120 are defined as underexposed candidate pixels, 120–200 is defined as the normal brightness range, and pixels above 220 are defined as bright candidate pixels. The global brightness imbalance of the current image is determined based on the number of pixels in each range, providing input for global brightness optimization.
[0036] The 1920×1080 pixel image was divided into 8×8 local histogram processing units. Within each unit, brightness frequency was counted, and brightness ranges with excessively high frequencies were cropped. The cropped pixels were then redistributed to other brightness levels to limit the occupancy of the searchlight's high-brightness areas on the overall grayscale dynamic range. A local histogram cropping limit factor of 2.0 was set, and a bilinear transition was implemented based on the brightness mapping relationship between adjacent processing units to avoid significant brightness abrupt changes at region boundaries. After equalization processing, the average brightness of the normal pipe wall area was adjusted from approximately 138 to approximately 142. The original brightness of approximately 226 in the area directly illuminated by the searchlight was compressed into a higher but unsaturated brightness range, resulting in more continuous grayscale levels in areas containing pipe wall scratches, corrosion spots, and potential cracks, thus obtaining a globally optimized brightness image.
[0037] The global brightness optimization image was divided into regions, and a local brightness analysis grid was established using image pixel coordinates. The entire 1920×1080 pixel image was divided into multiple fixed-size analysis regions, and the average brightness, brightness standard deviation, and proportion of bright pixels within each region were calculated. Using 120×120 pixels as local brightness analysis units, the average brightness of each region was calculated, and the average brightness difference between adjacent regions was used as a local brightness difference index. When the average brightness of a region was below 120, it was marked as an underexposed candidate region; when the average brightness was above 220 and the proportion of bright pixels reached a set ratio, it was marked as a locally overexposed candidate region; when the average brightness difference between adjacent regions reached 40 or more, it was determined as a candidate region for uneven illumination based on regional spatial continuity. Analysis revealed locally bright areas at the spotlight reflection location and brightness attenuation areas at the pipe wall edges, thus obtaining a set of local brightness anomaly regions containing different anomaly types and their pixel coordinate ranges.
[0038] Based on the local brightness of the image, the identified local brightness anomaly areas are classified and optimized. For overexposed areas, brightness compression is used to reduce the grayscale gain of high-brightness pixels, and the optimized maximum brightness is limited to below 245 to prevent grayscale saturation in the reflective areas of the metal pipe wall. For underexposed areas, the local brightness gain is increased based on the difference between the average brightness of the area and 120, so that the texture of the dark pipe wall comes into the identifiable brightness range. For areas with uneven illumination, gradual compensation is performed based on the average brightness difference between the current area and the adjacent normal area, so that the brightness of the compensated area gradually approaches that of the adjacent normal pipe wall area. The original average brightness of the potential defect area is about 128, which is increased to about 136 after local compensation, and the surrounding background area is adjusted from about 142 to about 145. The brightness difference between the two areas is still maintained and can be used for defect identification. After the anomaly area optimization, the brightness transition of each area is more continuous, resulting in a brightness-optimized standard image, while preserving the original defect texture and spatial position relationship.
[0039] The brightness-optimized standard image is subjected to texture detail enhancement and dynamic contrast optimization. Based on the metal texture characteristics of the pipe wall surface, a combination of local contrast enhancement and high-frequency detail enhancement is used to enhance the edges, scratches, cracks, and corrosion textures in the image. First, the gray-level gradient of each local region is calculated, and regions with significant gradient changes are selected as candidate regions for texture details, and the local contrast of these regions is increased. For continuous and smooth regions, the enhancement gain is limited to avoid excessive amplification of the processing texture of the metal pipe wall itself. For potential defect areas, the original edge gradient is about 18, which is increased to about 31 after texture enhancement, making the gray-level change between the small cracks and the surrounding pipe wall more obvious. For bright areas, the enhancement gain is limited to avoid the generation of false defect textures at reflective edges. After texture enhancement, local contrast adjustment, and bright area suppression, the continuity of the pipe wall texture, the clarity of defect edges, and local gray-level differences are improved simultaneously. Finally, the second inspection image is output, and the corresponding 12.0 m pipe position and original timestamp are retained.
[0040] In one embodiment, see Figure 3 The specific steps for performing frame-by-frame image visual feature recognition and texture feature analysis on the second inspection image to construct the pipe wall texture feature set for different frames are as follows: The second inspection image is decomposed into a temporal frame to obtain a continuous inspection image frame sequence. The continuous inspection image frame sequence is subjected to frame-by-frame image visual feature recognition to extract the brightness features, color features, texture features and edge features of the inner wall of the pipe, and generate multi-frame pipe wall visual feature information. Texture feature analysis is performed on the multi-frame pipe wall visual feature information to identify the degree of texture change, edge continuity and color anomaly, and to construct pipe wall texture feature sets for different frames.
[0041] In this embodiment, the second inspection images are time-sequentially organized according to the industrial camera's sampling frequency of 10 frames / s and the image acquisition timestamps. The second inspection images formed during the continuous inspection process are decomposed at the frame level according to the acquisition sequence to obtain a continuous inspection image frame sequence. Taking the inspection process of the inspection device moving from a position of 12.0 m to a position of 12.3 m at the pipe inlet as an example, under the condition of a moving speed of 0.15 m / s, this process lasts for about 2 seconds, corresponding to about 20 inspection images, and forming a continuous image frame sequence F1 to F20 according to the timestamps. Furthermore, the pipe position index is established for each frame using the moving distance information of the inspection device. For example, F1 corresponds to a position of about 12.000 m. Subsequent image frames are corrected for position according to the actual moving distance, so that the continuous image frames simultaneously have temporal order and pipe spatial position relationship. For the continuous inspection image frame sequence, the effective pipe wall region in each image frame is first extracted using a pipe inner wall boundary recognition method, excluding non-pipe wall regions formed by the inspection device's own structure and direct illumination by the searchlight. Then, visual features are extracted frame by frame for the effective pipe wall region. Brightness features are obtained by pixel grayscale statistics to obtain the average brightness, brightness standard deviation, and brightness distribution of the region; color features are characterized by RGB channels and chromaticity information after color space conversion to represent pipe wall color changes; texture features are described by local binary mode to describe the local texture structure of the pipe wall, and texture contrast and directionality are obtained by combining grayscale co-occurrence relationships; edge features are extracted by gradient edge detection to extract structural information such as pipe wall welds, cracks, and corrosion boundaries. Taking F1 to F20 as examples, brightness, color, texture, and edge feature vectors are formed for each frame, ultimately constituting multi-frame pipe wall visual feature information.
[0042] Based on the multi-frame visual feature information of the pipe wall corresponding to each image frame, the pipe wall is first divided into multiple corresponding analysis regions according to the axial position and circumferential area of the pipe, so that the same pipe wall region in different image frames can be feature aligned. For each corresponding region, the amount of brightness change and color difference between adjacent image frames is calculated, and the stability of abnormal changes is determined by the changes in consecutive frames. Taking a pipe wall region near 12.15 m as an example, if the average brightness of the corresponding region is basically stable in multiple consecutive frames, but the color feature continues to change towards the darker area, it is recorded as a color abnormality change; if a certain region only shows a significant brightness change in a single frame and immediately recovers in the next frame, it is recorded as a transient illumination change rather than directly as a defect feature. The degree of brightness change and color abnormality of each pipe wall region is formed through this continuous frame comparison.
[0043] After analyzing the brightness and color features, a further analysis of the changes in the pipe wall texture features in consecutive image frames is performed. Using the texture features obtained from the aforementioned local binary mode and gray-level co-occurrence relationship, texture similarity is calculated for pipe wall regions with spatial correspondence in adjacent image frames, and the degree of texture change is determined based on the texture differences between consecutive frames. When the same pipe wall region continuously exhibits linear texture, blocky texture, or locally coarse texture changes in consecutive image frames, the texture anomaly degree of the corresponding region is increased; when the texture change only occurs in a single image frame, and the texture in adjacent frames recovers rapidly, the corresponding anomaly degree is decreased. Taking the pipe wall region at 12.15 m as an example, if the same spatial region from F8 to F14 all exhibits elongated texture changes with the same direction, it is recorded as a continuous texture change region, and a corresponding texture change degree parameter is formed.
[0044] To analyze edge features in multi-frame visual feature information of pipe walls, edge pixels and directions are extracted from consecutive image frames. Edge positions in corresponding regions of adjacent frames are matched, and edge continuity length, directional consistency, and discontinuity are calculated. For normal weld seams or pipe wall structure edges, which typically exhibit stable directional and spatial continuity across consecutive image frames, they are retained as normal structural features. For newly formed edges caused by defects such as cracks or breaks, the focus is on analyzing whether they persist and form independent edge structures. Taking suspected anomalous regions in F8–F14 as examples, if a region exhibits a linear edge with gradually increasing length and consistent edge direction across adjacent frames, the anomalous edge weight is increased; if the edge only appears suddenly in one frame and disappears in the next, its anomalous weight is decreased. Finally, edge continuity features corresponding to each image frame are obtained. Combining the analysis results of brightness changes, color anomalies, texture changes, and edge continuity from different image frames, and using image frame numbers and pipe spatial locations as indices, various features are associated and encapsulated to construct a pipe wall texture feature set corresponding to different image frames. For the same pipe wall region, the degree of texture variation, edge continuity, and color anomaly are recorded simultaneously, enabling subsequent defect identification to utilize both single-frame visual features and continuous frame variation features. For example, a suspected anomaly region detected at 12.15 m can form three related features: "degree of texture variation, edge continuity, and color anomaly," while retaining its continuous variation records in F8–F14. This ultimately forms a multi-frame pipe wall texture feature set arranged according to time and pipe location.
[0045] In one embodiment, the specific steps for performing visual feature similarity matching of the pipe wall texture feature set based on the internal pipe defect database, conducting confidence comparison analysis, and marking the defect points to be tested are as follows: Based on the pipeline internal defect database, the pipe wall texture feature set is matched with the visual feature similarity of defects to obtain a suspected defect set of multiple frames of images. Defect confidence scores are calculated on the suspected defect set to obtain defect confidence scores for multiple frames of images; The confidence level of the defect is continuously compared across frames based on the confidence threshold. If the confidence level of the defect in three consecutive frames at the same location is not lower than the confidence threshold, it is marked as a defect point to be tested.
[0046] In this embodiment, a visual feature matching relationship is established between the multi-frame pipe wall texture feature set formed in the previous stage and the pipeline internal defect database. The defect database pre-stores standard visual features of defects such as cracks, corrosion, deformation, leakage, and damage, where each type of defect includes features such as brightness variation, color difference, texture structure, edge morphology, and spatial continuity. The pipe wall texture features in each image frame are divided into multiple regions to be matched according to the pipeline location, and similarity calculations are performed with the corresponding type features in the defect database. For example, in a pipe wall region approximately 12.15 m from the pipeline inlet, consecutive image frames F8 to F14 all detect elongated textures and continuous edges. The texture direction, edge continuity, and color anomaly degree of this region are matched with the standard features of cracks. When the overall similarity reaches the preset matching condition, the region is marked as a suspected crack defect, and a multi-frame suspected defect set is formed by combining the corresponding image frame number and pipeline location.
[0047] The similarity of visual features of defects is used as the basic confidence level, and further, the degree of texture variation, edge continuity, and color anomaly are introduced for joint correction. This ensures that the confidence level not only reflects the similarity between the current image frame and the standard features of defects, but also reflects the stability of the visual features of the region itself. Taking a suspected crack region at 12.15 m as an example, if an image frame has a similarity of 0.82 with the standard features of cracks, and at the same time has obvious texture variation and high edge continuity, then the overall defect confidence level of that frame is 0.86; if another image frame only has local color anomalies, and the texture and edge features are not obvious, then its confidence level may be 0.58. The defect confidence levels of the corresponding regions in F8 to F14 are calculated in the above manner to form a multi-frame defect confidence level sequence arranged according to image frames.
[0048] Based on the multi-frame defect confidence sequence, a defect confidence threshold of 0.75 is used to continuously compare suspected defects with the same spatial correspondence in the pipeline. Since the inspection device moves at 0.15 m / s and the industrial camera samples at 10 frames / s, the theoretical moving distance between adjacent image frames is approximately 0.015 m. Therefore, suspected defects in the corresponding pipe wall region in adjacent frames are associated as the same location. For example, a suspected crack near 12.15 m has defect confidences of 0.86, 0.89, and 0.84 in consecutive frames F8, F9, and F10, respectively. All three frames have a confidence level of no less than 0.75, and the suspected defect areas have a continuous spatial correspondence. Therefore, this area is marked as a defect point to be tested. If another suspected area only reaches 0.79 in F12, while it drops to 0.62 and 0.55 in F13 and F14, respectively, the three-frame confirmation condition is not met, and it is not marked as a defect point to be tested. The final result is a set of defects to be tested that includes pipe location indexes, associated image frames, and continuous confidence information.
[0049] In one embodiment, the specific steps for performing secondary image acquisition on the defect point to be tested and dividing the defect area image and background image are as follows: Based on the mobile inspection parameter set, the mobile speed parameter set of the pipeline inspection device is calculated. The mobile speed parameter set includes mobile speed, mobile direction, inspection mileage and data collection time. Based on the set of moving speed parameters, the spatiotemporal position of the defect point to be tested is converted to obtain the internal spatial coordinates of the defect point in the pipeline. Based on the internal spatial coordinates of the pipeline, the pipeline inspection device is adjusted to perform secondary image acquisition of the defect point to be tested, and a high-definition retest image is obtained. Defect point edge region identification is performed on high-definition retest images, and the edge contour lines of the defect region are extracted. Region image segmentation is performed based on the edge contour of the defect region to divide the defect region image and the background image.
[0050] In this embodiment, the moving speed parameters of the inspection device are calculated for a metal conveying pipeline with an inner diameter of 600 mm and a length of 180 m. The moving inspection parameters are centrally recorded as yaw angle, pitch angle, roll angle, moving speed, moving direction, inspection mileage, and acquisition time. The inertial measurement unit outputs attitude data at a frequency of 100 Hz, the odometer outputs movement data at a frequency of 50 Hz, and the industrial camera acquires images at a frequency of 10 frames / s. When the inspection device moves along the pipeline axis at a speed of 0.15 m / s, the theoretical axial displacement corresponding to adjacent image frames is 0.015 m; when the acquisition time interval is 1 s, the corresponding inspection mileage change is approximately 0.15 m. The timestamp of each image frame is correlated with the most recent odometer data, and the real-time inspection mileage at the time of image acquisition is obtained using adjacent mileage data interpolation. For example, if the pipeline mileages corresponding to consecutive image frames associated with a defect point to be tested are 12.105 m, 12.120 m and 12.135 m respectively, then the defect is determined to exist continuously along the pipeline axis based on the position changes of the three frames, and the movement speed parameter set of the region is formed by the corresponding acquisition time, movement direction and attitude parameters.
[0051] Using the axial position of the inspection device when it enters the pipeline as the zero point, the inspection mileage corresponding to the continuous image frames associated with the defect point to be measured is used as the basis for the axial position, and axial correction is performed by combining the fixed distance between the industrial camera installation position and the center of the inspection device. For the circumferential position of the pipe wall, the lateral center of the image is used as the circumferential reference of the pipe wall, and the circumferential position is calculated based on the pixel offset of the defect area center relative to the image center, the pipe inner diameter of 600 mm, and the yaw, pitch, and roll angles of the inspection device. Taking the defect point to be measured at a distance of about 12.15 m from the pipe inlet as an example, the lateral offset of the defect area center in the corresponding image is about 120 pixels. In the 1920×1080 pixel image, the circumferential position of the pipe wall corresponding to the defect is obtained according to the circumferential mapping relationship after the pipe wall is unfolded. Then, the position is averaged using the mileage data of three consecutive frames to reduce the positioning error of a single frame, so that the final spatial coordinates establish a stable correspondence with the actual pipe wall position near 12.15 m.
[0052] The inspection device calculates the return distance based on the axial coordinates of the defect point to be tested and its current position. When it enters the range of approximately 0.30 m before the target position, it gradually reduces its moving speed from 0.15 m / s and makes fine-tuning adjustments within a range of approximately 0.15 m before and after the defect point to keep the defect area near the center of the camera's field of view. The industrial camera continues to use a resolution of 1920×1080 pixels for retesting and acquisition. The searchlight maintains the aforementioned 800 lx illumination condition, and the illumination angle is fine-tuned based on the bright areas in the retesting images to avoid large-area specular reflections. During the secondary acquisition process, no fewer than 10 retesting images are continuously acquired. The frame with the most complete defect outline is selected as the high-definition retesting image based on the clarity of the defect edges, the stability of the regional brightness, and the image integrity. For example, for a suspected defect at 12.15 m, after the inspection device achieves stable positioning near the target position, it positions the defect area near the center of the image, thereby obtaining a retesting image for accurate edge recognition.
[0053] The effective pipe wall area was determined based on image brightness and pipe wall texture features, excluding the inspection device's own structure, areas directly illuminated by the searchlight, and invalid areas at the image edges. Then, a gradient edge detection method was used to extract pixels with significant grayscale changes. For the 1920×1080 pixel retest image, a 300×300 pixel area around the suspected defect center was selected as the key analysis area. Local gradient calculations were performed on this area, and an initial edge retention condition was set for continuous edges with a length of at least 10 pixels. Edges shorter than 10 pixels and appearing only in isolation were filtered out to reduce false detections caused by sensor noise and metal pipe wall particle texture. Taking a suspected crack at 12.15 m as an example, a continuous, slender edge with a length of approximately 86 pixels was extracted within the key analysis area, and the directional changes of adjacent edge pixels remained within a small range. Therefore, this continuous edge was identified as a candidate defect contour, and its complete edge contour line was extracted.
[0054] Using the extracted defect edge contour as the region segmentation boundary, the pixels inside and outside the contour are classified into different regions. For closed defect contours, all pixels inside the contour are divided into the defect region, and the adjacent pipe wall region outside the contour and no more than 50 pixels away from the contour edge is used as the background reference region to preserve the normal pipe wall texture around the defect. For slender defects such as cracks, the region is segmented according to the strip-shaped area formed by the edge contour, and a certain pixel range is extended on both sides of the crack to form the background region. Taking a suspected crack at 12.15 m as an example, the defect region is formed based on a continuous edge contour of about 86 pixels in length, and about 30 pixels of normal pipe wall are preserved on both sides as the background region. Finally, two types of data are obtained: the defect region image and the background image. These two images are then associated with and saved with the axial position and corresponding circumferential position of the 12.15 m pipe.
[0055] In one embodiment, the specific steps for classifying defects and calculating defects in the defect area image, generating defect assessment results, encapsulating the defect assessment results, and outputting a pipeline inspection defect report are as follows: Visual defect depth analysis is performed on the defect area image to obtain depth defect features; Defects are classified and identified based on deep defect features to determine the defect type, which includes cracks, corrosion, deformation, leakage and damage. Based on the defect region image, the length, width, area, and morphological change parameters of the defect region are calculated to obtain the defect morphological parameters; Defect severity is assessed based on defect morphology parameters and defect type, and defect assessment results are generated. The defect assessment results, the internal spatial coordinates of the defect point to be tested, and the image of the defect area are encapsulated to output a pipeline inspection defect report.
[0056] In this embodiment, the segmented defect region image is subjected to visual defect depth analysis to obtain depth defect features that can characterize the internal structure of the defect. First, the 1920×1080 pixel defect region image is converted into a grayscale image, and defect depth characterization parameters are established based on the differences in brightness, texture, and edge gradient between the defect region and the adjacent 50 pixel background region. Then, multiple sampling sections are set along the length of the defect contour line. For each section, the grayscale attenuation, texture roughness, and edge gradient changes between the defect center region and the normal pipe wall region are calculated, and the visual depth of the defect is determined according to the change trend of multiple consecutive sections. For a suspected crack located at 12.15 m, no less than 10 sections are set along the crack contour of approximately 86 pixels in length. Visual features of the crack center and the normal pipe walls on both sides are extracted at each section. When the crack center region continuously shows obvious brightness decrease and gradient abrupt change, the corresponding features are combined to form a depth defect feature. For regions that only have surface color changes and no continuous texture gradient changes, their depth defect response is reduced to distinguish surface stains from defects with actual structural features.
[0057] Based on the aforementioned deep defect features, defect classification and identification are performed by matching visual features with preset defect type features to determine the defect type. The preset defect types include cracks, corrosion, deformation, leakage, and breakage, and corresponding edge morphology, texture structure, color variation, and spatial distribution features are established for each. For example, cracks are mainly characterized by long, continuous edges and directional textures; corrosion is characterized by blocky or flaky rough textures accompanied by color anomalies; deformation is characterized by continuous displacement of the pipe wall contour or local geometric changes; leakage is characterized by local highlights, wet textures, and diffused edges; and breakage is characterized by irregular boundaries and missing textures in local areas. For the defect area at 12.15 m, the extracted 86-pixel continuous edges, texture direction, brightness variation, and deep defect features are matched with the above types for similarity. When the overall matching degree of the crack category is higher than that of other types and reaches the preset classification threshold of 0.75, the area is identified as a crack defect, and the matching results of other categories are retained as auxiliary classification information.
[0058] Using the edge contour of the defect area as the calculation boundary, the image pixel size is spatially converted to the pipe wall imaging scale corresponding to a pipe inner diameter of 600 mm. The maximum axial dimension, maximum lateral dimension, and internal area of the defect contour are calculated. For slender cracks, the ratio of defect length to maximum width and the change in contour direction are also calculated to describe the defect morphology. Taking a crack area at 12.15 m as an example, after image scale conversion, the actual crack length is found to be approximately 43 mm, the maximum width approximately 0.8 mm, and the defect area approximately 18 mm². 2Multiple contour points are extracted along the crack length direction, and the directional change of adjacent contour points is calculated. When there is local bending in the crack, the maximum directional change is used as one of the morphological change parameters. Through the above calculations, defect morphological parameters such as defect length, width, area, length-to-width ratio, and contour direction change are obtained.
[0059] Corresponding morphological evaluation conditions were set for different defect types, and the parameters of defect length, width, area, and morphological changes were jointly analyzed. For crack-type defects, the focus was on analyzing crack length, maximum width, area, and directional changes; for corrosion-type defects, the focus was on analyzing corrosion area, boundary irregularity, and texture changes; for deformation-type defects, the focus was on analyzing contour offset and morphological change range. For example, a crack identified at 12.15 m had a length of approximately 43 mm, a maximum width of approximately 0.8 mm, and an area of approximately 18 mm². 2 If no obvious bifurcation or large-scale expansion is observed, the crack is determined as a second-level defect based on the preset crack evaluation range, and the evaluation criteria such as crack length, maximum width, and area are recorded. If the crack length or width is found to exceed the corresponding level boundary during the retest, the defect level is readjusted according to the actual morphological parameters, thereby avoiding fixed level judgment based solely on the defect type.
[0060] Record the defect number, defect type, defect severity, defect length, maximum width, defect area, morphological change parameters, pipe axial position, pipe wall circumferential position, defect area image, and corresponding background image, and associate the acquisition time of the original inspection image and the high-definition retest image. Taking a crack defect at 12.15 m as an example, its identification result is recorded as crack type, second-level defect, associated with a defect length of approximately 43 mm, a maximum width of 0.8 mm, and a maximum width of 18 mm. 2 The defect area is recorded, along with the corresponding internal spatial coordinates of the pipeline and a re-measured image. After data encapsulation, a defect record for the pipeline is generated, sorted according to the pipeline's axial position, and a pipeline inspection defect report is output, including defect location, visual evidence, morphological parameters, and evaluation results.
[0061] In one embodiment, a pipeline inspection defect identification device based on visual images is provided, for performing the pipeline inspection defect identification method based on visual images as described above, including: The first processing module drives the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then performs geometric posture compensation and reverse deblurring processing, and outputs the first inspection image. The second processing module is used to enhance the texture details and dynamically optimize the contrast of the first inspection image, and output the second inspection image. The visual analysis module is used to perform frame-by-frame image visual feature recognition and texture feature analysis on the second inspection image, and to construct a pipe wall texture feature set for different frames. The confidence calculation module is used to perform visual feature similarity matching of defects in the pipe wall texture feature set based on the internal defect database of the pipe, and to perform confidence comparison analysis to mark the defect points to be tested. The secondary acquisition module is used to acquire secondary images of the defect points to be tested, and to divide the defect area image and the background image. The defect assessment module is used to classify defects and calculate defect morphology in the defect area image, generate defect assessment results, encapsulate the defect assessment results into data, and output a pipeline inspection defect report.
[0062] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0063] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0065] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.
[0066] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.
Claims
1. A method for pipeline inspection defect identification based on visual images, characterized in that, Includes the following steps: Drive the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then perform geometric posture compensation and reverse deblurring processing to output the first inspection image. The first inspection image is enhanced in texture detail and dynamically optimized in contrast, and the second inspection image is output. The second inspection image is subjected to frame-by-frame image visual feature recognition and texture feature analysis to construct a pipe wall texture feature set for different frames; Based on the pipeline internal defect database, the pipe wall texture feature set is subjected to defect visual feature similarity matching, and confidence comparison analysis is performed to mark the defect points to be tested. Secondary image acquisition is performed on the defect point to be tested to divide the defect area image and background image; The defect area image is classified and its morphology is calculated to generate a defect assessment result. The defect assessment result is then encapsulated and a pipeline inspection defect report is output.
2. The pipeline inspection defect identification method based on visual images according to claim 1, characterized in that, The specific steps for driving the mobile inspection device to enter the target pipeline to perform real-time visual acquisition of the pipeline interior, then performing geometric attitude compensation and reverse deblurring processing, and outputting the first inspection image are as follows: Drive the mobile inspection device into the target pipeline, turn on the front-end searchlight, and use the onboard industrial camera to perform real-time visual acquisition of the pipeline interior at a preset sampling frequency to obtain real-time inspection images of the pipeline interior. The mobile inspection device collects its own movement parameters based on the mobile self-inspection unit, thus obtaining the mobile inspection parameter set. On the local side of the mobile inspection device, geometric posture compensation and reverse deblurring are performed on the real-time inspection image inside the pipe based on the mobile inspection parameter set, and the first inspection image is output.
3. The pipeline inspection defect identification method based on visual images according to claim 2, characterized in that, The mobile self-test unit includes an inertial measurement unit, a mileage detection unit, and a motion control unit. The mobile inspection parameter set includes yaw angle, pitch angle, roll angle, movement speed, movement direction, and movement distance.
4. The pipeline inspection defect identification method based on visual images according to claim 2, characterized in that, The specific steps for performing geometric pose compensation and reverse deblurring on the real-time inspection image within the pipe based on the mobile inspection parameter set on the local side of the mobile inspection device, and outputting the first inspection image are as follows: The mobile inspection device extracts real-time inspection images of the pipe on the local side. Multiple reference points are obtained by marking the pipe inner wall structure reference points on the real-time inspection images inside the pipe. By matching adjacent image frames for multiple reference points and calculating the positional changes of subsequent image frames, the displacement, displacement direction, and displacement rate between image frames are obtained. The attitude change of the inspection device is analyzed based on the mobile inspection parameter set to obtain the attitude change amount. Motion state correlation analysis is performed based on attitude change and image frame displacement, displacement direction and displacement change rate to determine the degree of motion jitter of the mobile inspection device. Based on the degree of motion jitter, the jitter state is classified and identified to obtain the motion jitter type, which includes translational jitter, vertical posture jitter, and compound jitter. Based on the motion jitter type, jitter posture compensation calculation is performed to obtain a motion compensation parameter set, which includes image translation compensation parameters and image rotation compensation parameters. Motion blur calculation is performed based on the inter-frame displacement and displacement change rate of the image, and blur compensation operation is performed to output motion blur compensation parameters. Geometric posture compensation is performed on the real-time inspection image inside the pipe based on the motion compensation parameter set, and reverse deblurring is performed according to the motion blur compensation parameters to output the first inspection image.
5. The pipeline inspection defect identification method based on visual images according to claim 2, characterized in that, The specific steps for enhancing texture details and dynamically optimizing contrast of the first inspection image to output the second inspection image are as follows: The first inspection image is uploaded to the cloud image processing platform for global brightness distribution analysis to generate global brightness distribution features. Based on the global brightness distribution characteristics, the first inspection image is optimized by adaptive histogram equalization to obtain a globally optimized brightness image. The global brightness optimization image is divided into regions, and local brightness is calculated to obtain the local brightness of the image at multiple locations; Based on the local brightness of the image, regional brightness difference analysis is performed to obtain local brightness abnormal regions, which include local overexposure, underexposure and uneven lighting regions. Local brightness optimization is performed on the aforementioned areas of abnormal brightness to obtain a brightness-optimized standard image; The brightness-optimized standard image is enhanced with texture detail and dynamically optimized for contrast, and a second inspection image is output.
6. The pipeline inspection defect identification method based on visual images according to claim 5, characterized in that, The specific steps for performing frame-by-frame image visual feature recognition and texture feature analysis on the second inspection image to construct the pipe wall texture feature set for different frames are as follows: The second inspection image is decomposed into a temporal frame to obtain a continuous inspection image frame sequence. The continuous inspection image frame sequence is subjected to frame-by-frame image visual feature recognition to extract the brightness features, color features, texture features and edge features of the inner wall of the pipe, and generate multi-frame pipe wall visual feature information. Texture feature analysis is performed on the multi-frame pipe wall visual feature information to identify the degree of texture change, edge continuity and color anomaly, and to construct pipe wall texture feature sets for different frames.
7. The pipeline inspection defect identification method based on visual images according to claim 6, characterized in that, The specific steps for matching the visual features of the pipe wall texture feature set based on the internal defect database, performing confidence comparison analysis, and marking the defect points to be tested are as follows: Based on the pipeline internal defect database, the pipe wall texture feature set is matched with the visual feature similarity of defects to obtain a suspected defect set of multiple frames of images. Defect confidence scores are calculated on the suspected defect set to obtain defect confidence scores for multiple frames of images; The confidence level of the defect is continuously compared across frames based on the confidence threshold. If the confidence level of the defect in three consecutive frames at the same location is not lower than the confidence threshold, it is marked as a defect point to be tested.
8. The pipeline inspection defect identification method based on visual images according to claim 7, characterized in that, The specific steps for performing secondary image acquisition on the defect point to be tested, and dividing the defect area image and background image are as follows: Based on the mobile inspection parameter set, the mobile speed parameter set of the pipeline inspection device is calculated. The mobile speed parameter set includes mobile speed, mobile direction, inspection mileage and data collection time. Based on the set of moving speed parameters, the spatiotemporal position of the defect point to be tested is converted to obtain the internal spatial coordinates of the defect point in the pipeline. Based on the internal spatial coordinates of the pipeline, the pipeline inspection device is adjusted to perform secondary image acquisition of the defect point to be tested, and a high-definition retest image is obtained. Defect point edge region identification is performed on high-definition retest images, and the edge contour lines of the defect region are extracted. Region image segmentation is performed based on the edge contour of the defect region to divide the defect region image and the background image.
9. The pipeline inspection defect identification method based on visual images according to claim 8, characterized in that, The specific steps for classifying defects and calculating defect morphology in the defect area image, generating defect assessment results, encapsulating the defect assessment results, and outputting a pipeline inspection defect report are as follows: Visual defect depth analysis is performed on the defect area image to obtain depth defect features; Defects are classified and identified based on deep defect features to determine the defect type, which includes cracks, corrosion, deformation, leakage and damage. Based on the defect region image, the length, width, area, and morphological change parameters of the defect region are calculated to obtain the defect morphological parameters; Defect severity is assessed based on defect morphology parameters and defect type, and defect assessment results are generated. The defect assessment results, the internal spatial coordinates of the defect point to be tested, and the image of the defect area are encapsulated to output a pipeline inspection defect report.
10. A pipeline inspection defect identification device based on visual images, characterized in that, The method for performing pipeline inspection defect identification based on visual images as described in claim 1 includes: The first processing module drives the mobile inspection device into the target pipeline to perform real-time visual acquisition of the pipeline interior, then performs geometric posture compensation and reverse deblurring processing, and outputs the first inspection image. The second processing module is used to enhance the texture details and dynamically optimize the contrast of the first inspection image, and output the second inspection image. The visual analysis module is used to perform frame-by-frame image visual feature recognition and texture feature analysis on the second inspection image, and to construct a pipe wall texture feature set for different frames. The confidence calculation module is used to perform visual feature similarity matching of defects in the pipe wall texture feature set based on the internal defect database of the pipe, and to perform confidence comparison analysis to mark the defect points to be tested. The secondary acquisition module is used to acquire secondary images of the defect points to be tested, and to divide the defect area image and the background image. The defect assessment module is used to classify defects and calculate defect morphology in the defect area image, generate defect assessment results, encapsulate the defect assessment results into data, and output a pipeline inspection defect report.
