Automatic identification system of cracks on the inner wall of endoscope tube based on video deep learning

Abstract

The present invention discloses an automatic identification system for cracks in the inner wall of an endoscopic pipeline based on video deep learning, which relates to the technical field of pipeline detection. The present invention comprises an image acquisition module, an image preprocessing module, a pipeline inner wall model establishment module, a deep learning identification module and a result output module; the image acquisition module is used to use an autonomous mobile endoscope device to collect images inside the pipeline, and its multi-wheel drive structure, adaptive rubber tracks, high-precision sensors, etc. ensure the stability of the collection process; the present invention significantly improves the automation level and accuracy of pipeline inner wall crack detection, uses the autonomous mobile endoscope device in combination with stereo lighting and multi-angle image acquisition, ensures that high-quality pipeline inner wall images can be obtained even in complex environments, and provides high-quality input data for the deep learning model through calibration and registration, noise suppression and contrast enhancement of the image preprocessing module, thereby improving the accuracy and reliability of crack identification.

Description

Technical Field

[0001] The present invention relates to the field of pipeline detection technology, and in particular to an automatic recognition system for cracks on the inner wall of an endoscopic pipeline based on video deep learning. Background Art

[0002] In industrial production, pipeline systems are critical infrastructure for transporting various media (such as liquids and gases). They are widely used in numerous sectors, including the petrochemical industry, power generation, and water supply and drainage. The integrity of pipeline interior walls is directly related to production safety, efficiency, and environmental protection. For example, in the petrochemical industry, pipeline leaks can cause serious accidents such as fires and explosions, while also polluting the environment. In water supply systems, pipeline damage can lead to water waste and pollution. Therefore, timely and accurate detection of defects such as cracks in pipeline interior walls is crucial. However, traditional inspection methods have many shortcomings. Manual visual inspection relies on the experience and skills of inspectors, is highly subjective, has low inspection efficiency, and can easily lead to missed inspections due to fatigue caused by long work hours. Non-destructive testing techniques such as ultrasonic testing and radiographic testing can detect some defects, but they are complex, require specialized personnel to operate the equipment, have high requirements for pipeline surface finishes, and may pose certain health risks to personnel during the inspection process, making them unable to meet the needs of large-scale, rapid inspections.

[0003] With the rapid development of artificial intelligence (AI), video deep learning technology has shown tremendous potential in the field of industrial inspection. Deep learning algorithms have achieved remarkable results in tasks such as image recognition, classification, and object detection. In industrial visual inspection, deep learning-based methods can automatically learn features from images, making them more adaptable to complex scenarios and diverse defect types. For example, in some surface defect detection scenarios, deep learning models can accurately identify tiny scratches, cracks, and other defects with far greater accuracy and efficiency than traditional methods. Video deep learning technology can also process large amounts of image data, enabling real-time inspection and providing timely feedback on the condition of pipeline interior walls, providing more reliable support for industrial production. This technological development is prompting research into its application in pipeline interior crack detection to overcome the limitations of traditional inspection methods.

[0004] In order to solve the above defects, a technical solution is now provided. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of low accuracy, low efficiency and insufficient reliability of existing pipeline inner wall crack detection technology, and to propose an endoscopic pipeline inner wall crack automatic identification system based on video deep learning.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The automatic identification system for cracks on the inner wall of endoscope tubes based on video deep learning includes:

[0008] The image acquisition module uses an autonomous mobile endoscope to capture images inside the pipeline, and cooperates with a stereo lighting device and a multi-angle image acquisition module to ensure the acquisition of high-quality, all-round images of the pipeline inner wall;

[0009] The image preprocessing module calibrates and registers the collected multi-angle images to ensure geometric accuracy, suppress noise, and enhance contrast, providing high-quality input data for subsequent deep learning models;

[0010] The pipeline inner wall modeling module eliminates duplicate image data through feature point extraction and matching, image registration, selects the optimal image based on evaluation parameters, and then uses an algorithm to convert the processed image into point cloud data. Through filtering, surface reconstruction, and texture mapping, a model reflecting the actual appearance of the pipeline inner wall is constructed.

[0011] A deep learning recognition module is used to build an improved convolutional neural network model. By training it with different annotated data sets, it can identify cracks on the inner wall of the pipeline and improve its ability to identify cracks in different situations.

[0012] The result output module displays the crack identification results in a visual interface, highlights the crack location and range with different colors and marks, and generates an inspection report containing detailed crack information, providing data support for pipeline maintenance and repair.

[0013] Furthermore, the specific process performed by the image acquisition module is as follows:

[0014] Image data from within the pipeline is collected through an autonomously moving endoscope. The device uses a multi-wheel drive structure equipped with adaptive rubber tracks to adapt to the inner walls of pipelines of varying diameters and materials. The device maintains a stable speed and posture during crawling. A gyroscope and accelerometer are installed on the endoscope to monitor its position and motion within the pipeline in real time, transmitting this information via wireless communication.

[0015] The endoscope device is equipped with a three-dimensional lighting device, which consists of multiple LED light sources with independently controllable angles and brightness. These light sources are distributed around the endoscope device and automatically adjust the lighting angle and brightness according to the geometric shape and reflective characteristics of the inner wall of the pipe through an intelligent algorithm.

[0016] A multi-angle image acquisition module is constructed, including multiple cameras installed at different positions and angles of the endoscope device, which can simultaneously capture images of the inner wall of the pipeline from different perspectives; the cameras have the characteristics of large depth of field and low distortion, capturing images of the inner wall of the pipeline from close to long distances; at the same time, the camera acquisition frame rate is adaptively adjusted according to the pipeline crawling speed and detection requirements.

[0017] Furthermore, the specific steps of automatically adjusting the illumination angle by the intelligent algorithm in the image acquisition module are as follows:

[0018] Angle adjustment based on normal vector: Assume that the normal vector of the inner wall of the pipe at a certain point is The position coordinates of the endoscope device in the pipeline are (x, y, z); for each LED light source i, its initial direction vector is According to the principle of light reflection, the standard lighting angle makes the angle between the light and the normal vector perpendicular to obtain the strongest reflected light; calculate the dot product of each light source direction vector and the normal vector where θ i is the angle between the light source and the normal vector; by adjusting the angle of the light source, cosθ i Equal to; use the gradient descent optimization algorithm to adjust to minimize |cosθ i |, the update formula is Where α is the learning rate;

[0019] Angle adjustment based on ray tracing: Considering the multiple reflections of light on the inner wall of the pipe, each LED light source emits light to simulate the light propagation path within the pipe. When the light encounters the inner wall of the pipe, the direction of the reflected light is calculated based on the relationship between the angle of incidence and the angle of reflection. By tracing the propagation path of the light, it is analyzed to determine which areas are not receiving the required lighting or are shadowed. Based on the analysis results, the angle of the light source is adjusted to ensure that the light covers the inner wall of the pipe more evenly.

[0020] In areas with smooth inner walls of the pipe, the normal vector-based angle adjustment algorithm is first used to adjust the light source angle to a direction perpendicular to the inner wall, providing a good foundation for lighting. When encountering curved parts, connected parts or areas with geometric shapes in the pipe, the ray tracing algorithm is enabled. Taking into account the reflection and propagation path of light on the inner wall of complex shapes, the light source angle can be adjusted more accurately to ensure that these areas can also receive uniform lighting.

[0021] Furthermore, the specific steps of automatically adjusting the light brightness by the intelligent algorithm in the image acquisition module are as follows:

[0022] Brightness adjustment based on the illumination model: according to the inverse square law of illumination Where E is the illuminance, I is the light intensity, r is the distance from the light source to the illuminated point, and θ is the angle between the light and the normal of the illuminated surface. For each point P on the inner wall of the pipe, calculate the distance r from it to each LED light source. i and the angle θ i ; Given the initial intensity I0 of each light source, calculate the theoretical illuminance at that point Set a target illuminance E target, according to the actual measured illuminance E actual , calculate the brightness adjustment coefficient Then, adjust the brightness of each light source according to I i =kI0; When encountering a curved pipe section, the distance r close to the inner wall becomes smaller, and E calculated according to the above formula will increase. At this time, the brightness I of the light source on this side is reduced to maintain the balance of illumination.

[0023] Brightness adjustment based on image grayscale histogram: Get the grayscale histogram H(k) of the image captured by the camera in real time, where k is the grayscale level; calculate the mean of the grayscale histogram and standard deviation Set the target mean μ target and standard deviation σ target , calculate the brightness adjustment parameters according to the current μ and σ, specifically using the linear mapping function L new =aL old +b, where L old is the brightness of the original image, L new is the adjusted brightness, a and b are based on μ, σ, μ target and σ target The calculated coefficient is used to change the overall brightness of the image by adjusting the brightness parameter, and the brightness of the light source is fed back to the LED light source control system to adjust the brightness of the light source to obtain appropriate lighting conditions. For curved pipe sections, the brightness of the light source at the corresponding position is adjusted according to the changes in the grayscale histogram of different areas in the image.

[0024] In the initial stage, the basic brightness of the light source is set according to the illumination model. By calculating the distance and angle between the light source and each point on the inner wall of the pipe, the brightness of each LED light source is preliminarily determined according to the inverse square law of illumination, so that the entire inner wall of the pipe has a uniform illumination foundation on a macro scale. Then, during the image acquisition process, the grayscale histogram of the image captured by the camera is analyzed in real time. If the overall brightness or local brightness of the image is found to be not as expected, a brightness adjustment method based on the image grayscale histogram is used for fine-tuning. This combination, based on the consideration of the physical illumination model, is dynamically optimized according to the visual effect of the actual captured image, ensuring that images with better clarity and detail integrity can be obtained under different pipeline conditions.

[0025] Furthermore, the image preprocessing module performs the following steps:

[0026] Image calibration and registration: Calibrate and register images collected from multiple angles. Using known camera installation positions and angles, geometric transformation algorithms are used to map images from different perspectives into a unified coordinate system. Image distortion caused by lens distortion is corrected to ensure geometric accuracy and provide accurate input data for subsequent deep learning models.

[0027] Noise suppression and enhancement: Image noise suppression technology is used, combined with the characteristics of the pipeline inner wall image, to effectively remove noise from the image. A method based on wavelet transform and local adaptive threshold is used to suppress noise while retaining the edge information of the cracks. At the same time, histogram equalization and contrast stretching algorithms are used to enhance the contrast of the image, making the cracks more prominent in the image and easier to identify.

[0028] Furthermore, the specific operation steps of the pipeline inner wall model building module are as follows:

[0029] Eliminate duplicate image data:

[0030] Use scale-invariant feature transformation or accelerated robust feature algorithm to extract feature points from each image in the image data. Feature points are representative local structural information in the image and are invariant to image rotation, scaling, and illumination changes. Match feature points between adjacent images, calculate the distance and similarity between feature points, find corresponding feature point pairs in different images, use the nearest neighbor algorithm to determine the matching feature points, and set a threshold to eliminate false matches.

[0031] Based on the matched feature points, the images are registered using geometric transformation models such as affine transformation or perspective transformation, aligning images from different perspectives in the same coordinate system so that duplicated areas can accurately correspond. In the registered images, duplicates are identified and removed by comparing the similarity between the images and the content of the overlapping areas. An image difference algorithm is used to calculate the differences between adjacent images and remove duplicate pixel information. For overlapping areas, the images in the overlapping areas are evaluated to determine which image is retained as the optimal one.

[0032] When the image data is in video format, optical flow motion estimation technology is used to estimate the motion of the object by analyzing the motion vectors of the pixels in the image. Based on the motion trajectory of the endoscope in the pipe and the optical flow information, the repeated areas between adjacent frames are determined. For the same parts in consecutive frames, only the relevant image data of one frame is retained to reduce data redundancy.

[0033] Construct the inner wall model of the pipe:

[0034] Based on the characteristics of the image data and the geometry of the pipe's inner wall, a 3D reconstruction algorithm based on multi-view geometry is selected, including binocular stereo vision or multi-view stereo vision algorithms. When the texture information contained in the image data meets the requirements, a 3D reconstruction method based on structured light is used. For pipe objects, the reconstruction process is optimized based on their geometric constraints, including the circular cross-section of the pipe, which is used to reconstruct the entire pipe's inner wall by analyzing and fitting images of different cross-sections.

[0035] Through the three-dimensional reconstruction algorithm, the processed image data is converted into point cloud data. The point cloud is a collection of a series of points in three-dimensional space, each point contains spatial coordinates and color and texture information; the generated point cloud is filtered to remove noise points and abnormal points; specifically, statistical filtering and radius filtering methods are used for filtering.

[0036] Furthermore, the specific operation steps for evaluating the image of the overlapping area in the pipeline inner wall model building module are as follows:

[0037] Image evaluation is performed by comprehensively judging the image through the evaluation parameters obtained. The evaluation parameters include:

[0038] Clarity parameters: including gradient amplitude, high-frequency energy, and degree of focus. The gradients of the image in the horizontal and vertical directions are calculated using the Sobel operator, Scharr operator, or Laplace operator. The image is Fourier transformed to convert it to the frequency domain, and the high-frequency energy is calculated to evaluate the clarity of the image. A focus evaluation function is used to measure the degree of focus of the image. Specifically, the variance method is used to calculate the variance of the grayscale values ​​of the image pixels. The larger the variance, the more focused the image and the clearer the details. This is quantified using a value between 0 and 1. After normalizing the obtained gradient amplitude, high-frequency energy, and degree of focus, a triangle is established with the gradient amplitude, high-frequency energy, and degree of focus as the three sides of the triangle. The area of ​​the triangle is calculated and recorded as the clarity evaluation value, which is then used as the standard for measuring clarity.

[0039] Noise level parameter: By calculating the root mean square noise and signal-to-noise ratio of the image, and then calculating the ratio of the root mean square noise to the signal-to-noise ratio after normalization, the noise evaluation value is obtained, and this noise evaluation value is used as the standard for measuring the noise level;

[0040] Color parameters: Evaluated by color saturation and color consistency. Color saturation measures color vividness by calculating the saturation component in the HSV color space. Color consistency is measured by calculating the entropy of the color histogram. The obtained color saturation and the entropy of the color histogram are normalized and then the ratio is calculated to obtain the color evaluation value, which is used as the standard for measuring color parameters.

[0041] The obtained clarity evaluation value, noise evaluation value, and color evaluation value are calibrated as qx, zp, and ys, respectively, and after normalization, they are entered into the following formula: To obtain the comprehensive evaluation value ZGZ, where β1, β2, and β3 are the preset weight coefficients of the clarity evaluation value, noise evaluation value, and color evaluation value respectively, and the obtained comprehensive evaluation value ZGZ is used as the standard for measuring the image selection priority evaluation;

[0042] The different comprehensive evaluation values ​​obtained from the images in the overlapping area are sorted by size, and the image with the largest comprehensive evaluation value is selected as the optimal image.

[0043] Furthermore, the execution process of the deep learning recognition module is as follows:

[0044] A convolutional neural network model for identifying cracks on pipeline walls was constructed. This model was improved upon the classic CNN architecture by adding convolution kernels tailored to the texture features of pipeline wall images. The network structure employed a multi-scale feature fusion approach to simultaneously capture crack features of varying sizes and shapes. Furthermore, an attention mechanism was introduced to enable the model to automatically focus on cracked areas when processing images, improving recognition accuracy.

[0045] Collect images of pipeline inner wall cracks of different types, sizes, and severities as a training dataset; annotate these images with information including the location, length, width, and direction of the cracks; use data augmentation technology to increase the diversity of the dataset; and optimize the model parameters by training the customized model on a large-scale dataset for a long time, thereby improving the model's ability to recognize pipeline inner wall cracks in various situations.

[0046] Furthermore, the execution process of the result output module is as follows:

[0047] The crack identification results obtained by the deep learning recognition module are displayed on a visual interface. On the 3D model of the pipeline, the location and range of the cracks are highlighted with different colors and markers. The direction of the cracks is indicated by red lines, and the severity of the cracks is indicated by different shades of color, making it easier for inspectors to intuitively understand the distribution of cracks on the inner wall of the pipeline.

[0048] At the same time, an inspection report is generated, which includes the number of cracks, average length, maximum width, and location coordinate information in the pipeline; these reports are saved in the form of text files or spreadsheets to provide accurate data support for subsequent pipeline maintenance and repair work.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] (1) The present invention significantly improves the automation level and accuracy of crack detection on the inner wall of pipelines. It uses an autonomous mobile endoscope device in conjunction with stereoscopic lighting and multi-angle image acquisition to ensure that high-quality images of the inner wall of pipelines can be obtained even in complex environments. Through the calibration, registration, noise suppression and contrast enhancement of the image preprocessing module, it provides high-quality input data for the deep learning model, thereby improving the accuracy and reliability of crack identification.

[0051] (2) The present invention automatically adjusts the illumination angle and brightness through an intelligent algorithm, effectively responding to the geometric shape and reflective characteristics of the pipeline inner wall, and achieving a uniform and optimized lighting effect. This illumination adjustment strategy improves the clarity and detail integrity of the image, especially in the flat areas and complex shape areas of the pipeline inner wall. It can provide uniform illumination, further improving the accuracy of crack recognition. At the same time, the advanced image processing technology of the image preprocessing module, such as noise suppression based on wavelet transform and histogram equalization, enhances the crack characteristics and improves the recognition efficiency.

[0052] (3) In the present invention, the deep learning recognition module adopts an improved convolutional neural network model, which effectively captures the characteristics of cracks of different sizes and shapes through multi-scale feature fusion and attention mechanism, thereby improving the ability to recognize cracks in complex situations. The result output module displays the crack recognition results in an intuitive manner on the visual interface and generates a detection report containing detailed crack information, providing accurate data support for pipeline maintenance and repair. This intuitive result display and detailed report generation greatly facilitate the analysis and subsequent processing of cracks on the inner wall of the pipeline by detection personnel, thereby improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0054] Figure 1 This is the overall system block diagram of the present invention. DETAILED DESCRIPTION

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] It should be understood that the terms “include” and “comprising” used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0057] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0058] like Figure 1 As shown in FIG, the automatic recognition system for cracks on the inner wall of an endoscope pipe based on video deep learning includes an image acquisition module, an image preprocessing module, a pipe inner wall model building module, a deep learning recognition module, and a result output module;

[0059] The image acquisition module uses an autonomous mobile endoscope to capture images inside the pipeline. Its multi-wheel drive structure, adaptive rubber tracks, and high-precision sensors ensure a stable acquisition process. A stereo lighting device and multi-angle image acquisition module ensure high-quality, all-round images of the pipeline's inner wall.

[0060] Image data from within the pipeline is collected through an autonomously moving endoscope. The endoscope utilizes a multi-wheel drive structure equipped with adaptive rubber tracks that adapt to the inner walls of pipelines of varying diameters and materials. The endoscope maintains a stable speed and posture during movement, minimizing the impact of vibration and shaking on image quality. High-precision gyroscopes and accelerometers are installed on the endoscope to monitor its position and motion within the pipeline in real time, transmitting this information via wireless communication.

[0061] The endoscope device is equipped with a three-dimensional lighting device, which consists of multiple LED light sources with independently controllable angles and brightness. These light sources are distributed around the endoscope device and automatically adjust the lighting angle and brightness according to the geometric shape and reflective characteristics of the inner wall of the pipe through an intelligent algorithm. The lighting angle adjustment process is as follows:

[0062] Angle adjustment based on normal vector: Assume that the normal vector of the inner wall of the pipe at a certain point is The position coordinates of the endoscope device in the pipeline are (x, y, z); for each LED light source i, its initial direction vector is According to the principle of light reflection, the standard lighting angle makes the angle between the light and the normal vector perpendicular to obtain the strongest reflected light; calculate the dot product of each light source direction vector and the normal vector where θ i is the angle between the light source and the normal vector; by adjusting the angle of the light source, cosθ iApproaches 0 (i.e. the angle approaches 90°); uses the gradient descent optimization algorithm to adjust to minimize |cosθ i |, the update formula is Where α is the learning rate; Angle adjustment based on ray tracing: Considering the multiple reflections of light on the inner wall of the pipe, light is emitted from each LED light source to simulate the propagation path of light in the pipe. When the light encounters the inner wall of the pipe, the direction of the reflected light is calculated based on the relationship between the incident angle and the reflection angle. By tracing the propagation path of the light, it is analyzed which areas receive substandard lighting or have shadows. The angle of the light source is adjusted based on the analysis results so that the light can more evenly cover the inner wall of the pipe. In areas with flat inner walls of the pipe, the normal vector-based angle adjustment algorithm is first used to quickly adjust the light source angle to a direction close to perpendicular to the inner wall, providing a good foundation for lighting. When encountering curved parts, connecting parts, or areas with special geometric shapes of the pipe, the ray tracing algorithm is enabled again. The reflection and propagation path of light on the complex inner wall are more carefully considered, so that the light source angle can be more accurately adjusted to ensure that these special areas also receive uniform lighting. This combination fully utilizes the advantages of both algorithms, ensuring the accuracy of the basic lighting angle while being able to cope with complex situations.

[0063] The light intensity adjustment process is as follows: Brightness adjustment based on the illumination model: According to the inverse square law of illumination Where E is the illuminance, I is the intensity of the light source, r is the distance from the light source to the illuminated point, and θ is the angle between the light source and the normal of the illuminated surface. For each point P on the inner wall of the pipe, calculate the distance r from it to each LED light source. i and the angle θ i Given the initial intensity I0 of each light source, calculate the theoretical illuminance at that point Set a target illuminance E target , according to the actual measured illuminance E actual (obtained by the light sensor installed on the endoscope device) and calculate the brightness adjustment coefficient Then, adjust the brightness of each light source according to I i =kI0. When encountering a curved pipe section, the distance r closer to the inner wall becomes smaller, and the E calculated according to the above formula will increase. At this time, the brightness I of the light source on this side is reduced (i.e., k is reduced) to maintain the balance of illumination. Brightness adjustment based on the image grayscale histogram: The grayscale histogram H(k) of the image captured by the camera is obtained in real time, where k is the grayscale level. The mean of the grayscale histogram is calculated. and standard deviation Set the target mean μ target and standard deviation σ target , calculate the brightness adjustment parameters according to the current μ and σ, specifically using the linear mapping function Lnew =aL old +b, where L old is the brightness of the original image, L new is the adjusted brightness, a and b are based on μ, σ, μ target and σ target The calculated coefficient; the overall brightness of the image is changed by adjusting the brightness parameter, and at the same time, it is fed back to the LED light source control system to adjust the brightness of the light source to obtain appropriate lighting conditions; for the curved pipe section, the brightness of the light source at the corresponding position is adjusted in a targeted manner according to the changes in the grayscale histogram of different areas in the image; in the initial stage, the basic brightness of the light source is set according to the illumination model, and the brightness of each LED light source is preliminarily determined according to the inverse square law of illumination by calculating the distance and angle from the light source to each point on the inner wall of the pipe, so that the entire inner wall of the pipe has a uniform illumination basis at a macro level; then, during the image acquisition process, the grayscale histogram of the image captured by the camera is analyzed in real time. When it is found that the overall brightness or local brightness of the image does not meet expectations, the brightness adjustment method based on the image grayscale histogram is used for fine-tuning; this combination can be dynamically optimized according to the visual effect of the actual captured image based on the consideration of the physical illumination model, ensuring that images with good clarity and detail integrity can be obtained under different pipeline conditions.

[0064] A multi-angle image acquisition module is constructed, including multiple high-resolution, high-frame-rate cameras. These cameras are installed at different positions and angles of the endoscope device and can simultaneously capture images of the inner wall of the pipeline from different perspectives. The camera lens adopts a special optical design with a large depth of field and low distortion, which can clearly capture images of the inner wall of the pipeline from close to long distances. At the same time, the camera's acquisition frame rate is adaptively adjusted according to the pipeline crawling speed and detection requirements, up to 120 frames per second, to ensure that any possible crack information is not missed.

[0065] The image preprocessing module calibrates and registers the collected multi-angle images to ensure geometric accuracy, while suppressing noise and enhancing contrast, providing high-quality input data for subsequent deep learning models;

[0066] Image calibration and registration: Calibrate and register images collected from multiple angles. Using known camera installation positions and angles, geometric transformation algorithms map images from different perspectives into a unified coordinate system. Image distortion caused by factors like lens distortion is corrected to ensure geometric accuracy and provide accurate input data for subsequent deep learning models.

[0067] Noise suppression and enhancement: Image noise suppression technology is used, combined with the characteristics of pipeline inner wall images, to effectively remove noise from the image. A method based on wavelet transform and local adaptive thresholding is used to suppress noise while retaining the edge information of the cracks. In addition, histogram equalization and contrast stretching algorithms are used to enhance the contrast of the image, making the cracks more prominent in the image and easier to identify.

[0068] The pipeline inner wall modeling module eliminates duplicate image data through feature point extraction and matching, image registration, and other methods. It then selects the optimal image based on evaluation parameters and uses a suitable algorithm to convert the processed image into point cloud data. Through filtering, surface reconstruction, and texture mapping, it constructs a model that accurately reflects the actual appearance of the pipeline inner wall. The specific process is as follows:

[0069] Eliminate duplicate image data: Use the Scale Invariant Feature Transform (SIFT) or Speeded Up Robust Features (SURF) algorithm to extract feature points from each image in the image data. These feature points represent representative local structural information in the image and are invariant to image rotation, scaling, illumination changes, etc.; match feature points between adjacent images, calculate the distance and similarity between feature points, find corresponding feature point pairs in different images, use the nearest neighbor algorithm to determine the matching feature points, and set a threshold to eliminate false matches; based on the matched feature points, use the geometric transformation model of affine transformation or perspective transformation to align images from different perspectives in the same coordinate system so that duplicate areas can accurately correspond; in the registered images, identify and remove duplicate areas by comparing the similarity between the images and the content of the overlapping areas; use the image difference algorithm to calculate the difference between adjacent images and remove duplicate pixel information; for overlapping areas, evaluate the images in the overlapping areas and determine which image is retained as the optimal image. Image evaluation is based on a comprehensive judgment of the image evaluation parameters, which include:

[0070] Clarity parameters: including gradient amplitude, high-frequency energy and degree of focus. The gradient of the image in the horizontal and vertical directions is calculated by using the Sobel operator, Scharr operator or Laplace operator; the image is Fourier transformed, converted into the frequency domain, and the high-frequency energy is calculated to evaluate the clarity of the image; the focus evaluation function is used to measure the degree of focus of the image. Specifically, the variance method is used to calculate the variance of the grayscale value of the image pixels. The larger the variance, the more focused the image and the clearer the details. It is quantified by a value of 0-1; after normalizing the obtained gradient amplitude, high-frequency energy and degree of focus, a triangle is established with the gradient amplitude, high-frequency energy and degree of focus as the three sides of the triangle, and the area of ​​the triangle is calculated and recorded as the clarity evaluation value. As a standard for measuring clarity; Noise level parameter: By calculating the root mean square noise and signal-to-noise ratio of the image, the ratio of the root mean square noise to the signal-to-noise ratio is calculated after normalization to obtain the noise evaluation value, and this noise evaluation value is used as the standard for measuring the noise level; Color parameter: Evaluated by color saturation and color consistency. Color saturation is measured by calculating the saturation component in the HSV color space to measure the color vividness; Color consistency is measured by statistically analyzing the entropy of the color histogram; The obtained color saturation and the entropy of the color histogram are normalized and the ratio is calculated to obtain the color evaluation value, and this color evaluation value is used as the standard for measuring color parameters; The obtained clarity evaluation value, noise evaluation value and color evaluation value are then calibrated as qx, zp and ys respectively, and after normalization, they are inserted into the following formula: To obtain the comprehensive evaluation value ZGZ, where β1, β2, and β3 are the preset weight coefficients of the clarity evaluation value, noise evaluation value, and color evaluation value, respectively, and the obtained comprehensive evaluation value ZGZ is used as the standard for measuring the image selection priority evaluation; the different comprehensive evaluation values ​​obtained from the images in the overlapping area are sorted according to size, and the image with the largest comprehensive evaluation value is selected as the optimal image.

[0071] When the image data is in video format, optical flow motion estimation technology is used to estimate the motion of the object by analyzing the motion vectors of the pixels in the image. Based on the motion trajectory of the endoscope in the pipe and the optical flow information, the repeated areas between adjacent frames are determined. For the same parts in consecutive frames, only the relevant image data of one frame is retained, thereby reducing data redundancy.

[0072] Construct a model of the inner wall of a pipeline: According to the characteristics of the image data and the geometric shape of the inner wall of the pipeline, select a 3D reconstruction algorithm based on multi-view geometry, including binocular stereo vision or multi-eye stereo vision algorithm; if the image data contains sufficient texture information, use a 3D reconstruction method based on structured light; for pipeline objects, optimize the reconstruction process in combination with its geometric constraints, including utilizing the circular cross-sectional features of the pipeline to reconstruct the entire inner wall of the pipeline by analyzing and fitting images of different cross-sections; convert the processed image data into point cloud data through the 3D reconstruction algorithm. A point cloud is a collection of a series of points in a 3D space, each point containing spatial coordinates, color, and texture information; filter the generated point cloud to remove noise points and abnormal points; specific use Statistical filtering, radius filtering and other methods; surface reconstruction algorithms, including Poisson surface reconstruction or moving least squares surface reconstruction, are used to convert point cloud data into a continuous surface model. These algorithms fit a smooth surface to represent the inner wall of the pipe based on the distribution of the point cloud; during the surface reconstruction process, the connectivity and smoothness of each area of ​​the inner wall of the pipe are analyzed to ensure that the model can accurately reflect the actual appearance of the inner wall of the pipe, and the smoothness of the surface and the degree of detail retention are controlled by adjusting the parameters of the algorithm; finally, texture mapping is performed on the generated inner wall model of the pipe, and the texture information in the original image is mapped to the reconstructed surface model, so that the model can more realistically present the specific appearance of each area of ​​the inner wall of the pipe, including detailed features such as color and texture.

[0073] The deep learning recognition module is used to build an improved convolutional neural network model, which is trained with a large number of labeled data sets to enable it to accurately identify cracks on the inner wall of the pipeline and improve the ability to identify cracks in different situations;

[0074] A convolutional neural network (CNN) model for identifying cracks in pipeline walls was constructed. This model, based on a classic CNN architecture, was modified by adding convolution kernels tailored to the texture features of pipeline wall images. The network structure employed a multi-scale feature fusion approach, enabling it to simultaneously capture crack features of varying sizes and shapes. Furthermore, an attention mechanism was introduced, enabling the model to automatically focus on areas where cracks are likely to exist when processing images, improving recognition accuracy. A large number of images of pipeline wall cracks of varying types, sizes, and severity were collected as a training dataset. These images were annotated with information including the location, length, width, and direction of the cracks. Data augmentation techniques, such as rotation, flipping, and scaling, were used to increase the diversity of the dataset. By training the customized CNN model over a long period of time on a large dataset, the model's parameters were optimized, enhancing the model's ability to identify pipeline wall cracks in a variety of situations.

[0075] The result output module displays the crack identification results on a visual interface, highlights the crack location and range with different colors and markers, and generates a detection report containing detailed crack information, providing data support for pipeline maintenance and repair;

[0076] The crack identification results obtained by the deep learning recognition module are displayed on a visual interface. On the 3D pipeline model, the location and extent of cracks are highlighted with different colors and markers. For example, a red line indicates the direction of the crack, and different shades of color indicate the severity of the crack, allowing inspectors to intuitively understand the distribution of cracks on the pipeline wall. A detailed inspection report is also generated, including information such as the number of cracks, average length, maximum width, and location coordinates within the pipeline. These reports can be saved as text files or spreadsheets, providing accurate data support for subsequent pipeline maintenance and repair work.

[0077] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning, characterized by: include: The image acquisition module uses an autonomous mobile endoscope to capture images inside the pipeline, and cooperates with a stereo lighting device and a multi-angle image acquisition module to ensure the acquisition of high-quality, all-round images of the pipeline inner wall; The image preprocessing module calibrates and registers the collected multi-angle images to ensure geometric accuracy, suppress noise, and enhance contrast, providing high-quality input data for subsequent deep learning models; The pipeline inner wall modeling module eliminates duplicate image data through feature point extraction and matching, image registration, selects the optimal image based on evaluation parameters, and then uses an algorithm to convert the processed image into point cloud data. Through filtering, surface reconstruction, and texture mapping, a model reflecting the actual appearance of the pipeline inner wall is constructed. A deep learning recognition module is used to build an improved convolutional neural network model. By training it with different annotated data sets, it can identify cracks on the inner wall of the pipeline and improve its ability to identify cracks in different situations. The result output module displays the crack identification results in a visual interface, highlights the crack location and range with different colors and marks, and generates an inspection report containing detailed crack information, providing data support for pipeline maintenance and repair.

2. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 1 is characterized in that: The specific process performed by the image acquisition module is as follows: Image data from within the pipeline is collected through an autonomously moving endoscope. The device uses a multi-wheel drive structure equipped with adaptive rubber tracks to adapt to the inner walls of pipelines of varying diameters and materials. The device maintains a stable speed and posture during crawling. A gyroscope and accelerometer are installed on the endoscope to monitor its position and motion within the pipeline in real time, transmitting this information via wireless communication. The endoscope device is equipped with a three-dimensional lighting device, which consists of multiple LED light sources with independently controllable angles and brightness. These light sources are distributed around the endoscope device and automatically adjust the lighting angle and brightness according to the geometric shape and reflective characteristics of the inner wall of the pipe through an intelligent algorithm. A multi-angle image acquisition module is constructed, including multiple cameras installed at different positions and angles of the endoscope device, which can simultaneously capture images of the inner wall of the pipeline from different perspectives; the cameras have the characteristics of large depth of field and low distortion, capturing images of the inner wall of the pipeline from close to long distances; at the same time, the camera acquisition frame rate is adaptively adjusted according to the pipeline crawling speed and detection requirements.

3. The automatic identification system for inner wall cracks of endoscope tubes based on video deep learning according to claim 2 is characterized in that: The specific steps of automatically adjusting the illumination angle by the intelligent algorithm in the image acquisition module are as follows: Angle adjustment based on normal vector: Assume that the normal vector of the inner wall of the pipe at a certain point is The position coordinates of the endoscope device in the pipeline are (x, y, z); for each LED light source i, its initial direction vector is According to the principle of light reflection, the standard lighting angle makes the angle between the light and the normal vector perpendicular to obtain the strongest reflected light; Calculate the dot product of each light source direction vector and the normal vector where θ i is the angle between the light source and the normal vector; by adjusting the angle of the light source, cosθ i The difference from 0 is within the preset difference range; The gradient descent optimization algorithm is used to adjust to minimize |cosθ i |, the update formula is Where α is the learning rate; Angle adjustment based on ray tracing: Considering the multiple reflections of light on the inner wall of the pipe, each LED light source emits light to simulate the light propagation path within the pipe. When the light encounters the inner wall of the pipe, the direction of the reflected light is calculated based on the relationship between the angle of incidence and the angle of reflection. By tracing the propagation path of the light, it is analyzed to determine which areas are not receiving the required lighting or are shadowed. Based on the analysis results, the angle of the light source is adjusted to ensure that the light covers the inner wall of the pipe more evenly. In areas with smooth inner walls of the pipe, the normal-based angle adjustment algorithm is first used to adjust the light source angle to a direction perpendicular to the inner wall, providing a good foundation for lighting. When encountering curved sections, connected sections, or areas with geometric shapes in the pipe, the ray tracing algorithm is then enabled. Taking into account the reflection and propagation path of light on the inner walls of complex shapes, the light source angle can be adjusted more accurately to ensure that these areas can also receive uniform lighting.

4. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 2 is characterized in that: The specific steps of automatically adjusting the light intensity by the intelligent algorithm in the image acquisition module are as follows: Brightness adjustment based on the illumination model: according to the inverse square law of illumination Where E is the illuminance, I is the light intensity, r is the distance from the light source to the illuminated point, and θ is the angle between the light and the normal of the illuminated surface. For each point P on the inner wall of the pipe, calculate the distance r from it to each LED light source. i and the angle θ i ; Given the initial intensity I0 of each light source, calculate the theoretical illuminance at that point Set a target illuminance E target , according to the actual measured illuminance E actual , calculate the brightness adjustment coefficient Then, adjust the brightness of each light source according to I i =kI0; When encountering a curved pipe section, the distance r close to the inner wall becomes smaller, and E calculated according to the above formula will increase. At this time, the brightness I of the light source close to the inner wall is reduced to maintain the balance of illumination. Brightness adjustment based on image grayscale histogram: Get the grayscale histogram H(k) of the image captured by the camera in real time, where k is the grayscale level; calculate the mean of the grayscale histogram and standard deviation Set the target mean μ target and standard deviation σ target , calculate the brightness adjustment parameters according to the current μ and σ, specifically using the linear mapping function L new =aL old +b, where L old is the brightness of the original image, L new is the adjusted brightness, a and b are based on μ, σ, μ target and σ target The calculated coefficient is used to change the overall brightness of the image by adjusting the brightness parameter, and the brightness is fed back to the LED light source control system to adjust the brightness of the light source to obtain matching lighting conditions. For curved pipe sections, the brightness of the light source at the corresponding position is adjusted according to the changes in the grayscale histogram of different areas in the image. In the initial stage, the base brightness of the light source is set according to the illumination model. By calculating the distance and angle between the light source and each point on the inner wall of the pipe, the brightness of each LED light source is preliminarily determined according to the inverse square law of illumination, so that the entire inner wall of the pipe has a uniform illumination foundation on a macro scale. Then, during the image acquisition process, the grayscale histogram of the image captured by the camera is analyzed in real time. If the overall brightness or local brightness of the image is found to be not as expected, a brightness adjustment method based on the image grayscale histogram is used for fine-tuning. This combination, based on the consideration of the physical illumination model, is dynamically optimized according to the visual effect of the actual captured image, ensuring that images with better clarity and detail integrity can be obtained under different pipeline conditions.

5. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 1 is characterized in that: The image preprocessing module performs the following steps: Image calibration and registration: Calibrate and register images collected from multiple angles. Using known camera installation positions and angles, geometric transformation algorithms are used to map images from different perspectives into a unified coordinate system. Image distortion caused by lens distortion is corrected to ensure geometric accuracy and provide accurate input data for subsequent deep learning models. Noise suppression and enhancement: Image noise suppression technology is used, combined with the characteristics of the pipeline inner wall image, to effectively remove noise from the image. A method based on wavelet transform and local adaptive threshold is used to suppress noise while retaining the edge information of the cracks. At the same time, histogram equalization and contrast stretching algorithms are used to enhance the contrast of the image, making the cracks more prominent in the image and easier to identify.

6. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 1 is characterized in that: The specific operation steps of the pipeline inner wall model building module are as follows: Eliminate duplicate image data: Use scale-invariant feature transformation or accelerated robust feature algorithm to extract feature points from each image in the image data. Feature points are representative local structural information in the image and are invariant to image rotation, scaling, and illumination changes. Match feature points between adjacent images, calculate the distance and similarity between feature points, find corresponding feature point pairs in different images, use the nearest neighbor algorithm to determine the matching feature points, and set a threshold to eliminate false matches. Based on the matched feature points, the images are registered using geometric transformation models such as affine transformation or perspective transformation, aligning images from different perspectives in the same coordinate system so that duplicated areas can accurately correspond. In the registered images, duplicate areas are identified and removed by comparing the similarity between the images and the content of the overlapping areas. An image difference algorithm is used to calculate the differences between adjacent images and remove duplicate pixel information. For overlapping areas, the images in the overlapping areas are evaluated to determine which image is retained as the optimal image; When the image data is in video format, optical flow motion estimation technology is used to estimate the motion of the object by analyzing the motion vectors of the pixels in the image. Based on the motion trajectory of the endoscope in the pipe and the optical flow information, the repeated areas between adjacent frames are determined. For the same parts in consecutive frames, only the relevant image data of one frame is retained to reduce data redundancy. Construct the inner wall model of the pipe: Based on the characteristics of the image data and the geometry of the pipe's inner wall, a 3D reconstruction algorithm based on multi-view geometry is selected, including binocular stereo vision or multi-view stereo vision algorithms. When the texture information contained in the image data meets the requirements, a 3D reconstruction method based on structured light is used. For pipe objects, the reconstruction process is optimized based on their geometric constraints, including the circular cross-section of the pipe, which is used to reconstruct the entire pipe's inner wall by analyzing and fitting images of different cross-sections. Through the three-dimensional reconstruction algorithm, the processed image data is converted into point cloud data. The point cloud is a collection of a series of points in three-dimensional space, each point contains spatial coordinates and color and texture information; the generated point cloud is filtered to remove noise points and abnormal points; specifically, statistical filtering and radius filtering methods are used for filtering.

7. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 6 is characterized in that: The specific steps for evaluating the image of the overlapping area in the pipeline inner wall model building module are as follows: Image evaluation is performed by comprehensively judging the image through the evaluation parameters obtained. The evaluation parameters include: Clarity parameters: including gradient amplitude, high-frequency energy, and degree of focus. The gradients of the image in the horizontal and vertical directions are calculated using the Sobel operator, Scharr operator, or Laplace operator. The image is Fourier transformed to convert it to the frequency domain, and the high-frequency energy is calculated to evaluate the clarity of the image. A focus evaluation function is used to measure the degree of focus of the image. Specifically, the variance method is used to calculate the variance of the grayscale values ​​of the image pixels. The larger the variance, the more focused the image and the clearer the details. This is quantified using a value between 0 and 1. After normalizing the obtained gradient amplitude, high-frequency energy, and degree of focus, a triangle is established with the gradient amplitude, high-frequency energy, and degree of focus as the three sides of the triangle. The area of ​​the triangle is calculated and recorded as the clarity evaluation value, which is then used as the standard for measuring clarity. Noise level parameter: By calculating the root mean square noise and signal-to-noise ratio of the image, and then calculating the ratio of the root mean square noise to the signal-to-noise ratio after normalization, the noise evaluation value is obtained, and this noise evaluation value is used as the standard for measuring the noise level; Color parameters: Evaluated by color saturation and color consistency. Color saturation measures color vividness by calculating the saturation component in the HSV color space. Color consistency is measured by calculating the entropy of the color histogram. The obtained color saturation and the entropy of the color histogram are normalized and then the ratio is calculated to obtain the color evaluation value, which is used as the standard for measuring color parameters. The obtained clarity evaluation value, noise evaluation value, and color evaluation value are calibrated as qx, zp, and ys, respectively, and after normalization, they are entered into the following formula: To obtain the comprehensive evaluation value ZGZ, where β1, β2, and β3 are the preset weight coefficients of the clarity evaluation value, noise evaluation value, and color evaluation value respectively, and the obtained comprehensive evaluation value ZGZ is used as the standard for measuring the image selection priority evaluation; The different comprehensive evaluation values ​​obtained from the images in the overlapping area are sorted by size, and the image with the largest comprehensive evaluation value is selected as the optimal image.

8. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 1 is characterized in that: The execution process of the deep learning recognition module is as follows: A convolutional neural network model for identifying cracks on pipeline walls was constructed. This model was improved upon the classic CNN architecture by adding convolution kernels tailored to the texture features of pipeline wall images. The network structure employed a multi-scale feature fusion approach to simultaneously capture crack features of varying sizes and shapes. Furthermore, an attention mechanism was introduced to enable the model to automatically focus on cracked areas when processing images, improving recognition accuracy. Collect images of pipeline inner wall cracks of different types, sizes, and severities as a training dataset; annotate these images with information including the location, length, width, and direction of the cracks; use data augmentation technology to increase the diversity of the dataset; and optimize the model parameters by training the customized model on a large-scale dataset for a long time, thereby improving the model's ability to recognize pipeline inner wall cracks in various situations.

9. The automatic identification system for cracks on the inner wall of an endoscope tube based on video deep learning according to claim 1 is characterized in that: The execution process of the result output module is as follows: The crack identification results obtained by the deep learning recognition module are displayed on a visual interface. On the 3D model of the pipeline, the location and range of the cracks are highlighted with different colors and markers. The direction of the cracks is indicated by red lines, and the severity of the cracks is indicated by different shades of color, making it easier for inspectors to intuitively understand the distribution of cracks on the inner wall of the pipeline. At the same time, an inspection report is generated, which includes the number of cracks, average length, maximum width, and location coordinate information in the pipeline; these reports are saved in the form of text files or spreadsheets to provide accurate data support for subsequent pipeline maintenance and repair work.

Images