A pipeline weak defect enhancement method, device, equipment, medium and product

Abstract

The present application relates to the technical field of pipeline detection, and discloses a pipeline weak defect enhancement method, device, equipment, medium and product, comprising: acquiring an original defect image of a pipeline, and obtaining a standard defect image through a dark channel prior algorithm; performing multi-scale division on the image to obtain a plurality of standard defect image blocks of each scale; determining a preset feature dictionary according to the material of the defect, and performing sparse coding on the standard defect image blocks according to the defect features in the dictionary to obtain the sparse coefficients corresponding to each standard defect image block under each scale; reconstructing the standard defect image blocks according to the sparse coefficients under each scale and the preset feature dictionary under each scale, splicing the reconstructed image blocks, and obtaining an enhanced image; the present application divides the defect image into multiple scales, calculates the sparse coefficients of the defect image blocks under each scale of each material, splices the defect image blocks, and obtains an enhanced image, thereby realizing the enhancement of the weak defect image.

Description

Technical Field

[0001] This invention relates to the field of pipeline inspection technology, specifically to a method, apparatus, equipment, medium, and product for enhancing pipeline micro-defects. Background Technology

[0002] As the core transportation carrier for oil and gas, water supply and drainage, and municipal engineering, pipelines are directly related to industrial production and people's livelihood. Pipelines, buried underground or exposed to complex environments for extended periods, are prone to developing minute defects such as hairline cracks and slight pitting. Due to the often harsh environments inside pipelines, including low illumination, silt cover, and pervasive moisture, the contrast of the acquired raw images is low, and these minute defect features are masked by background noise and environmental interference, resulting in low accuracy in fault identification and making them difficult to use directly for subsequent defect identification. Summary of the Invention

[0003] This invention provides a method, apparatus, equipment, medium, and product for enhancing the identification of weak defects in pipelines, in order to solve the problem that existing defect images have low accuracy in identifying weak defects and cannot be directly used for defect identification.

[0004] In a first aspect, the present invention provides a method for enhancing weak defects in pipelines, comprising: The original defect image of the pipeline is obtained, and the standard defect image is obtained by calculating the original defect image through the dark channel prior algorithm containing gradient constraint terms. The standard defect image is divided into multiple scales to obtain multiple standard defect image blocks corresponding to each scale. Based on the material of the defects in the standard defect image patch, a preset feature dictionary for multiple scales is determined. Based on the defect features in the preset feature dictionary, sparse coding is performed on the standard defect image patch to obtain the sparse coefficients corresponding to each standard defect image patch at each scale. The sparse coding is calculated based on structural similarity constraints and gradient constraints. Based on the sparsity coefficients and preset feature dictionaries at each scale, standard defect image blocks are linearly combined and reconstructed, and the reconstructed image blocks at each scale are stitched together and fused to obtain an enhanced image.

[0005] The dark channel prior algorithm, which includes gradient constraints, removes environmental interference such as water vapor and silt. It also uses gradient information to protect the edge details of minor defects, preventing slight defects from being erased during the dehazing process. Furthermore, the material-adaptive preset feature dictionary ensures that the defect features of pipes made of different materials can be accurately matched, avoiding errors caused by feature mismatch. Finally, by using sparse coding to calculate sparse coefficients, the defect image patch is reconstructed, thereby enhancing the minor defects and providing clear image information for subsequent defect identification.

[0006] In one optional implementation, the original defect image of the pipeline is obtained, and a standard defect image is obtained by calculating the original defect image using a dark channel prior algorithm that includes gradient constraint terms, including: Dark channel calculations are performed on the original defect image to obtain a dark channel image, and atmospheric light value data are determined based on the brightness of each pixel in the dark channel image. The gradient optimization term is determined based on the gradient value and gradient constraint coefficient of each pixel in the original defect image. Transmittance data for each pixel is obtained by using atmospheric light value data and gradient optimization terms to calculate transmittance. Based on the transmittance data and atmospheric light value data of each pixel, denoising calculations are performed on each pixel in the original defect image. Finally, the denoised pixels are stitched together to obtain a standard defect image.

[0007] First, the ambient light interference is objectively quantified by calculating the dark channel and estimating the atmospheric light value. Then, a gradient optimization term is introduced to participate in the transmittance calculation. This enables adaptive weakening of the dehazing intensity at defect edges with large gradients to preserve the subtle defect details at the edges, and normal dehazing at flat backgrounds with small gradients to fully remove water vapor interference. Finally, the reverse restoration from the original image to a clean image is completed by using transmittance and atmospheric light value, resulting in a standard defect image that can simultaneously meet the requirements of removing interference and protecting details.

[0008] In one optional implementation, standard defect image blocks are sparsely encoded according to defect features in a preset feature dictionary to obtain sparse coefficients corresponding to each standard defect image block at each scale, including: Based on the pixel differences, structural similarities, and gradient differences between each standard defect image block and its corresponding reconstructed image block, the fidelity term, structural similarity constraint term, and gradient constraint term are determined respectively. The sparse coding objective function is determined by the sum of the fidelity term, the structural similarity constraint term, and the gradient constraint term. The sparse coding objective function is solved iteratively to obtain the sparse coefficients corresponding to each standard defect image block that minimizes the value of the sparse coding objective function.

[0009] By incorporating structural similarity constraints and gradient constraints into the sparse coding objective function, the process of solving sparse coefficients is simultaneously guided by the fidelity terms, structural similarity constraints, and gradient constraints. This ensures that the reconstructed image patch maintains high consistency with the original image patch in three dimensions: grayscale value, texture structure, and defect edges, effectively avoiding structural distortion and edge blurring problems.

[0010] In one alternative implementation, the method further includes: The sparse coefficient values ​​are filtered based on the sparsity threshold. Sparse coefficient values ​​with absolute values ​​less than the sparsity threshold are set to zero, resulting in the filtered sparse coefficients.

[0011] In one optional implementation, the method for constructing the preset feature dictionary includes: Obtain a set of training image patches, as well as the position information of each training image patch in the set and the defect probability of each position in the pipeline. All training image patches in the set have the same scale and the same material. Based on the location information of each training image block and the defect probability at each location within the pipeline, the defect probability of each training image block is determined, and the spatial position weight of each training image block is determined based on the defect probability and the distance from the central axis of the pipeline. Linear reconstruction is performed on each training image patch using the dictionary matrix to be optimized and the sparse coefficient matrix to be optimized to obtain the reconstructed image patch, and the reconstruction error is determined based on the reconstructed image patch and the training image patch. Based on the product of the reconstruction error of each training image patch and the spatial location weight, as well as the sparsity of the sparse coefficients, a dictionary optimization objective function is constructed. The dictionary optimization objective function is solved iteratively to obtain the optimal dictionary matrix and optimal sparsity coefficients that minimize the sum of reconstruction error and sparsity. The optimal dictionary matrix is ​​then used as the preset feature dictionary.

[0012] By determining the spatial position weights based on the defect probability and distance from the central axis of the pipeline at the location of each training image patch, the dictionary training process can prioritize training the defect region. That is, the error of the high-weight region is amplified, while the error of the low-weight region is reduced. By using the dictionary matrix to be optimized and the sparse coefficient matrix for linear reconstruction and constructing a dictionary optimization objective function that combines weighted reconstruction error and sparsity, the dictionary training is guided by both the accuracy of defect image reconstruction and the simplicity of sparse coefficients. The optimal dictionary matrix is ​​obtained through iterative solution and saved as a preset feature dictionary. This ensures that the dictionary atoms are optimized for defect features in three dimensions: material, scale, and defect location, making the optimized feature dictionary more comprehensive and accurate.

[0013] In one alternative implementation, the method further includes: Obtain the pixel mean of the enhanced image; The fine-tuning term is determined by multiplying the difference between each pixel in the enhanced image and the pixel mean with the gain coefficient. The enhanced image is then optimized based on the fine-tuning terms to obtain the optimized enhanced image.

[0014] By introducing adaptive gain adjustment after image stitching, the average pixel value is calculated based on the brightness distribution of each image as a baseline. Brighter areas are brightened and darker areas are darkened, thereby increasing the contrast between the defect area and the background area in a targeted manner. This makes the originally subtle defects more visually prominent and improves the accuracy of subsequent defect recognition.

[0015] Secondly, the present invention provides a pipeline weak defect enhancement device, comprising: The image acquisition module is used to acquire the original defect image of the pipeline and to calculate the standard defect image by using a dark channel prior algorithm that includes gradient constraint terms. The scale segmentation module is used to perform multi-scale segmentation on the standard defect image to obtain multiple standard defect image blocks corresponding to each scale. The sparse coefficient matrix confirmation module is used to determine the corresponding preset feature dictionaries for multiple scales based on the material of the defects in the standard defect image blocks, and to perform sparse coding on the standard defect image blocks based on the defect features in the preset feature dictionaries to obtain the sparse coefficients corresponding to each standard defect image block at each scale. The sparse coding is calculated based on structural similarity constraints and gradient constraints. The image reconstruction module is used to linearly combine and reconstruct standard defect image blocks according to the sparse coefficients and preset feature dictionaries at each scale, and then stitch and fuse the reconstructed image blocks at each scale to obtain an enhanced image.

[0016] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform a pipeline weak defect enhancement method of the first aspect or any corresponding embodiment described above.

[0017] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform a pipeline weak defect enhancement method according to the first aspect or any corresponding embodiment described above.

[0018] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute a pipeline weak defect enhancement method as described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0019] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of a pipeline weak defect enhancement method according to an embodiment of the present invention; Figure 3 This is a structural block diagram of a pipeline weak defect enhancement device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0023] As an optional application scenario of this invention, such as Figure 1 As shown, this pipeline weak defect enhancement system may include at least one terminal device and at least one server. Figure 1 The system is illustrated in the example, which includes a computer 101, a mobile terminal 102, and a server 103, and the terminal devices such as the computer 101 and the mobile terminal 102 are connected to the server 103 through a network 110.

[0024] Specifically, the terminal device can be a smartphone, tablet, laptop, PDA, desktop computer, game console, smart TV, smart wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, etc. Server 103 can be a standalone physical server, a server cluster, a distributed system, or a cloud server providing cloud services. Network 110 can be a wired or wireless network, examples of which include, but are not limited to, the Internet, corporate intranet, local area network, wide area network, mobile communication network, and combinations thereof.

[0025] This invention provides a method for enhancing weak defects in pipelines. The method involves dividing the defect image into multiple scales, calculating the sparsity coefficient of the defect image blocks at each scale for each material, and then stitching the defect image blocks together to obtain an enhanced image. This method aims to enhance the weak defect image and make the weak defect features clearer.

[0026] According to an embodiment of the present invention, a method for enhancing weak defects in a pipeline is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] This embodiment provides a method for enhancing weak defects in pipelines. Figure 2 This is a flowchart of a pipeline weak defect enhancement method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the original defect image of the pipeline, and calculate the standard defect image by using the dark channel prior algorithm containing gradient constraint terms.

[0028] The original defect image refers to the raw image of the pipe's inner wall directly acquired by the pipeline robot using its high-sensitivity camera, containing background noise and environmental interference. The gradient constraint term incorporates image gradient information—the degree of grayscale change between a pixel and its surrounding pixels—into the transmittance calculation, adaptively reducing the dehazing effort in the defect edge region, thus preserving subtle edge details during dehazing. The dark channel prior algorithm is used to dehaze and remove interference from the original defect image, resulting in a clear base image. The standard defect image refers to the defect image after denoising using the dark channel prior algorithm.

[0029] Step S202: Divide the standard defect image into multiple scales to obtain multiple standard defect image blocks corresponding to each scale.

[0030] Multi-scale partitioning refers to the operation of dividing a preprocessed standard defect image into multiple local image blocks according to several preset scales. A standard defect image block refers to the local images formed at different scales after the standard defect image has been partitioned into multiple scales, and it is the basic computational unit for subsequent sparse coding and reconstruction processing.

[0031] Because of the large size differences in subtle defects, multi-scale segmentation allows for the comprehensive capture of defect features of different sizes. Smaller scales are used to capture details, while larger scales are used to extract contours, effectively reducing computational load. Without multi-scale segmentation, multi-scale feature extraction is impossible, and the computational load is high and efficiency is low.

[0032] For example, the multi-scale approach can be divided into three scales: small scale, medium scale, and large scale. Three scale levels are set: small scale (8×8 pixels), medium scale (16×16 pixels), and large scale (32×32 pixels). The block size at each scale is half the size of the corresponding scale (i.e., 4, 8, and 16 pixels), ensuring that overlapping areas retain feature correlation, resulting in image patch sets at each scale. ,in For an 8×8 block set, For a 16×16 block set, The set is a 32×32 block set. The pixel size division is only for example and is not a limitation.

[0033] Step S203: Based on the material of the defects in the standard defect image blocks, determine the corresponding preset feature dictionaries for multiple scales, and perform sparse coding on the standard defect image blocks according to the defect features in the preset feature dictionaries to obtain the sparse coefficients corresponding to each standard defect image block at each scale. The sparse coding is calculated based on the structural similarity constraint term and the gradient constraint term.

[0034] The material of the defect refers to the manufacturing material of the pipe's inner wall, which can include various materials such as concrete, PVC, and cast iron. Each material corresponds to multiple pre-defined feature dictionaries at various scales. The material determines the type of dictionary used. Through material recognition, the dictionary corresponding to the material is called from the pre-defined dictionary library for sparse coding and reconstruction, avoiding mismatch problems caused by different materials. The pre-defined feature dictionary refers to a defect feature template library independently constructed for each pipe material and each scale during the training phase. Each dictionary consists of a pre-defined number of atoms, and each atom is a vector with the same dimension as the image patch. As an example, the pre-defined number can be 256. Sparse coding refers to the process of solving for the optimal sparse coefficients for each standard defect image patch under the condition of a fixed pre-defined feature dictionary, so that the reconstructed image patch has a high degree of consistency with the standard defect image patch in the three dimensions of pixels, structure, and edges. Sparse coefficients are weight vectors obtained through sparse coding, used to measure the contribution of each dictionary atom to the reconstruction of the current image patch. Each image patch has a sparse coefficient value with a preset number of dimensions. According to the example, it can be 256 dimensions, that is, the sparse coefficient is composed of sparse coefficient values ​​with 256 dimensions.

[0035] The similarity constraint term is used to ensure that the reconstructed image and the original image maintain consistency in overall texture, brightness, and contrast structure, while the gradient constraint term is used to ensure that the reconstructed image and the original image maintain consistency in sharpness at defect edge locations.

[0036] Step S204: Based on the sparsity coefficients at each scale and the preset feature dictionary at each scale, the standard defect image blocks are linearly combined and reconstructed, and the reconstructed image blocks at each scale are stitched and fused to obtain an enhanced image.

[0037] Using sparse coefficients obtained from sparse coding at various scales as weights, feature atoms at the corresponding scales in the pre-defined feature dictionary are superimposed and combined according to their weights to reassemble a standard defect image patch at the current scale. Then, the image patches reconstructed at multiple scales are combined into a complete image, resulting in three enhanced images of the same size but with different focuses. These enhanced images are then weighted and averaged pixel-by-pixel to form a final fused image. The enhanced image refers to a clear image output after preprocessing, multi-scale sparse coding reconstruction, and weighted fusion of the original defect image, where defect features are enhanced and background noise is suppressed.

[0038] For example, reconstructing any image patch by calling atoms from the dictionary based on sparse coefficients can be represented as:

[0039] in, For reconstructed image patches at k scales, For a feature dictionary with k scales, represents the sparsity coefficients at scale k. As an example, k can be 1, 2, or 3, where k=1 represents a small scale, k=2 represents a medium scale, and k=3 represents a large scale.

[0040] After stitching together reconstructed image patches of the same scale to obtain a complete reconstructed image, the reconstructed images at different scales are then fused, which can be represented as:

[0041] in, To enhance the image, This is the reconstructed image after small-scale stitching. This is a reconstructed image after mesoscale stitching. This is a reconstructed image after large-scale stitching, where the weight coefficients are set according to the defect representation ability of each scale, with the highest weight for the medium scale to balance details and global features.

[0042] This embodiment provides a method for enhancing subtle defects in pipelines. It removes environmental interference such as water vapor and silt using a dark channel prior algorithm incorporating gradient constraints, and utilizes gradient information to protect the edge details of subtle defects, preventing minor defects from being erased during defogging. Furthermore, a material-adaptive preset feature dictionary ensures accurate matching of defect features from pipelines of different materials, avoiding errors caused by feature mismatch. Finally, sparse coding is used to calculate sparse coefficients and reconstruct the defect image patch, thus enhancing subtle defects and providing clear image information for subsequent defect identification.

[0043] In an optional embodiment, step S201 involves acquiring the original defect image of the pipeline and calculating a standard defect image using a dark channel prior algorithm that includes gradient constraint terms. Specifically, this includes: Step a1: Perform dark channel calculation on the original defect image to obtain the dark channel image, and determine the atmospheric light value data based on the brightness of each pixel in the dark channel image.

[0044] Dark channel computation involves taking a local neighborhood centered on each pixel in the original defective image, identifying all pixel values ​​within that neighborhood across the R, G, and B color channels, and using the minimum value as the dark channel value for that pixel. This results in a dark channel map reflecting the "fog density" across the image. The local neighborhood is typically 3×3, which, compared to the traditional 5×5 neighborhood used in dark channel priors, reduces the neighborhood size to preserve subtle defect details. The traditional 5×5 neighborhood over-averages the defective pixels with the surrounding background pixels, masking defect details. Reducing the neighborhood allows for precise capture of subtle grayscale differences between the defect and the background, thus preserving the outline of the defect. Atmospheric light value data refers to the parameters used in the dark channel prior algorithm to estimate the ambient light intensity, representing the original pixel brightness values ​​of the foggested areas in the image.

[0045] For example, the formula for calculating the dark channel can be expressed as:

[0046] in, For the dark channel image at pixel position The value at that location, Represents the R, G, and B channels. For the original defect image in pixels The brightness value of color channel c at the location. For The 3×3 neighborhood centered on the center.

[0047] For example, the 0.1% of pixels with the highest brightness in the dark channel are selected, and the maximum brightness value of these pixels in the original image is used as the atmospheric light value. For pipeline scenarios, The value range is controlled within [200, 255] to avoid excessive dehazing leading to distortion of defect features.

[0048] Step a2: Determine the gradient optimization term based on the gradient value and gradient constraint coefficient of each pixel in the original defect image.

[0049] The gradient optimization term is used in the transmittance calculation of the dark channel prior algorithm to adjust the dehazing intensity according to the gradient value of each pixel in the image, so that the dehazing intensity of the defect edge area is weakened, thereby preserving the edge details of minor defects while removing environmental interference such as water vapor and silt.

[0050] For example, the gradient optimization term can be expressed as: ,in, for The gradient value of a pixel. This is the gradient constraint coefficient, which can take a value of 0.05.

[0051] The gradient value of a pixel can be calculated in the following way:

[0052] in, for The gradient value of a pixel. For horizontal gradient, The vertical gradient is denoted as: The horizontal gradient can be represented as: The vertical gradient can be expressed as: .

[0053] Step a3: Calculate the transmittance using atmospheric light value data and gradient optimization terms to obtain the transmittance data for each pixel.

[0054] The transmittance data of each pixel is used to measure the degree of contamination of the original image by environmental interference such as water vapor and fog at each pixel location. Finally, a clear image without fog is restored through reverse calculation.

[0055] For example, the transmittance data of each pixel can be calculated in the following way:

[0056] in, for Pixel transmittance data, This is the dehazing weighting factor, and its value can be 0.85. For the original image, This represents the atmospheric light value corresponding to color channel c. This is a gradient optimization term.

[0057] Step a4: Based on the transmittance data and atmospheric light value data of each pixel, perform denoising calculations on each pixel in the original defect image, and finally stitch together the denoised pixels to obtain a standard defect image.

[0058] For example, denoising can be calculated according to the following formula:

[0059] in, After noise reduction Pixels For the original image, Atmospheric light value, for Pixel transmittance data, This represents the minimum transmittance, which can be 0.1.

[0060] This embodiment provides a method for enhancing subtle defects in pipelines. First, it objectively quantifies ambient light interference through dark channel calculation and atmospheric light value estimation. Then, it introduces a gradient optimization term to participate in transmittance calculation. This achieves adaptive weakening of dehazing at defect edges with large gradients to preserve subtle defect details at the edges, and normal dehazing at flat backgrounds with small gradients to fully remove water vapor interference. Finally, it completes the reverse restoration from the original image to a clean image through transmittance and atmospheric light value, obtaining a standard defect image that can simultaneously meet the requirements of interference removal and detail preservation.

[0061] In an optional embodiment, step S203 involves sparsely encoding standard defect image blocks based on defect features in a preset feature dictionary to obtain sparse coefficients corresponding to each standard defect image block at each scale. Specifically, this includes: Step b1: Based on the pixel differences, structural similarities, and gradient differences between each standard defect image block and the corresponding reconstructed image block, determine the fidelity term, structural similarity constraint term, and gradient constraint term, respectively.

[0062] Pixel difference refers to the difference in grayscale values ​​at each corresponding pixel location between the original image patch and the reconstructed image patch. The smaller the error, the closer the reconstructed image is to the original image at the pixel level. Structural similarity refers to the overall similarity between the original image patch and the reconstructed image patch in terms of brightness, contrast, and texture structure. Gradient difference refers to the difference in the sharpness of edges between the original image patch and the reconstructed image patch. If the edges of the reconstructed image are blurrier than those of the original image, large gradient values ​​will appear in the residual at the edge locations.

[0063] For example, a fidelity item can be represented as: ,in, For a single image patch, For feature dictionary, is the sparsity coefficient.

[0064] The structural similarity constraint term can be expressed as: ,in, The structural similarity (SSIM) constraint coefficients are... For a single image patch, For feature dictionary, The sparsity coefficient is . For the original image patch With reconstructed image patches Structural similarity.

[0065] The gradient constraint term can be expressed as: ,in, This is the gradient constraint coefficient, which can take a value of 0.1.

[0066] For example, structural similarity can be calculated according to the following formula. :

[0067] in, , , They are respectively , The mean, , They are variance, For covariance, , The denominator should be a constant to avoid zero.

[0068] Step b2: Determine the sparse coding objective function based on the sum of the fidelity term, the structural similarity constraint term, and the gradient constraint term.

[0069] The sparse coding objective function is used to solve for the optimal sparse coefficients for each standard defect image block under the condition of a fixed preset feature dictionary, so that the reconstructed image block achieves the best performance in three dimensions: "pixel accuracy, structural similarity, and edge fidelity". The smaller the value of the sparse coding objective function, the better the sparse coefficients are.

[0070] For example, the sparse coding objective function can be expressed as:

[0071] in, To ensure authenticity, For structural similarity constraints, This is the gradient constraint term.

[0072] Step b3 involves iteratively solving the sparse coding objective function to obtain the sparse coefficients corresponding to each standard defect image block that minimizes the value of the sparse coding objective function.

[0073] For example, the following formula can be used for iterative solution:

[0074] in, This is the updated sparse coefficient vector. This is the current sparse coefficient vector. The iteration step size, also known as the learning rate, is used to control the magnitude of each adjustment. The gradient of the objective function needs to be recalculated at each iteration. As an example, a non-negativity constraint can also be introduced simultaneously. This avoids feature distortion caused by negative coefficients.

[0075] The gradient of the objective function can be calculated using the following formula:

[0076] in, The gradient of the objective function. The derivative of the fidelity term indicates the direction in which pixel differences influence coefficient adjustments. The derivative of the structural similarity constraint term indicates the direction in which structural differences guide the adjustment of the coefficients. The derivative of the gradient constraint term is represented, and the direction in which marginal differences guide the adjustment of the coefficients is indicated. For the current reconstructed image patch, To reconstruct the residual between the image patch and the original image patch, This is the transpose of the dictionary matrix. The gradient of the structural similarity loss with respect to the sparse coefficients. The spatial gradient of the residual image reflects the difference between the reconstructed image and the original image at edge locations. Yes The derivative is taken again to map the marginal differences back to the coefficient space.

[0077] For example, the objective function value is calculated after each iteration. The convergence threshold is reached when the difference between the function values ​​of two iterations is less than the maximum number of iterations. Stop iterating when the optimal sparse coefficients are obtained. .

[0078] The difference in function values ​​during iteration being less than the convergence threshold can be expressed as:

[0079] in, After the (t+1)th iteration update, the new sparse coefficient vector Substitute the calculated function value into the sparse coding objective function. After the t-th iteration update, the new sparse coefficient vector will be... Substitute the calculated function value into the sparse coding objective function. This is the preset convergence threshold.

[0080] This embodiment provides a method for enhancing weak defects in pipelines. By incorporating structural similarity constraints and gradient constraints into the sparse coding objective function, the solution process for sparse coefficients is simultaneously guided by the fidelity terms, structural similarity constraints, and gradient constraints. This ensures that the reconstructed image patch maintains high consistency with the original image patch in three dimensions: grayscale value, texture structure, and defect edges, effectively avoiding structural distortion and edge blurring problems.

[0081] In an optional embodiment, the method further includes: Step c1: Filter the sparse coefficient values ​​in the sparse coefficients according to the sparse threshold, and set the sparse coefficient values ​​with absolute values ​​less than the sparse threshold to zero, so as to obtain the filtered sparse coefficients.

[0082] The sparsity threshold is a preset positive constant used to filter the sparse coefficient values ​​in the optimal sparse coefficients obtained by iterative solution, and to remove invalid coefficients that represent noise or irrelevant information.

[0083] For example, the sparsity threshold can be taken as follows: The absolute value is less than The coefficients are set to 0 to further suppress invalid sparse coefficient values ​​corresponding to noise, while retaining the valid coefficient values ​​corresponding to defect features, thus obtaining the final sparse coefficients. .

[0084] In an optional embodiment, the method for constructing the preset feature dictionary in step S203 specifically includes: Step d1: Obtain the training image patch set, as well as the position information of each training image patch in the training image patch set and the defect probability of each position in the pipeline. The training image patches in the training image patch set have the same scale and the same material.

[0085] The training image patch set refers to the collection of all image patches obtained at each scale after dividing the standard sample images corresponding to each material into multiple scales. This collection is used to train the dictionary for each corresponding scale. The location information of each training image patch refers to its spatial coordinates in the original complete sample image and its corresponding position in the physical space of the pipe. The defect probability at each location within the pipe refers to the likelihood of a defect occurring at different physical locations on the pipe's inner wall, obtained by statistically analyzing a large number of historical labeled samples.

[0086] For example, separate sample libraries for weak defects can be constructed for concrete, PVC, and cast iron. Each sample library can contain a preset number of samples; for instance, this could be 1000 samples, including 500 hairline cracks and 500 minor pitting corrosion samples, but this is not a limitation. The sample size can be uniformly cropped to 64×64 pixels, and each sample is normalized and its mean is zeroed out to obtain a sample matrix. ,in For the number of samples, in the previous example, , Where 4096 = 64 × 64. The multi-scale division method is the same as in step S202, and will not be repeated here.

[0087] Step d2: Determine the defect probability of each training image block based on the position information of each training image block and the defect probability of each position in the pipeline, and determine the spatial position weight of each training image block based on the defect probability and the distance from the central axis of the pipeline.

[0088] Spatial location weights refer to a comprehensive importance score assigned to each training image patch. This score guides the algorithm during dictionary training to focus on defect-prone areas and suppress interference from background and edge regions. Since the location information of image patches can be mapped to the physical location of the pipeline, and the defect probability distribution at each location within the pipeline is obtained in advance through statistical analysis of a large number of historical labeled samples, the defect locations of each pixel patch can be determined by combining these factors. The defect probability is used to prioritize the allocation of dictionary capacity during dictionary atom updates to reconstruct features of defect-prone areas, thereby ensuring that the learned dictionary atoms focus on locations within the pipeline where defects are truly likely to occur.

[0089] For example, spatial location weights can be calculated using the following formula:

[0090] in, Spatial location weights, This is the weighting coefficient, which can take a value of 0.7. This represents the defect probability weight. For defective areas, the value can be 0.9-1.0, and for non-defective areas, the value can be 0.1-0.3. As a distance-weighted system, with the pipeline's centerline as the reference, the closer to the centerline, the greater the weight. The larger the value, the greater the range (0.6-1.0), and the smaller the value in the edge region (0.1-0.5), the better to avoid edge noise interfering with dictionary generation.

[0091] Step d3: Linear reconstruction of each training image patch is performed using the dictionary matrix to be optimized and the sparse coefficient matrix to be optimized to obtain the reconstructed image patch, and the reconstruction error is determined based on the reconstructed image patch and the training image patch.

[0092] The reconstruction error is used to quantify the precision of atoms in the current dictionary and the reconstruction quality of defective regions, providing a numerical basis for subsequent column-by-column updating of dictionary atoms according to the error direction.

[0093] Step d4: Construct a dictionary optimization objective function based on the product of the reconstruction error and spatial location weight of each training image patch, and the sparsity of the sparse coefficients.

[0094] For example, the dictionary optimization objective function can be expressed as:

[0095] in, This is the spatial location weight matrix. Indicates reconstruction error, For Hadama accumulation, This is the sparsity regularization coefficient, which can take a value of 0.01. This is used to measure how many atoms are used for each image patch; the smaller the value, the better the feature extraction.

[0096] Step d5 involves iteratively solving the dictionary optimization objective function to obtain the optimal dictionary matrix and optimal sparsity coefficients that minimize the sum of reconstruction error and sparsity. The optimal dictionary matrix is ​​then used as the preset feature dictionary.

[0097] By alternating optimization, the sparse coefficients are first calculated by fixing the dictionary, and then the dictionary atoms are updated column by column by fixing the sparse coefficients. This process is repeated to continuously reduce the objective function value until the preset number of iterations is reached, at which point the objective function converges to the minimum value. The output dictionary matrix and sparse coefficients are then the optimal solution.

[0098] For example, the process of first fixing the dictionary and calculating the sparse coefficients can be performed using the Orthogonal Matching Pursuit (OMP) algorithm. After fixing the sparse coefficients, the following formula can be used to iterate over each atom in the dictionary:

[0099] in, This refers to the k-th column of the canonical matrix D, i.e., the k-th dictionary atom. This is the spatial location weight matrix, with the same size as the training sample matrix X. This is the reconstruction error generated when reconstructing all training image patches using the remaining 255 atoms after removing the contribution of the k-th atom. The transpose of the k-th row of the sparse coefficient matrix transforms the original 1×N row vector into an N×1 column vector. This is the sum of the squares of all coefficient values ​​in the current row.

[0100] For example, in each iteration of training, the current dictionary D is fixed first, and the sparse coefficients are obtained by solving the OMP algorithm for each training image patch. That is, each image patch independently selects the most matching atoms from 256 atoms and calculates the corresponding weights, so that the coefficients of all patches together form a coefficient matrix. .

[0101] Then fix the coefficient matrix Update the dictionary column by column, updating the k-th atom. When, calculate and remove The reconstruction error after contribution is decomposed using SVD and used to update atoms. Simultaneously, the coefficients of the corresponding rows are updated, and this method is used to update all 256 atoms column by column.

[0102] This embodiment provides a method for enhancing weak defects in pipelines. By determining the spatial position weights based on the defect probability and distance from the pipeline's central axis of each training image patch, the dictionary training process can prioritize training the defect region. That is, the error of high-weight regions is amplified, while the error of low-weight regions is reduced. By using the dictionary matrix to be optimized and the sparse coefficient matrix for linear reconstruction and constructing a dictionary optimization objective function that combines weighted reconstruction error and sparsity, the dictionary training is guided by both the accuracy of defect image reconstruction and the simplicity of sparse coefficients. The optimal dictionary matrix is ​​obtained through iterative solution and saved as a preset feature dictionary. This ensures that the dictionary atoms are optimized for defect features in three dimensions: material, scale, and defect location, making the optimized feature dictionary more comprehensive and accurate.

[0103] In an optional embodiment, the method further includes: Step e1: Obtain the average pixel value of the enhanced image.

[0104] The arithmetic mean of the grayscale values ​​of all pixels in an enhanced image represents the overall brightness level of the entire image.

[0105] Step e2: Determine the fine-tuning term based on the product of the difference between each pixel in the enhanced image and the pixel mean, and the gain coefficient.

[0106] The fine-tuning term is used to calculate the amount of adjustment a pixel needs to be made based on how much it deviates from the average brightness of the image.

[0107] For example, the fine-tuning term can be represented as:

[0108] in, To enhance the pixels in the image , To enhance the average pixel value of the image, This is the gain coefficient, which can take a value of 0.15.

[0109] Step e3: Enhance the image by fine-tuning the image to obtain the optimized enhanced image.

[0110] For example, enhancements and optimizations can be performed according to the following formula:

[0111] in, For the optimized enhanced image pixels, To enhance the pixels in the image , This is for fine-tuning. The optimized enhanced image is obtained by integrating and stitching together the pixels of each optimized enhanced image.

[0112] For example, the fused image blocks are stitched together, and bilinear interpolation is used to process overlapping areas between blocks to eliminate stitching artifacts. For the optimized enhanced image, signal-to-noise ratio (SNR), contrast enhancement (CR), and sharpness enhancement (ED) of defect areas can also be calculated to optimize and adjust the structural similarity constraint coefficients. or gradient constraint coefficient .

[0113] As an example, ,Target ; The target CR is ≥40%; ED is the mean gradient of the defect area edge, and the target ED is ≥20. This is the square of the signal intensity in the defect region of the image, used to measure the intensity level of the defect signal. This is the square of the intensity of the background noise region in the image, used to measure the level of noise interference. This represents the image contrast, used to quantify the difference in brightness between defects and the background in the image after enhancement processing. The contrast of the preprocessed image is used to quantify the degree of brightness difference between defects and the background in the image after the denoising process is completed, i.e. before entering dictionary reconstruction enhancement.

[0114] This embodiment provides a method for enhancing subtle defects in pipelines. By introducing adaptive gain adjustment after image stitching, the average pixel value is calculated based on the brightness distribution of each image as a baseline. Brighter areas are brightened and darker areas are darkened, thereby specifically increasing the contrast between the defect area and the background area. This makes subtle defects that are not easily noticeable more visually prominent and improves the accuracy of subsequent defect identification.

[0115] This embodiment also provides a pipeline weak defect enhancement device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated for details already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0116] This embodiment provides a device for enhancing minor defects in pipelines, such as... Figure 3 As shown, it includes: The image acquisition module 301 is used to acquire the original defect image of the pipeline and to calculate the standard defect image by using a dark channel prior algorithm containing gradient constraint terms. The scale division module 302 is used to divide the standard defect image into multiple scales to obtain multiple standard defect image blocks corresponding to each scale. The sparse coefficient matrix confirmation module 303 is used to determine the corresponding preset feature dictionary for multiple scales based on the material of the defect in the standard defect image block, and to perform sparse coding on the standard defect image block based on the defect features in the preset feature dictionary to obtain the sparse coefficients corresponding to each standard defect image block at each scale. The sparse coding is calculated based on the structural similarity constraint term and the gradient constraint term. The image reconstruction module 304 is used to perform linear combination reconstruction of standard defect image blocks based on the sparse coefficients and preset feature dictionaries at each scale, and then stitch and fuse the reconstructed image blocks at each scale to obtain an enhanced image.

[0117] The pipeline weak defect enhancement device provided in this embodiment of the invention can execute the pipeline weak defect enhancement method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0118] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0119] The following is a detailed reference. Figure 4 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0120] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0121] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a memory 408, or installed from a ROM 402. When the computer program is executed by the processor 401, it performs the functions defined in a pipeline weakness defect enhancement method according to embodiments of the present invention.

[0122] Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0123] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, it implements the pipeline weak defect enhancement method shown in the above embodiments.

[0124] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0125] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for enhancing weak defects in pipelines, characterized in that, The method includes: The original defect image of the pipeline is obtained, and the standard defect image is obtained by calculating the original defect image through the dark channel prior algorithm containing gradient constraint terms. The standard defect image is divided into multiple scales to obtain multiple standard defect image blocks corresponding to each scale. Based on the material of the defects in the standard defect image block, a preset feature dictionary for multiple scales is determined, and based on the defect features in the preset feature dictionary, sparse coding is performed on the standard defect image block to obtain the sparse coefficients corresponding to each standard defect image block at each scale. The sparse coding is calculated based on structural similarity constraints and gradient constraints. Based on the sparse coefficients at each scale and the preset feature dictionary at each scale, the standard defect image blocks are linearly combined and reconstructed, and the reconstructed image blocks at each scale are stitched and fused to obtain an enhanced image.

2. The method according to claim 1, characterized in that, Obtain the original defect image of the pipeline, and calculate the standard defect image by using a dark channel prior algorithm that includes gradient constraint terms, including: Dark channel calculation is performed on the original defect image to obtain a dark channel image, and atmospheric light value data is determined based on the brightness of each pixel in the dark channel image. The gradient optimization term is determined based on the gradient value and gradient constraint coefficient of each pixel in the original defect image; Transmittance is calculated using the atmospheric light value data and the gradient optimization term to obtain the transmittance data of each pixel; Based on the transmittance data of each pixel and the atmospheric light value data, denoising calculations are performed on each pixel in the original defect image. Finally, the denoised pixels are stitched together to obtain a standard defect image.

3. The method according to claim 1, characterized in that, Based on the defect features in the preset feature dictionary, sparse coding is performed on the standard defect image blocks to obtain the sparse coefficients corresponding to each standard defect image block at each scale, including: Based on the pixel differences, structural similarities, and gradient differences between each standard defect image block and its corresponding reconstructed image block, fidelity terms, structural similarity constraints, and gradient constraints are determined respectively. The sparse coding objective function is determined based on the sum of the fidelity term, the structural similarity constraint term, and the gradient constraint term. The sparse coding objective function is solved iteratively to obtain the sparse coefficients corresponding to each standard defect image block that minimizes the value of the sparse coding objective function.

4. The method according to claim 3, characterized in that, The method further includes: The sparse coefficient values ​​in the sparse coefficients are filtered according to the sparse threshold, and the sparse coefficient values ​​with an absolute value less than the sparse threshold are set to zero, thus obtaining the filtered sparse coefficients.

5. The method according to claim 1, characterized in that, The method for constructing the preset feature dictionary includes: Obtain a set of training image blocks, as well as the position information of each training image block in the set and the defect probability of each position in the pipeline. The training image blocks in the set have the same scale and the same material. Based on the position information of each training image block and the defect probability at each position in the pipeline, the defect probability of each training image block is determined, and based on the defect probability and the distance from the central axis of the pipeline, the spatial position weight of each training image block is determined. The training image blocks are linearly reconstructed using the dictionary matrix to be optimized and the sparse coefficient matrix to be optimized to obtain reconstructed image blocks, and the reconstruction error is determined based on the reconstructed image blocks and the training image blocks. Based on the product of the reconstruction error and spatial location weight of each training image patch, and the sparsity of the sparse coefficients, a dictionary optimization objective function is constructed. The dictionary optimization objective function is iteratively solved to obtain the optimal dictionary matrix and optimal sparsity coefficients that minimize the sum of the reconstruction error and sparsity. The optimal dictionary matrix is ​​then used as the preset feature dictionary.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the average pixel value of the enhanced image; The fine-tuning term is determined by multiplying the difference between each pixel in the enhanced image and the pixel mean with the gain coefficient. The enhanced image is optimized based on the fine-tuning terms to obtain the optimized enhanced image.

7. A device for enhancing the appearance of minor defects in pipelines, characterized in that, The device includes: The image acquisition module is used to acquire the original defect image of the pipeline and to calculate the standard defect image by using the dark channel prior algorithm containing gradient constraint terms. The scale division module is used to divide the standard defect image into multiple scales to obtain multiple standard defect image blocks corresponding to each scale. The sparse coefficient matrix confirmation module is used to determine the corresponding preset feature dictionaries for multiple scales based on the material of the defects in the standard defect image blocks, and to perform sparse coding on the standard defect image blocks based on the defect features in the preset feature dictionaries to obtain the sparse coefficients corresponding to each standard defect image block at each scale. The sparse coding is calculated based on structural similarity constraints and gradient constraints. The image reconstruction module is used to perform linear combination reconstruction of the standard defect image blocks according to the sparse coefficients at each scale and the preset feature dictionary at each scale, and to stitch and fuse the reconstructed image blocks at each scale to obtain an enhanced image.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform a pipeline weak defect enhancement method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform a pipeline weak defect enhancement method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform a pipeline weak defect enhancement method according to any one of claims 1 to 6.

Images