A method, computer device and storage medium for deduplicating similar images in a pipeline

Abstract

The application discloses a pipeline inner similar image deduplication method, a computer device and a storage medium, relates to the technical field of image processing, and aims at solving the problems that the existing image deduplication method is harsh and not robust, and the effect of processing underground pipeline environment data is poor. The application first detects a defect area in a pipeline in real time, then verifies defect type determination and relative position, then cuts the defect area, then fuses to determine similarity, and finally sorts the clarity and deduplicates according to a similar image set. The application can deduplicate similar images, does not need the same images in an image database, can well solve the problem that there is redundancy in urban underground pipeline environment data, has important practical value, and accurately realizes the deduplication of similar images of underground pipelines by combining an AI deep model and a fusion calculation method of two traditional similarities, and has strong robustness for rigid body transformation of images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method, computer device, and storage medium for deduplication of similar images within a pipeline. Background Technology

[0002] In urban underground pipeline operations, a massive amount of images of the pipeline environment needs to be collected for subsequent maintenance and repair. Therefore, underground pipeline robots are required to capture these images and identify different types of defects to facilitate subsequent operations. However, when capturing images of pipeline defects, the same defect may be photographed multiple times. Some images are taken with the industrial camera's default focus, while others are taken after digital zoom (2x or even 3x magnification). These images contain a large amount of redundant information, necessitating the removal of images of the same defect and retaining the clearest one.

[0003] According to research and analysis, current image deduplication schemes are performed on databases containing identical images. One type uses traditional cryptographic principles, while the other uses perceptual hashing. However, current schemes suffer from two serious drawbacks.

[0004] Disadvantage 1: The current solution requires that the database to be processed contains exactly the same images, which is often impossible to meet in real underground pipeline scenarios.

[0005] Disadvantage 2: The processing logic based on perceptual hashing is not robust to operations such as image translation and scaling, and is prone to failure due to rigid transformations of the image. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to provide a method, computer device, and storage medium for deduplication of similar images within pipelines, thereby solving the problems that existing image deduplication methods have stringent requirements, lack robustness, and are ineffective in processing underground pipeline environmental data.

[0007] A method for deduplicating similar images within a pipeline includes the following steps:

[0008] S1: Real-time detection of defect areas inside the pipeline;

[0009] S2: Verify whether images of defects of the same type and relative position are the same defect;

[0010] S3: Cropping the defect region from the image with the same defect obtained in step S2;

[0011] S4: Classify the cropped region images with the same defect and then perform a comprehensive similarity determination;

[0012] S5: Select a set of similar images based on the comprehensive similarity map obtained in step S4, then calculate the sharpness of each image in the set of similar images, and sort and remove duplicates from the images in the set of similar images based on the sharpness.

[0013] Preferably, step S1 includes:

[0014] The YOLO deep learning object detection algorithm is used to determine pipe defects and the areas of pipe defects in pipe images in real time.

[0015] Preferably, step S2 includes:

[0016] Based on the YOLO deep learning detection algorithm, existing defect types in pipeline images are labeled and trained. Then, the images to be tested are used for inference and discrimination to obtain the defect type and detection box position in each pipeline image. Next, the relative position of defects of the same type is verified. The verification mechanism is as follows: first, the image region is divided into eight quadrants and numbered sequentially. Then, it is judged whether the center point coordinates of the detection box of images with the same type of defect fall into the same quadrant region. If they fall into the same quadrant region, the image defects are considered to be in close positions, so the next step can be continued; otherwise, they are considered not to be the same defect, the verification image is deleted, and the process returns to the first step.

[0017] Preferably, step S3 includes:

[0018] The AI ​​deep learning detection model is used to infer and determine the defective region in the image, which is described in the form of a bounding box (bbox). Then, the region is cropped from the original image and the subsequent process continues.

[0019] Preferably, step S4 includes:

[0020] S4.1: Group the cropped region images of the same defect obtained in step S3 together, and then perform cosine similarity determination on each group; the cosine similarity determination formula is as follows:

[0021]

[0022] Where A and B represent different image vectors, A i and B i These represent the components of vectors A and B, respectively.

[0023] S4.2: Perform structural similarity judgment on the cropped region images of the same defect after calculating cosine similarity. The formula for calculating structural similarity is:

[0024] SSIM(x,y)=[l(x,y)]α ·[c(x,y)] β ·[s(x,y)] γ

[0025] Where x and y represent the two images to be compared, l(x,y) represents the brightness measurement value, c(x,y) represents the contrast measurement value, s(x,y) represents the structure measurement value, and the three parameters α, β, and γ are used to represent the importance of these three modules;

[0026] To simplify the formula, let

[0027] α=β=γ=1,C3=C2 / 2

[0028] The simplified formula for SSIM can be obtained as follows:

[0029]

[0030] Where μ x and μ y σ is the mean of the x and y values ​​of the image. xy Let C1 and C2 represent the covariance of the two images, where C1 and C2 are constants.

[0031] S4.3: The cosine similarity obtained in step S4.1 and the structural similarity obtained in step S4.2 are weighted and combined to obtain the comprehensive similarity. The calculation formula is as follows:

[0032] final_score=α×cos(x,y)+β×ssim(x,y)

[0033] Where final_score represents the combined weighted similarity value obtained by fusing the two similarity calculation methods, and α and β represent the importance of cosine similarity and structural similarity, respectively.

[0034] Preferably, step S5 includes: determining a candidate set of similar images based on comprehensive similarity and a threshold, wherein images with a comprehensive similarity greater than the threshold are selected as candidates for similar images, and images with a comprehensive similarity less than the threshold are ignored and no longer considered; if the comprehensive similarity of all images in the candidate set is less than the threshold, no further processing is performed, and the process returns to step one; for each image in the candidate set with a comprehensive similarity greater than the threshold, Laplacian filtering is performed to obtain the response variance value corresponding to each image; based on the calculated Laplacian response variance value, the images are sorted by sharpness, and the image with the highest sharpness is determined to be the one to be retained, and other images are removed.

[0035] The preferred method for dividing the space into eight quadrants is as follows:

[0036] Based on the coordinates of the center point of the image, each quadrant is divided at 45°, resulting in a total of 8 quadrants. The regions are numbered starting from 1 and going counterclockwise or clockwise from above the positive half of the x-axis.

[0037] A computer device for deduplicating similar images within a pipeline includes a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to implement a method for deduplicating similar images within a pipeline.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements a method for deduplication of similar images within a pipeline.

[0039] The beneficial effects of this invention include:

[0040] 1. It can deduplicate similar images without requiring identical images in the image database, effectively solving the problem of redundancy in urban underground pipeline environmental data, and has significant practical value.

[0041] 2. By combining AI deep models with two traditional similarity fusion calculation methods, the deduplication of similar images of underground pipelines is accurately achieved, and the images are highly robust to rigid body transformations.

[0042] 3. The overall judgment process of this patent application is very rigorous. It elaborates on the image judgment and selection logic of similar defect types from top to bottom and from large to small, as well as the acquisition of a candidate set of similar images based on comprehensive similarity, and sorting and selecting the best one based on their clarity.

[0043] 4. Compared with existing image deduplication technologies, the process of this patent application demonstrates a more accurate deduplication judgment effect and is less demanding on the image database to be deduplicated. It is more practical and easier to use in real-world scenarios. This patent application has very practical value for image acquisition and processing of urban underground pipelines and related operations. Attached Figure Description

[0044] Figure 1 This is a flowchart of a method for deduplicating similar images within a pipeline, as described in Example 1.

[0045] Figure 2 Example 1 involves the detection of defect areas (detachment) within a pipeline.

[0046] Figure 3 Example 1 involves the detection of defect areas (deposits) within a pipeline.

[0047] Figure 4 This is a schematic diagram of the relative position verification of defects involved in Example 1.

[0048] Figure 5 This is a schematic diagram of the image being divided into 8 quadrants in Example 1.

[0049] Figure 6 Example 1 involves a coordinate system and quadrant diagram representation of the defect area.

[0050] Figure 7 This is a representation of the detection frame for the defect involved in Example 1.

[0051] Figure 8 This is a schematic diagram of the defect area trimming involved in Example 1.

[0052] Figure 9 This is a 1x digital zoom image from Example 1.

[0053] Figure 10 This is a 2x digital zoom image from Example 1.

[0054] Figure 11 This is a 3x digital zoom image from Example 1.

[0055] Figure 12 Example 1 involves the cropping region roi1 corresponding to the 1x digital zoom image obtained through steps S1-S3.

[0056] Figure 13 Example 1 involves the cropping region roi2 corresponding to the 2x digital zoom image obtained after processing in steps S1-S3.

[0057] Figure 14 Example 1 involves the cropping region roi3 corresponding to the 3x digital zoom image obtained after processing in steps S1-S3. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0059] Example 1

[0060] The following is in conjunction with the appendix Figure 1 - Appendix Figure 8 Specific embodiments of the present invention will be described in detail;

[0061] S1: Real-time detection of defect areas inside pipelines

[0062] In this embodiment, two types of pipe defects are involved. The first type is "pipe detachment," such as... Figure 2 As shown, the second type is

[0063] "Pipeline deposits" such as Figure 3 As shown, the YOLO deep learning object detection algorithm can be used to determine pipe defects and the areas of pipe defects in images in real time.

[0064] S2: Defect Type Determination and Relative Location Verification

[0065] Using the YOLO detection algorithm based on deep learning, two existing defect types are labeled and trained. Then, the images to be tested are used for inference and discrimination to obtain the defect type and detection box position in each pipeline image. Finally, the relative positions of the identified defects of the same type are verified.

[0066] Verification mechanism Figure 4 As shown: The image area is divided into eight quadrants, numbered as shown in the figure. The red boxes in the figure represent defect areas, and the red dots represent the center points of the detection boxes. When the center point coordinates of two defect detection boxes of the same type fall within the same quadrant area, it is determined that the defect locations are relatively close, so the next step can be carried out; otherwise, it is considered that they are not the same defect.

[0067] The method for dividing the space into eight quadrants is as follows:

[0068] Using the center point coordinates of the image as a reference, divide the image into eight quadrants, each space representing a 45° angle. The quadrants are numbered counter-clockwise from the positive x-axis, starting from 1 and going up to 8. This completes the division of the image into eight quadrants. (See reference for details.) Figure 5 As shown.

[0069] S3: Defect Area Trimming

[0070] When using an AI deep learning detection model for inference and judgment, the defect area will be outlined in the current image, such as... Figure 8 As shown, it is described in the form of a bounding box (bbox), and its coordinate system representation is as follows: Figure 6 As shown.

[0071] Among them, {x min ,y min} represents the top-left corner coordinates of the detected defect region, and width and height represent the width and height of the defect region; then, this region is cropped from the original image, and the cropping result is as follows. Figure 8As shown, proceed with the subsequent process.

[0072] S4: Similarity Fusion Determination

[0073] Cosine similarity:

[0074] After obtaining the defect category of the detection box and performing relative position verification, the cropped region images of the same defect are grouped together. Figure 9 , Figure 10 , Figure 11 After performing the above steps, the corresponding Figure 12 , Figure 13 , Figure 14 Let roi1, roi2, and roi3 be the three roi3, and determine their cosine similarity.

[0075] The principle of cosine similarity determination:

[0076] Cosine similarity is based on the pixel values ​​of images. It analyzes whether the pixel values ​​of two images are close to each other to indicate whether they are similar. In essence, it projects the pixel information of two images into a high-dimensional space, represents it as a vector, and then calculates the angle between the two vectors. The smaller the angle, the more similar they are.

[0077] The formula for determining cosine similarity is as follows:

[0078]

[0079] Where A and B represent different image vectors, A i and B i These represent the components of vectors A and B, respectively.

[0080] The specific cosine similarity calculation results are shown in Table 1;

[0081] Table 1. Results of Cosine Similarity Calculation

[0082]

[0083] Structural similarity:

[0084] Structural similarity is an evaluation metric that measures image similarity from three aspects: brightness, contrast, and structure. Its similarity determination results are significantly better than those calculated by perceptual hashing and cosine similarity. Since defect data collection in urban underground pipeline scenarios is valuable, ensuring the accuracy of data collection and deduplication is paramount. Therefore, this patent uses structural similarity to integrate multiple traditional similarity determination algorithms, thereby ensuring the accuracy of the algorithm.

[0085] SSIM(x,y)=[l(x,y)]α ·[c(x,y)] β ·[s(x,y)] γ

[0086] Where x and y represent the two images to be compared, l(x,y) represents the brightness measurement, c(x,y) represents the contrast measurement, and s(x,y) represents the structure measurement. The three parameters α, β, and γ are used to represent the importance of these three modules.

[0087] To simplify the formula, let

[0088] α=β=γ=1,C3=C2 / 2

[0089] The simplified formula for SSIM can be obtained as follows:

[0090]

[0091] Where μ x and μ y σ is the mean of the x and y values ​​of the image. xy The covariance of two images is represented by C1 and C2, which are constants, usually set to 0.01. The structural similarity index reflects the structural attributes of objects in a scene from the perspective of image composition, and models it as a combination of three different factors: brightness, contrast, and structure. The mean is used as an estimate of brightness, the standard deviation as an estimate of contrast, and the covariance as an estimate of structure. The specific structural similarity calculation results are shown in Table 2.

[0092] Table 2 Results of Structural Similarity Calculation

[0093] Matching images SSIM similarity <roi1,roi2> 0.8802 <roi1,roi3> 0.8210 <roi2,roi3> 0.8408

[0094] Based on dataset testing, this patent combines cosine similarity and structural similarity in a weighted manner, and the final similarity calculation formula is as follows:

[0095] final_score=α×cos(x,y)+β×ssim(x,y)

[0096] Where final_score represents the comprehensive weighted similarity value obtained by fusing the two similarity calculation methods, and α and β represent the importance of cosine similarity and structural similarity, respectively. Based on a large number of repeated experiments and statistics in the test set, this patent sets α to 0.80, β to 0.20, and the threshold to 0.85. When the value of the comprehensive similarity calculation is greater than this set threshold, the images to be compared are determined to be similar images; otherwise, they are considered to be dissimilar images.

[0097] S5: Sorting and Deduplication of Similar Image Sets Based on Image Clarity

[0098] After determining the candidate set of similar images based on comprehensive similarity, Laplacian filtering is applied to each image in the candidate set to obtain the response variance value for each image. This value represents the degree of blurriness of the image; the larger the value, the greater the likelihood that the image is sharp. Laplacian filtering is used to calculate the second derivative of the image, highlighting regions with rapid gradient changes. A high variance in an image indicates a broad response, suggesting that the image is in focus and has minimal blurring. Conversely, a low response variance indicates a small response spread, suggesting the absence of clear edge boundaries and a higher probability that the image is blurry. The Laplacian convolution kernel is shown below.

[0099]

[0100] For the candidate set of similar images, the Laplacian method was used to obtain the variance of the response, and the results are shown in Table 3.

[0101] Table 3

[0102] Candidate Images Laplace response variance roi1 742.4041 roi2 450.5875 roi3 470.2135

[0103] Based on the calculated Laplacian response variance, the images are sorted by sharpness, and the image with the highest sharpness is selected as the one to be retained. Therefore, the original image corresponding to the image is retained, and the other images are removed. This completes the deduplication of similar images in the urban underground pipeline environment.

[0104] Example 2

[0105] A computer device for deduplicating similar images within a pipeline includes a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to implement a method for deduplicating similar images within a pipeline as described in Embodiment 1.

[0106] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for deduplicating similar images within a pipeline, characterized in that, Includes the following steps: S1: Real-time detection of defect areas inside the pipeline and saving defect images to the database; S2: Verify the defect types and relative positions of defect images in the database to determine whether images of the same type of defect belong to the same defect; S3: Cropping the defect region from the image with the same defect obtained in step S2; S4: Classify the cropped region images with the same defect and then perform a comprehensive similarity determination; S5: Select a set of similar images based on the comprehensive similarity obtained in step S4, then calculate the sharpness of each image in the set of similar images, and sort and remove duplicates of the images in the set of similar images according to the sharpness. Step S2 includes: Based on the YOLO deep learning detection algorithm, existing defect types in pipeline images are labeled and trained. Then, the images to be tested are used for inference and discrimination to obtain the defect type and detection box position in each pipeline image. Then, the relative position of defects of the same type is verified. The verification mechanism is as follows: first, the image region is divided into eight quadrants and numbered in sequence. Then, it is judged whether the center point coordinates of the detection box of images with the same type of defect fall into the same quadrant region. If they fall into the same quadrant region, the image defects are considered to be in close positions, so the next step can be continued; otherwise, they are considered not to be the same defect, the image is ignored and the process returns to the first step. The method for dividing the image into eight quadrants is as follows: using the coordinates of the center point of the image as a reference, divide the image into quadrants every 45°, and a total of 8 quadrants can be divided. The regions are numbered starting from the positive half of the x-axis and counting counterclockwise or clockwise, from 1 to 8.

2. The method for deduplication of similar images within a pipeline according to claim 1, characterized in that, Step S1 includes: The YOLO deep learning object detection algorithm is used to determine pipe defects and the areas of pipe defects in pipe images in real time.

3. The method for deduplicating similar images within a pipeline according to claim 1, characterized in that, Step S3 includes: The AI ​​deep learning detection model is used to infer and determine the defective region in the image, which is described in the form of a bounding box. Then, the region is cropped out from the original image and the subsequent process continues.

4. The method for deduplicating similar images within a pipeline according to claim 1, characterized in that, Step S4 includes: S4.1: Group the cropped region images of the same defect obtained in step 3 together, and then perform cosine similarity determination on each group; the cosine similarity determination formula is as follows: ； Where A and B represent different image vectors. and Let A and B represent the components of vectors A and B, respectively. S4.2: Perform structural similarity judgment on the cropped region images of the same defect after calculating cosine similarity. The formula for calculating structural similarity is: ； Where x and y represent the two images to be compared. Indicates the measured brightness value. This represents the contrast measurement value. These represent structural measurements, and the three parameters... This is used to indicate the importance of these three modules; To simplify the formula, let ； The simplified formula for SSIM can be obtained as follows: ； in and It is the mean of the x and y values ​​of the image. This represents the covariance of two images. and It is a constant; S4.3: The cosine similarity obtained in step S4.1 and the structural similarity obtained in step S4.2 are weighted and combined to obtain the comprehensive similarity. The calculation formula is as follows: ； in This represents the combined weighted similarity value obtained by integrating these two similarity calculation methods. and These represent the importance of cosine similarity and structural similarity, respectively.

5. The method for deduplication of similar images within a pipeline according to claim 1, characterized in that, Step S5 includes: determining a candidate set of similar images based on comprehensive similarity and a threshold, wherein images with comprehensive similarity greater than the threshold are selected as candidates for similar images, and images with comprehensive similarity less than the threshold are ignored and no longer considered; If the overall similarity of all images in the candidate set of similar images is less than the threshold, no further processing is performed, and the process returns to step one. For each image in the candidate set of similar images with an overall similarity greater than the threshold, Laplacian filtering is performed to obtain the response variance value corresponding to each image. Based on the calculated Laplacian response variance value, the images are sorted by sharpness, and the image with the highest sharpness is determined to be the one to be retained. Other images are then removed.

6. A computer device for deduplicating similar images within a pipeline, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to implement the method of any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images