Pareto front-driven municipal pipeline defect intelligent identification method and device

Abstract

The invention provides a Pareto front-driven municipal pipeline defect intelligent identification method and device. The method comprises the following steps: acquiring a to-be-detected image of a municipal pipeline; performing feature extraction on the to-be-detected image to generate a first feature vector; according to the first feature vector, a plurality of dictionary libraries and final state feature coefficient vectors of the dictionary libraries, calculating error values between the first feature vector and the dictionary libraries; the construction method of the dictionary database comprises the following steps: collecting a sample image set with pipeline defect information; performing feature extraction on sample images in the sample image set to generate a second feature vector set; based on a second-generation non-dominated sorting genetic algorithm, according to the second feature vector set, constructing a dictionary database and a final state coefficient vector of the dictionary database; screening a minimum error value from the error values; and determining a defect detection result of the to-be-detected image according to the target dictionary database corresponding to the minimum error value. According to the technology, the accuracy and efficiency of municipal drainage pipeline defect identification are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a Pareto frontier-driven municipal pipeline defect intelligent identification method and device. Background Art

[0002] During long-term service, municipal drainage pipes are prone to functional and structural defects inside the pipes due to the acidic or corrosive substances they transport in the water, including but not limited to rupture, leakage, deformation, corrosion, misalignment, disconnection, foreign body intrusion, concealed connection of branches, and sediment accumulation.

[0003] Such defects can lead to reduced pipeline carrying capacity, leakage and environmental pollution, and even safety hazards such as road collapse. Efficient and accurate detection methods are urgently needed to guide repair decisions. Summary of the Invention

[0004] The purpose of the present invention is to provide a Pareto front-driven intelligent identification method and device for municipal pipeline defects to alleviate technical problems such as reduced pipeline carrying capacity, leakage pollution, and even road collapse, which can cause safety hazards. It also provides an efficient and accurate means of defect detection for municipal pipelines to guide repair decisions.

[0005] In a first aspect, an embodiment of the present invention provides a Pareto front-driven intelligent identification method for municipal pipeline defects, comprising: collecting images to be inspected of municipal pipelines; performing feature extraction on the images to be inspected to generate a first feature vector; calculating the error value between the first feature vector and the multiple dictionary libraries based on the first feature vector, multiple preset dictionary libraries, and the final-state feature coefficient vectors of the multiple dictionary libraries; a method for constructing the dictionary libraries comprises: collecting a set of sample images with pipeline defect information; performing feature extraction on the sample images in the sample image set to generate a second feature vector set; constructing the dictionary libraries and the final-state coefficient vectors of the dictionary libraries based on the second feature vector set based on a second-generation non-dominated sorting genetic algorithm; screening out the minimum error value from the error values; and determining the defect detection result of the image to be inspected based on the target dictionary library corresponding to the minimum error value.

[0006] In a preferred embodiment of the present invention, based on the second generation non-dominated sorting genetic algorithm, the step of constructing the above-mentioned dictionary library and the final state coefficient vector of the above-mentioned dictionary library according to the above-mentioned second eigenvector set includes: determining the atomic unit with the same number of columns as the above-mentioned second eigenvector according to the second eigenvector in the above-mentioned second eigenvector set; constructing the initial dictionary library according to the above-mentioned atomic unit; calculating the intermediate state coefficient vector according to the above-mentioned second eigenvector, the above-mentioned initial dictionary library and the randomly generated initial coefficient vector; the above-mentioned initial coefficient vector has the same dimension as the above-mentioned second eigenvector; constructing the above-mentioned dictionary library and the above-mentioned final state coefficient vector according to the above-mentioned intermediate state coefficient vector and the above-mentioned initial dictionary library.

[0007] In a preferred embodiment of the present invention, the step of calculating the intermediate state coefficient vector based on the above-mentioned second eigenvector, the above-mentioned initial dictionary library and the randomly generated initial coefficient vector includes: step A1: splicing the above-mentioned atomic units of the above-mentioned initial dictionary library into a first vector code according to a preset column order; and, using binary coding or real number coding for the above-mentioned initial coefficient vector to obtain a second vector code; the number of constrained non-zero elements of the above-mentioned second vector code does not exceed a preset sparsity; step A2: randomly splicing the above-mentioned first vector code and the above-mentioned second vector code to generate an initial population; step A3: based on the Pareto dominance relationship, screening target individuals that meet the preset screening criteria from the above-mentioned initial population; step A4: performing an evolution operation on the above-mentioned target individuals to obtain evolved individuals; step A5: repeating the above-mentioned steps A1 to A4 until a preset number of iterations is reached or the frontier change of the Pareto dominance relationship is less than a preset threshold, to obtain a final individual; step A6: determining the above-mentioned intermediate state coefficient vector based on the above-mentioned final individual.

[0008] In a preferred embodiment of the present invention, the step of performing an evolution operation on the target individual to obtain an evolved individual includes: performing a crossover operation or a mutation operation on the target first vector code and the target second vector code corresponding to the target individual to obtain the evolved individual.

[0009] In a preferred embodiment of the present invention, the step of performing a crossover operation on the target first vector code and the target second vector code corresponding to the target individual to obtain the evolved individual includes: randomly exchanging the columns of the target first vector code corresponding to the target individual to obtain an updated first vector code; and, using a uniform crossover method to exchange the columns of the target second vector code corresponding to the target individual, retaining the non-zero parameters in the target second vector code, to obtain an updated second vector code; and determining the evolved individual based on the updated first vector code and the updated second vector code.

[0010] In a preferred embodiment of the present invention, the step of performing a mutation operation on the target first vector code and the target second vector code corresponding to the above-mentioned target individual to obtain an evolved individual includes: randomly perturbing the numerical value of the column of the target first vector code corresponding to the above-mentioned target individual to obtain an updated third vector code; flipping the non-zero parameters of the target second vector code corresponding to the above-mentioned target individual with a preset probability to obtain an updated fourth vector code; and determining the evolved individual based on the above-mentioned updated third vector code and the above-mentioned updated fourth vector code.

[0011] In a preferred embodiment of the present invention, the step of determining the evolved individual based on the updated first vector code and the updated second vector code includes: eliminating individuals whose updated first vector code and the updated second vector code do not satisfy preset constraints to obtain a vector code that has been constrained; and determining the vector code that has been constrained as the evolved individual.

[0012] In a preferred embodiment of the present invention, the above pipeline defect information includes: no defect, leakage, rupture, undulation, misalignment and defect level information corresponding to the above no defect, leakage, rupture, undulation and misalignment respectively.

[0013] In a second aspect, an embodiment of the present invention provides a Pareto front-driven intelligent identification device for municipal pipeline defects, comprising: a data acquisition module for collecting images to be inspected of municipal pipelines; an identification module for performing feature extraction on the above-mentioned images to be inspected to generate a first feature vector; calculating the error value between the above-mentioned first feature vector and the above-mentioned multiple dictionary libraries based on the above-mentioned first feature vector, multiple preset dictionary libraries and the final-state feature coefficient vectors of the above-mentioned multiple dictionary libraries; a method for constructing the above-mentioned dictionary libraries comprising: collecting a set of sample images with pipeline defect information; performing feature extraction on the sample images in the above-mentioned sample image set to generate a second feature vector set; constructing the above-mentioned dictionary library and the final-state coefficient vector of the above-mentioned dictionary library based on the above-mentioned second feature vector set based on the second generation non-dominated sorting genetic algorithm; screening out the minimum error value from the above-mentioned error values; and determining the defect detection result of the above-mentioned image to be inspected based on the target dictionary library corresponding to the above-mentioned minimum error value.

[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned defect detection method for the government pipeline.

[0015] The embodiments of the present invention have the following beneficial technical effects: Embodiments of the present invention provide a Pareto-front-driven intelligent municipal pipeline defect identification method and device, comprising: collecting an image of a municipal pipeline to be inspected; performing feature extraction on the image to be inspected to generate a first feature vector; calculating the error between the first feature vector and the multiple dictionary libraries based on the first feature vector, multiple pre-set dictionary libraries, and the final-state feature coefficient vectors of the multiple dictionary libraries; constructing the dictionary libraries includes: collecting a set of sample images containing pipeline defect information; performing feature extraction on the sample images in the sample image set to generate a second set of feature vectors; constructing the dictionary libraries and the final-state coefficient vectors of the dictionary libraries based on the second set of feature vectors using a second-generation non-dominated sorting genetic algorithm; selecting the minimum error value from the error values; and determining the defect detection result for the image to be inspected based on the target dictionary library corresponding to the minimum error value. This technology achieves accurate defect identification of municipal pipeline images by comparing the first feature vector with a pre-built dictionary database, thereby improving the accuracy and efficiency of municipal drainage pipeline defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A schematic flow chart of a Pareto front-driven municipal pipeline defect intelligent identification method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a construction process of a dictionary library and a final state coefficient vector of the dictionary library provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a Pareto front-driven municipal pipeline defect intelligent identification device provided by an embodiment of the present invention; Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0018] Icons: 31 - data acquisition module; 32 - identification module; 41 - memory; 42 - processor; 43 - bus; 44 - communication interface. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Municipal sewer pipes, as a core component of the underground network within urban infrastructure, have long shouldered the heavy responsibility of collecting and transporting domestic sewage, industrial wastewater, and rainwater. However, due to the frequent presence of acidic, alkaline, or corrosive components in the water they transport (such as industrial wastewater, acid rain infiltration, or saline groundwater), coupled with complex external environmental influences, pipes inevitably develop various functional and structural defects over their long service lives. These defects not only threaten the normal operation of the pipes but can also lead to cascading environmental and public safety risks, necessitating the urgent need for scientific detection methods to address them.

[0021] Based on this, an embodiment of the present invention provides a Pareto front-driven intelligent identification method and device for municipal pipeline defects. This technology accurately identifies defects in municipal pipeline images by comparing the first eigenvector with a pre-built dictionary database, thereby improving the accuracy and efficiency of municipal drainage pipeline defect identification.

[0022] Example 1 In this embodiment, Figure 1 A schematic flow chart of a Pareto front-driven municipal pipeline defect intelligent identification method provided in an embodiment of the present invention.

[0023] Depend on Figure 1 As can be seen, the method includes: Step S101: Collect an image of a municipal pipeline to be inspected.

[0024] In this embodiment, the image to be detected is generally collected by a camera. Here, the image to be detected is generally an internal image of a municipal pipeline.

[0025] Step S102: extracting features from the image to be detected to generate a first feature vector.

[0026] Here, the above step S102 includes: extracting color features, pixel features, texture features and shape features of the above image to be detected to generate a first feature vector.

[0027] Step S103: Calculate the error value between the first feature vector and the multiple dictionary libraries based on the first feature vector, multiple preset dictionary libraries, and the final-state feature coefficient vectors of the multiple dictionary libraries. The method for constructing the dictionary libraries includes: collecting a sample image set with pipeline defect information; performing feature extraction on the sample images in the sample image set to generate a second feature vector set; and constructing the dictionary library and the final-state coefficient vector of the dictionary library based on the second feature vector set based on a second-generation non-dominated sorting genetic algorithm.

[0028] The above pipeline defect information includes: no defect, leakage, rupture, undulation, misalignment, and defect level information corresponding to the above no defect, leakage, rupture, undulation, and misalignment respectively. In this embodiment, feature extraction is performed on the sample images in the sample image set to generate a second feature vector set, which includes extracting color features, pixel features, texture features, and shape features of the sample images to generate the second feature vector set.

[0029] Here, the step of extracting the color features of the sample image includes: extracting the color features of the sample image through color channels and a brightness model constructed by the color channels.

[0030] Specifically, the brightness model is I=0.21R+0.72G+0.07B, which is a weighted average of the RGB channels. Here, I is the brightness model, R is the red channel, G is the green channel, and B is the blue channel.

[0031] Furthermore, the step of extracting pixel features of the above-mentioned sample image includes: dividing the sample image into multiple image blocks based on preset parameters; determining the pixel average, pixel variance and pixel kurtosis of each of the above-mentioned image blocks according to the preset weights of the corresponding areas of the above-mentioned image blocks and the pixel values ​​of each of the above-mentioned image blocks; the multiplication result obtained by multiplying the preset weights of the above-mentioned multiple image blocks is 1; and determining the pixel features of the sample image based on the above-mentioned pixel average, the above-mentioned pixel variance and the above-mentioned pixel kurtosis.

[0032] Here, the pixel value of each of the above image blocks is determined by the ratio of the number of pixels having a preset pixel intensity in the image block to the number of pixels in the image block.

[0033] Among them, the step of determining the pixel average, pixel variance and pixel kurtosis of each of the above-mentioned image blocks based on the preset weights of the areas corresponding to the above-mentioned image blocks and the pixel values ​​of each of the above-mentioned image blocks includes: calculating the pixel average of each of the above-mentioned image blocks based on the preset weights of the areas corresponding to the above-mentioned image blocks and the pixel values ​​of each of the above-mentioned image blocks; using a sliding window technology of a preset size, calculating the local variance and the overall variance of each pixel point in each of the above-mentioned image blocks based on the pixel values ​​of each of the above-mentioned image blocks, the above-mentioned pixel average and the window pixel average corresponding to the above-mentioned sliding window; calculating the above-mentioned pixel variance based on the above-mentioned local variance and the above-mentioned overall variance; calculating the above-mentioned pixel kurtosis based on the above-mentioned pixel variance, the pixel value of each of the above-mentioned image blocks and the above-mentioned pixel average.

[0034] Furthermore, the step of calculating the local variance and the overall variance of each pixel point in each of the above-mentioned image blocks based on the pixel value of each of the above-mentioned image blocks, the above-mentioned pixel average value, and the window pixel average value corresponding to the above-mentioned sliding window includes: calculating the above-mentioned local variance based on the following formula:

[0035] in, is the above local variance, is the preset size of the sliding window, is the pixel value of the i+mth row and j+nth column in the above image block, is the average value of the pixels in the sliding window when the center pixel of the sliding window is in the i-th row and j-th column; The overall variance is calculated based on the following formula:

[0036] in, is the overall variance, M is the number of rows in the image block, N is the number of columns in the image block, I(i,j) is the pixel value in the i-th row and j-th column in the image block, is the average value of the above pixels; The step of calculating the pixel variance according to the local variance and the global variance includes: The above pixel variance is calculated based on the following formula:

[0037] in, is the above pixel variance, is the preset coefficient.

[0038] Furthermore, the step of extracting shape features of the sample image includes: extracting first-order invariant moments, second-order invariant moments, third-order invariant moments, fourth-order invariant moments, fifth-order invariant moments, sixth-order invariant moments and seventh-order invariant moments of the sample image; and determining the shape features of the sample image based on the above-mentioned first-order invariant moments, second-order invariant moments, third-order invariant moments, fourth-order invariant moments, fifth-order invariant moments, sixth-order invariant moments and seventh-order invariant moments.

[0039] Furthermore, the step of extracting the first-order invariant moment, the second-order invariant moment, the third-order invariant moment, the fourth-order invariant moment, the fifth-order invariant moment, the sixth-order invariant moment, and the seventh-order invariant moment of the sample image includes: extracting the first-order invariant moment of the sample image based on the following formula:

[0040] in, is the first-order invariant moment mentioned above, is the second-order central moment of the sample image in the horizontal direction, The second-order central moment of the sample image in the vertical direction; The second-order invariant moment of the sample image is extracted based on the following formula:

[0041] in, is the second-order invariant moment mentioned above, is the first-order central moment in the horizontal direction and the first-order central moment in the vertical direction; The third-order invariant moment of the sample image is extracted based on the following formula:

[0042] in, is the third-order invariant moment mentioned above, is the third-order central moment in the horizontal direction, is the third-order central moment in the vertical direction, is the second-order central moment in the horizontal direction and the first-order central moment in the vertical direction, is the first-order central moment in the horizontal direction and the second-order central moment in the vertical direction; The fourth-order invariant moment of the sample image is extracted based on the following formula:

[0043] in, is the fourth-order invariant moment mentioned above; The fifth-order invariant moment of the sample image is extracted based on the following formula:

[0044] in, is the fifth-order invariant moment mentioned above; The sixth-order invariant moment of the sample image is extracted based on the following formula:

[0045] wherein, is the sixth-order invariant moment mentioned above; The seventh-order invariant moment of the sample image is extracted based on the following formula:

[0046] wherein, is the seventh-order invariant moment mentioned above.

[0047] Further, the step of extracting the texture feature of the sample image comprises: converting the sample image into a gray-scale image; calculating a gray-scale co-occurrence matrix of a preset direction according to the gray-scale image; and calculating a Haralick texture feature corresponding to each of the directions according to the gray-scale co-occurrence matrix.

[0048] Wherein, the GLCM is generated from three angles: 0°, 45° and 90°.

[0049] The GLCM of two pixel intensity values in the pipeline image at a distance d and a direction is defined as , which represents the probability of the pixel pair with pixel intensity values I m and I n appearing at a distance d and an angle . Wherein, is the ratio of the number of (I m , I n ) pairs at a distance d and an angle to the total number of pixel pairs.

[0050] Wherein, N is the number of pixel intensity values; I m and I n : two different gray levels of the gray-scale image mentioned above; d: the distance between the pixel pairs; φ: the direction angle between the pixel pairs; Five Haralick texture features are obtained from the GLCM, specifically including: ; wherein, ; ; ; ; ; Among them, ASM is the angular second moment, CON is the contrast, IDF is the inverse difference moment, Corr is the correlation, and Var is the variance.

[0051] GLCM is calculated for three different angles θ, namely 0°, 45° and 90°, and GLCM is the sum of two pixel intensity values ​​(I m , I n ) is generated for three distance values ​​d=1, 2 and 3.

[0052] Therefore, 9 gray-level co-occurrence matrices are generated for each image block. Furthermore, 5 texture features are calculated from each gray-level co-occurrence matrix, resulting in 45 texture feature vectors, denoted as WT.

[0053] In some embodiments, after the step of extracting the color features, pixel features, texture features and shape features of the sample image to generate a second feature vector set, the method includes: normalizing the second feature vector set to obtain normalized features; and constructing the dictionary library and the final state coefficient vector of the dictionary library based on the second feature vector set based on a second-generation non-dominated sorting genetic algorithm, including: constructing the dictionary library and the final state coefficient vector of the dictionary library based on the normalized features based on a second-generation non-dominated sorting genetic algorithm.

[0054] Furthermore, the step of normalizing the above-mentioned second feature vector set to obtain normalized features includes: determining the number of groups according to the feature dimension of the above-mentioned second feature vector set; the square of the above-mentioned number of groups is equal to the above-mentioned feature dimension; dividing the above-mentioned second feature vector set into multiple groups of sub-feature combinations according to the above-mentioned number of groups; calculating the mean and variance of each group of the above-mentioned sub-feature combinations; normalizing the above-mentioned second feature vector set according to the mean and variance of the above-mentioned sub-feature combinations to obtain intermediate normalized features; adjusting the above-mentioned intermediate normalized features based on preset scaling coefficients and offset coefficients to obtain normalized features.

[0055] For ease of understanding, assume there are N second eigenvector set samples, each consisting of color, texture, and shape features, resulting in a feature dimension of 64. Assuming there are 8 groups, each group corresponds to 8 features. Then, for each second eigenvector set sample, calculate the mean and variance of each feature group.

[0056] The mean and variance of each set of features are calculated using the following formula: ;

[0057] in, is the mean of the eigenvalues ​​of the g-th group, is the variance of the g-th group eigenvalue, is the eigenvalue of the gth group of the i-th second eigenvector set sample, H and W are the width and height of the feature map, H=W=1.

[0058] Next, the second feature vector set is normalized according to the mean and variance of the sub-feature combination to obtain an intermediate normalized feature; the intermediate normalized feature is adjusted based on a preset scaling factor and offset factor to obtain a normalized feature.

[0059] The calculation formula is:

[0060]

[0061] Among them, the above is the eigenvalue of the intermediate state normalized feature of the gth group of the i-th second eigenvector set sample, is a non-zero positive number, is the scaling factor, is the offset coefficient, is the normalized feature.

[0062] Step S104: Filter out the minimum error value from the above error values.

[0063] Step S105: determining the defect detection result of the image to be detected according to the target dictionary library corresponding to the minimum error value.

[0064] An embodiment of the present invention provides a Pareto front-driven intelligent identification method for municipal pipeline defects, comprising: collecting an image of a municipal pipeline to be inspected; performing feature extraction on the image to be inspected to generate a first feature vector; calculating the error between the first feature vector and the multiple dictionary libraries based on the first feature vector, multiple pre-set dictionary libraries, and the final-state feature coefficient vectors of the multiple dictionary libraries; constructing the dictionary libraries includes: collecting a set of sample images containing pipeline defect information; performing feature extraction on the sample images in the sample image set to generate a second set of feature vectors; constructing the dictionary libraries and the final-state coefficient vectors of the dictionary libraries based on the second set of feature vectors using a second-generation non-dominated sorting genetic algorithm; selecting the minimum error value from the error values; and determining the defect detection result for the image to be inspected based on the target dictionary library corresponding to the minimum error value. This technology achieves accurate defect identification of municipal pipeline images by comparing the first feature vector with a pre-built dictionary database, thereby improving the accuracy and efficiency of municipal drainage pipeline defect identification.

[0065] Example 2 On the basis of the above embodiment, the embodiment of the present invention focuses on the process of constructing the above dictionary library and the final state coefficient vector of the above dictionary library based on the second generation non-dominated sorting genetic algorithm according to the above second eigenvector set. Figure 2 A schematic diagram of a dictionary library and a construction process of a final state coefficient vector of the dictionary library provided by an embodiment of the present invention.

[0066] Depend on Figure 2 As you can see, the process includes: Step S201: determining, based on the second eigenvector in the second eigenvector set, an atomic unit having the same number of columns as the second eigenvector.

[0067] Step S202: constructing an initial dictionary library based on the above atomic units.

[0068] Step S203: Calculate an intermediate state coefficient vector based on the second eigenvector, the initial dictionary library, and a randomly generated initial coefficient vector; the initial coefficient vector has the same dimension as the second eigenvector.

[0069] Step S204: constructing the dictionary library and the final state coefficient vector according to the intermediate state coefficient vector and the initial dictionary library.

[0070] In one embodiment, the step of calculating the intermediate state coefficient vector based on the above-mentioned second eigenvector, the above-mentioned initial dictionary library and the randomly generated initial coefficient vector includes: step A1: splicing the above-mentioned atomic units of the above-mentioned initial dictionary library into a first vector code according to a preset column order; and, using binary encoding or real number encoding for the above-mentioned initial coefficient vector to obtain a second vector code; the number of constrained non-zero elements of the above-mentioned second vector code does not exceed a preset sparsity; step A2: randomly splicing the above-mentioned first vector code and the above-mentioned second vector code to generate an initial population; step A3: based on the Pareto dominance relationship, screening target individuals that meet the preset screening criteria from the above-mentioned initial population; step A4: performing an evolution operation on the above-mentioned target individuals to obtain evolved individuals; step A5: repeating the above-mentioned steps A1 to A4 until a preset number of iterations is reached or the frontier change of the Pareto dominance relationship is less than a preset threshold, thereby obtaining a final individual; step A6: determining the above-mentioned intermediate state coefficient vector based on the above-mentioned final individual.

[0071] The step A3 includes calculating a first calculation result of the first objective function and a second calculation result of the second objective function corresponding to each individual in the initial population; the first objective function is: ;|| ||o<L;The above second objective function is: ;|| ||o<L. Wherein, L is the preset sparsity, D is the dictionary matrix corresponding to the above individual, F1 represents the above first calculation result, Y is the above second eigenvector, is the initial coefficient vector, F2 represents the second calculation result, the second calculation result is the Frobenius norm between the second eigenvector and the dictionary matrix, Di is the target dictionary matrix corresponding to the i-th target pipeline defect information in the individual, i is the target coefficient vector corresponding to the target pipeline defect information in the initial coefficient vector; the first calculation result and the second calculation result are divided into multiple non-dominated hierarchies based on the Pareto dominance relationship; the congestion degree of the target first calculation result and the target second calculation result corresponding to each layer in the non-dominated hierarchy is calculated; based on the non-dominated hierarchy and the target congestion degree corresponding to the non-dominated hierarchy, a tournament selection method is used to screen target individuals that meet the preset screening criteria.

[0072] The step of performing an evolution operation on the target individual to obtain an evolved individual includes: performing a crossover operation or a mutation operation on the target first vector code and the target second vector code corresponding to the target individual to obtain the evolved individual.

[0073] Furthermore, the step of performing a crossover operation on the target first vector code and the target second vector code corresponding to the target individual to obtain the evolved individual includes: randomly exchanging the columns of the target first vector code corresponding to the target individual to obtain an updated first vector code; and, using a uniform crossover method to exchange the columns of the target second vector code corresponding to the target individual, retaining the non-zero parameters in the target second vector code, to obtain an updated second vector code; and determining the evolved individual based on the updated first vector code and the updated second vector code.

[0074] Furthermore, the step of performing a mutation operation on the target first vector code and the target second vector code corresponding to the above-mentioned target individual to obtain the evolved individual includes: randomly perturbing the numerical value of the column of the target first vector code corresponding to the above-mentioned target individual to obtain an updated third vector code; flipping the non-zero parameters of the target second vector code corresponding to the above-mentioned target individual with a preset probability to obtain an updated fourth vector code; and determining the evolved individual based on the above-mentioned updated third vector code and the above-mentioned updated fourth vector code.

[0075] In some examples, the step of determining the evolved individual based on the updated first vector code and the updated second vector code includes: eliminating individuals whose updated first vector code and the updated second vector code do not satisfy preset constraints to obtain a vector code that has been constrained; and determining the vector code that has been constrained as the evolved individual.

[0076] For ease of understanding, the present invention further provides the following examples for detailed description: First, assume that a municipal pipeline inspection system collects sample images of the following three types of defects (simplified to 3 samples, with a feature dimension of 64): Among them, the defect category is: leakage, and the corresponding second eigenvector is = [0.8, 0.2, 0.5, ...,0.3]; the defect category is: rupture, and the corresponding second eigenvector is = [0.1, 0.9, 0.4, ..., 0.7]; the defect category is: undulation, and the corresponding second eigenvector is = [0.6, 0.4, 0.3, ..., 0.1].

[0077] Then, based on the second eigenvector above, three dictionary libraries are constructed (one dictionary for each defect type), each dictionary contains 5 atoms (K=5), and the atomic dimension is 64. For example, the leakage dictionary library Initialized as:

[0078] Sparsity constraint: Set the sparsity L=2, that is, each coefficient vector has at most 2 non-zero elements.

[0079] Then, the NSGA-II optimization process is carried out: First, chromosome encoding is performed, that is, the individual contains the dictionary library D and the intermediate state coefficient vector α, The atoms of are expanded into continuous vectors by column. For example, The encoding of is: 0.7, 0.3, ..., 0.4, 0.2, 0.8, ..., 0.5, ..., 0.1, 0.2, ..., 0.8. Then, the intermediate state coefficient vector is encoded: the α of each sample is encoded in binary, for example = [1, 0, 1, 0, 0] (indicates the use of the 1st and 3rd atoms of ).

[0080] Next, perform population initialization, that is, generate the initial population (assuming the population size is 4): Individual 1: Initial Atom+ = [1, 0, 1, 0, 0]; Instance 2: Random Atom+ = [0, 1, 0, 1, 0]; Instance 3: Random Atom+ = [1, 1, 0, 0, 0]; Instance 4: Hybrid Atom+ = [0, 0, 1, 1, 0].

[0081] Then, taking sample leakage as an example, calculate the objective function of individual 1:

[0082] Next, traverse all dictionary libraries and calculate the minimum error:

[0083] Furthermore, comparing all individuals and , divide the Pareto hierarchy. For example, individual 1 ( =0.25, = 0.25) may belong to the first layer. The crowding degree is the distribution density of the calculated solutions within the same layer. The greater the distance between individuals, the higher the crowding degree.

[0084] Furthermore, a genetic operation is performed, that is, crossover is to exchange the atomic columns of the two parent dictionary libraries and mix the coefficient vectors. For example, after the parent individual 1 and individual 2 are crossed, the generated child dictionary library contains the first 3 columns of the parent individual 1 and The last two columns of . Mutation is to randomly perturb the values ​​of dictionary atoms or the non-zero positions of coefficient vectors. For example, from becomes .

[0085] Furthermore, iterative optimization is performed, i.e., the above steps are repeated until the termination condition is met (e.g., the maximum number of iterations = 100 or the error convergence threshold ε = 0.01). The final Pareto front solution set may contain the following individuals: Solution A: =0.18, =0.20 (balanced); Solution B: =0.15, =0.25 (focusing on sparsity); Solution C: =0.22, =0.15 (focusing on classification accuracy); Finally, a classification decision is made, that is, the comprehensive optimal solution is selected from the Pareto frontier and classified using its dictionary library and sparse coefficients: Input new image feature Y new = [0.7, 0.3, ..., 0.4] Calculate the reconstruction error with each dictionary library: Error1=||Y new -D1α1||=0.18, Error2=0.25, Error3=0.30; Therefore, the category corresponding to the minimum error is selected: leakage.

[0086] Furthermore, after obtaining the solution to the minimum value of the objective function, the latest coefficient vector α is obtained, and then the dictionary matrix D is updated. The updating steps are as follows: For each dictionary atom dk, the update process includes: Construct the error matrix: Ek=XΩk-DΩkαΩk; where Ωk is the set of indices of all signals using the dictionary atom dk, XΩk is the subset of all signals using dk, DΩk is the subset of dictionary atoms corresponding to the indices in Ωk, and αΩk is the subset of the corresponding coefficient vectors.

[0087] Furthermore, SVD update is performed: [U, S, V] = SVD (Ek) Take the first column of U as the new atom as dk, and the product of the first element of S and the first column of V as the new sparse coefficient αk,i.

[0088] The present invention uses embodiments to illustrate the above steps in detail; Image original input matrix , each column represents a set of feature vectors; Initialized dictionary matrix, , each column represents an atomic unit; The first set of eigenvectors x1=[1,4,7]T is known; x1≈Dα1

[0089] Through genetic algorithm, we can get: , then we can get:

[0090] Similarly, we can get:

[0091]

[0092] Then we can get:

[0093] The atom d1 of the dictionary D is used by all signals, so we calculate the product of DΩ1 and αΩ1:

[0094] Next, update the dictionary D. Given d1=[1,0,1]T, construct the error matrix E1:

[0095] Perform SVD decomposition on E1: E1=UΣVT The solution is:

[0096]

[0097]

[0098] The updated atom dk′ can be obtained by taking the first column of the U matrix, that is, dk′=U[:,0]dk′=U[:,0]=[-0.69 0.34 0.64]T. At the same time, the updated sparse coefficient αΩk can be obtained by multiplying Σ(1,1) by the first column of the V matrix, that is, αΩk=Σ(1,1)V[:,0]= [5.85 0.00 0.00]T.

[0099] Repeat the above steps until the minimum value of F is obtained.

[0100] The reconstructed dictionary matrix error is obtained by calculating the Frobenius norm between the original input matrix X and the data matrix Da reconstructed using the dictionary matrix D and the sparse coefficient matrix X. The error calculation formula is as follows: Error = ||X-Da||F; Calculate the Frobenius norm. The specific calculation formula is as follows: Where m and n are the number of rows and columns of the original input matrix X, respectively. ij is the element in row i and column j of X, (Dα) ij is the element in row i and column j of the reconstruction matrix.

[0101] If the error of the reconstructed dictionary matrix is ​​less than a preset threshold ε, it is considered to have converged, otherwise the dictionary matrix needs to be further updated.

[0102] Through the reconstruction and optimization of the dictionary library matrix, it has been clearly explained how to optimize the training of dictionary library matrices of different types and levels.

[0103] According to the above steps, a powerful and complete dictionary database is constructed. The defect categories of pipeline defect image category i and its defect level Di include: defect-free image, background noise image, leakage, rupture, undulation, misalignment, etc.; the defect levels include: first-level defect image, second-level defect image and third-level defect image.

[0104] Then, for the images of the above categories and defect levels, a dictionary database D is established respectively. Each dictionary database has a corresponding coefficient vector a. It is assumed that there are n dictionary databases D in total.

[0105] Examples are shown in Table 1 below: Table 1

[0106] The classification can be achieved by The feature vector of the input image is Y, and the errors between n dictionary databases D are calculated respectively.

[0107] Error=||Y-Da||F Analyze and compare the Error values. The defect type corresponding to the smallest error is the type of the image.

[0108] As shown in Table 2 below, when the input is Y1, E1=0.15, the minimum; otherwise, E2=0.47, E3=0.32, it can be concluded that the input image defect type is leakage.

[0109] Table 2

[0110] An embodiment of the present invention provides a method for constructing the above-mentioned dictionary library and the final state coefficient vector of the above-mentioned dictionary library based on the second eigenvector set based on the second eigenvector in the above-mentioned second eigenvector set, including: determining an atomic unit with the same number of columns as the above-mentioned second eigenvector based on the second eigenvector in the above-mentioned second eigenvector set; constructing an initial dictionary library based on the above-mentioned atomic unit; calculating an intermediate state coefficient vector based on the above-mentioned second eigenvector, the above-mentioned initial dictionary library and a randomly generated initial coefficient vector; the above-mentioned initial coefficient vector has the same dimension as the above-mentioned second eigenvector; constructing the above-mentioned dictionary library and the above-mentioned final state coefficient vector based on the above-mentioned intermediate state coefficient vector and the above-mentioned initial dictionary library. This method realizes the co-evolution of the dictionary library and the coefficient vector through a multi-objective optimization algorithm based on NSGA-II, significantly improving the sparsity, accuracy and global optimization ability of the feature representation, thereby achieving better representation efficiency and computational robustness in complex data modeling.

[0111] Example 3 Based on the above embodiments, Figure 3 A structural schematic diagram of a Pareto frontier driven intelligent identification device for municipal pipeline defects is provided for an embodiment of the present application.

[0112] As seen, the device comprises: Figure 3 A data acquisition module 31 for collecting images to be detected of the municipal pipeline. An identification module 32 for extracting features from the images to be detected to generate a first feature vector; calculating error values between the first feature vector and a plurality of dictionary libraries according to the first feature vector, the plurality of dictionary libraries, and terminal state feature coefficient vectors of the plurality of dictionary libraries; the construction method of the dictionary library comprising: collecting a sample image set with pipeline defect information; extracting features from sample images in the sample image set to generate a second feature vector set; constructing the dictionary library and the terminal state coefficient vector of the dictionary library based on the second generation of non-dominated sorting genetic algorithm according to the second feature vector set; selecting the smallest error value from the error values; determining the defect detection result of the images to be detected according to the target dictionary library corresponding to the smallest error value.

[0113] Wherein, the data acquisition module 31 is connected with the identification module 32.

[0114] In one of the embodiments, the identification module 32 is further configured to determine an atomic unit with the same column number as a second feature vector in the second feature vector set according to the second feature vector; construct an initial dictionary library according to the atomic unit; calculate an intermediate state coefficient vector according to the second feature vector, the initial dictionary library, and a randomly generated initial coefficient vector; the initial coefficient vector has the same dimension as the second feature vector; and construct the dictionary library and the terminal state coefficient vector according to the intermediate state coefficient vector and the initial dictionary library.

[0115]

[0116] ​In one embodiment, the identification module 32 is also used for step A1: splicing the above-mentioned atomic units of the above-mentioned initial dictionary library into a first vector code according to the preset order of columns; and, using binary coding or real number coding for the above-mentioned initial coefficient vector to obtain a second vector code; the number of constrained non-zero elements of the above-mentioned second vector code does not exceed the preset sparsity; step A2: randomly splicing the above-mentioned first vector code and the above-mentioned second vector code to generate an initial population; step A3: based on the Pareto dominance relationship, screening target individuals that meet the preset screening criteria from the above-mentioned initial population; step A4: performing an evolution operation on the above-mentioned target individuals to obtain evolved individuals; step A5: repeating the above-mentioned steps A1 to A4 until the preset number of iterations is reached or the frontier change of the Pareto dominance relationship is less than the preset threshold, and the final individual is obtained; step A6: determining the above-mentioned intermediate state coefficient vector based on the above-mentioned final individual.

[0117] In one embodiment, the identification module 32 is further configured to perform a crossover operation or a mutation operation on the target first vector code and the target second vector code corresponding to the target individual to obtain an evolved individual.

[0118] In one embodiment, the identification module 32 is also used to randomly exchange the columns of the target first vector code corresponding to the above-mentioned target individual to obtain an updated first vector code; and, use a uniform crossover method to exchange the columns of the target second vector code corresponding to the above-mentioned target individual, retain the non-zero parameters in the above-mentioned target second vector code, and obtain an updated second vector code; determine the evolved individual based on the above-mentioned updated first vector code and the above-mentioned updated second vector code.

[0119] In one embodiment, the identification module 32 is also used to randomly perturb the numerical value of the column of the target first vector code corresponding to the above-mentioned target individual to obtain an updated third vector code; flip the non-zero parameters of the target second vector code corresponding to the above-mentioned target individual with a preset probability to obtain an updated fourth vector code; and determine the evolved individual based on the above-mentioned updated third vector code and the above-mentioned updated fourth vector code. In one embodiment, the identification module 32 is further used to eliminate individuals whose updated first vector codes and updated second vector codes do not meet preset constraints, and obtain vector codes that have been constrained; and determine the vector codes that have been constrained as evolved individuals.

[0120] The Pareto-front-driven intelligent municipal pipeline defect identification device provided in the embodiments of the present invention shares the same technical features as the Pareto-front-driven intelligent municipal pipeline defect identification method provided in the aforementioned embodiments, and thus solves the same technical problems and achieves the same technical effects. Those skilled in the art will clearly understand that, for ease and brevity of description, the specific operating procedures of the device described above can be referenced to the corresponding procedures in the aforementioned method embodiments and will not be further described here.

[0121] Example 4 This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of a Pareto front-driven municipal pipeline defect intelligent identification method.

[0122] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a Pareto front-driven municipal pipeline defect intelligent identification method.

[0123] See also Figure 4 The structure diagram of an electronic device shown in the figure includes: a memory 41 and a processor 42. The memory 41 stores a computer program that can be run on the processor 42. When the processor executes the computer program, the steps provided by the above-mentioned Pareto front-driven municipal pipeline defect intelligent identification method are implemented.

[0124] like Figure 4 As shown, the device further includes: a bus 43 and a communication interface 44, and a processor 42, a communication interface 44 and a memory 41 are connected via the bus 43; the processor 42 is used to execute executable modules stored in the memory 41, such as computer programs.

[0125] The memory 41 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive. The communication connection between the device network element and at least one other network element is achieved through at least one communication interface 44 (which may be wired or wireless), and may use the Internet, a wide area network, a local area network, a metropolitan area network, etc.

[0126] The bus 43 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0127] Memory 41 is used to store programs, and processor 42 executes the programs after receiving execution instructions. The methods performed by the Pareto front-driven municipal pipeline defect intelligent identification device disclosed in any of the aforementioned embodiments of the present invention can be applied to or implemented by processor 42. Processor 42 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the aforementioned method can be completed by hardware integrated logic circuits in processor 42 or by software instructions. The aforementioned processor 42 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 41, and processor 42 reads the information in memory 41 and, in conjunction with its hardware, completes the steps of the method described above.

[0128] Furthermore, an embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor 42, the machine-executable instructions prompt the processor 42 to implement the above-mentioned Pareto front-driven municipal pipeline defect intelligent identification method.

[0129] The electronic device and computer-readable storage medium provided by the embodiments of the present invention have the same technical features, and therefore can solve the same technical problems and achieve the same technical effects.

[0130] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0131] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

Claims

1. A Pareto front-driven municipal pipeline defect intelligent identification method, characterized in that: include: Collect images of municipal pipelines to be inspected; Performing feature extraction on the image to be detected to generate a first feature vector; Calculating error values ​​between the first feature vector and the plurality of dictionary libraries according to the first feature vector, a plurality of preset dictionary libraries, and final-state feature coefficient vectors of the plurality of dictionary libraries; The dictionary library construction method includes: collecting a sample image set containing pipeline defect information; performing feature extraction on sample images in the sample image set to generate a second feature vector set; and constructing the dictionary library and a final state coefficient vector of the dictionary library based on the second feature vector set using a second-generation non-dominated sorting genetic algorithm; Filtering out a minimum error value from the error values; The defect detection result of the image to be detected is determined according to the target dictionary library corresponding to the minimum error value.

2. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 1 is characterized in that: The step of constructing the dictionary library and the final state coefficient vector of the dictionary library according to the second eigenvector set based on the second generation non-dominated sorting genetic algorithm includes: determining, according to the second eigenvector in the second eigenvector set, an atomic unit having the same number of columns as the second eigenvector; Constructing an initial dictionary library based on the atomic units; Calculating an intermediate state coefficient vector based on the second eigenvector, the initial dictionary library, and a randomly generated initial coefficient vector; the initial coefficient vector has the same dimension as the second eigenvector; The dictionary library and the final state coefficient vector are constructed according to the intermediate state coefficient vector and the initial dictionary library.

3. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 2 is characterized in that: The step of calculating the intermediate state coefficient vector according to the second eigenvector, the initial dictionary library, and the randomly generated initial coefficient vector comprises: Step A1: splicing the atomic units of the initial dictionary library into a first vector code according to a preset column order; and, using binary encoding or real number encoding on the initial coefficient vector to obtain a second vector code; the number of non-zero elements of the second vector code is constrained not to exceed a preset sparsity; Step A2: randomly concatenate the first vector code and the second vector code to generate an initial population; Step A3: Based on the Pareto dominance relationship, target individuals meeting the preset screening criteria are screened from the initial population; Step A4: performing an evolution operation on the target individual to obtain an evolved individual; Step A5: Repeat steps A1 to A4 until a preset number of iterations is reached or the frontier change of the Pareto dominance relationship is less than a preset threshold, thereby obtaining a final individual; Step A6: Determine the intermediate state coefficient vector based on the final individual.

4. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 3 is characterized in that: The step of performing an evolution operation on the target individual to obtain an evolved individual includes: A crossover operation or a mutation operation is performed on the target first vector code and the target second vector code corresponding to the target individual to obtain an evolved individual.

5. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 4 is characterized in that: The step of performing a cross operation on the target first vector code and the target second vector code corresponding to the target individual to obtain an evolved individual includes: Randomly swapping the columns of the target first vector code corresponding to the target individual to obtain an updated first vector code; and swapping the columns of the target second vector code corresponding to the target individual using a uniform crossover method, retaining the non-zero parameters in the target second vector code, to obtain an updated second vector code; An evolved individual is determined according to the updated first vector code and the updated second vector code.

6. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 5 is characterized in that: The step of performing a mutation operation on the target first vector code and the target second vector code corresponding to the target individual to obtain an evolved individual includes: Randomly perturbing the value of the column of the target first vector code corresponding to the target individual to obtain an updated third vector code; Flipping the non-zero parameters of the target second vector code corresponding to the target individual with a preset probability to obtain an updated fourth vector code; An evolved individual is determined according to the updated third vector code and the updated fourth vector code.

7. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 5 is characterized in that: The step of determining the evolved individual according to the updated first vector code and the updated second vector code comprises: Eliminating individuals whose updated first vector code and updated second vector code do not satisfy preset constraints, to obtain a vector code that has undergone constraint processing; The constrained vector is encoded and determined as an evolved individual.

8. The Pareto front-driven municipal pipeline defect intelligent identification method according to claim 1, characterized in that: The pipeline defect information includes: no defect, leakage, rupture, undulation, misalignment, and defect level information corresponding to the no defect, leakage, rupture, undulation, and misalignment, respectively.

9. A Pareto front-driven municipal pipeline defect intelligent identification device, characterized in that: include: A data acquisition module, used to collect images of municipal pipelines to be inspected; A recognition module, configured to extract features from the image to be detected and generate a first feature vector; Calculating error values ​​between the first feature vector and the plurality of dictionary libraries according to the first feature vector, a plurality of preset dictionary libraries, and final-state feature coefficient vectors of the plurality of dictionary libraries; The dictionary library construction method includes: collecting a sample image set containing pipeline defect information; performing feature extraction on sample images in the sample image set to generate a second feature vector set; constructing the dictionary library and the final state coefficient vector of the dictionary library based on the second feature vector set based on a second-generation non-dominated sorting genetic algorithm; screening out a minimum error value from the error values; and determining a defect detection result of the image to be inspected based on a target dictionary library corresponding to the minimum error value.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the Pareto front-driven municipal pipeline defect intelligent identification method described in any one of claims 1 to 8.

Images