Pipeline image classification method and device and electronic equipment

Abstract

The invention provides a pipeline image classification method and apparatus, and an electronic device. The method comprises the steps of collecting a pipeline image; extracting a first image feature vector of the pipeline image; calculating error values between the first image feature vector and the dictionary databases according to the first image feature vector, the dictionary databases and final state sparse vectors of the dictionary databases; screening a minimum error value from the error values; determining defect classification of the pipeline image according to the target dictionary database corresponding to the minimum error value; the construction method of the dictionary database comprises the following steps: acquiring an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing a dictionary database and a final state sparse vector of the dictionary database according to the second image feature vector. According to the method, pipeline image features are compared with the pre-constructed dictionary database, pipeline image defects are classified through the sparse representation technology, and the accuracy and efficiency of pipeline detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a pipeline image classification method, device and electronic equipment. Background Art

[0002] Currently, due to the presence of acidic or corrosive water in pipe networks, functional or structural defects can develop within drainage pipes over long periods of service. These defects include ruptures, leakage, deformation, corrosion, misalignment, disconnection, intrusion, concealed branch connections, foreign material intrusion, deposits, and obstructions. These defects can cause significant damage to the pipes, impacting their normal operation. Therefore, pipeline inspections are necessary, typically using methods such as television inspections, periscope inspections, sonar inspections, and manual observation. Specifically, this involves using video footage captured inside the pipes to identify the type of defect and facilitate appropriate repair measures.

[0003] Specifically, pipeline image classification methods primarily include support vector machines, convolutional neural networks, and particle swarm optimization algorithms. Support vector machines only support binary classification problems. While they can be extended to multi-classification problems, this increases programming complexity. Convolutional neural network models can be trained with tens of thousands of samples, offering strong data processing capabilities and a high level of intelligence. However, they still have some limitations. For example, the wide variety of pipeline defect types and complex pipeline environments result in low accuracy for deep learning-based automatic drainage pipeline defect identification.

[0004] Overall, existing pipeline image classification methods suffer from high programming complexity and low accuracy. Summary of the Invention

[0005] The purpose of the present invention is to provide a pipeline image classification method, device and electronic equipment to alleviate the technical problems of high programming complexity and low accuracy in existing pipeline image classification methods, reduce programming difficulty and increase classification accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a pipeline image classification method, comprising: acquiring a pipeline image; extracting a first image feature vector of the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and final-state sparse vectors of the multiple dictionary databases; screening a minimum error value from the error values; determining a defect classification of the pipeline image based on a target dictionary database corresponding to the minimum error value; a method for constructing the dictionary database comprising: obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector.

[0007] In a preferred embodiment of the present invention, the step of constructing the above-mentioned dictionary database and the final-state sparse vector of the above-mentioned dictionary database based on the above-mentioned second image feature vector includes: determining atomic units with the same number of columns as the above-mentioned second image feature vector based on the above-mentioned second image feature vector; constructing an initial dictionary database based on the above-mentioned atomic units; calculating an intermediate-state sparse vector based on the above-mentioned second image feature vector, the above-mentioned initial dictionary database and a randomly generated initial sparse vector; the above-mentioned initial sparse vector has the same dimension as the above-mentioned second image feature; constructing the above-mentioned dictionary database and the above-mentioned final-state sparse vector based on the above-mentioned intermediate-state sparse vector and the above-mentioned initial dictionary database.

[0008] In a preferred embodiment of the present invention, the step of calculating the intermediate sparse vector based on the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector includes: Step A1: calculating the calculation result of a preset objective function and the intermediate sparse vector corresponding to the calculation result based on the second image feature vector, the initial dictionary database, and the initial sparse vector; the objective function is: ;|| || o <L; where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the above calculation result, X is the second image feature vector, is the above-mentioned initial sparse vector; step A2: evaluating the performance parameters of each subset in the above-mentioned initial sparse vector based on a preset defined fitness function; step A3: screening the target subset in the above-mentioned initial sparse vector whose performance parameters are greater than a preset parameter threshold; step A4: performing genetic operations on the above-mentioned target subset to obtain a sub-intermediate sparse vector; step A5: repeating the above-mentioned steps A1 to A4 for a preset number of iterations until the value of the above-mentioned calculation result is minimized; step A6: determining the sub-intermediate sparse vector corresponding to the above-mentioned calculation result with the minimum value as the above-mentioned intermediate sparse vector.

[0009] In a preferred embodiment of the present invention, the above genetic operation includes: performing crossover, mutation and replacement operations on the above target subset.

[0010] In a preferred embodiment of the present invention, the step of constructing the dictionary database and the final-state sparse vector based on the intermediate-state sparse vector and the initial dictionary database includes: step B1: constructing an error matrix based on the intermediate-state sparse vector and the initial dictionary database; step B2: performing singular value decomposition on the error matrix to obtain an intermediate-state dictionary database and an updated sparse vector; step B3: calculating the Frobenius norm between the intermediate-state dictionary database and the second image feature vector; step B4: determining whether the Frobenius norm is less than a preset threshold; step B5: when the Frobenius norm is greater than or equal to the preset threshold, repeating steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determining the updated sparse vector and the intermediate-state dictionary database corresponding to the Frobenius norm less than the preset threshold as the final-state sparse vector and the dictionary database, respectively.

[0011] In a preferred embodiment of the present invention, the step of extracting the first image feature vector of the pipeline image includes: extracting color features and texture features of the pipeline image; and determining the first image feature vector based on the color features and the texture features.

[0012] In a preferred embodiment of the present invention, the step of extracting color features of the above-mentioned pipeline image includes: extracting color features of the above-mentioned pipeline image through color channels and a brightness model constructed by the above-mentioned color channels; the step of extracting texture features of the above-mentioned pipeline image includes: converting the above-mentioned pipeline image into a grayscale image; calculating a grayscale co-occurrence matrix of a preset direction based on the above-mentioned grayscale image; and calculating the Haralick texture feature corresponding to each of the above-mentioned directions based on the above-mentioned grayscale co-occurrence matrix.

[0013] In a preferred embodiment of the present invention, the above defect information includes: no defect, leakage, crack, undulation, misalignment and defect level information corresponding to the above no defect, leakage, crack, undulation and misalignment respectively.

[0014] In a second aspect, an embodiment of the present invention further provides a pipeline image classification device, comprising: a data acquisition module for collecting pipeline images; a type determination module for extracting a first image feature vector of the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and the final-state sparse vectors of the multiple dictionary databases; screening a minimum error value from the error values; determining a defect classification of the pipeline image based on a target dictionary database corresponding to the minimum error value; a method for constructing the dictionary database comprising: obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the pipeline image classification method.

[0016] The embodiments of the present invention have the following beneficial technical effects: Embodiments of the present invention provide a pipeline image classification method, device, and electronic device, comprising: capturing a pipeline image; extracting a first image feature vector from the pipeline image; calculating the error between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and the final-state sparse vectors of the multiple dictionary databases; selecting a minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The dictionary database is constructed by obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector. This method compares pipeline image features with a pre-constructed dictionary database and utilizes sparse representation technology to accurately classify pipeline image defects, thereby improving the accuracy and efficiency of pipeline inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a pipeline image classification method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a process for constructing a dictionary database and a final-state sparse vector of the dictionary database provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of a pipeline image classification device provided by an embodiment of the present invention; Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0019] Icons: 31 - data acquisition module; 32 - type determination module; 41 - memory; 42 - processor; 43 - bus; 44 - communication interface. DETAILED DESCRIPTION To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Because the water transported by pipelines contains acidic or corrosive substances, long-term service can lead to various functional or structural defects within drainage pipes, such as ruptures, leakage, deformation, and corrosion, seriously affecting their normal operation. To detect these defects, methods such as television inspection, pipeline periscope inspection, sonar inspection, and manual observation are commonly used. By interpreting internal pipeline video, the defect type can be identified and repair measures can be taken. Existing pipeline image classification methods mainly include support vector machines, convolutional neural networks, and particle swarm optimization algorithms. However, support vector machines are complex to program and difficult to extend to multi-classification problems. While convolutional neural networks have strong data processing capabilities and a high degree of intelligence, their automatic recognition accuracy is low due to the wide variety of pipeline defects and complex environments. Overall, existing methods suffer from high programming complexity and low accuracy.

[0021] Based on this, embodiments of the present invention provide a pipeline image classification method, apparatus, and electronic device. This method compares pipeline image features with a pre-built dictionary database and utilizes sparse representation technology to accurately classify pipeline image defects, thereby improving the accuracy and efficiency of pipeline inspection. To facilitate understanding, a pipeline image classification method is first introduced.

[0022] Example 1 In this embodiment, Figure 1 A schematic diagram of a pipeline image classification method provided by an embodiment of the present invention.

[0023] Depend on Figure 1 As can be seen, the method includes: Step S101: Acquire pipeline images.

[0024] Step S102: extracting a first image feature vector of the pipeline image.

[0025] In actual operation, the step S102 includes: extracting color features and texture features of the pipeline image; and determining the first image feature vector based on the color features and the texture features.

[0026] Furthermore, the step of extracting color features of the above-mentioned pipeline image includes: extracting color features of the above-mentioned pipeline image through color channels and a brightness model constructed by the above-mentioned color channels; the step of extracting texture features of the above-mentioned pipeline image includes: converting the above-mentioned pipeline image into a grayscale image; calculating the grayscale co-occurrence matrix of a preset direction based on the above-mentioned grayscale image; and calculating the Haralick texture features corresponding to each of the above-mentioned directions based on the above-mentioned grayscale co-occurrence matrix.

[0027] Specifically, the brightness model is I=0.21R+0.72G+0.07B, which is a weighted average of RGB channels.

[0028] Among them, I is the brightness model, R is the red channel, G is the green channel, and B is the blue channel.

[0029] Then, the pixel features of the first-order histogram of the image block of the pipeline image are extracted, and the calculation is as follows: Among them, the above pixel features are represented by a first-order histogram: {P(I k ); k=1,2,…,N} Among them, I k is the brightness model of the kth image block, and the total number of image blocks is N.

[0030] Three different features are obtained from each first-order histogram: ; ; ; in, is the average value, is the variance, is the kurtosis.

[0031] Therefore, 12 color feature values ​​can be obtained based on the pipeline image.

[0032] Furthermore, the present invention generates GLCM from the following three angles: 0°, 45° and 90°.

[0033] The GLCM of two pixel intensity values ​​in the above pipeline image at a distance d and direction ϕ is defined as P(Im,In,d,ϕ), which represents the probability of the occurrence of a pixel pair with pixel intensity values ​​of Im and In at a distance d and an angle ϕ.

[0034] Where N is the number of pixel intensity values; Im and In: two different gray levels of the above grayscale image; d: distance between pixel pairs; φ: direction angle between pixel pairs; Five Haralick texture features are obtained from GLCM, including: ; ; ; ; ; Among them, ASM is the angular second moment, CON is the contrast, IDF is the inverse difference moment, Corr is the correlation, and Var is the variance of the random variable.

[0035] The GLCM is calculated for three different angles θ, namely 0°, 45° and 90°, and the GLCM is generated from two pixel intensity values ​​(Im, In) at three distance values ​​d=1, 2 and 3.

[0036] Therefore, 9 gray-level co-occurrence matrices are generated for each image block. Furthermore, 5 texture features are calculated from each gray-level co-occurrence matrix, resulting in 45 texture feature vectors, denoted as WT.

[0037] Furthermore, 12 color features and 45 texture features are accumulated to form 57 image feature vectors represented as X. Therefore, the total number of entries of the first image feature vector of an image block is 57.

[0038] Step S103: calculating error values ​​between the first image feature vector and the plurality of dictionary databases according to the first image feature vector, a plurality of preset dictionary databases, and final-state sparse vectors of the plurality of dictionary databases.

[0039] Step S104: Filter the minimum error value from the above error values.

[0040] Step S105: Determine the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database includes: obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and a final sparse vector of the dictionary database based on the second image feature vector.

[0041] The defect information includes: no defect, leakage, crack, undulation, misalignment, and defect level information corresponding to the no defect, leakage, crack, undulation, and misalignment, respectively.

[0042] For ease of understanding, this application is explained in detail through the following actual cases: Assume that the above-mentioned multiple dictionary databases are the data in the following Table 1:

[0043] First, by inputting the first image feature vector, the error between the first image feature vector and each dictionary database is calculated respectively:

[0044] Where Error is the error, Y is the first image feature vector, D is the dictionary database, and a is the final sparse vector of the dictionary database.

[0045] Then, analyze and compare the error values. The defect type corresponding to the minimum error value is the defect classification of the pipeline image: As shown in Table 2 below, when the input first image feature vector is Y1, the corresponding error is 0.1, and the defect of the output pipeline image is classified as leakage.

[0046] Table 2

[0047] An embodiment of the present invention provides a pipeline image classification method, comprising: acquiring a pipeline image; extracting a first image feature vector from the pipeline image; calculating the error between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple pre-set dictionary databases, and the final-state sparse vectors of the multiple dictionary databases; selecting a minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The dictionary database is constructed by obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector. This method compares pipeline image features with a pre-constructed dictionary database and utilizes sparse representation technology to accurately classify pipeline image defects, thereby improving the accuracy and efficiency of pipeline inspection.

[0048] Example 2 In this embodiment, the process of constructing a dictionary database and a final-state sparse vector of the dictionary database is mainly introduced. Figure 2 A schematic diagram of a process for constructing a dictionary database and a final-state sparse vector of the dictionary database provided by an embodiment of the present invention.

[0049] Depend on Figure 2 As can be seen, the method includes: Step S201: determining, based on the second image feature vector, atomic units having the same number of columns as the second image feature vector.

[0050] To ensure that the number of atomic units in the dictionary matches the number of columns in the second image feature vector, we first need to understand the specific structure of the second image feature vector. Assume that the second image feature vector is a C-dimensional vector, meaning it has C columns, each representing a specific feature in the image (such as texture, color, or shape). Next, a dictionary containing C atomic units is constructed for this C-dimensional vector. Each atomic unit can be considered a basic pattern that describes local or global image features. During the initialization phase, these C atomic units can be generated randomly or based on prior knowledge. These atomic units serve as the basic elements of the dictionary, used to represent and reconstruct image features. During the optimization process, these atomic units are continuously adjusted to best fit the eigenvalues ​​of each column in the second image feature vector. Ultimately, each atomic unit will capture an important aspect of the image feature vector, allowing the entire dictionary to efficiently and accurately represent the complex image features. In short, determining the number of atomic units to match the number of columns in the second image feature vector ensures that the basis number of the dictionary matches the feature vector dimensionality, allowing for effective modeling and compression of image features during subsequent sparse representation and optimization. This not only improves the accuracy of feature representation but also enhances the overall performance of the model.

[0051] Step S202: constructing an initial dictionary database based on the above atomic units.

[0052] Here, based on the determined C atomic units, an initial dictionary database containing these atomic units is initialized for subsequent feature representation and optimization processes.

[0053] Step S203: Calculating an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector has the same dimension as the second image feature.

[0054] Step S204: constructing the dictionary database and the final-state sparse vector according to the intermediate-state sparse vector and the initial dictionary database.

[0055] In actual operation, the step of calculating the intermediate sparse vector according to the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector includes: Step A1: Calculate the calculation result of a preset objective function and the intermediate sparse vector corresponding to the calculation result based on the second image feature vector, the initial dictionary database and the initial sparse vector.

[0056] The above objective function is: ;|| || o <L; Wherein, L is the preset sparsity, D is the dictionary library matrix corresponding to the above initial dictionary database, F represents the above calculation result, X is the above second image feature vector, is the above initial sparse vector.

[0057] Step A2: Evaluate the performance parameters of each subset in the above initial sparse vector based on a preset defined fitness function.

[0058] Step A3: Filter the target subset whose performance parameter in the initial sparse vector is greater than a preset parameter threshold.

[0059] Step A4: Perform genetic operations on the target subset to obtain a sparse vector of the intermediate state; Step A5: Repeat the above steps A1 to A4 for a preset number of iterations until the value of the above calculation result is the minimum.

[0060] Step A6: Determine the sub-intermediate state sparse vector corresponding to the calculation result with the smallest value as the intermediate state sparse vector.

[0061] Furthermore, the above genetic operation includes: performing crossover, mutation and replacement operations on the above target subset.

[0062] In some embodiments, the step of constructing the dictionary database and the final-state sparse vector based on the intermediate-state sparse vector and the initial dictionary database includes: step B1: constructing an error matrix based on the intermediate-state sparse vector and the initial dictionary database; step B2: performing singular value decomposition on the error matrix to obtain an intermediate-state dictionary database and an updated sparse vector; step B3: calculating the Frobenius norm between the intermediate-state dictionary database and the second image feature vector; step B4: determining whether the Frobenius norm is less than a preset threshold; step B5: when the Frobenius norm is greater than or equal to the preset threshold, repeating steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determining the updated sparse vector and the intermediate-state dictionary database corresponding to the Frobenius norm less than the preset threshold as the final-state sparse vector and the dictionary database, respectively.

[0063] The error matrix is ​​constructed based on the intermediate state sparse vector and the initial dictionary database according to the following formula:

[0064] Where Ek is the error matrix, dk is the atomic unit, Ωk is the index set of all signals using the atomic unit, XΩk is the subset of all signals using the atomic unit, DΩk is the subset of the atomic units corresponding to the indices in Ωk, and αΩk is the sparse coefficient subset corresponding to the intermediate state sparse vector.

[0065] Furthermore, the error matrix is ​​subjected to singular value decomposition based on the following formula to obtain the intermediate state dictionary database and the updated sparse vector:

[0066] Among them, U, S and V are matrices after the error matrix is ​​decomposed by singular value.

[0067] Take the first column of U as the new atomic unit, and the product of the first element of S and the first column of V is the updated sparse vector.

[0068] The present invention uses the following actual case to illustrate the above steps in detail: Assume that the second image feature vector is , each column represents a set of feature vectors; The initial dictionary database is , each column represents an atomic unit; The first set of eigenvectors x1=[1,4,7]T is known; x1≈Dα1

[0069] Through genetic algorithm, we can get: , then we can get:

[0070] Similarly, we can get:

[0071]

[0072] Then we can get:

[0073] in, is a subset of the intermediate state sparse vector.

[0074] The atomic cells of the initial dictionary database are used by all signals, so the following calculation is performed:

[0075] Next, update the initial dictionary database. Given d1=[1,0,1]T, construct the error matrix E1:

[0076] Performing singular value decomposition on E1, we get the following results:

[0077]

[0078]

[0079] Thus, the updated atomic unit can be obtained, which is expressed as ,Right now At the same time, the updated sparse vector αΩk can be obtained by multiplying Σ(1,1) by the first column of the V matrix, that is, .

[0080] Repeat the above steps until the objective function reaches its minimum value.

[0081] Next, the error of the intermediate dictionary database is calculated. The error of the intermediate dictionary database is obtained by calculating the Frobenius norm between the second image feature vector and the initial dictionary database and the intermediate dictionary database. The error calculation formula is as follows: Error=||X−Da||F; Calculate the Frobenius norm. The specific calculation formula is as follows:

[0082] Where m and n are the number of rows and columns of the second image feature vector, respectively. is the element in row i and column j of the second image feature vector, It is the element in row i and column j in the intermediate state dictionary database.

[0083] Furthermore, if the Frobenius norm is less than a preset threshold ε, it is considered to have converged, otherwise the intermediate state dictionary database needs to be further updated.

[0084] An embodiment of the present invention provides a method for constructing a dictionary database and its final sparse vector, comprising: determining, based on the second image feature vector, atomic units having the same number of columns as the second image feature vector; constructing an initial dictionary database based on the atomic units; calculating an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector having the same dimension as the second image feature; and constructing the dictionary database and final sparse vector based on the intermediate sparse vector and the initial dictionary database. By constructing the initial dictionary database based on the second image feature vector and iteratively optimizing the sparse vector, this method achieves efficient and accurate representation of pipeline image features, thereby improving the accuracy and reliability of pipeline defect classification.

[0085] Example 3 Based on the above embodiments, Figure 3 A schematic diagram of the structure of a pipeline image classification device provided by an embodiment of the present invention.

[0086] Depend on Figure 3 As can be seen, the device includes: The data acquisition module 31 is used to collect pipeline images.

[0087] The type determination module 32 is configured to extract a first image feature vector of the pipeline image; calculate an error value between the first image feature vector and the plurality of dictionary databases based on the first image feature vector, a plurality of preset dictionary databases, and the final-state sparse vectors of the plurality of dictionary databases; select a minimum error value from the error values; and determine a defect classification of the pipeline image based on a target dictionary database corresponding to the minimum error value. A method for constructing the dictionary database includes: obtaining an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector.

[0088] The data acquisition module 31 is connected to the type determination module 32 .

[0089] In one embodiment, the device also includes: a dictionary construction module connected to the type determination module 32; the dictionary construction module is also used to determine, based on the above-mentioned second image feature vector, atomic units with the same number of columns as the above-mentioned second image feature vector; construct an initial dictionary database based on the above-mentioned atomic units; calculate an intermediate sparse vector based on the above-mentioned second image feature vector, the above-mentioned initial dictionary database and a randomly generated initial sparse vector; the above-mentioned initial sparse vector has the same dimension as the above-mentioned second image feature; and construct the above-mentioned dictionary database and the above-mentioned final sparse vector based on the above-mentioned intermediate sparse vector and the above-mentioned initial dictionary database.

[0090] In one embodiment, the dictionary construction module is further configured to perform step A1: calculating a calculation result of a preset objective function and the intermediate sparse vector corresponding to the calculation result based on the second image feature vector, the initial dictionary database, and the initial sparse vector; the objective function is: ;|| || o <L; where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the above calculation result, X is the second image feature vector, is the above-mentioned initial sparse vector; step A2: evaluating the performance parameters of each subset in the above-mentioned initial sparse vector based on a preset defined fitness function; step A3: screening the target subset in the above-mentioned initial sparse vector whose performance parameters are greater than a preset parameter threshold; step A4: performing genetic operations on the above-mentioned target subset to obtain a sub-intermediate sparse vector; step A5: repeating the above-mentioned steps A1 to A4 for a preset number of iterations until the value of the above-mentioned calculation result is minimized; step A6: determining the sub-intermediate sparse vector corresponding to the above-mentioned calculation result with the minimum value as the above-mentioned intermediate sparse vector.

[0091] In one embodiment, the dictionary construction module is further used to execute step B1: construct an error matrix based on the above-mentioned intermediate state sparse vector and the above-mentioned initial dictionary database; step B2: perform singular value decomposition on the above-mentioned error matrix to obtain the intermediate state dictionary database and the updated sparse vector; step B3: calculate the Frobenius norm between the above-mentioned intermediate state dictionary database and the above-mentioned second image feature vector; step B4: determine whether the above-mentioned Frobenius norm is less than a preset threshold; step B5: when the above-mentioned Frobenius norm is greater than or equal to the preset threshold, repeat the above-mentioned steps A1 to A6 and the above-mentioned steps B1 to B5 until the above-mentioned Frobenius norm is less than the above-mentioned preset threshold, and determine the updated sparse vector and the intermediate state dictionary database corresponding to the above-mentioned Frobenius norm less than the above-mentioned preset threshold as the above-mentioned final state sparse vector and the above-mentioned dictionary database, respectively.

[0092] In one embodiment, the type determination module 32 is further configured to extract color features and texture features of the pipeline image; and determine the first image feature vector based on the color features and the texture features.

[0093] In one embodiment, the type determination module 32 is further used to extract the color features of the above-mentioned pipeline image through the color channel and the brightness model constructed by the above-mentioned color channel; the step of extracting the texture features of the above-mentioned pipeline image includes: converting the above-mentioned pipeline image into a grayscale image; calculating the grayscale co-occurrence matrix of a preset direction based on the above-mentioned grayscale image; and calculating the Haralick texture features corresponding to each of the above-mentioned directions based on the above-mentioned grayscale co-occurrence matrix.

[0094] The pipeline image classification device provided in the embodiments of the present invention shares the same technical features as the pipeline image classification method provided in the aforementioned embodiments, and thus solves the same technical problems and achieves the same technical effects. Those skilled in the art will clearly understand that, for ease and brevity of description, the specific operating procedures of the device described above can be referenced to the corresponding procedures in the aforementioned method embodiments and will not be further described here.

[0095] Example 4 This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of the pipeline image classification method.

[0096] This embodiment provides a computer-readable storage medium storing a computer program, which implements the steps of the pipeline image classification method when executed by a processor.

[0097] See also Figure 4 The structure diagram of an electronic device shown in the figure includes: a memory 41 and a processor 42. The memory 41 stores a computer program that can be run on the processor 42. When the processor executes the computer program, the steps provided by the above-mentioned pipeline image classification method are implemented.

[0098] like Figure 4 As shown, the device further includes: a bus 43 and a communication interface 44, and a processor 42, a communication interface 44 and a memory 41 are connected via the bus 43; the processor 42 is used to execute executable modules stored in the memory 41, such as computer programs.

[0099] The memory 41 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive. The communication connection between the device network element and at least one other network element is achieved through at least one communication interface 44 (which may be wired or wireless), and may use the Internet, a wide area network, a local area network, a metropolitan area network, etc.

[0100] The bus 43 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0101] Memory 41 is used to store programs, and processor 42 executes the programs after receiving execution instructions. The methods performed by the pipeline image classification device disclosed in any of the aforementioned embodiments of the present invention can be applied to or implemented by processor 42. Processor 42 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the aforementioned method can be completed by hardware integrated logic circuits in processor 42 or software instructions. The aforementioned processor 42 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 41, and processor 42 reads the information in memory 41 and, in conjunction with its hardware, completes the steps of the method described above.

[0102] Furthermore, an embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor 42, the machine-executable instructions prompt the processor 42 to implement the above-mentioned pipeline image classification method.

[0103] The electronic device and computer-readable storage medium provided by the embodiments of the present invention have the same technical features, and therefore can solve the same technical problems and achieve the same technical effects.

[0104] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0105] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

Claims

1. A pipeline image classification method, characterized in that: include: Collect pipeline images; extracting a first image feature vector of the pipeline image; Calculating error values ​​between the first image feature vector and the plurality of dictionary databases according to the first image feature vector, a plurality of preset dictionary databases, and final-state sparse vectors of the plurality of dictionary databases; Selecting a minimum error value from the error values; determining a defect classification of the pipeline image according to a target dictionary database corresponding to the minimum error value; The method for constructing the dictionary database includes: obtaining an original pipeline image carrying defect information; Feature extraction is performed on the original pipeline image to obtain a second image feature vector; and the dictionary database and a final-state sparse vector of the dictionary database are constructed based on the second image feature vector.

2. The pipeline image classification method according to claim 1, characterized in that: The step of constructing the dictionary database and the final-state sparse vector of the dictionary database according to the second image feature vector includes: determining, according to the second image feature vector, an atomic unit having the same number of columns as the second image feature vector; constructing an initial dictionary database based on the atomic units; Calculating an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector has the same dimension as the second image feature; The dictionary database and the final-state sparse vector are constructed according to the intermediate-state sparse vector and the initial dictionary database.

3. The pipeline image classification method according to claim 2, characterized in that: The step of calculating an intermediate sparse vector according to the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector comprises: Step A1: calculating a calculation result of a preset objective function and the intermediate sparse vector corresponding to the calculation result according to the second image feature vector, the initial dictionary database, and the initial sparse vector; The objective function is: ;|| || o <L; Wherein, L is the preset sparsity, D is the dictionary library matrix corresponding to the initial dictionary database, F represents the calculation result, X is the second image feature vector, is the initial sparse vector; Step A2: evaluating the performance parameters of each subset in the initial sparse vector based on a preset defined fitness function; Step A3: screening the target subset in the initial sparse vector whose performance parameter is greater than a preset parameter threshold; Step A4: performing genetic operations on the target subset to obtain a sub-intermediate state sparse vector; Step A5: Repeating steps A1 to A4 for a preset number of iterations until the value of the calculation result is minimized; Step A6: Determine the sub-intermediate state sparse vector corresponding to the calculation result with the smallest value as the intermediate state sparse vector.

4. The pipeline image classification method according to claim 3, characterized in that: The genetic operation includes: performing crossover, mutation and replacement operations on the target subset.

5. The pipeline image classification method according to claim 3, characterized in that: The step of constructing the dictionary database and the final-state sparse vector according to the intermediate-state sparse vector and the initial dictionary database includes: Step B1: constructing an error matrix according to the intermediate state sparse vector and the initial dictionary database; Step B2: performing singular value decomposition on the error matrix to obtain an intermediate state dictionary database and update the sparse vector; Step B3: calculating the Frobenius norm between the intermediate dictionary database and the second image feature vector; Step B4: determining whether the Frobenius norm is less than a preset threshold; Step B5: When the Frobenius norm is greater than or equal to a preset threshold, repeat steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determine the updated sparse vector and the intermediate-state dictionary database corresponding to the Frobenius norm that is less than the preset threshold as the final-state sparse vector and the dictionary database, respectively.

6. The pipeline image classification method according to claim 1, characterized in that: The step of extracting a first image feature vector of the pipeline image comprises: Extracting color features and texture features of the pipeline image; The first image feature vector is determined according to the color feature and the texture feature.

7. The pipeline image classification method according to claim 2, characterized in that: The step of extracting the color features of the pipeline image comprises: Extracting color features of the pipeline image through color channels and a brightness model constructed by the color channels; The step of extracting texture features of the pipeline image comprises: converting the pipeline image into a grayscale image; Calculating a gray level co-occurrence matrix in a preset direction according to the gray level image; The Haralick texture feature corresponding to each of the directions is calculated according to the gray level co-occurrence matrix.

8. The pipeline image classification method according to claim 1, characterized in that: The defect information includes: no defect, leakage, crack, undulation, misalignment, and defect level information corresponding to the no defect, leakage, crack, undulation, and misalignment, respectively.

9. A pipeline image classification device, characterized in that: include: A data acquisition module, used for collecting pipeline images; a type determination module configured to extract a first image feature vector of the pipeline image; and calculate an error value between the first image feature vector and the plurality of dictionary databases based on the first image feature vector, a plurality of preset dictionary databases, and final-state sparse vectors of the plurality of dictionary databases; Selecting a minimum error value from the error values; determining a defect classification of the pipeline image according to a target dictionary database corresponding to the minimum error value; The method for constructing the dictionary database includes: obtaining an original pipeline image carrying defect information; Feature extraction is performed on the original pipeline image to obtain a second image feature vector; and the dictionary database and a final-state sparse vector of the dictionary database are constructed based on the second image feature vector.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the pipeline image classification method according to any one of claims 1 to 8.

Images