Pipeline image classification methods, devices and electronic equipment

Abstract

This invention provides a method, apparatus, and electronic device for pipeline image classification, comprising: acquiring a pipeline image; extracting a first image feature vector from the pipeline image; calculating an error value between the first image feature vector and multiple dictionary databases based on the first image feature vector, multiple dictionary databases, and the final state sparse vectors of the multiple dictionary databases; selecting the minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing a dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector. This method improves the accuracy and efficiency of pipeline detection by comparing pipeline image features with a pre-constructed dictionary database and using sparse representation technology to classify pipeline image defects.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a pipeline image classification method, apparatus, and electronic device. Background Technology

[0002] Currently, because the water transported by pipeline networks contains acidic or corrosive substances, functional or structural defects can occur inside drainage pipes during long-term operation. These defects include ruptures, leaks, deformation, corrosion, misalignment, disconnection, intrusion, concealed branch connections, foreign object penetration, sedimentation, and obstructions. These defects can cause varying degrees of damage to the pipelines, affecting their normal operation. Therefore, pipeline inspection is necessary, typically using methods such as television inspection, pipeline periscope inspection, sonar inspection, and manual observation. Specifically, this includes analyzing video footage of the pipeline's interior to identify the type of defect and facilitate appropriate repair measures.

[0003] Specifically, methods for pipe image classification mainly include Support Vector Machines (SVMs), Convolutional Neural Networks (CNNs), and Particle Swarm Optimization (PSO). SVMs only support binary classification problems; while they can be extended to multi-class classification, this increases programming complexity. CNN models can handle tens of thousands of training samples, have strong data processing capabilities, and a high degree of intelligence, but they still have some limitations. For example, the accuracy of automatic defect identification in drainage pipes based on deep learning is relatively low due to the diverse types of pipe defects and the complexity of pipe environments.

[0004] Overall, existing pipeline image classification methods suffer from high programming complexity and low accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a pipeline image classification method, apparatus, and electronic device to alleviate the technical problems of high programming complexity and low accuracy in existing pipeline image classification methods, thereby reducing programming difficulty and increasing classification accuracy.

[0006] In a first aspect, embodiments of the present invention provide a pipeline image classification method, comprising: acquiring a pipeline image; extracting a first image feature vector from the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, a preset multiple dictionary databases, and the final state sparse vectors of the multiple dictionary databases; selecting the minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database comprises: acquiring an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector.

[0007] In a preferred embodiment of the present invention, the step of constructing the dictionary database and the final state sparse vector of the dictionary database based on the second image feature vector includes: determining atomic units with the same number of columns as the second image feature vector based on the second image feature vector; constructing an initial dictionary database based on the atomic units; calculating an intermediate state sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector having the same dimension as the second image feature vector; and constructing the dictionary database and the final state sparse vector based on the intermediate state sparse vector and the initial dictionary database.

[0008] In a preferred embodiment of the present invention, the step of calculating an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector includes: Step A1: Calculating the result of a preset objective function and the corresponding intermediate sparse vector based on the second image feature vector, the initial dictionary database, and the initial sparse vector; the objective function is: ;|| || o <L; where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the calculation result, and X is the second image feature vector. The initial sparse vector is defined as follows: Step A2: Evaluate the performance parameters of each subset in the initial sparse vector based on the preset defined fitness function; Step A3: Filter the target subsets in the initial sparse vector whose performance parameters are greater than the preset parameter threshold; Step A4: Perform genetic operations on the target subsets to obtain the sub-intermediate sparse vectors; Step A5: Repeat steps A1 to A4 for the preset number of iterations until the value of the calculated result is minimized; Step A6: Determine the sub-intermediate sparse vector corresponding to the calculated result with the minimum value as the intermediate sparse vector.

[0009] In a preferred embodiment of the present invention, the genetic operation includes performing crossover, mutation, and replacement operations on the target subset.

[0010] In a preferred embodiment of the present invention, the step of constructing the dictionary database and the final state sparse vector based on the intermediate state sparse vector and the initial dictionary database includes: Step B1: Constructing an error matrix based on the intermediate state sparse vector and the initial dictionary database; Step B2: Performing singular value decomposition on the error matrix to obtain the intermediate state dictionary database and the updated sparse vector; Step B3: Calculating the Frobenius norm between the intermediate state dictionary database and the second image feature vector; Step B4: Determining whether the Frobenius norm is less than a preset threshold; Step B5: When the Frobenius norm is greater than or equal to the preset threshold, repeating steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determining the updated sparse vector and the intermediate state dictionary database corresponding to the Frobenius norm less than the preset threshold as the final state sparse vector and the dictionary database, respectively.

[0011] In a preferred embodiment of the present invention, the step of extracting the first image feature vector of the pipeline image includes: extracting the color features and texture features of the pipeline image; and determining the first image feature vector based on the color features and texture features.

[0012] In a preferred embodiment of the present invention, the step of extracting the color features of the pipeline image includes: extracting the color features of the pipeline image through color channels and a luminance model constructed from the color channels; the step of extracting the texture features of the pipeline image includes: converting the pipeline image into a grayscale image; calculating a grayscale co-occurrence matrix for a preset direction based on the grayscale image; and calculating Haralick texture features corresponding to each of the aforementioned directions based on the grayscale co-occurrence matrix.

[0013] In a preferred embodiment of the present invention, the defect information includes: no defect, leakage, crack, undulation, misalignment, and defect level information corresponding to the above-mentioned no defect, leakage, crack, undulation, and misalignment.

[0014] Secondly, embodiments of the present invention also provide a pipeline image classification device, comprising: a data acquisition module for acquiring pipeline images; a type determination module for extracting a first image feature vector from the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, a preset multiple dictionary databases, and the final state sparse vectors of the multiple dictionary databases; selecting the minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value; the method for constructing the dictionary database comprises: acquiring an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, which includes a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the pipeline image classification method.

[0016] The embodiments of the present invention have the following beneficial technical effects:

[0017] This invention provides a method, apparatus, and electronic device for pipeline image classification, comprising: acquiring a pipeline image; extracting a first image feature vector from the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and the final state sparse vectors of the multiple dictionary databases; selecting the minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector. This method, by comparing pipeline image features with a pre-constructed dictionary database, utilizes sparse representation technology to achieve accurate classification of pipeline image defects, thereby improving the accuracy and efficiency of pipeline detection. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a pipeline image classification method provided in an embodiment of the present invention.

[0020] Figure 2 A schematic diagram illustrating the process of constructing a dictionary database and the final state sparse vector of the dictionary database, provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of a pipeline image classification device provided in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0023] Icons: 31-Data acquisition module; 32-Type determination module; 41-Memory; 42-Processor; 43-Bus; 44-Communication interface. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Because the water transported by the pipeline network contains acidic or corrosive substances, long-term service leads to various functional or structural defects inside the drainage pipes, such as ruptures, leaks, deformations, and corrosion, severely affecting their normal operation. To detect these defects, methods such as television inspection, pipeline periscope inspection, sonar inspection, and manual observation are commonly used. By interpreting video footage of the inside of the pipeline, the type of defect is identified, and remedial measures can be taken. Existing pipeline image classification methods mainly include support vector machines, convolutional neural networks, and particle swarm optimization algorithms. However, support vector machines are complex to program and difficult to extend to multi-classification problems. While convolutional neural networks have strong data processing capabilities and high intelligence, their automatic identification accuracy is low due to the wide variety of pipeline defects and the complexity of the environment. Overall, existing methods suffer from high programming complexity and low accuracy.

[0026] Based on this, embodiments of the present invention provide a pipeline image classification method, apparatus, and electronic device. This method compares pipeline image features with a pre-constructed dictionary database and utilizes sparse representation techniques to achieve accurate classification of pipeline image defects, thereby improving the accuracy and efficiency of pipeline inspection. For ease of understanding, a pipeline image classification method is first introduced.

[0027] Example 1

[0028] In this embodiment, Figure 1 This is a flowchart illustrating a pipeline image classification method provided in an embodiment of the present invention.

[0029] Depend on Figure 1 As seen, the method includes:

[0030] Step S101: Acquire pipeline image.

[0031] Step S102: Extract the first image feature vector of the above pipeline image.

[0032] In actual operation, step S102 includes: extracting the color features and texture features of the pipeline image; and determining the first image feature vector based on the color features and texture features.

[0033] Furthermore, the step of extracting the color features of the pipeline image includes: extracting the color features of the pipeline image through color channels and a brightness model constructed from the color channels; the step of extracting the texture features of the pipeline image includes: converting the pipeline image into a grayscale image; calculating a grayscale co-occurrence matrix for a preset direction based on the grayscale image; and calculating the Haralick texture features corresponding to each of the aforementioned directions based on the grayscale co-occurrence matrix.

[0034] Specifically, the brightness model mentioned above is I=0.21R+0.72G+0.07B, which is a weighted average of the RGB channels.

[0035] Where I represents the luminance model, R represents the red channel, G represents the green channel, and B represents the blue channel.

[0036] Then, pixel features are extracted from the first-order histogram of the pipeline image patch, calculated as follows:

[0037] The aforementioned pixel features are represented by a first-order histogram:

[0038] {P(I k );k=1,2,…,N}

[0039] Among them, I k Let N be the brightness model for the k-th image patch, and let N be the total number of image patches.

[0040] Three distinct features were obtained from each first-order histogram:

[0041] ;

[0042] ;

[0043] ;

[0044] in, This is the average value. For variance, For peak value.

[0045] Therefore, 12 color feature values ​​can be obtained based on the pipeline image.

[0046] Furthermore, the present invention generates GLCM from the following three angles: 0°, 45°, and 90°.

[0047] In the pipeline image above, the GLCM of two pixel intensity values ​​at a distance d and an angle ϕ is defined as P(Im,In,d,ϕ), which represents the probability of a pixel pair with pixel intensity values ​​Im and In occurring at a distance d and an angle φ.

[0048] Where N is the number of pixel intensity values;

[0049] Im and In: Two different gray levels in the grayscale image above;

[0050] d: Distance between pixel pairs;

[0051] φ: directional angle between pixel pairs;

[0052] Five Haralick texture features were obtained from GLCM, specifically:

[0053] ;

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] Where ASM is the second moment of the angle, CON is the contrast, IDF is the inverse difference moment, Corr is the correlation, and Var is the variance of the random variable.

[0059] GLCM is calculated for three different angles θ, namely 0°, 45° and 90°, and GLCM is generated by two pixel intensity values ​​(Im, In) at three distance values ​​d=1, 2 and 3.

[0060] Therefore, nine gray-level co-occurrence matrices were generated for each image patch. Furthermore, five texture features were calculated from each gray-level co-occurrence matrix, resulting in 45 texture feature vectors, denoted as WT.

[0061] Furthermore, the 12 color features and 45 texture features are combined to form a total of 57 image feature vectors, represented as X. Therefore, the total number of entries in the first image feature vector of an image patch is 57.

[0062] Step S103: Calculate the error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, the preset multiple dictionary databases, and the final state sparse vectors of the multiple dictionary databases.

[0063] Step S104: Select the smallest error value from the above error values.

[0064] Step S105: Determine the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value; the method for constructing the dictionary database includes: acquiring the original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vector of the dictionary database based on the second image feature vector.

[0065] The aforementioned defect information includes: no defects, leakage, cracking, undulation, misalignment, and the defect level information corresponding to the aforementioned no defects, leakage, cracking, undulation, and misalignment.

[0066] To facilitate understanding, this application will be described in detail through the following practical examples:

[0067] Assume the above dictionary databases contain the data shown in Table 1 below:

[0068]

[0069] First, by inputting the first image feature vector, the error between the first image feature vector and each dictionary database is calculated:

[0070]

[0071] Where Error is the error, Y is the first image feature vector, D is the dictionary database, and a is the final state sparse vector of the dictionary database.

[0072] Then, the error values ​​are analyzed and compared. The defect type corresponding to the smallest error value is the defect classification of the pipeline image.

[0073] As shown in Table 2 below, when the first input image feature vector is Y1, the corresponding error is 0.1, and the defect classification of the output pipeline image is leakage.

[0074] Table 2

[0075]

[0076] This invention provides a pipeline image classification method, comprising: acquiring a pipeline image; extracting a first image feature vector from the pipeline image; calculating an error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and the final state sparse vectors of the multiple dictionary databases; selecting the minimum error value from the error values; and determining the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector. This method, by comparing pipeline image features with a pre-constructed dictionary database, utilizes sparse representation technology to achieve accurate classification of pipeline image defects, thereby improving the accuracy and efficiency of pipeline detection.

[0077] Example 2

[0078] This embodiment focuses on the process of constructing a dictionary database and the final state sparse vectors of the dictionary database. Figure 2 This is a schematic diagram illustrating the process of constructing a dictionary database and the final state sparse vector of the dictionary database, as provided in an embodiment of the present invention.

[0079] Depend on Figure 2 As seen, the method includes:

[0080] Step S201: Based on the second image feature vector, determine the atomic units with the same number of columns as the second image feature vector.

[0081] To ensure the number of atomic units in the dictionary matches the number of columns in the second image feature vector, it's first necessary to understand the specific structure of the second image feature vector. Assume the second image feature vector is a C-dimensional vector, meaning it has C columns, each representing a specific feature in the image (e.g., texture, color, or shape). Next, a dictionary containing C atomic units needs to be constructed for this C-dimensional vector. Each atomic unit can be viewed as a basic pattern describing local or global features of the image. During initialization, these C atomic units can be created randomly or based on prior knowledge. These atomic units will serve as the basic elements of the dictionary, used to represent and reconstruct image features. During optimization, these atomic units are continuously adjusted to fit each column of feature values ​​in the second image feature vector as closely as possible. Ultimately, each atomic unit will be able to capture a key aspect of the image feature vector, enabling the entire dictionary to efficiently and accurately represent the complex features of the image. In short, determining the number of atomic units to match the number of columns in the second image feature vector ensures that the number of base units in the dictionary matches the dimension of the feature vector, allowing for efficient modeling and compression of image features during subsequent sparse representation and optimization. This not only improves the accuracy of feature representation but also enhances the overall performance of the model.

[0082] Step S202: Construct an initial dictionary database based on the above atomic units.

[0083] Here, based on the determined C atomic units, an initial dictionary database containing these atomic units is initialized for subsequent feature representation and optimization processes.

[0084] Step S203: Calculate the intermediate sparse vector based on the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector; the initial sparse vector has the same dimension as the second image feature vector.

[0085] Step S204: Based on the above intermediate sparse vector and the above initial dictionary database, construct the above dictionary database and the above final sparse vector.

[0086] In practice, the steps for calculating the intermediate sparse vector based on the aforementioned second image feature vector, the aforementioned initial dictionary database, and the randomly generated initial sparse vector include:

[0087] Step A1: Based on the second image feature vector, the initial dictionary database, and the initial sparse vector, calculate the result of the preset objective function and the intermediate sparse vector corresponding to the calculation result.

[0088] The objective function is as follows: ;|| || o<L;

[0089] Where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the calculation result, and X is the second image feature vector. Let be the initial sparse vector mentioned above.

[0090] Step A2: Evaluate the performance parameters of each subset in the initial sparse vector based on the preset fitness function.

[0091] Step A3: Filter the target subset of the initial sparse vectors whose performance parameters are greater than the preset parameter threshold.

[0092] Step A4: Perform genetic operations on the above target subset to obtain the sparse vector of the sub-intermediate state;

[0093] Step A5: Repeat steps A1 to A4 above to reach the preset number of iterations until the value of the above calculation result is minimized.

[0094] Step A6: The sparse vector of the sub-intermediate state corresponding to the calculation result with the smallest value is determined as the intermediate sparse vector.

[0095] Furthermore, the genetic operations described above include crossover, mutation, and replacement operations on the target subset.

[0096] In some implementations, the step of constructing the dictionary database and the final state sparse vector based on the intermediate state sparse vector and the initial dictionary database includes: Step B1: Constructing an error matrix based on the intermediate state sparse vector and the initial dictionary database; Step B2: Performing singular value decomposition on the error matrix to obtain the intermediate state dictionary database and the updated sparse vector; Step B3: Calculating the Frobenius norm between the intermediate state dictionary database and the second image feature vector; Step B4: Determining whether the Frobenius norm is less than a preset threshold; Step B5: When the Frobenius norm is greater than or equal to the preset threshold, repeating steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determining the updated sparse vector and the intermediate state dictionary database corresponding to the Frobenius norm less than the preset threshold as the final state sparse vector and the dictionary database, respectively.

[0097] The error matrix is ​​constructed based on the aforementioned intermediate sparse vectors and the aforementioned initial dictionary database, using the following formula:

[0098]

[0099] Where Ek is the error matrix, dk is the atomic unit, Ωk is the set of indices of all signals using the atomic unit, XΩk is a subset of all signals using the atomic unit, DΩk is a subset of the atomic units corresponding to the indices in Ωk, and αΩk is a subset of the sparse coefficients corresponding to the above intermediate sparse vector.

[0100] Furthermore, singular value decomposition is performed on the above error matrix based on the following formula to obtain the intermediate state dictionary database and update the sparse vector:

[0101]

[0102] Where U, S, and V are the matrices obtained by singular value decomposition of the error matrix.

[0103] Take the first column of U as the new atomic unit, and the product of the first element of S and the first column of V is the updated sparse vector.

[0104] This invention provides a method for constructing a dictionary database and a final-state sparse vector of the dictionary database, comprising: determining atomic units with the same number of columns as the second image feature vector based on the second image feature vector; constructing an initial dictionary database based on the atomic units; calculating an intermediate-state sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; wherein the initial sparse vector has the same dimension as the second image feature vector; and constructing the dictionary database and the final-state sparse vector based on the intermediate-state sparse vector and the initial dictionary database. This method achieves efficient and accurate representation of pipeline image features by constructing an initial dictionary database based on the second image feature vector and iteratively optimizing the sparse vector, thereby improving the accuracy and reliability of pipeline defect classification.

[0105] Example 3

[0106] Based on the above embodiments, Figure 3 This is a schematic diagram of the structure of a pipeline image classification device provided in an embodiment of the present invention.

[0107] Depend on Figure 3 As seen, the device includes:

[0108] Data acquisition module 31 is used to acquire pipeline images.

[0109] The type determination module 32 is used to extract a first image feature vector from the pipeline image; calculate the error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases, and the final state sparse vectors of the multiple dictionary databases; select the minimum error value from the error values; and determine the defect classification of the pipeline image based on the target dictionary database corresponding to the minimum error value. The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; performing feature extraction on the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vectors of the dictionary database based on the second image feature vector.

[0110] The data acquisition module 31 is connected to the type determination module 32.

[0111] In one embodiment, the apparatus further includes a dictionary construction module connected to the type determination module 32; the dictionary construction module is further configured to determine, based on the second image feature vector, atomic units with the same number of columns as the second image feature vector; construct an initial dictionary database based on the atomic units; calculate an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector has the same dimension as the second image feature vector; and construct the dictionary database and the final sparse vector based on the intermediate sparse vector and the initial dictionary database.

[0112] In one embodiment, the dictionary construction module is further configured to perform step A1: based on the second image feature vector, the initial dictionary database, and the initial sparse vector, calculate the result of a preset objective function and the intermediate sparse vector corresponding to the calculation result; the objective function is: ;|| || o <L; where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the calculation result, and X is the second image feature vector. The initial sparse vector is defined as follows: Step A2: Evaluate the performance parameters of each subset in the initial sparse vector based on the preset defined fitness function; Step A3: Filter the target subsets in the initial sparse vector whose performance parameters are greater than the preset parameter threshold; Step A4: Perform genetic operations on the target subsets to obtain the sub-intermediate sparse vectors; Step A5: Repeat steps A1 to A4 for the preset number of iterations until the value of the calculated result is minimized; Step A6: Determine the sub-intermediate sparse vector corresponding to the calculated result with the minimum value as the intermediate sparse vector.

[0113] In one embodiment, the dictionary construction module is further configured to perform step B1: construct an error matrix based on the intermediate sparse vector and the initial dictionary database; step B2: perform singular value decomposition on the error matrix to obtain the intermediate dictionary database and the updated sparse vector; step B3: calculate the Frobenius norm between the intermediate dictionary database and the second image feature vector; step B4: determine whether the Frobenius norm is less than a preset threshold; step B5: when the Frobenius norm is greater than or equal to the preset threshold, repeat steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold, and determine the updated sparse vector and the intermediate dictionary database corresponding to the Frobenius norm less than the preset threshold as the final sparse vector and the dictionary database, respectively.

[0114] In one embodiment, the type determination module 32 is further configured to extract the color features and texture features of the pipeline image; and determine the first image feature vector based on the color features and texture features.

[0115] In one embodiment, the type determination module 32 is further configured to extract color features of the pipeline image through color channels and a brightness model constructed from the color channels; the step of extracting texture features of the pipeline image includes: converting the pipeline image into a grayscale image; calculating a grayscale co-occurrence matrix for a preset direction based on the grayscale image; and calculating Haralick texture features corresponding to each of the aforementioned directions based on the grayscale co-occurrence matrix.

[0116] The pipeline image classification device provided in this embodiment of the invention has the same technical features as the pipeline image classification method provided in the above embodiments, and therefore can solve the same technical problems and achieve the same technical effects. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0117] Example 4

[0118] This embodiment provides an electronic device, including a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of a pipeline image classification method.

[0119] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a pipeline image classification method.

[0120] See Figure 4 The diagram shows the structure of an electronic device, which includes a memory 41 and a processor 42. The memory 41 stores a computer program that can run on the processor 42. When the processor executes the computer program, it implements the steps provided by the pipeline image classification method described above.

[0121] like Figure 4 As shown, the device also includes a bus 43 and a communication interface 44, with the processor 42, the communication interface 44 and the memory 41 connected via the bus 43; the processor 42 is used to execute executable modules, such as computer programs, stored in the memory 41.

[0122] The memory 41 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 44 (which may be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0123] Bus 43 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0124] The memory 41 stores the program, and the processor 42 executes the program after receiving the execution instruction. The method executed by the pipeline image classification device disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 42, or implemented by the processor 42. The processor 42 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 42 or by instructions in the form of software. The processor 42 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 41, and processor 42 reads information from memory 41 and, in conjunction with its hardware, completes the steps of the above method.

[0125] Furthermore, this embodiment of the invention also provides a machine-readable storage medium storing machine-executable instructions. When these machine-executable instructions are invoked and executed by the processor 42, they cause the processor 42 to implement the aforementioned pipeline image classification method.

[0126] The electronic devices and computer-readable storage media provided in the embodiments of the present invention have the same technical features, so they can also solve the same technical problems and achieve the same technical effects.

[0127] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0128] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

Claims

1. A method for classifying pipe images, characterized in that, include: Acquire pipeline images; Extract the first image feature vector from the pipeline image; Based on the first image feature vector, a plurality of preset dictionary databases, and the final state sparse vectors of the plurality of dictionary databases, calculate the error value between the first image feature vector and the plurality of dictionary databases; Select the minimum error value from the error values; Based on the target dictionary database corresponding to the minimum error value, the defect classification of the pipeline image is determined; The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vector of the dictionary database based on the second image feature vector. The steps of constructing the dictionary database and the final-state sparse vector of the dictionary database based on the second image feature vector include: Based on the second image feature vector, determine the atomic units with the same number of columns as the second image feature vector; Based on the atomic units, construct an initial dictionary database; Based on the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector, an intermediate sparse vector is calculated; the initial sparse vector has the same dimension as the second image feature vector. Based on the intermediate sparse vector and the initial dictionary database, construct the dictionary database and the final sparse vector; The step of calculating the intermediate sparse vector based on the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector includes: Step A1: Based on the second image feature vector, the initial dictionary database, and the initial sparse vector, calculate the result of the preset objective function and the corresponding sub-intermediate sparse vector; The objective function is: ;|| || o <L; Where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the calculation result, and X is the second image feature vector. Let be the initial sparse vector; Step A2: Evaluate the performance parameters of each subset in the initial sparse vector based on the preset defined fitness function; Step A3: Filter the target subset of the initial sparse vector whose performance parameter is greater than a preset parameter threshold; Step A4: Perform genetic operations on the target subset to obtain the sparse vector of the sub-intermediate state; Step A5: Repeat steps A1 to A4 until the preset number of iterations is reached, until the value of the calculation result is minimized; Step A6: Determine the sparse vector of the intermediate state corresponding to the calculation result with the smallest value as the intermediate state sparse vector.

2. The pipeline image classification method according to claim 1, characterized in that, The genetic operations include crossover, mutation, and replacement operations on the target subset.

3. The pipeline image classification method according to claim 1, characterized in that, The steps of constructing the dictionary database and the final state sparse vector based on the intermediate state sparse vector and the initial dictionary database include: Step B1: Construct an error matrix based on the intermediate sparse vector and the initial dictionary database; Step B2: Perform singular value decomposition on the error matrix to obtain an intermediate state dictionary database and update the sparse vector; Step B3: Calculate the Frobenius norm between the intermediate state dictionary database and the second image feature vector; Step B4: Determine whether the Frobenius norm is less than a preset threshold; Step B5: When the Frobenius norm is greater than or equal to a preset threshold, repeat steps A1 to A6 and steps B1 to B5 until the Frobenius norm is less than the preset threshold. Then, update the sparse vector and intermediate state dictionary database corresponding to the Frobenius norm that is less than the preset threshold, and determine them as the final state sparse vector and the dictionary database, respectively.

4. The pipeline image classification method according to claim 1, characterized in that, The step of extracting the first image feature vector of the pipeline image includes: Extract the color and texture features of the pipeline image; The first image feature vector is determined based on the color features and the texture features.

5. The pipeline image classification method according to claim 4, characterized in that, The step of extracting the color features of the pipeline image includes: The color features of the pipeline image are extracted using the color channels and the brightness model constructed from the color channels. The step of extracting texture features from the pipeline image includes: Convert the pipeline image into a grayscale image; Calculate the gray-level co-occurrence matrix in a preset direction based on the gray-level image; Based on the gray-level co-occurrence matrix, calculate the Haralick texture feature corresponding to each direction.

6. The pipeline image classification method according to claim 1, characterized in that, The defect information includes: no defects, leakage, cracks, undulations, misalignments, and the defect level information corresponding to the no defects, leakage, cracks, undulations, and misalignments.

7. A pipeline image classification device, characterized in that, include: The data acquisition module is used to collect images of the pipeline. The type determination module is used to extract the first image feature vector of the pipeline image; and calculate the error value between the first image feature vector and the multiple dictionary databases based on the first image feature vector, multiple preset dictionary databases and the final state sparse vectors of the multiple dictionary databases. Select the minimum error value from the error values; Based on the target dictionary database corresponding to the minimum error value, the defect classification of the pipeline image is determined; The method for constructing the dictionary database includes: acquiring an original pipeline image carrying defect information; extracting features from the original pipeline image to obtain a second image feature vector; and constructing the dictionary database and the final state sparse vector of the dictionary database based on the second image feature vector. The type determination module is used to determine atomic units with the same number of columns as the second image feature vector based on the second image feature vector; construct an initial dictionary database based on the atomic units; calculate an intermediate sparse vector based on the second image feature vector, the initial dictionary database, and a randomly generated initial sparse vector; the initial sparse vector has the same dimension as the second image feature vector; and construct the dictionary database and the final sparse vector based on the intermediate sparse vector and the initial dictionary database. The step of calculating the intermediate state sparse vector based on the second image feature vector, the initial dictionary database, and the randomly generated initial sparse vector includes: Step A1: Calculating the result of a preset objective function and the corresponding sub-intermediate state sparse vector based on the second image feature vector, the initial dictionary database, and the initial sparse vector; the objective function is: ;|| || o <L; where L is the preset sparsity, D is the dictionary matrix corresponding to the initial dictionary database, F represents the calculation result, and X is the second image feature vector. The initial sparse vector is defined as follows: Step A2: Evaluate the performance parameters of each subset in the initial sparse vector based on a preset defined fitness function; Step A3: Filter the target subsets in the initial sparse vector whose performance parameters are greater than a preset parameter threshold; Step A4: Perform a genetic operation on the target subsets to obtain a sub-intermediate sparse vector; Step A5: Repeat steps A1 to A4 for a preset number of iterations until the value of the calculation result is minimized; Step A6: Determine the sub-intermediate sparse vector corresponding to the calculation result with the minimum value as the intermediate sparse vector.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the pipeline image classification method according to any one of claims 1 to 6.

Images