Underground drainage pipe defect detection method, device and computer equipment

Abstract

The present invention relates to a method, device and computer equipment for detecting defects in underground drainage pipes. The method for detecting defects in underground drainage pipes performs multi-level triage detection and integrates an uncommon pipe defect detection model learned from small samples, significantly improving the ability to detect defects in underground drainage pipes. Compared with the existing single detection model, the multi-level triage detection can more reasonably utilize the characteristics of the drainage pipe defect detection model based on the convolutional neural network, and implement detection in a guided manner according to the actual situation of the sample library and the applicable conditions of the detection model, so as to achieve accurate identification of normal and defective conditions and reasonable diversion detection, and then accurately subdivide common defects and uncommon defects for the screened defect image set, thereby improving detection efficiency and detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of pipeline detection, and in particular to a method, a device and a computer equipment for detecting defects in underground drainage pipelines. Background Art

[0002] The urban underground drainage network is one of the important infrastructures to ensure the normal operation of the city, which directly affects the urban environment, social economy and people's life safety. With the continuous acceleration of the urbanization process, the long-term use of drainage pipes will cause various defects such as pipe blockage, pipe corrosion, cracks, etc., which will lead to safety accidents such as road collapse and waterlogging, seriously affecting the operation of the city and the normal life of citizens, and even causing immeasurable economic losses. Therefore, regular inspection and evaluation of drainage pipes and maintenance of defective pipes are necessary means to ensure the safe operation of the drainage pipe system. At present, the use of pipeline robot closed-circuit television (CCTV) for pipeline defect detection is one of the commonly used pipeline defect detection technologies.

[0003] However, the interpretation of pipeline defects and health status assessment based on CCTV inspection videos is still mainly done manually, which requires high professional skills from technicians and has problems such as high cost and low efficiency. Although computer vision algorithms have been applied to the field of drainage pipe inspection, the internal environment of drainage pipes is extremely complex. Traditional visual algorithms and machine learning algorithms only analyze a small number of defect features, resulting in missed detection and false detection. The detection model has poor generalization ability and high false detection rate, and cannot meet the accuracy requirements of drainage pipe inspection. Summary of the invention

[0004] Based on this, it is necessary to provide an underground drainage pipe defect detection method, device and computer equipment that improve generalization ability and reduce false detection rate.

[0005] In order to achieve the above-mentioned purpose, on the one hand, the embodiment of the present application provides a method for detecting defects in underground drainage pipes, comprising the steps of:

[0006] A small sample pipeline defect detection model is constructed; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results based on the image tensor features, and output the pipeline defect type based on the score of the detailed feature measurement results;

[0007] Use the small sample learning dataset to train the small sample pipeline defect detection model;

[0008] Acquire detection images of underground drainage pipes;

[0009] Input the inspection image into the normal / defect binary classification model and output the classification result;

[0010] Input the detection image with the classification result of defect into the common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0011] The detection images with similarity values ​​less than the preset value are input into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0012] In one of the embodiments, the small sample learning dataset includes an uncommon defect sample query set, a support set, and a common defect auxiliary set;

[0013] The steps of training a small sample pipeline defect detection model using a small sample learning dataset include:

[0014] Randomly selecting a first number of categories from the common defect auxiliary set, and selecting a second number of samples in each category to form a training task;

[0015] Generate a pre-trained model based on each training task;

[0016] Transfer the knowledge learned by the pre-trained model to the training task of training a small sample pipeline defect detection model.

[0017] In one embodiment, the steps are also included:

[0018] Extracting first image tensor features of images in a query set of uncommon defect samples and second image tensor features of any defect category in a support set;

[0019] Calculating the cosine similarity between the detail description operator of the first image tensor feature and the detail description operator of the second image tensor feature;

[0020] According to the cosine similarity, the metric value between the image in the query set of uncommon defect samples and any defect category in the support set is obtained;

[0021] According to the measurement value, the pipeline defect category of the inspection image is determined.

[0022] In one embodiment, in the step of calculating the cosine similarity between the detail description operator of the first image tensor feature and the detail description operator of the second image tensor feature, the cosine similarity is obtained based on the following formula:

[0023]

[0024] in, is the cosine similarity; x iis a detailed description operator of the first image tensor feature; It is the detailed description operator of the second image tensor feature.

[0025] In one embodiment, in the step of obtaining a metric value between an image in a query set of uncommon defect samples and any defect category in a support set according to cosine similarity, the metric value is obtained based on the following formula:

[0026]

[0027] Ψ(q) is the first image tensor feature, c is any defect category in the support set; g(Ψ(q), c) is the metric value; n is the product of the height and width of the image; k is the number of the nearest detail description operators, which can be a natural number greater than 0.

[0028] In one embodiment, the step of determining the pipeline defect category of the inspection image according to the metric value includes:

[0029] The defect category corresponding to the highest metric value is determined as the defect category of the image in the uncommon defect sample query set.

[0030] In one embodiment, the steps are also included:

[0031] Receive inspection videos of underground drainage pipes;

[0032] Extract key frames from the detection video and convert the key frames into a detection image set;

[0033] The detection image set is preprocessed to obtain the detection image of the underground drainage pipe.

[0034] In one embodiment, the step of preprocessing the detection image set to obtain the detection image of the underground drainage pipe includes:

[0035] The detection image sets are processed with data enhancement in sequence to obtain detection images of underground drainage pipes.

[0036] On the one hand, an embodiment of the present invention provides an underground drainage pipe defect detection device, comprising:

[0037] A construction module is used to construct a small sample pipeline defect detection model; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results based on the image tensor features, and output the pipeline defect type based on the score of the detailed feature measurement results;

[0038] A training module, used to train a small sample pipeline defect detection model using a small sample learning data set;

[0039] An acquisition module, used for acquiring a detection image of an underground drainage pipe;

[0040] The first model processing module is used to input the inspection image into the normal / defect binary classification model and output the classification result;

[0041] The second model processing module is used to input the detection image with the classification result of defect into the common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0042] The third model processing module is used to input the detection image with a similarity value less than a preset value into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0043] On the other hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned underground drainage pipe defect detection method when executing the computer program.

[0044] One of the above technical solutions has the following advantages and beneficial effects:

[0045] The above-mentioned underground drainage pipeline defect detection method performs multi-level triage detection and integrates the uncommon pipeline defect detection model learned from small samples, which significantly improves the underground drainage pipeline defect detection capability. Compared with the existing single detection model, the multi-level triage detection can more reasonably utilize the characteristics of the drainage pipeline defect detection model based on the convolutional neural network, and carry out guided detection according to the actual situation of the sample library and the applicable conditions of the detection model, so as to achieve accurate identification and reasonable diversion detection of normal and defective parts, and then accurately segment common defects and uncommon defects for the selected defect image set, which can improve the detection efficiency and detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 is a schematic flow chart of a method for detecting defects in underground drainage pipes in one embodiment;

[0049] Figure 2is a schematic flow chart of steps for training a small sample pipeline defect detection model using a small sample learning data set in one embodiment;

[0050] Figure 3 It is a structural diagram of a network model for identifying defects in a small sample pipeline in an embodiment;

[0051] Figure 4 is a schematic diagram of a deep detail description operator in one embodiment;

[0052] Figure 5 is a schematic flow chart of the steps of acquiring a detection image of an underground drainage pipe in an embodiment;

[0053] Figure 6 4 is a block diagram of an underground drainage pipe defect detection device in one embodiment. DETAILED DESCRIPTION

[0054] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0056] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0057] It can be understood that the “connection” in the following embodiments should be understood as “electrical connection”, “communication connection”, etc. if the connected circuits, modules, units, etc. have electrical signals or data transmission between each other.

[0058] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.

[0059] At present, when deep learning technology is used for automatic detection of underground drainage pipeline defects, the lack of drainage pipeline defect sample sets and the imbalance of samples between defect types greatly reduce the generalization ability and recognition robustness of pipeline defect recognition models. At present, domestic and foreign researchers have applied deep learning technology to automatic recognition of underground drainage pipeline defects, but there are few studies on intelligent recognition of relatively rare pipeline defects. In order to solve the problem of small and unbalanced sample sets, a common solution is to enhance small sample pipeline defects by data enhancement methods such as flipping, scaling, and cropping of scarce sample sets. With the development of generative adversarial network technology, some studies have introduced transfer learning and generative adversarial networks to expand pipeline defect sample data sets and then train recognition networks. However, data enhancement technology has supplemented the deficiency of small samples to a certain extent, but the generated defect images and original images are still highly correlated. How to improve the generalization ability and recognition robustness of small sample drainage pipeline defect detection models is still a problem worth exploring. The underground drainage pipeline defect detection method proposed in this application can effectively solve the above problems.

[0060] In one embodiment, Figure 1 As shown, a method for detecting defects in underground drainage pipes is provided, comprising the steps of:

[0061] S110, constructing a small sample pipeline defect detection model; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results according to the image tensor features, and output the pipeline defect type according to the score of the detailed feature measurement results;

[0062] Among them, the feature extraction network is used to input the pipeline defect data X and output the image feature Ψ(X), which is specifically expressed in the form of a tensor, that is, the image tensor feature mentioned above. Furthermore, the detailed feature measurement result is obtained in the measurement module, and finally the defect type is identified by the score of the detailed feature measurement result. The feature extraction network can select a general convolutional neural network model.

[0063] S120, training a small sample pipeline defect detection model using a small sample learning data set;

[0064] S130, acquiring a detection image of the underground drainage pipe;

[0065] Specifically, the detection image can be obtained by any means in the art. For example, it can be obtained by shooting by a pipeline robot. It can also be obtained by detecting a video and further converting it into a detection image.

[0066] S140, inputting the inspection image into a normal / defect binary classification model and outputting a classification result;

[0067] Among them, the normal / defect binary classification model is used to divide the detection image into defect and normal. The classification results are also defect and normal. When training the normal / defect binary classification model, a general convolutional neural network model, such as ResNet and its variants, Conv64F, etc., can be selected as the feature extraction network for drainage pipe defect recognition, and the network model is trained by continuously adjusting the hyperparameters of the model. Set indicators for evaluating model performance, input validation set data to verify the optimal defect detection model obtained after training, and scientifically evaluate the recognition robustness and generalization ability of the model in order to obtain the optimal network detection model. It is worth noting that in the process of training the normal / defect binary classification model, all subdivided defects will be uniformly marked as defects to participate in training, and normal samples will not participate in training in the process of common pipeline defect detection models.

[0068] Specifically, the sample library used for training can be established based on the following means: collecting drainage pipe defect data samples; the sample library needs to pre-process such images when adopting them. The specific contents include data cleaning and data enhancement: Among them, data cleaning mainly performs frame processing on the detection video, the purpose is to convert the video into RGB three-channel images, and remove black edges, logos and annotated text on the pipeline defect sample data that may affect the training effect. Data enhancement mainly performs random horizontal and vertical flipping, rotation, contrast adjustment and size scaling on small sample pipeline defect images; then the drainage pipe defect samples are manually classified and labeled and stored, and detailed annotations are made for the non-defective drainage pipe detection samples: normal (ZC), and the defective drainage pipe detection samples: defective (QX). The pipeline defect samples are further subdivided into structural defects and functional defect samples of drainage pipeline defects: rupture (PL), deformation (BX), corrosion (FS), misalignment (CK), undulation (QF), disconnection (TJ), material falling off (TL), branch pipe concealed connection (AJ), foreign body penetration (CR), leakage (SL), deposition (CJ), scaling (JG), obstacle (ZW), residual wall and dam roots (CQ), tree roots (SG), slag (FZ), etc., and are randomly selected according to a certain proportion to divide the training set, validation set and test set.

[0069] S150, inputting the detection image with the classification result of defect into a common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0070] Specifically, the normal / defect binary classification model performs the first-level intelligent detection on the inspection image set to screen out the inspection images with pipeline defects; if the inspection image has no defects, it determines whether the inspection of all images is completed. If not, it continues to inspect the next inspection image until the inspection of all images is completed.

[0071] Furthermore, the common pipeline defect detection model detects common pipeline defects according to the similarity value, specifically, confirms the pipeline defect category with the highest similarity as the defect category of the detection image.

[0072] S160, inputting the detection image with a similarity value less than a preset value into a trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0073] Specifically, the similarity value can be a fixed value or a dynamic value. In one example, the dynamic value can be set specifically by the following steps: construct an objective function including a false positive rate and a missed detection rate; wherein, the false positive rate is the proportion of uncommon defects detected as common defects; the missed detection rate is the proportion of common defects detected as uncommon defects. The false positive rate and the missed detection rate are combined to form an evaluation index that comprehensively considers both. Then the multi-objective particle swarm optimization algorithm optimizes the objective function. These algorithms can find a set of non-inferior solutions considering multiple objectives, that is, no solution dominates another solution, and none of these solutions has better or worse performance. During the optimization process, a series of candidate solutions are generated, each corresponding to a set of thresholds for similarity values. The combination of these thresholds will be designed as a solution that minimizes or maximizes the objective function. Finally, the best solution is selected from all non-inferior solutions as the final detection threshold. This selection can be based on a lower missed detection rate. The objective function is a function that is optimized in an optimization problem, which is usually a mathematical formula used to measure the quality of the objective. In this case, the objective function is used to measure the comprehensive performance of the false positive rate and the missed detection rate, and the best detection threshold is determined based on these performance indicators. Specifically, the objective function can be in the following form: Objective function = weight 1 × false positive rate + weight 2 × missed detection rate, where false positive rate and missed detection rate are two key performance indicators of the model, which are used to measure the accuracy and reliability of the detection system. Weight 1 and weight 2 are parameters used to balance the relationship between over-detection rate and missed detection rate, and their selection can be determined according to actual needs and application scenarios. For example, if you are more sensitive to the false positive rate, you can give the false positive rate a higher weight; conversely, if you are more concerned about missed detection, you can adjust the weight to reduce the missed detection rate. By optimizing the objective function, you can find a set of parameters that minimize or maximize the objective function, that is, the optimal detection threshold. Such a combination of parameters can achieve a reasonable balance between over-detection rate and missed detection rate, thereby obtaining more accurate and reliable detection results.

[0074] Furthermore, if the similarity requirement is met, that is, the similarity between the pipeline defect type and the detection image is greater than or equal to the preset value, it is determined whether the detection of all images is completed. If not, the detection of the next inspection image is continued until the detection of all images is completed. It should be noted that after the small sample pipeline defect detection model is detected, it is determined whether the detection of all images is completed. If not, the detection of the next inspection image is continued until the detection of all images is completed.

[0075] The above-mentioned underground drainage pipeline defect detection method performs multi-level triage detection and integrates the uncommon pipeline defect detection model learned from small samples, which significantly improves the underground drainage pipeline defect detection capability. Compared with the existing single detection model, the multi-level triage detection can more reasonably utilize the characteristics of the drainage pipeline defect detection model based on the convolutional neural network, and carry out guided detection according to the actual situation of the sample library and the applicable conditions of the detection model, so as to achieve accurate identification and reasonable diversion detection of normal and defective parts, and then accurately segment common defects and uncommon defects for the selected defect image set, which can improve the detection efficiency and detection accuracy.

[0076] In one of the embodiments, the small sample learning dataset includes an uncommon defect sample query set, a support set, and a common defect auxiliary set;

[0077] like Figure 2 As shown in FIG. 1 , the steps of training a small sample pipeline defect detection model using a small sample learning dataset include:

[0078] S210, randomly selecting a first number of categories from the common defect auxiliary set, and selecting a second number of samples in each category to form a training task;

[0079] S220, generating a pre-training model based on each training task;

[0080] S230, transferring the knowledge learned by the pre-trained model to the training task of training a small sample pipeline defect detection model.

[0081] Specifically, during the training of the small sample pipeline defect detection model, multiple tasks are constructed for training. Each episodic task randomly extracts N-way K-shot samples from the common defect auxiliary set, that is, N categories, K number of samples in each category; N and K are the first number and the second number mentioned above, and the extracted samples are divided into a support set and a query set. The model learned through thousands of training tasks is the pre-trained model, which can be used to classify images in the uncommon defect sample query set through knowledge transfer. Furthermore, during the test process, the pre-trained model can be randomly selected from the common defect auxiliary set for testing.

[0082] In one embodiment, the steps are also included:

[0083] Extracting first image tensor features of images in a query set of uncommon defect samples and second image tensor features of any defect category in a support set;

[0084] Specifically, this step can be performed by a feature extraction network.

[0085] Calculating the cosine similarity between the detail description operator of the first image tensor feature and the detail description operator of the second image tensor feature;

[0086] For details, please refer to the small sample pipeline defect detection model. Figure 3 This step can be performed by the measurement module of the small sample pipeline defect detection model. According to the cosine similarity, the measurement value between the image in the uncommon defect sample query set and any defect category in the support set is obtained; this application considers the detailed features of the pipeline defect image, such as Figure 4 As shown in the figure, the deep detail description operator is used to represent the images of the uncommon pipeline defect query set and the support set. The Ψ(X) output by the feature extraction network is an n*d dimensional tensor. Among them, n=h*w, h represents the height of the image, w represents the width of the image, and d represents the number of channels output by the model. Ψ(X) can be represented as a set of n d-dimensional deep detail description operators: where x i Represents the i-th detail description operator in the set n.

[0087] Traverse and calculate and filter the similarity of the nearest depth detail description operator. The input query set image q and a defect category c in the known support set are extracted through the feature extraction network, and the output feature tensors are Ψ(q) and Ψ(c) respectively. For each detail description operator x of Ψ(q) i and the detailed description operator of Ψ(c) Use the following formula:

[0088]

[0089] in, is the cosine similarity; x i is a detailed description operator of the first image tensor feature; It is the detailed description operator of the second image tensor feature.

[0090] Calculate the cosine similarity between the detailed description operators respectively. The closer the cosine value is to 1, the higher the similarity between the two operators. Conversely, the lower the similarity is. i k nearest neighbor distance (most similar) detail description operators The value of k is usually 3 or 5.

[0091] According to the measurement value, the pipeline defect category of the inspection image is determined.

[0092] Specifically, the step of determining the pipeline defect category of the inspection image according to the metric value includes: determining the defect category corresponding to the highest metric value as the defect category of the image in the uncommon defect sample query set.

[0093] In one example, in the step of obtaining a metric value between an image in a query set of uncommon defect samples and any defect category in a support set according to cosine similarity, the metric value is obtained based on the following formula:

[0094]

[0095] Ψ(q) is the first image tensor feature, c is any defect category in the support set; g(Ψ(q), c) is the metric value; n is the product of the height and width of the image; k is the number of the nearest detail description operators, which can be a natural number greater than 0.

[0096] In one embodiment, if Figure 5 As shown, the steps include:

[0097] S510, receiving a detection video of an underground drainage pipe;

[0098] S520, extracting key frames in the detection video and converting the key frames into a detection image set;

[0099] S530: pre-process the detection image set to obtain a detection image of the underground drainage pipe.

[0100] In one embodiment, the step of preprocessing the detection image set to obtain the detection image of the underground drainage pipe includes:

[0101] The detection image sets are processed with data enhancement in sequence to obtain detection images of underground drainage pipes.

[0102] Specifically, data enhancement mainly performs random horizontal and vertical flipping, rotation, contrast adjustment, and size scaling on small sample pipeline defect images.

[0103] In one embodiment, Figure 6 As shown, an underground drainage pipeline defect detection device is also provided, including:

[0104] A construction module is used to construct a small sample pipeline defect detection model; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results based on the image tensor features, and output the pipeline defect type based on the score of the detailed feature measurement results;

[0105] A training module, used to train a small sample pipeline defect detection model using a small sample learning data set;

[0106] An acquisition module, used for acquiring a detection image of an underground drainage pipe;

[0107] The first model processing module is used to input the inspection image into the normal / defect binary classification model and output the classification result;

[0108] The second model processing module is used to input the detection image with the classification result of defect into the common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0109] The third model processing module is used to input the detection image with a similarity value less than a preset value into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0110] For the specific definition of the underground drainage pipe defect detection device, please refer to the definition of the underground drainage pipe defect detection method above, which will not be repeated here. Each module in the above-mentioned underground drainage pipe defect detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0111] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0112] A small sample pipeline defect detection model is constructed; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results based on the image tensor features, and output the pipeline defect type based on the score of the detailed feature measurement results;

[0113] Use the small sample learning dataset to train the small sample pipeline defect detection model;

[0114] Acquire detection images of underground drainage pipes;

[0115] Input the inspection image into the normal / defect binary classification model and output the classification result;

[0116] Input the detection image with the classification result of defect into the common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0117] The detection images with similarity values ​​less than the preset value are input into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0118] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0119] A small sample pipeline defect detection model is constructed; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results based on the image tensor features, and output the pipeline defect type based on the score of the detailed feature measurement results;

[0120] Use the small sample learning dataset to train the small sample pipeline defect detection model;

[0121] Acquire detection images of underground drainage pipes;

[0122] Input the inspection image into the normal / defect binary classification model and output the classification result;

[0123] Input the detection image with the classification result of defect into the common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value;

[0124] The detection images with similarity values ​​less than the preset value are input into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image.

[0125] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0126] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0127] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0128] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0133] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0134] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for detecting defects in underground drainage pipes, characterized in that: include: Constructing a small sample pipeline defect detection model; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain detailed feature measurement results according to the image tensor features, and output the pipeline defect type according to the score of the detailed feature measurement results; Training the small sample pipeline defect detection model using a small sample learning data set; Acquire detection images of underground drainage pipes; Input the inspection image into a normal / defective binary classification model and output a classification result; Input the detection image with the classification result of defect into a common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value; The detection image with a similarity value less than a preset value is input into a trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image; wherein the similarity value can be obtained by the following steps: constructing an objective function including a false positive rate and a missed detection rate; wherein the false positive rate is the proportion of uncommon defects detected as common defects; the missed detection rate is the proportion of common defects detected as uncommon defects; optimizing the objective function using a multi-objective particle swarm optimization algorithm to obtain a set of non-inferior solutions; and selecting the best solution from the non-inferior solutions based on a lower missed detection rate as the similarity value.

2. The underground drainage pipe defect detection method according to claim 1, characterized in that: The small sample learning data set includes an uncommon defect sample query set, a support set, and a common defect auxiliary set; The step of training the small sample pipeline defect detection model using a small sample learning data set includes: Randomly selecting a first number of categories from the common defect auxiliary set, and selecting a second number of samples in each of the categories to form a training task; Based on each of the training tasks, generate a pre-training model; The knowledge learned by the pre-trained model is transferred to the training task of training a small sample pipeline defect detection model.

3. The underground drainage pipe defect detection method according to claim 2, characterized in that: Also includes the steps: Extracting a first image tensor feature of an image in the uncommon defect sample query set and a second image tensor feature of any defect category in the support set; Calculating the cosine similarity between the detail description operator of the first image tensor feature and the detail description operator of the second image tensor feature; According to the cosine similarity, a metric value between the image in the uncommon defect sample query set and any defect category in the support set is obtained; According to the measurement value, the pipeline defect category of the inspection image is determined.

4. The underground drainage pipe defect detection method according to claim 3, characterized in that: In the step of calculating the cosine similarity between the detail description operator of the first image tensor feature and the detail description operator of the second image tensor feature, the cosine similarity is obtained based on the following formula: in, is the cosine similarity; x i A detailed description operator for the first image tensor feature; A detailed description operator for the second image tensor feature.

5. The underground drainage pipe defect detection method according to claim 3, characterized in that: In the step of obtaining a metric value between the image in the uncommon defect sample query set and any defect category in the support set according to the cosine similarity, the metric value is obtained based on the following formula: Ψ(q) is the first image tensor feature, c is any defect category in the support set; g(Ψ(q), c) is the measurement value; n is the product of the height and width of the image; k is the number of the nearest detail description operators, which can be a natural number greater than 0.

6. The underground drainage pipe defect detection method according to claim 3, characterized in that: The steps of determining the pipeline defect category of the inspection image according to the measurement value include: The defect category corresponding to the highest metric value is determined as the defect category of the image in the uncommon defect sample query set.

7. The underground drainage pipe defect detection method according to claim 1, characterized in that: Also includes the steps: Receive inspection videos of underground drainage pipes; Extracting key frames from the detection video and converting the key frames into a detection image set; The detection image set is preprocessed to obtain the detection image of the underground drainage pipe.

8. According to the underground drainage pipe defect detection method of claim 7, the step of preprocessing the detection image set to obtain the detection image of the underground drainage pipe comprises: Data enhancement processing is performed on the detection image set to obtain a detection image of the underground drainage pipe.

9. An underground drainage pipe defect detection device, characterized in that: include: A construction module is used to construct a small sample pipeline defect detection model; the small sample pipeline defect detection model includes a feature extraction network and a measurement module; the feature extraction network is used to receive pipeline defect data and output image tensor features; the measurement module is used to obtain a detailed feature measurement result according to the image tensor feature, and output the pipeline defect type according to the score of the detailed feature measurement result; A training module, used to train the small sample pipeline defect detection model using a small sample learning data set; An acquisition module, used for acquiring a detection image of an underground drainage pipe; A first model processing module, used for inputting the detection image into a normal / defect binary classification model and outputting a classification result; A second model processing module is used to input the detection image with the classification result of defect into a common pipeline defect detection model; wherein the common pipeline defect detection model confirms the pipeline defect type according to the similarity value; The third model processing module is used to input the detection image with a similarity value less than a preset value into the trained small sample pipeline defect detection model to obtain the current defect type corresponding to the detection image; wherein the similarity value can be obtained by the following steps: constructing an objective function including a false positive rate and a missed detection rate; wherein the false positive rate is the proportion of uncommon defects detected as common defects; the missed detection rate is the proportion of common defects detected as uncommon defects; optimizing the objective function using a multi-objective particle swarm optimization algorithm to obtain a set of non-inferior solutions; and selecting the best solution from the non-inferior solutions based on a lower missed detection rate as the similarity value.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the underground drainage pipe defect detection method according to any one of claims 1 to 8 are implemented.

Images