Pipeline leakage point detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a pipeline leakage point detection method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring a visible light image and an infrared image of a target detection area; performing pipeline area identification on the visible light image by adopting a preset pipeline detection model to obtain a pipeline area image; cutting the infrared image based on the pipeline area image to obtain a to-be-leaked point detection image; and detecting the image to be subjected to leakage point detection by adopting a preset leakage point detection model to obtain a pipeline leakage point detection result. According to the invention, the preset pipeline detection model and the preset leakage point detection model are adopted to detect the pipeline and the leakage points respectively, so that single or multiple leakage points in the pipeline can be detected quickly and accurately, and the detection precision and speed are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of nuclear engineering pipeline monitoring, and in particular to a pipeline leakage detection method, device, storage medium and electronic equipment. Background Art

[0002] The reactor building process pipelines are dispersed along the plant building, and are mainly responsible for the transmission of reactor process liquids. When the reactor coolant pressure boundary is damaged and a loss of coolant accident occurs, it will have a great impact on the safety of the reactor and the surrounding environment. The consequences of a reactor loss of coolant vary with the size, location and initial state of the breach, and can be roughly divided into micro breaches, small breaches, medium breaches and large breaches. A loss of coolant accident will cause the temperature and humidity in the plant to rise, and the working state of the internal equipment to change. At the same time, the radioactivity level in the plant will increase significantly, and the working environment of the equipment in the affected area will deteriorate, causing long-term impacts on reactor safety.

[0003] When the reactor is running, most of the reactor's primary loop process pipelines will release a large amount of radioactive substances, and it is impossible to rely on human inspections to detect loss of coolant. The traditional method of using instruments to monitor flow is mainly used to monitor pipeline leaks, but the traditional method cannot detect breach accidents in time, especially small breaches. Summary of the invention

[0004] In view of this, the present invention provides a pipeline leakage detection method, device, storage medium and electronic equipment, the main purpose of which is to solve the current problem of inaccurate and untimely monitoring of reactor pipeline leakage.

[0005] In order to solve the above problems, the present application provides a pipeline leakage detection method, comprising: Acquire visible light images and infrared images of the target detection area; Using a preset pipeline detection model to perform pipeline area recognition on the visible light image to obtain a pipeline area image; Cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; The preset leak point detection model is used to detect the leak point detection image to obtain the pipeline leak point detection result.

[0006] Optionally, acquiring a visible light image and an infrared image of the target detection area specifically includes: Acquire the visible light original image and infrared original image of the target detection area; Using a nonlinear stationary wavelet filtering method to filter the visible light original image and the infrared original image respectively, to obtain an initial visible light filtered image and an initial infrared filtered image; The initial visible light filtered image and the initial infrared filtered image are smoothed by using a bilateral filtering method to obtain the visible light image and the infrared image.

[0007] Optionally, before using a preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain a pipeline region image, the method further includes: constructing a preset pipeline detection model; The construction of the preset pipeline detection model specifically includes: Acquire a first visible light sample image set of the target detection area; Performing expansion processing on the first visible light sample image set to obtain a second visible light sample image set; Preprocessing the second visible light sample image set to obtain a third visible light sample image set; Using a binary K-means clustering algorithm to perform clustering processing on the third visible light sample image set to obtain a preset anchor frame for training the preset pipeline detection model; Based on the third visible light sample image set and the preset anchor frame, an improved YOLOv4 algorithm is used to perform model training on a preset neural network model to obtain the preset pipeline detection model; Among them, the preset pipeline detection model includes a main feature extraction network, a feature fusion network and a detection head network.

[0008] Optionally, the using a preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain a pipeline region image specifically includes: Performing initial feature extraction on the visible light image using an input layer of a main feature extraction network of a preset pipeline detection model to obtain a first feature image; Using a plurality of densely connected network layers of a main feature extraction network of a preset pipeline detection model to extract features from the first feature image to obtain a second feature image; Using a feature fusion network of a preset pipeline detection model to perform feature fusion processing on the second feature image to obtain a third feature image; The pipeline region in the third feature image is identified by using a detection head network of a preset pipeline detection model to obtain the pipeline region image.

[0009] Optionally, the detection head network using the preset pipeline detection model identifies the pipeline area in the third feature image to obtain the pipeline area image, specifically including: Predicting parameters for adjusting a preset anchor frame based on image features of the third feature image to obtain fine-tuning parameter values; Adjusting the preset anchor box based on the fine-tuning parameter value to obtain a target bounding box; The third feature image is cut based on the target bounding box to obtain the pipeline area image.

[0010] Optionally, the step of cutting the infrared image based on the pipeline area image to obtain the image of the leak point to be detected specifically includes: Performing reverse positioning processing on the infrared image based on the pipeline area image to obtain a target infrared area corresponding to the pipeline area image; The infrared image is segmented based on the target infrared region to obtain the image for leak point detection.

[0011] Optionally, the using a preset leak detection model to detect the leak detection image to be detected to obtain a pipeline leak detection result specifically includes: Using the input layer of the main feature extraction network of the preset leak detection model to perform initial feature extraction on the leak detection image to obtain a fourth feature image; Using a plurality of densely connected network layers of a main feature extraction network of a preset leak detection model to extract features from the fourth feature image to obtain a fifth feature image; Using a feature fusion network of a preset leak detection model to perform feature fusion processing on the fifth feature image to obtain a sixth feature image; The detection head network of the preset pipeline detection model is used to identify the leakage point area in the sixth feature image to obtain the pipeline leakage point detection result.

[0012] In order to solve the above problems, the present application provides a pipeline leakage detection device, comprising: An acquisition module, used to acquire visible light images and infrared images of the target detection area; A pipeline identification module, used to identify the pipeline area of ​​the visible light image using a preset pipeline detection model to obtain a pipeline area image; A cutting module, used for cutting the infrared image based on the pipeline area image to obtain an image of a leak point to be detected; The leak detection module is used to detect the leak detection image using a preset leak detection model to obtain a pipeline leak detection result.

[0013] In order to solve the above problem, the present application provides a storage medium, characterized in that the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above pipeline leakage detection method are implemented.

[0014] In order to solve the above problem, the present application provides an electronic device, characterized in that it includes at least a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above pipeline leakage detection method when executing the computer program on the memory.

[0015] Beneficial effects of the present application: The present application obtains visible light images and infrared images of the target detection area; uses a preset pipeline detection model to identify the pipeline area on the visible light image to obtain a pipeline area image; cuts the infrared image based on the pipeline area image to obtain a leak detection image; uses a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result. The present application uses a preset pipeline detection model and a preset leak detection model to detect pipelines and leaks respectively, and can quickly and accurately detect single and multiple leaks in the pipeline; it can realize automated intelligent detection of single and multiple leaks in infusion pipelines, and perform image denoising through nonlinear stationary wavelet and bilateral filtering algorithms, thereby improving detection accuracy and speed.

[0016] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A schematic diagram of a pipeline leak detection method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of a pipeline leak detection method provided by another embodiment of the present application is shown; Figure 3 A structural block diagram of a pipeline leakage detection device provided in another embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] Various aspects and features of the present application are described herein with reference to the accompanying drawings.

[0019] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0020] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0021] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0022] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0023] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0024] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0025] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0026] The present application provides a pipeline leak detection method, such as Figure 1 As shown, including: Step S101: Acquire a visible light image and an infrared image of a target detection area; In the specific implementation process of this step, a camera can be used to collect images of the target detection area. The camera can be a Smart CAOS camera. The Smart CAOS camera can simultaneously collect visible light images, near infrared images (NIR) and short wave infrared (SWIR) linear high dynamic range (HDR) images for relative identification of transparent test liquids. The visible light original image and infrared original image of the target detection area are collected by the Smart CAOS camera. The visible light original image and the infrared original image are filtered and denoised to obtain the visible light image and the infrared image.

[0027] Step S102: using a preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain a pipeline region image; In the specific implementation process of this step, the input layer of the main feature extraction network of the preset pipeline detection model is used to perform initial feature extraction on the visible light image to obtain the first feature image; the DenseNet121 convolutional neural network is used as the main feature extraction network of the improved YOLOv4 algorithm, and the main feature extraction network includes an input layer, multiple densely connected network layers, including convolutional layers, dense blocks, transition layers and classifiers. The dense connection mechanism is adopted to directly connect the layers in each dense block in a feedforward manner. Each layer in the DenseBlock is densely connected to all subsequent layers, and the output of the previous layer is used as the input of the current layer. In DenseNet121, there are 4 DenseBlocks, with the numbers of 6, 12, 24 and 16 respectively. DenseNet121 has few network parameters. Due to the design of DenseBlock, the number of FeatureMaps output by each convolutional layer in DenseBlock is very small. At the same time, this connection method makes the transmission of features and gradients more effective and the network easier to train. The main reason for the disappearance of gradients is that the input information and gradient information are transmitted between many layers. The use of DenseBlock connection is equivalent to directly connecting each layer to Input and Loss, which reduces the occurrence of gradient disappearance. The first feature image is extracted by multiple densely connected network layers of the main feature extraction network of the preset pipeline detection model to obtain the second feature image; the feature fusion network of the preset pipeline detection model is used to perform feature fusion processing on the second feature image to obtain the third feature image; the detection head network of the preset pipeline detection model is used to identify the pipeline area in the third feature image to obtain the pipeline area image.

[0028] Step S103: cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; In the specific implementation process of this step, the infrared image is reversely positioned based on the pipeline area image to obtain a target infrared area corresponding to the pipeline area image; the infrared image is segmented based on the target infrared area to obtain the image of the leak point to be detected.

[0029] Step S104: using a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result.

[0030] In the specific implementation process of this step, the input layer of the main feature extraction network of the preset leakage detection model is used to perform initial feature extraction on the image to be leak detected, so as to obtain a fourth feature image; multiple densely connected network layers of the main feature extraction network of the preset leakage detection model are used to perform feature extraction on the fourth feature image, so as to obtain a fifth feature image; the feature fusion network of the preset leakage detection model is used to perform feature fusion processing on the fifth feature image, so as to obtain a sixth feature image; the detection head network of the preset pipeline detection model is used to identify the leakage area in the sixth feature image, so as to obtain the pipeline leakage detection result.

[0031] The present application obtains a visible light image and an infrared image of the target detection area; uses a preset pipeline detection model to identify the pipeline area on the visible light image to obtain a pipeline area image; cuts the infrared image based on the pipeline area image to obtain a leak detection image; uses a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result. The present application uses a preset pipeline detection model and a preset leak detection model to detect pipelines and leaks respectively, and can quickly and accurately detect single and multiple leaks in the pipeline; it can realize automated intelligent detection of single and multiple leaks in infusion pipelines, and perform image noise reduction through nonlinear stationary wavelet and bilateral filtering algorithms, thereby improving detection accuracy and speed.

[0032] Another embodiment of the present application provides another pipeline leak detection method, such as Figure 2 As shown, including: Step S201: acquiring a visible light original image and an infrared original image of a target detection area; In the specific implementation of this step, the target detection area can be the leak-prone pipeline area of ​​the reactor building. Smart CAOS cameras can be used to collect images, collect visible light image data and infrared image data of the process pipelines of the reactor building, and the visible light image and infrared image are one-to-one corresponding. These image data are transmitted to the data processing center, and the data center uses machine vision algorithms to detect and locate targets. When a nuclear reactor has a loss of coolant accident, the data center transmits relevant alarm information to the client of the reactor main control room for relevant alarm reminders.

[0033] Step S202: using a nonlinear stationary wavelet filtering method to filter the visible light original image and the infrared original image respectively to obtain an initial visible light filtered image and an initial infrared filtered image; In the specific implementation process of this step, first, wavelet decomposition is performed on the visible light original image and the infrared original image to obtain multiple first frequency sub-bands corresponding to the visible light original image and multiple second frequency sub-bands corresponding to the infrared original image; wavelet decomposition can decompose the image into multiple frequency sub-bands, including low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-band contains the main structural information of the image, while the high-frequency sub-band contains the details and edge information of the image. Nonlinear processing is performed on each of the first frequency sub-bands and the multiple second frequency sub-bands to remove noise, and multiple third frequency sub-bands corresponding to the visible light original image and multiple fourth frequency sub-bands corresponding to the infrared original image are obtained. Nonlinear processing is performed on each frequency sub-band to remove noise while retaining important features of the image. Nonlinear processing can be implemented in a variety of ways, such as using threshold processing, median filtering or adaptive filtering. The selection of these methods depends on the type of noise and the characteristics of the image. After nonlinear processing, each third frequency sub-band and each fourth frequency sub-band are respectively reconstructed by wavelet to obtain the initial visible light filtered image and the initial infrared filtered image. The reconstruction process is to merge the processed low-frequency and high-frequency sub-bands back into a complete filtered initial visible light filtered image and initial infrared filtered image.

[0034] Step S203: using a bilateral filtering method to smooth the initial visible light filtered image and the initial infrared filtered image to obtain the visible light image and the infrared image; In the specific implementation process of this step, a bilateral filter kernel is determined, and the bilateral filter kernel includes a spatial domain kernel and an intensity domain kernel. The spatial domain kernel determines the weight according to the spatial distance between pixels, while the intensity domain kernel determines the weight according to the intensity difference between pixels. The spatial domain kernel is usually a Gaussian function, and its weight decreases as the distance between pixels increases. The intensity domain kernel is also a Gaussian function, but its weight decreases as the intensity difference between pixels increases. For each pixel, its spatial distance and intensity difference with other pixels in the neighborhood are calculated; the weights are calculated using the spatial domain kernel and the intensity domain kernel respectively, and then the two are multiplied to obtain the final weight. Using the calculated weights, each pixel in the initial filtered image is weighted averaged to obtain a smoothed pixel value. After bilateral filtering, the initial visible light filtered image will become a smoother visible light image. Similarly, the initial infrared filtered image will also become a smoother infrared image.

[0035] Step S204: constructing a preset pipeline detection model; In the specific implementation process of this step, the first visible light sample image set of the target detection area is collected and obtained; the first visible light sample image set is expanded to obtain the second visible light sample image set; in order to avoid the insufficient data set causing the model to fit during the training process, the data set is expanded by diagonal mirror transformation, random rotation, and brightness change (bright and dark). Since the rotation of the image will cause part of the information to be lost, and it is not easy to judge whether the rotated image still contains our complete target. Therefore, the original image is first filled with the length of the long and short sides to form a square, so that the target information in the original image will not be lost during the rotation of the filled image. Data expansion can make the training data as close to the real data as possible, thereby improving the prediction accuracy. In addition, data expansion can make the network learning more robust, so that the model has stronger generalization ability. The second visible light sample image set is preprocessed to obtain the third visible light sample image set; the visible light image and the infrared image are preprocessed separately, including image denoising, contrast increase, etc. Nonlinear stationary filtering and bilateral filtering algorithms can be used for image denoising. Nonlinear stationary wavelet filtering can effectively remove noise from an image while retaining the edge information of the image. Bilateral filtering can further smooth the image, reduce noise, and maintain the detailed features of the image. The third visible light sample image set is clustered using a binary K-means clustering algorithm to obtain a preset anchor frame for training the preset pipeline detection model. The preset anchor frame is a priori frame, a predefined frame used in a target detection task, and their size and proportion are extracted from the data set by a clustering algorithm to better match the target size distribution in the data set. A set of priori frames are regenerated using a binary K-means clustering algorithm to improve the detection capability of small leaks. On the COCO data set, the original YOLOv4 algorithm uses the priori frame parameters pre-calculated on the data set. However, when using different data sets (such as indoor scenes or surface defect data sets of aircraft engine parts, the plant environment of the present invention), these priori frames may no longer be applicable because they are significantly different from the target size and type in the new data set. Therefore, the priori frame needs to be recalculated. The Bisecting K-means clustering algorithm is an improved K-means clustering method that can overcome the limitation of the K-means algorithm that it is easy to fall into the local minimum. In the target detection task, the Bisecting K-means clustering algorithm is used to regenerate the prior box suitable for a specific data set. For example, in the improved YOLOv4 algorithm, the Bisecting K-means clustering algorithm is used to regenerate 9 cluster centers as the prior box parameters. These regenerated prior boxes are closely related to the training model. As one of the inputs of the model, they help the model better understand and predict targets of different sizes and shapes.Specifically, the role of the prior frame is to provide a reference frame so that the model can more accurately locate and classify targets during prediction. By adjusting the size and proportion of the prior frame, the detection accuracy of the model on targets of different scales can be improved. The binary K-means clustering algorithm analyzes the COCO dataset to obtain the prior frame, which is generated by re-clustering to meet the needs of a specific dataset and plays a key role in the training model, helping the model to better detect targets. Based on the third visible light sample image set and the preset anchor frame, the preset neural network model is trained using the improved YOLOv4 algorithm to obtain the preset pipeline detection model; wherein the preset pipeline detection model includes a main feature extraction network, a feature fusion network and a detection head network. DenseNet121 is introduced as the feature extraction network of YOLOv4. DenseNet mainly consists of convolutional layers, dense blocks, transition layers and classifiers. It adopts a dense connection mechanism to directly connect the layers in each dense block in a feedforward manner. Each layer in DenseBlock is densely connected to all subsequent layers, and the output of the previous layer is used as the input of the current layer. In DenseNet121, there are 4 DenseBlocks, with the numbers being 6, 12, 24 and 16 respectively. DenseNet121 has fewer network parameters. Due to the design of DenseBlock, the number of FeatureMaps output by each convolutional layer in DenseBlock is small. At the same time, this connection method makes the transmission of features and gradients more effective, and the network is easier to train, because the main reason for the disappearance of gradients is that the input information and gradient information are transmitted between many layers. The use of DenseBlock connection is equivalent to directly connecting each layer to Input and Loss, which reduces the occurrence of gradient disappearance. Specifically, in DenseNet121, there are 4 DenseBlocks, with the numbers being 6, 12, 24 and 16 respectively. These DenseBlocks play a key role in feature extraction. DenseNet uses dense connections to enable the output features of each layer to be used by all subsequent layers, thereby enhancing the reusability and transferability of features. This design helps alleviate the problem of gradient disappearance and enables low-level features to be directly propagated to the output layer, so that they can be better strengthened and utilized to play a role in feature reuse and strengthening. The different number of stacking times of each DenseBlock results in different output feature map sizes and number of channels. For example, the feature map resolution output at the end of DenseBlock3 is 14×14, while the feature map resolution output at the end of DenseBlock4 is 7×7. This size change helps to extract features of different scales at different levels, thereby enhancing the expressiveness of the model.After each DenseBlock, there is a transition layer, which performs dimensionality reduction through a 1×1 convolution layer and downsampling through an average pooling layer. This design not only controls the number of channels of the feature map, but also further reduces the size of the feature map, making the model more efficient when processing high-dimensional data. In some applications, such as fracture site detection, global information is crucial for feature extraction. DenseNet121 uses dense connections to better transmit global information to each layer of the network, thereby improving the accuracy of feature extraction. Compared with other network structures (such as ResNet), DenseNet has fewer parameters, which makes the model less likely to fall into overfitting during training, thereby improving the generalization ability of the model. The four DenseBlocks in DenseNet121 effectively enhance the ability of feature extraction through dense connections, feature reuse, transition layer design, and extraction of global information, making the model perform well in various computer vision tasks.

[0036] Step S205: constructing a preset leak point detection model; During the specific implementation of this step, a first infrared sample image set of the target detection area is collected and obtained; the first infrared sample image set is expanded to obtain a second infrared sample image set; the second infrared sample image set is preprocessed to obtain a third infrared sample image set; the third infrared sample image set is clustered using a binary K-means clustering algorithm to obtain a preset anchor frame for training the preset leakage detection model; based on the third infrared sample image set and the preset anchor frame, an improved YOLOv4 algorithm is used to train a preset neural network model to obtain the preset leakage detection model; wherein the preset leakage detection model includes a main feature extraction network, a feature fusion network and a detection head network.

[0037] Step S206: using a preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain a pipeline region image; In the specific implementation process of this step, the input layer of the main feature extraction network of the preset pipeline detection model is used to perform initial feature extraction on the visible light image to obtain the first feature image; specifically, the input visible light image is preliminarily processed through the initial convolution layer, batch normalization layer, ReLU activation function and maximum pooling layer to extract the preliminary first feature image. The first feature image is extracted by multiple densely connected network layers of the main feature extraction network of the preset pipeline detection model to obtain the second feature image; the multiple densely connected network layers include at least four densely connected network layers, and the number of convolution modules DenseBlock contained in each densely connected network layer is: 6, 12, 24 and 16 respectively; the first Dense Block: contains 6 fully connected layers DenseLayer, and each fully connected layer DenseLayer is densely connected. The size of the feature map in this Block is consistent, and dense connections are used between layers. The preliminary feature map is further extracted and fused through multiple 3x3 convolution layers (there may be a 1x1 convolution layer in front for dimensionality reduction and reducing the amount of calculation). Transition layer: Located between the first and second Dense Blocks, it reduces the number of channels and spatial size of the feature map through a 1x1 convolution layer and an average pooling layer. The second Dense Block: Contains 12 DenseLayers, and also uses dense connections to further extract and fuse the feature maps processed by the transition layer. The third and fourth Dense Blocks: contain 24 and 16 DenseLayers respectively, and continue to use dense connections to extract and fuse features. The transition layer after each Dense Block will further reduce the number of channels and spatial size of the feature map. Global average pooling: After the last Dense Block, the feature map is converted into the second feature image through a global average pooling layer. The feature fusion network of the preset pipeline detection model is used to perform feature fusion processing on the second feature image to obtain a third feature image; the feature fusion network Neck part of the preset pipeline detection model is used to perform feature fusion processing to obtain the third feature image; specifically, the feature fusion network Neck part includes a spatial pyramid pooling network and a path aggregation network; the spatial pyramid pooling network is used to perform multi-scale pooling processing on the third feature image to obtain each local feature; each local feature is spliced ​​to obtain a first feature vector; the first feature vector is fused using a top-down and bottom-up feature fusion method to obtain the third feature image. The detection head network of the preset pipeline detection model is used to identify the pipeline area in the third feature image to obtain the pipeline area image.Specifically, based on the image features of the third feature image, parameters for adjusting the preset anchor frame are predicted to obtain fine-tuning parameter values; based on the fine-tuning parameter values, the preset anchor frame is adjusted to obtain a target bounding box; based on the target bounding box, the third feature image is cut to obtain the pipeline area image.

[0038] Step S207: cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; In the specific implementation process of this step, the infrared image is reversely positioned based on the pipeline area image to obtain a target infrared area corresponding to the pipeline area image; The infrared image is segmented based on the target infrared region to obtain the image for leak point detection.

[0039] Step S208: using a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result.

[0040] In the specific implementation process of this step, the input layer of the main feature extraction network of the preset leakage detection model is used to perform initial feature extraction on the image to be leak detected, so as to obtain a fourth feature image; multiple densely connected network layers of the main feature extraction network of the preset leakage detection model are used to perform feature extraction on the fourth feature image, so as to obtain a fifth feature image; the feature fusion network of the preset leakage detection model is used to perform feature fusion processing on the fifth feature image, so as to obtain a sixth feature image; the detection head network of the preset pipeline detection model is used to identify the leakage area in the sixth feature image, so as to obtain the pipeline leakage detection result.

[0041] This application obtains visible light images and infrared images of the target detection area; uses a preset pipeline detection model to identify the pipeline area of ​​the visible light image to obtain a pipeline area image; cuts the infrared image based on the pipeline area image to obtain a leak detection image; uses a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result. This application uses a preset pipeline detection model and a preset leak detection model to detect pipelines and leaks respectively, and can quickly and accurately detect single and multiple leaks in the pipeline; it can realize automated intelligent detection of single and multiple leaks in the infusion pipeline, and use nonlinear stationary wavelet and bilateral filtering algorithms to reduce image noise, thereby improving detection accuracy and speed. Ultimately, the reactor plant process pipeline leakage monitoring and positioning is achieved, so that relevant alarms are promptly issued to operating personnel.

[0042] In the specific implementation process, the present application provides a pipeline leakage detection device, such as Figure 3 As shown, including: An acquisition module 1 is used to acquire a visible light image and an infrared image of a target detection area; A pipeline identification module 2 is used to identify the pipeline area of ​​the visible light image using a preset pipeline detection model to obtain a pipeline area image; A cutting module 3 is used to cut the infrared image based on the pipeline area image to obtain an image of a leak point to be detected; The leak detection module 4 is used to detect the leak detection image by using a preset leak detection model to obtain a pipeline leak detection result.

[0043] In the specific implementation process, the acquisition module 1 is specifically used to: collect and obtain the visible light original image and the infrared original image of the target detection area; use a nonlinear stationary wavelet filtering method to filter the visible light original image and the infrared original image respectively to obtain an initial visible light filtered image and an initial infrared filtered image; use a bilateral filtering method to smooth the initial visible light filtered image and the initial infrared filtered image to obtain the visible light image and the infrared image.

[0044] In a specific implementation process, the device also includes: a construction module, which is specifically used to: collect and obtain a first visible light sample image set of the target detection area; expand the first visible light sample image set to obtain a second visible light sample image set; preprocess the second visible light sample image set to obtain a third visible light sample image set; cluster the third visible light sample image set using a binary K-means clustering algorithm to obtain a preset anchor frame for training the preset pipeline detection model; based on the third visible light sample image set and the preset anchor frame, use an improved YOLOv4 algorithm to train a preset neural network model to obtain the preset pipeline detection model; wherein the preset pipeline detection model includes a main feature extraction network, a feature fusion network and a detection head network.

[0045] In the specific implementation process, the pipeline identification module 2 is specifically used to: use the input layer of the main feature extraction network of the preset pipeline detection model to perform initial feature extraction on the visible light image to obtain a first feature image; use multiple densely connected network layers of the main feature extraction network of the preset pipeline detection model to extract features from the first feature image to obtain a second feature image; use the feature fusion network of the preset pipeline detection model to perform feature fusion processing on the second feature image to obtain a third feature image; use the detection head network of the preset pipeline detection model to identify the pipeline area in the third feature image to obtain the pipeline area image.

[0046] In the specific implementation process, the pipeline identification module 2 is also used to: predict the parameters used to adjust the preset anchor frame based on the image features of the third feature image to obtain fine-tuning parameter values; adjust the preset anchor frame based on the fine-tuning parameter values ​​to obtain a target bounding box; and cut the third feature image based on the target bounding box to obtain the pipeline area image.

[0047] In the specific implementation process, the cutting module 3 is specifically used to: perform reverse positioning processing on the infrared image based on the pipeline area image to obtain a target infrared area corresponding to the pipeline area image; and perform segmentation processing on the infrared image based on the target infrared area to obtain the image of the leak point to be detected.

[0048] In the specific implementation process, the leakage detection module 4 is specifically used to: use the input layer of the main feature extraction network of the preset leakage detection model to perform initial feature extraction on the image to be leaked, and obtain a fourth feature image; use multiple densely connected network layers of the main feature extraction network of the preset leakage detection model to extract features from the fourth feature image, and obtain a fifth feature image; use the feature fusion network of the preset leakage detection model to perform feature fusion processing on the fifth feature image, and obtain a sixth feature image; use the detection head network of the preset pipeline detection model to identify the leakage area in the sixth feature image, and obtain the pipeline leakage detection result.

[0049] The present application obtains a visible light image and an infrared image of the target detection area; uses a preset pipeline detection model to identify the pipeline area on the visible light image to obtain a pipeline area image; cuts the infrared image based on the pipeline area image to obtain a leak detection image; uses a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result. The present application uses a preset pipeline detection model and a preset leak detection model to detect pipelines and leaks respectively, and can quickly and accurately detect single and multiple leaks in the pipeline; it can realize automated intelligent detection of single and multiple leaks in infusion pipelines, and perform image noise reduction through nonlinear stationary wavelet and bilateral filtering algorithms, thereby improving detection accuracy and speed.

[0050] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented: Step 1: Obtain visible light images and infrared images of the target detection area; Step 2: using a preset pipeline detection model to perform pipeline area recognition on the visible light image to obtain a pipeline area image; Step 3: cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; Step 4: Use a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result.

[0051] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0052] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0053] The specific implementation process of the above method steps can refer to the embodiments of any of the above pipeline leak detection methods, and this embodiment will not be repeated here.

[0054] The present application obtains a visible light image and an infrared image of the target detection area; uses a preset pipeline detection model to identify the pipeline area on the visible light image to obtain a pipeline area image; cuts the infrared image based on the pipeline area image to obtain a leak detection image; uses a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result. The present application uses a preset pipeline detection model and a preset leak detection model to detect pipelines and leaks respectively, and can quickly and accurately detect single and multiple leaks in the pipeline; it can realize automated intelligent detection of single and multiple leaks in infusion pipelines, and perform image noise reduction through nonlinear stationary wavelet and bilateral filtering algorithms, thereby improving detection accuracy and speed.

[0055] Another embodiment of the present application provides an electronic device, which may be a server, and includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external client via a network connection. When the electronic device program is executed by the processor, it implements the functions or steps of a pipeline leak detection method server side.

[0056] In one embodiment, an electronic device is provided, which may be a client. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external server via a network connection. When the electronic device program is executed by the processor, the functions or steps on the client side of a pipeline leak detection method are implemented.

[0057] Another embodiment of the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the following method steps when executing the computer program in the memory: Step 1: Obtain visible light images and infrared images of the target detection area; Step 2: using a preset pipeline detection model to perform pipeline area recognition on the visible light image to obtain a pipeline area image; Step 3: cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; Step 4: Use a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result.

[0058] The specific implementation process of the above method steps can refer to the embodiments of any of the above pipeline leak detection methods, and this embodiment will not be repeated here.

[0059] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A pipeline leak detection method, characterized in that: include: Acquire visible light images and infrared images of the target detection area; Using a preset pipeline detection model to perform pipeline area recognition on the visible light image to obtain a pipeline area image; Cutting the infrared image based on the pipeline area image to obtain an image of the leak point to be detected; The preset leak point detection model is used to detect the leak point detection image to obtain the pipeline leak point detection result.

2. The method according to claim 1, characterized in that The step of acquiring the visible light image and the infrared image of the target detection area specifically includes: Acquire the visible light original image and infrared original image of the target detection area; Using a nonlinear stationary wavelet filtering method to filter the visible light original image and the infrared original image respectively, to obtain an initial visible light filtered image and an initial infrared filtered image; The initial visible light filtered image and the initial infrared filtered image are smoothed by using a bilateral filtering method to obtain the visible light image and the infrared image.

3. The method according to claim 1, characterized in that Before using the preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain the pipeline region image, the method further includes: constructing a preset pipeline detection model; The construction of the preset pipeline detection model specifically includes: Acquire a first visible light sample image set of the target detection area; Performing expansion processing on the first visible light sample image set to obtain a second visible light sample image set; Preprocessing the second visible light sample image set to obtain a third visible light sample image set; Using a binary K-means clustering algorithm to perform clustering processing on the third visible light sample image set to obtain a preset anchor frame for training the preset pipeline detection model; Based on the third visible light sample image set and the preset anchor frame, an improved YOLOv4 algorithm is used to perform model training on a preset neural network model to obtain the preset pipeline detection model; Among them, the preset pipeline detection model includes a main feature extraction network, a feature fusion network and a detection head network.

4. The method according to claim 1, characterized in that The using of a preset pipeline detection model to perform pipeline region recognition on the visible light image to obtain a pipeline region image specifically includes: Performing initial feature extraction on the visible light image using an input layer of a main feature extraction network of a preset pipeline detection model to obtain a first feature image; Using a plurality of densely connected network layers of a main feature extraction network of a preset pipeline detection model to extract features from the first feature image to obtain a second feature image; Using a feature fusion network of a preset pipeline detection model to perform feature fusion processing on the second feature image to obtain a third feature image; The pipeline region in the third feature image is identified by using a detection head network of a preset pipeline detection model to obtain the pipeline region image.

5. The method according to claim 4, characterized in that The detection head network using the preset pipeline detection model identifies the pipeline area in the third feature image to obtain the pipeline area image, specifically including: Predicting parameters for adjusting a preset anchor frame based on image features of the third feature image to obtain fine-tuning parameter values; Adjusting the preset anchor box based on the fine-tuning parameter value to obtain a target bounding box; The third feature image is cut based on the target bounding box to obtain the pipeline area image.

6. The method according to claim 1, characterized in that The step of cutting the infrared image based on the pipeline area image to obtain the image of the leak point to be detected specifically includes: Performing reverse positioning processing on the infrared image based on the pipeline area image to obtain a target infrared area corresponding to the pipeline area image; The infrared image is segmented based on the target infrared region to obtain the image for leak point detection.

7. The method according to claim 1, characterized in that The method of using a preset leak detection model to detect the leak detection image to obtain a pipeline leak detection result specifically includes: Using the input layer of the main feature extraction network of the preset leak detection model to perform initial feature extraction on the leak detection image to obtain a fourth feature image; Using a plurality of densely connected network layers of a main feature extraction network of a preset leak detection model to extract features from the fourth feature image to obtain a fifth feature image; Using a feature fusion network of a preset leak detection model to perform feature fusion processing on the fifth feature image to obtain a sixth feature image; The detection head network of the preset pipeline detection model is used to identify the leakage point area in the sixth feature image to obtain the pipeline leakage point detection result.

8. A pipeline leakage detection device, characterized in that: include: An acquisition module, used to acquire visible light images and infrared images of the target detection area; A pipeline identification module, used to identify the pipeline area of ​​the visible light image using a preset pipeline detection model to obtain a pipeline area image; A cutting module, used for cutting the infrared image based on the pipeline area image to obtain an image of a leak point to be detected; The leak detection module is used to detect the leak detection image using a preset leak detection model to obtain a pipeline leak detection result.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the pipeline leakage detection method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the pipeline leakage detection method according to any one of claims 1 to 7 when executing the computer program on the memory.

Images