U-shaped pipeline inner wall foreign matter detection system and method based on non-vision-field vision

Abstract

The invention discloses a system and a method for detecting foreign matters on the inner wall of a U-shaped pipeline based on non-vision-field vision. The system comprises an excitation source, a detector and a data processing device. The excitation source is arranged at one port of the U-shaped pipeline, and light emitted by the excitation source enters from the port and is received by the detector arranged at the other port of the U-shaped pipeline after being reflected for at least three times on the inner wall. And the data processing device preprocesses the signal received by the detector and inputs the processed data into the neural network model, and finally the model outputs the scale and position estimation of the foreign matter. Contact type operation of a traditional detection means is avoided, complex instruments and equipment are not needed, the detection cost is reduced, the detection requirements of complex structures such as a U-shaped pipeline can be met, and efficient detection of blind area foreign matter can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of non-line-of-sight visual detection, and relates to a foreign body detection system and method, and more specifically to a U-shaped pipe inner wall foreign body detection system and method based on non-line-of-sight vision. Background Art

[0002] When there is an obstruction in the optical path between the camera and the imaging target, the target cannot be captured by traditional line-of-sight imaging technology. Non-line-of-sight imaging (NLOS) technology provides an innovative solution. By capturing the multiple scattered signals of light on the intermediate medium, this technology can realize the recognition, positioning and reconstruction of hidden targets, showing bright application prospects in many fields such as remote sensing monitoring, autonomous driving and emergency rescue.

[0003] Non-line-of-sight imaging (NLOS) technology can be divided into two categories: transient non-line-of-sight imaging and steady-state non-line-of-sight imaging, depending on the temporal resolution of the excitation and detection equipment used. In transient non-line-of-sight imaging, an extremely short pulse light source is used to illuminate the relay surface, and a time-resolved detector is used to capture the information of multiple bounced photons. Recently, researchers have proposed a variety of methods to reconstruct images of these hidden targets, including but not limited to ellipsoidal tomography, inverse rendering, phasor field virtual waves, and deep learning-based methods. Despite this, steady-state NLOS imaging technology is becoming an increasingly popular solution due to its low cost, high efficiency, and ease of deployment.

[0004] Steady-state non-line-of-sight imaging is a complex problem. Traditional physical modeling-based methods often need to impose additional conditional restrictions, such as introducing occlusions, to reduce the ill-posedness of the problem. Deep learning-based methods learn the mapping relationship from input to output in a data-driven way, and can establish an effective mapping model without explicit physical modeling, which has good flexibility and adaptability. In addition, although deep learning-based methods take a long time in the model training phase, they can achieve fast reasoning in the testing phase, which helps to deploy the model in actual scenarios. When the training data set is rich and reliable, the performance of deep learning-based methods can often surpass traditional physical modeling-based methods.

[0005] Detection of foreign matter on the inner wall of a pipeline is crucial to ensure the normal operation of the pipeline, because the operating efficiency is often affected by the accumulation of impurities, corrosion or intrusion of foreign matter in the pipeline, and in severe cases even leads to equipment failure and safety hazards. However, traditional pipeline detection mostly relies on ultrasonic, pulsed eddy current, magnetic flux leakage technology or endoscopic equipment, which have some limitations, mainly including: complicated operation, endoscopic equipment requires manual operation by professionals, and it is difficult to fully cover the detection in complex curved pipes; contact detection is required, which increases the difficulty of detection and equipment loss; the equipment is expensive, the maintenance cost is high, and it is difficult to use efficiently in a small space. The present invention is based on steady-state non-field of view imaging technology to detect foreign matter on the inner wall of the pipeline, aiming to achieve non-contact rapid foreign matter detection, thereby improving industrial production efficiency. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a U-shaped pipe inner wall foreign body detection system and method based on non-line-of-sight vision. The present invention can overcome the operational complexity and contact measurement limitations of traditional detection methods and greatly improve detection efficiency and reliability.

[0007] The technical solution adopted by the present invention is as follows:

[0008] A U-shaped pipe inner wall foreign body detection system based on non-line-of-sight vision, comprising: an excitation source, a detector and a data processing device;

[0009] The excitation source adopts optical excitation and is deployed at one port of the U-shaped pipe to emit light source;

[0010] The detector is deployed at another port of the U-shaped pipe to receive optical signals;

[0011] The data processing device is connected to the detector and is used to receive and process the signal from the detector to obtain the detection result of the foreign matter on the inner wall of the pipeline. The detection result includes the size and position information of the foreign matter.

[0012] Furthermore, the excitation source adopts a continuous wave laser with a wavelength of 532nm and a power of not less than 100mW, ensuring sufficient light intensity for signal detection after multiple reflections in the pipeline.

[0013] Furthermore, the light emitted by the excitation source is received by the detector after being reflected at least three times by the inner wall of the U-shaped pipe, and the optical signal received by the detector carries information related to the structure of the inner wall of the pipe.

[0014] Furthermore, the detector uses an industrial camera with a complementary metal oxide semiconductor (CMOS) image sensor, which can convert the optical signal after multiple reflections into a digital image signal and transmit it to the data processing device through a data line.

[0015] Furthermore, a deep neural network model is deployed on the data processing device, and the deep neural network model uses the visual geometry network (VGG19) as the backbone network.

[0016] A method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-line-of-sight vision is implemented by using the above-mentioned U-shaped pipe inner wall foreign matter detection system based on non-line-of-sight vision, and comprises the following steps:

[0017] The excitation source emits light, and the light is received by the detector after being reflected at least three times in the U-shaped pipe;

[0018] The detector receives the optical signal, converts the optical signal into a digital image signal, and transmits the digital image signal to the data processing device via a data line;

[0019] The data processing device preprocesses the digital image signal, inputs the preprocessed digital signal into a trained deep neural network model, and the deep neural network model outputs the detection result of foreign matter on the inner wall of the pipeline.

[0020] Furthermore, the deep neural network model training method is:

[0021] Acquire a training sample set, wherein the training sample set includes sample digital images to be identified and training labels corresponding to each sample digital image;

[0022] The preset deep neural network model is trained according to the training sample set and a preset training algorithm to obtain the deep neural network model.

[0023] Furthermore, the tasks of the deep neural network model include foreign object scale recognition tasks and foreign object position estimation tasks.

[0024] Furthermore, the fully connected layer output of the deep neural network model varies according to different task requirements; for the foreign object scale recognition task, the deep neural network model outputs the foreign object category confidence, which is used to judge the size category of the foreign object; for the foreign object position estimation task, the deep neural network model outputs the foreign object position estimation value, which is used to locate the specific position of the foreign object.

[0025] Furthermore, during the training process of the deep neural network model, a cross entropy loss function is used for the foreign object scale recognition task; and an MSE loss function is used for the foreign object position estimation task.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) Highly efficient non-contact detection, through optical excitation and multiple reflections of signals, can detect foreign objects on the inner wall of the U-shaped pipe, avoiding the contact operation of traditional detection methods.

[0028] (2) The structure is simple and the cost is low. It only uses conventional equipment such as continuous lasers and industrial cameras. No complex instruments are required, which reduces the detection cost.

[0029] (3) High detection efficiency. The deep neural network model helps to achieve efficient detection of foreign objects.

[0030] (4) Adaptable to complex structures: The system can process multiple reflection signals in the pipeline and adapt to the detection needs of complex structures such as U-shaped pipelines. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of the structure of a non-line-of-sight visual pipeline inner wall foreign body detection system provided in an embodiment of the present invention.

[0032] Figure 2 Schematic diagram of a device for amplifying data of foreign matter on the inner wall of a non-visual-area visual pipeline provided in an embodiment of the present invention.

[0033] Figure 3 A schematic diagram of light transmission on the inner wall of a non-viewing visual pipe provided in an embodiment of the present invention.

[0034] Figure 4 A partially enlarged schematic diagram of a device for amplifying data on foreign matter on the inner wall of a non-viewing area pipeline provided in an embodiment of the present invention.

[0035] Figure 5 A schematic diagram of the placement of foreign objects in a circular detection background area provided by an embodiment of the present invention.

[0036] Figure 6 A schematic diagram of the size classification of foreign matter on the inner wall of a non-visual area pipeline provided in an embodiment of the present invention.

[0037] Figure 7 A schematic diagram of a confusion matrix for identifying the size of foreign objects using a visual geometry network provided in an embodiment of the present invention.

[0038] Figure 8 A schematic diagram of a confusion matrix for identifying the size of foreign matter using a decision tree method provided in an embodiment of the present invention.

[0039] Fig. 9 A schematic diagram of a confusion matrix for identifying the size of foreign objects using the random forest method provided in an embodiment of the present invention.

[0040] Fig.10 A schematic diagram of a confusion matrix for identifying the size of foreign matter using a support vector machine method provided in an embodiment of the present invention.

[0041] Fig.11 A schematic diagram of a confusion matrix for identifying the size of foreign objects using the nearest neighbor method provided in an embodiment of the present invention.

[0042] Fig.12 The error vector diagram of the visual geometry network used in the embodiment of the present invention to identify the position of foreign objects.

[0043] Fig.13 This is an error vector diagram of foreign body position identification using the decision tree method provided in an embodiment of the present invention.

[0044] Fig.14 The error vector diagram of the random forest method provided in the embodiment of the present invention for identifying the position of foreign objects.

[0045] Fig.15 An error vector diagram of foreign body position identification using the support vector machine method provided in an embodiment of the present invention.

[0046] Fig.16 This is an error vector diagram of the nearest neighbor method for identifying the position of a foreign object provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention will be further explained in detail below in conjunction with the accompanying drawings and specific embodiments. Example

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 As shown, an embodiment of the present invention provides a U-shaped pipe inner wall foreign body detection system based on non-line-of-sight vision, including an excitation source, a detector and a data processing device; wherein the excitation source adopts optical excitation and is deployed at one port of the U-shaped pipe, and the light emitted by it is emitted into the U-shaped pipe from the port and is received by the detector deployed at the other port of the U-shaped pipe after experiencing at least three reflections on the inner wall of the U-shaped pipe; the data processing device preprocesses the signal received by the detector, and the preprocessed signal is input into a trained neural network model, and the neural network model outputs the detection result of the foreign body on the inner wall of the pipe, wherein the detection result includes two aspects, one is the scale recognition of the foreign body, and the other is the position estimation of the foreign body.

[0050] Next, the structure and function of each of the above parts are described in detail:

[0051] 1. Motivation source:

[0052] In this embodiment, the excitation source adopts a continuous wave laser with a wavelength of 532nm and a power of not less than 100mW, ensuring sufficient light intensity for signal detection after multiple reflections in the pipeline, wherein the light is emitted from the laser, first reaches the diffuse reflection area 1, and undergoes the first reflection, and then enters the invisible area 2 (this area is the main detection area for foreign matter on the inner wall of the pipeline), and undergoes several reflections in this area, after being reflected in the invisible area 2, the light reaches the diffuse reflection area 3, and undergoes reflection again, and finally, the reflected light reaches the detector located at the other end of the U-shaped pipeline and is received by it.

[0053] 2. Detector:

[0054] The detector adopts an industrial camera with a complementary metal oxide semiconductor image sensor, which can convert optical signals after multiple reflections into digital image signals and transmit them to the data processing device through a data line.

[0055] 3. Data processing device:

[0056] The data processing device uses an industrial computer to perform denoising on the received digital image signal to eliminate the interference signal introduced by environmental noise. The processed signal is input into a deep neural network model with a visual geometry network as the backbone network for feature extraction. The output of the fully connected layer of the deep neural network model varies according to different task requirements. For the foreign object scale recognition task, the confidence of the category is output to determine the size category of the foreign object. For the foreign object position estimation task, the foreign object position estimation value in a predefined spatial coordinate system is output to locate the specific position of the foreign object.

[0057] The present invention uses a deep learning method to analyze non-visual signals, so the amount of data used for model training must be of a certain size, such as Figure 2 As shown, in order to enhance the feature extraction capability of the deep neural network for foreign matter on the inner wall of a U-shaped pipe, an embodiment of the present invention provides a data amplification device for acquiring a training sample set, wherein the data amplification device includes an excitation source, a detector, a data processing device, and a rotational translation stage.

[0058] The excitation source and detector are the same as those in the above detection system. An inscribed circular hole is opened on the inner wall of the U-shaped pipe, and the size of the hole is Figure 4 The circular rotating table shown has the same background, and the circular foreign object is located on the circular background. When collecting data on foreign objects of different sizes and positions, the light emitted by the excitation source from one port of the U-shaped pipe enters the U-shaped pipe and is received by the detector deployed at the other port of the U-shaped pipe after experiencing at least three reflections on the inner wall of the U-shaped pipe. The data processing device pre-processes the signal received by the detector and assigns a corresponding label to each data, and the label includes the scale level and position of the foreign object.

[0059] Specifically, the data amplification device cuts a circular part in the detection area of ​​the inner wall of the pipe as a circular background, and constructs different foreign objects on it for obtaining training samples. The circular background can be rotated so that the same foreign object is located at different positions in the detection area, thereby amplifying the data volume of the training samples. To ensure that the position of the foreign object is controllable, the device uses a stepping servo rotating platform. Fixing the circular background on the rotating table and mounting it on the optical XYZ three-displacement platform can ensure that the circular background remains in the same position each time data is collected. The only thing that changes is the position of the foreign object on the inner wall, and the position of the foreign object can be controlled by itself.

[0060] During data collection, see Figure 6 In this embodiment, six sizes of black foreign bodies are provided, namely, 25mm, 30mm, 35mm, 40mm, 45mm and 50mm in diameter. Figure 5 On the circular background, a mark with a radius of 5 mm is taken as the center of the foreign body. Each position of the foreign body on the radius is collected at an interval of 9° as the rotating platform rotates one circle.

[0061] In this embodiment, the camera is controlled by an image acquisition program written in Python, and the rotating platform is controlled by writing controller instructions. Finally, a large number of foreign body image data of different sizes and located in different detection areas are obtained, and corresponding scale level labels and position labels are assigned to each image, where the position label consists of the angle of the circular detection area where the foreign body is located and the distance from the center of the circular detection area.

[0062] The original image obtained by the above-mentioned amplification device needs to be preprocessed by background difference denoising and the ROI area is extracted. Then, the image is cropped, mean filtered, homography corrected, resized, etc. according to the ROI area in turn to obtain the image as the data set of this embodiment.

[0063] In the foreign body size recognition task, black foreign bodies with diameters of 25mm, 30mm, 35mm, 40mm, 45mm, and 50mm are coded into six categories from 0 to 5. Both the training set and the test set contain foreign bodies of six different sizes, but the positions of the foreign bodies in the test set have all appeared in the training set. In the foreign body position estimation task, the radial distance and angle from the center of the foreign body to the center of the circle are extracted as position information. The foreign bodies in the training set and the test set use different scales to test the generalization performance of the model.

[0064] In this embodiment, the training batch size is set to 8, the number of iterations is set to 100 times, and the learning rate is set to 0.0001. In terms of the selection of the loss function, the present invention needs to solve two specific tasks, namely, the foreign body scale recognition task and the foreign body position estimation task. Among them, the foreign body scale recognition task is a classification task, and the cross entropy loss function is used, which can measure the difference between the distribution of the true value and the distribution of the predicted value, and optimize the model by minimizing this difference. The expression of the cross entropy loss function is as follows:

[0065]

[0066] In the formula, and Represent the probability distribution of the true label and the predicted result respectively, is the probability of the true label, is the predicted probability.

[0067] The foreign body position estimation task is a regression task, which uses the MSE loss function, which can quantify the difference between the predicted value and the actual value. The expression of the MSE function is as follows:

[0068]

[0069] In the formula, is the true value of the sample point, is the value predicted by the model, is the number of samples. The advantage of MSE is that it is highly sensitive to outliers in the model.

[0070] This embodiment uses a deep neural network model with a visual geometry network as the backbone network to extract features from foreign objects, and uses the stochastic gradient descent (SGD) method to train the network. In the training process of the foreign object scale recognition model, the number of iterations is set to 100, the training set contains 480 images, the validation set contains 120 images, and the test set contains 486 images. The completion of the training can be judged by observing the decrease of the loss function. If the loss function converges, it indicates that the training effect has been basically stable.

[0071] Apply the trained deep learning model to the test set to obtain the final recognition effect and draw the confusion matrix as shown in Figure 7 As shown in the figure, 0-5 correspond to 25mm, 30mm, 35mm, 40mm, 45mm, and 50mm round foreign bodies, respectively. The ordinate is the true value, and the abscissa is the predicted value. The left diagonal of the matrix represents successful classification, and the numbers in each of the remaining small rectangular blocks represent the type and number of misclassifications. By calculation, it can be concluded that the overall recognition accuracy of the model is 71.6%.

[0072] In addition, this embodiment also compares the effects of some common machine learning methods, including decision tree, random forest, support vector machine, and nearest neighbor method. The data sets used for training and testing of the above four methods are consistent with the method used in the present invention. Figure 8 The recognition accuracy of the decision tree method shown is 61.9%. Fig. 9 The recognition accuracy of the random forest method shown is 51.9%, as shown in Fig.10 The recognition accuracy of the nearest neighbor method shown is 53.1%. Fig.11 The recognition accuracy of the support vector machine method shown is 52.1%, which shows that the deep learning method used in the present invention is better than the machine learning method in the comparative case in the foreign body scale recognition task.

[0073] For the task of estimating the position of foreign objects, the model will predict the radial position and angle of the foreign objects respectively. In this embodiment, the number of training times is set to 100, the training set contains 418 images, the validation set contains 105 images, and the test set contains 563 images. Similarly, when the loss function of the model converges, it can be shown that the training results are basically stable. The trained deep learning model is applied to the test set, and its prediction results are as follows: Fig.12 As shown, the root mean square error of the angle prediction is 52.5°, and the root mean square error of the radial position prediction is 7.0 mm.

[0074] This embodiment also uses decision trees, random forests, support vector machines, and nearest neighbor methods to predict and analyze the location of foreign objects. Fig.13 The root mean square error of the angle prediction of the decision tree method shown is 129.2°, and the root mean square error of the radial position prediction is 11.9 mm, as shown in Fig.14 The random forest model shown has a root mean square error of 92.3° for angle prediction and 8.7 mm for radial position prediction. Fig.15 The root mean square error of the angle prediction of the support vector machine model shown is 101.3°, and the root mean square error of the radial position prediction is 7.1 mm, as shown in Fig.16 The root mean square error of the angle prediction of the nearest neighbor model shown is 107.4°, and the root mean square error of the radial position prediction is 8.8 mm. It can be seen from the results that in the task of foreign body position estimation, the deep learning method adopted in the present invention is better than the machine learning method in the comparative case.

[0075] The above is only one solution of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A U-shaped pipe inner wall foreign body detection system based on non-visual field vision, characterized in that: include: Excitation source, detector and data processing device; The excitation source adopts optical excitation and is deployed at one port of the U-shaped pipe to emit light source; The detector is deployed at another port of the U-shaped pipe to receive optical signals; The data processing device is connected to the detector and is used to receive and process the signal from the detector to obtain the detection result of the foreign matter on the inner wall of the pipeline. The detection result includes the size and position information of the foreign matter.

2. According to claim 1, a U-shaped pipe inner wall foreign body detection system based on non-visual field vision is characterized in that: The excitation source is a continuous wave laser with a wavelength of 532nm and a power of not less than 100mW.

3. According to claim 1, a U-shaped pipe inner wall foreign body detection system based on non-visual field vision is characterized in that: The light emitted by the excitation source is received by the detector after being reflected at least three times on the inner wall of the U-shaped pipe. The optical signal received by the detector carries information related to the structure of the inner wall of the pipe.

4. According to claim 1, a U-shaped pipe inner wall foreign body detection system based on non-visual field vision is characterized in that: The detector adopts an industrial camera with a complementary metal oxide semiconductor image sensor, and the camera can convert the optical signal after multiple reflections into a digital image signal and transmit it to the data processing device through a data line.

5. According to claim 1, a U-shaped pipe inner wall foreign body detection system based on non-visual field vision is characterized in that: A deep neural network model is deployed on the data processing device, and the deep neural network model uses a visual geometry network as a backbone network.

6. A method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-line-of-sight vision, implemented by using the U-shaped pipe inner wall foreign matter detection system based on non-line-of-sight vision as described in any one of claims 1 to 5, characterized in that: The following steps are involved: The excitation source emits light, and the light is received by the detector after being reflected at least three times in the U-shaped pipe; The detector receives the optical signal, converts the optical signal into a digital image signal, and transmits the digital image signal to the data processing device via a data line; The data processing device preprocesses the digital image signal, inputs the preprocessed digital signal into a trained deep neural network model, and the deep neural network model outputs the detection result of foreign matter on the inner wall of the pipeline.

7. The method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-visual field vision according to claim 6 is characterized in that: The deep neural network model training method is: Acquire a training sample set, wherein the training sample set includes sample digital images to be identified and training labels corresponding to each sample digital image; The preset deep neural network model is trained according to the training sample set and a preset training algorithm to obtain the deep neural network model.

8. The method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-visual field vision according to claim 6 is characterized in that: The tasks of the deep neural network model include foreign object scale recognition task and foreign object position estimation task.

9. The method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-visual field vision according to claim 8, characterized in that: The fully connected layer output of the deep neural network model varies according to different task requirements; for the foreign object scale identification task, the deep neural network model outputs the foreign object category confidence, which is used to judge the size category of the foreign object; for the foreign object position estimation task, the deep neural network model outputs the foreign object position estimation value, which is used to locate the specific position of the foreign object.

10. The method for detecting foreign matter on the inner wall of a U-shaped pipe based on non-visual field vision according to claim 8, characterized in that: During the training process of the deep neural network model, the cross entropy loss function is used for the foreign object scale recognition task; and the MSE loss function is used for the foreign object position estimation task.

Images