Periscopic vault monitoring system

Abstract

The periscopic vault monitoring system comprises a monitoring device, an image acquisition module and an image recognition module connected with the image acquisition module, relates to the technical field of arch crown monitoring, and solves the technical problem that an existing arch crown monitoring mode cannot directly and quickly know the arch crown pouring condition. The monitoring video during tunnel vault pouring is obtained in real time through the monitoring device, the depth-of-field detection image and the corresponding monitoring time are extracted from the monitoring video in a picture frame extraction mode, processing of redundant data can be reduced, and therefore real-time monitoring can be carried out, and the efficiency of tunnel vault pouring is improved. The problem that repairing of the area with the pouring problem after pouring forming is troublesome is avoided, and meanwhile the accuracy of the monitoring result can be guaranteed by analyzing the detection information.

Description

Technical Field

[0001] The invention belongs to the field of architecture and relates to a vault monitoring technology, in particular to a periscope vault monitoring system. Background Art

[0002] Tunnel projects account for a large proportion of construction in various places. Since the main function of the tunnel vault is to bear the pressure from the top of the tunnel and maintain the overall stability of the tunnel, in the fields of tunnel construction such as construction and mining, the construction quality of the vault is directly related to the safety and stability of the tunnel. Therefore, the vault laying operation is a key process to ensure structural safety and construction quality. Uneven pouring of the tunnel vault will lead to the appearance of cavities, which will pose a serious threat to the overall structural safety of the tunnel.

[0003] At present, the pouring monitoring method for tunnel vaults is usually to calculate the required amount of concrete to be poured, and then compare the actual amount of concrete used to monitor the pouring of the vault. However, this method cannot directly understand whether the internal pouring of the tunnel vault is uniform. Although pressure sensors or displacement meters are installed on the formwork or supporting structure, the concrete filling situation can be inferred through mechanical data. However, this method can only detect the local state of the sensor installation position, and the data can only stabilize after the concrete solidifies. Therefore, the pouring situation of the vault cannot be obtained in time. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a periscope vault monitoring system for solving the technical problem that the current method of monitoring the vault cannot directly and quickly understand the vault casting situation. The present invention obtains the monitoring video of the tunnel vault casting in real time through a monitoring device, and extracts the depth of field detection image and the corresponding monitoring time from the monitoring video by means of picture frame extraction. By analyzing the detection information, the above-mentioned problem can be solved by ensuring the accuracy of the monitoring result.

[0005] To achieve the above-mentioned object, a first aspect of the present invention provides a periscope vault monitoring system, comprising: a monitoring device, an image acquisition module and an image recognition module connected thereto; The image acquisition module is used to extract the monitoring video of the vault pouring taken by the monitoring device, extract some monitoring information from the monitoring video, and send the monitoring information to the image recognition module; wherein the monitoring information includes the depth of field monitoring image and the monitoring time; The image recognition module is used to classify the detection images in the monitoring information, extract and analyze the image features, and obtain the pouring information of the top supply.

[0006] It should be noted that the position of the monitoring device moves according to the distance from the arch casting end.

[0007] Preferably, the depth of field monitoring image is a picture in the monitoring video within a monitoring time, and the monitoring time is the shooting time of the depth of field monitoring image.

[0008] In order to further ensure accuracy, multiple frames can be extracted when pouring at the same position of the arch. This method can not only reduce the amount of redundant data and improve processing efficiency, but also ensure its accuracy to a certain extent.

[0009] Preferably, the extracting and analyzing image features after classifying the detection images in the monitoring information includes: Classifying a number of vault monitoring images through an image classification model to obtain a number of monitoring image sets; the pouring position of each group of monitoring image sets is the same; Generate a depth map for all the vault monitoring images in each set of detection images using a monocular depth estimation model; obtain the shooting distance between the vault monitoring image and the monitoring device, and extract the pouring image from the vault monitoring image based on the shooting distance and the depth map; The pouring information is obtained according to the pouring images and monitoring time corresponding to the vault monitoring images in the plurality of monitoring image sets; wherein the pouring information includes the uniformity of the pouring amount and the pouring speed.

[0010] It should be noted that when monitoring is just started, due to the small amount of data, historical data can be introduced when analyzing and comparing the vault monitoring images in each set of monitoring images. However, it should be noted that the introduced historical data needs to ensure the consistency of the data of the tunnel vault. If they are consistent, manual correction is required. The "same casting position" mentioned above refers to the area with the same vault cross-section position at different depths of the tunnel vault.

[0011] Preferably, the classifying of the plurality of vault monitoring images comprises: Constructing an image classification model: constructing an image classification model based on a convolutional neural network and introducing an attention mechanism, obtaining a number of vault monitoring images and the classification labels corresponding to the vault monitoring images through a database; integrating the vault monitoring images and the corresponding classification labels into a training data set and a test data set; training the image classification model through the training data set, and testing the trained image classification model through the test data set; adjusting the image classification model according to the test accuracy, and finally obtaining an image classification model with the vault monitoring image as input and the classification label as output; Image classification: Integrate the vault monitoring images with the same output classification labels to obtain several groups of monitoring image sets.

[0012] It should be noted that the classification label corresponding to the vault monitoring image is determined according to the position of the casting end of the vault, that is, the classification labels corresponding to casting ends at different positions are different.

[0013] Preferably, extracting the pouring image from the vault monitoring image according to the shooting distance and the depth map includes: Extract the vault monitoring image and the corresponding depth map; obtain the gray value of each pixel in the depth map, and calculate the monitoring distance of each pixel from the monitoring device according to the linear mapping relationship between the gray value and the distance; The monitoring distance is compared with the preset casting distance interval; when the monitoring distance is within the casting distance interval, the pixel corresponding to the monitoring distance is marked as the target pixel; when the monitoring distance is outside the casting distance area, the pixel corresponding to the monitoring distance is marked as an irrelevant pixel, and the irrelevant pixel is corrected based on the positional relationship of the target pixel; The target pixels of each vault monitoring image in the monitoring image set are extracted and counted to obtain a pouring image.

[0014] The pouring distance interval is determined based on the shooting distance and the farthest distance of the free fall of concrete during the vault pouring; and the linear mapping relationship between the gray value and the distance is: , where d is the monitoring distance, g is the gray value of the pixel, and d max is the distance of the farthest object from the monitoring device in the monitoring image, d min It is the distance of the object closest to the monitoring device in the monitoring image.

[0015] Preferably, the correction of irrelevant pixels based on the positional relationship of the target pixels includes: Extract the area covered and surrounded by the target pixel and mark it as the target area; determine whether all the irrelevant pixels in the target area are target pixels; if yes, mark the corresponding irrelevant pixels as target pixels; if not, keep the mark of the irrelevant pixels.

[0016] Preferably, the method for acquiring the pouring information includes: Extract the target pixel number of the pouring image corresponding to each vault monitoring image in a plurality of groups of detection image sets and the time difference between pouring two identical vault areas; Obtaining measurement data for measuring the number of target pixels and the degree of dispersion of time differences in each set of detection images; wherein the measurement data may be standard deviation, variance, range, etc.; The measurement data corresponding to each target pixel number and each time difference are compared with the preset threshold value to obtain the pouring information.

[0017] Specifically, the uniformity of pouring volume includes uniformity and differences; the uniformity of pouring speed also includes uniformity and differences, and the differences can be graded and differentiated according to the difference between the measured data and the corresponding preset threshold value. The time difference can reflect whether the pouring time between the two vault pouring areas is too different.

[0018] Preferably, the method for acquiring the pouring information includes: Extracting the target pixel number of the casting image corresponding to each vault monitoring image and the time difference between each vault monitoring image and the adjacent vault monitoring image from a plurality of groups of detection images; The vault monitoring image and the corresponding target pixel data and time difference are input into the prediction model to obtain the output label, and the corresponding pouring information is matched according to the output label. Among them, the prediction model is built based on the recurrent neural network model.

[0019] Preferably, the prediction model is constructed by: The BP neural network model is trained by standard training data, and the trained BP neural network model is marked as a prediction model; wherein the standard training data consists of training input data and training output data, the training input data is the vault monitoring image and the corresponding target pixel data and time difference, and the training output data is the output label corresponding to the training input data.

[0020] It should be noted that the training input data is historical data, and the training output data is manually labeled based on the input data.

[0021] Preferably, the image recognition module is further used to: determine the availability of the depth of field monitoring image; when the monitoring information is unavailable, adjust the monitoring parameters of the monitoring device and reacquire the monitoring information; The method to determine the usability of the depth of field monitoring image is to perform Fourier transform on the image and extract the high-frequency components. The clarity of the depth of field monitoring image is determined by calculating the proportion of the high-frequency components and comparing them with a preset threshold. The lower the proportion of the high-frequency components, the blurrier the image.

[0022] Preferably, the monitoring device includes a periscope, an image acquisition and transmission device, a slewing bearing, a guide rod and a servo motor; Compared with the prior art, the beneficial effects of the present invention are as follows: by acquiring the monitoring video of the tunnel vault pouring in real time through the monitoring device, and extracting the depth of field detection image and the corresponding monitoring time from the monitoring video by means of picture frame extraction, the processing of redundant data volume can be reduced, so that real-time monitoring can be performed, and the troublesome problem of repairing the area with pouring problems after pouring and forming can be avoided. At the same time, the accuracy of the monitoring result can be guaranteed by analyzing the detection information; and in order to obtain the uniformity of each pouring area of ​​the tunnel vault, the pouring image is extracted from the concrete image in the depth of field monitoring image by grayscale-distance mapping, and the pouring image corresponding to the pouring at the same position is compared, so that the area of ​​each same pouring position at the depth of the tunnel vault can be compared to measure whether there is a large difference in the pouring amount of concrete of the vault. In this way, independent analysis of different pouring areas is achieved, data mixing at different positions can be avoided, and the accuracy of monitoring is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is a schematic diagram of the process of Example 1 of the present invention; Figure 2 This is a schematic diagram of the process of Embodiment 3 of the present invention; Figure 3 It is a schematic diagram of the vault monitoring device of the present invention.

[0025] In the figure: 1. Periscope device; 2. Image acquisition and transmission device; 3. Rotary bearing; 4. Guide rod; 5. Servo motor. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figure 1-Figure 3 , the present invention provides the following embodiments: Example 1 A periscope vault monitoring system comprises: a monitoring device, an image acquisition module and an image recognition module connected thereto; The image acquisition module: extracts the monitoring video of the vault pouring shot by the image acquisition and transmission device 2 in the monitoring device through 4G transmission, extracts some monitoring information from the monitoring video, and sends the some monitoring information to the image recognition module; wherein the monitoring information includes the depth of field monitoring image and the monitoring time; The image recognition module is used to classify the detection images in the monitoring information, extract and analyze the image features, and obtain the pouring information of the top supply; The monitoring device includes a periscope 1, an image acquisition and transmission device 2, a slewing bearing 3, a guide rod 4 and a servo motor 5; the design and performance of the periscope 1 play a key role in the image acquisition capability of the system. Therefore, the objective lens and the eyepiece of the periscope 1 both use high-resolution infrared cameras to ensure that clear and delicate image information can be obtained. The high-resolution infrared camera can capture more details. In addition, the optical performance of the objective lens and the eyepiece also has good light transmittance, low distortion and other characteristics to ensure the quality and authenticity of the image. At the same time, the design of the prism should take into account the minimization of light loss to improve the light efficiency of the system. The image acquisition and transmission device 2 is a camera device with a 4G module. The slewing bearing 3 is used to control its overall rotation, and the guide rod 4 mainly guides the direction of movement to ensure that the monitoring equipment can accurately move and adjust in the arch area of ​​the lining trolley to achieve comprehensive monitoring of the arch. The servo motor 5 drives the movement of the monitoring device.

[0028] It should be noted that the above-mentioned depth of field monitoring image is a picture in the monitoring video during the monitoring time, and the monitoring time is the shooting time of the depth of field monitoring image. In order to further ensure the accuracy, as another preferred embodiment, multiple frames of pictures can be extracted when pouring at the same position of the arch.

[0029] Furthermore, after the detection images in the monitoring information are classified, image features are extracted and analyzed, including: Using an image classification model to classify a number of vault monitoring images according to the positions when the vault is poured, to obtain a number of monitoring image sets; the pouring positions of each group of monitoring image sets are the same; Specifically, the method for constructing the image classification model includes: Constructing an image classification model: constructing an image classification model based on a convolutional neural network and introducing an attention mechanism, obtaining a number of vault monitoring images and the classification labels corresponding to the vault monitoring images through a database; integrating the vault monitoring images and the corresponding classification labels into a training data set and a test data set; training the image classification model with the training data set, and testing the trained image classification model with the test data set; adjusting the image classification model according to the test accuracy, and finally obtaining an image classification model with the vault monitoring images as input and the classification labels as output; Image classification: Integrate the vault monitoring images with the same output classification labels to obtain several groups of monitoring image sets.

[0030] Then, the MiDaS v3.1 pre-trained model is used to generate monocular depth maps for all the vault monitoring images in each set of detection images; The specific implementation code is as follows: import torch from midas.model_loader import load_model device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model, transform = load_model("MiDaS_small", device) image = load_image("vault_monitor_001.jpg")# Load the original image input_tensor = transform(image).unsqueeze(0).to(device) with torch.no_grad(): depth_map = model(input_tensor) # Output depth map relative depth value) depth_map = depth_map.squeeze().cpu().numpy() Acquire the shooting distance between the vault monitoring image and the monitoring device, and extract the pouring image from the vault monitoring image according to the shooting distance and the depth map; Specifically, the pouring image is extracted from the vault monitoring image according to the shooting distance and the depth map, including: Extract the vault monitoring image and the corresponding depth map; obtain the gray value of each pixel in the depth map, and calculate the monitoring distance of each pixel from the monitoring device according to the linear mapping relationship between the gray value and the distance; The monitoring distance is compared with the preset casting distance interval; when the monitoring distance is within the casting distance interval, the pixel corresponding to the monitoring distance is marked as the target pixel; when the monitoring distance is outside the casting distance area, the pixel corresponding to the monitoring distance is marked as an irrelevant pixel, and the irrelevant pixel is corrected based on the positional relationship of the target pixel; The target pixels of each vault monitoring image in the monitoring image set are counted and extracted to obtain a pouring image.

[0031] The pouring distance interval is determined based on the shooting distance and the farthest distance of the free fall of concrete during the vault pouring; and the linear mapping relationship between the gray value and the distance is: , where d is the monitoring distance, g is the gray value of the pixel, and d max is the distance of the farthest object from the monitoring device in the monitoring image, d min It is the distance of the object closest to the monitoring device in the monitoring image.

[0032] Before construction, the farthest point d in the scene is measured by a laser rangefinder. max = 15m (e.g. top of the vault) and the nearest point d min =2m (such as the reference object position), the monocular depth estimation model used in this embodiment, so the grayscale value of the pixel is 0 (black) → the farthest distance, and 255 (white) → the closest distance.

[0033] The pouring information is obtained according to the pouring images and monitoring time corresponding to the vault monitoring images in the plurality of monitoring image sets; wherein the pouring information includes the uniformity of the pouring amount and the pouring speed.

[0034] The code implemented by the specific pixel classification logic is as follows: def classify_pixels(depth_map, D, d_min=2.0, d_max=15.0): # Calculate the actual distance of each pixel normalized_depth = (depth_map - np.min(depth_map)) / (np.max(depth_map) - np.min(depth_map)) d_pixels = d_max - normalized_depth (d_max - d_min) # Set the pouring distance interval lower_bound = D - 0.5 # Unit: meter upper_bound = D + 0.3 # Generate binary mask mask = np.where((d_pixels >= lower_bound) & (d_pixels <= upper_bound), 1, 0) # Morphological closing operation to fill holes kernel = np.ones((5,5), np.uint8) mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel) return mask The correction of irrelevant pixels based on the positional relationship of the target pixels mentioned in this implementation includes: Extract the area covered and surrounded by the target pixel and mark it as the target area; determine whether the irrelevant pixels in the target area are all target pixels; if yes, mark the corresponding irrelevant pixels as target pixels; if not, keep the mark of the irrelevant pixels; in this embodiment, the irrelevant pixels are eight pixels around the irrelevant pixel.

[0035] For example, if a pixel is marked as an “irrelevant pixel” (mask=0), but there are more than 5 “target pixels” (mask=1) in its surrounding 8 neighborhoods, it will be corrected to a target pixel.

[0036] Specifically, the methods for obtaining pouring information include: Extract the target pixel number of the pouring image corresponding to each vault monitoring image in a plurality of groups of detection image sets and the time difference between pouring two identical vault areas; Obtain the variance that measures the number of target pixels and the degree of dispersion of each time difference in each set of detection images; The number of each target pixel and the variance corresponding to each time difference are compared with the preset threshold value to obtain the pouring information.

[0037] Example 2

[0038] Different from the above-mentioned embodiment, the method for obtaining the pouring information in the present application includes: Extracting the target pixel number of the casting image corresponding to each vault monitoring image and the time difference between each vault monitoring image and the adjacent vault monitoring image from a plurality of groups of detection images; The vault monitoring image and the corresponding target pixel data and time difference are input into the prediction model to obtain the output label, and the corresponding pouring information is matched according to the output label. Among them, the prediction model is built based on the recurrent neural network model.

[0039] The specific method of constructing the prediction model includes: The BP neural network model is trained by standard training data, and then the trained model is tested by standard test data. When the test accuracy reaches a preset threshold, the trained BP neural network model is marked as a prediction model; wherein the standard training data is composed of training input data and training output data, the training input data is the vault monitoring image and the corresponding target pixel data and time difference, and the training output data is the output label of the corresponding training input data; and the standard test data has the same composition as the standard training data.

[0040] The pouring information obtained through the artificial intelligence model can be iteratively optimized with new data, gradually improving the adaptability to different project scenarios.

[0041] Example 3

[0042] The difference from the above embodiment is that in order to further improve the accuracy of its pouring information.

[0043] The image recognition module is also used to: determine the availability of the depth of field monitoring image; and when the monitoring information is unavailable, adjust the monitoring parameters of the monitoring device and reacquire the monitoring information.

[0044] The method to determine the usability of the depth of field monitoring image is to perform Fourier transform on the image and extract the high-frequency components. The clarity of the depth of field monitoring image is determined by calculating the proportion of the high-frequency components and comparing them with a preset threshold. The lower the proportion of the high-frequency components, the blurrier the image.

[0045] Of course, rapid screening can also be carried out through manual judgment.

[0046] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0047] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A periscope vault monitoring system, characterized in that: include: A monitoring device, an image acquisition module, and an image recognition module connected thereto; The image acquisition module is used to extract the monitoring video of the vault pouring taken by the monitoring device, extract some monitoring information from the monitoring video, and send the monitoring information to the image recognition module; wherein the monitoring information includes the depth of field monitoring image and the monitoring time; The image recognition module is used to classify the detection images in the monitoring information, extract and analyze the image features, and obtain the pouring information of the top supply.

2. A periscope vault monitoring system according to claim 1, characterized in that: The depth-of-field monitoring image is a picture in the monitoring video during the monitoring time, and the monitoring time is the shooting time of the depth-of-field monitoring image.

3. A periscope vault monitoring system according to claim 1, characterized in that: The step of extracting and analyzing image features after classifying the detection images in the monitoring information includes: Classifying a plurality of vault monitoring images according to the positions when the vault is poured, to obtain a plurality of monitoring image sets; the pouring positions of each group of monitoring image sets are the same; Generate a depth map for all the vault monitoring images in each set of detection images using a monocular depth estimation model; obtain the shooting distance between the vault monitoring image and the monitoring device, and extract the pouring image from the vault monitoring image based on the shooting distance and the depth map; The pouring information is obtained according to the pouring images and monitoring time corresponding to the vault monitoring images in the plurality of monitoring image sets; wherein the pouring information includes the uniformity of the pouring amount and the pouring speed.

4. A periscope vault monitoring system according to claim 3, characterized in that: The method of classifying the plurality of vault monitoring images is to classify them through an image classification model; The image classification model is constructed in a manner including: Constructing an image classification model: constructing an image classification model based on a convolutional neural network and introducing an attention mechanism, obtaining a number of vault monitoring images and the classification labels corresponding to the vault monitoring images through a database; integrating the vault monitoring images and the corresponding classification labels into a training data set and a test data set; training the image classification model through the training data set, and testing the trained image classification model through the test data set; adjusting the image classification model according to the test accuracy, and finally obtaining an image classification model with the vault monitoring image as input and the classification label as output; Image classification: Integrate the vault monitoring images with the same output classification labels to obtain several groups of monitoring image sets.

5. The periscope vault monitoring system according to claim 3, characterized in that: The method of extracting a pouring image from the vault monitoring image according to the shooting distance and the depth map includes: Extract the vault monitoring image and the corresponding depth map; obtain the gray value of each pixel in the depth map, and calculate the monitoring distance of each pixel from the monitoring device according to the linear mapping relationship between the gray value and the distance; The monitoring distance is compared with the preset casting distance interval; when the monitoring distance is within the casting distance interval, the pixel corresponding to the monitoring distance is marked as the target pixel; when the monitoring distance is outside the casting distance area, the pixel corresponding to the monitoring distance is marked as an irrelevant pixel, and the irrelevant pixel is corrected based on the positional relationship of the target pixel; The target pixels of each vault monitoring image in the monitoring image set are extracted and counted to obtain a pouring image.

6. A periscope vault monitoring system according to claim 5, characterized in that: The correcting of irrelevant pixels based on the positional relationship of the target pixels includes: Extract the area covered and surrounded by the target pixel and mark it as the target area; determine whether all the irrelevant pixels in the target area are target pixels; if yes, mark the corresponding irrelevant pixels as target pixels; if not, keep the mark of the irrelevant pixels.

7. A periscope vault monitoring system according to claim 3, characterized in that: The method for obtaining the pouring information includes: Extract the target pixel number of the pouring image corresponding to each vault monitoring image in a plurality of groups of detection image sets and the time difference between pouring two identical vault areas; Obtain measurement data for measuring the number of pixels of each target and the degree of dispersion of each time difference in each set of detection images; The measurement data corresponding to each target pixel number and each time difference are compared with the preset threshold value to obtain the pouring information.

8. The periscope vault monitoring system according to claim 3, characterized in that: The method for obtaining the pouring information includes: Extracting the target pixel number of the casting image corresponding to each vault monitoring image and the time difference between each vault monitoring image and the adjacent vault monitoring image from a plurality of groups of detection images; The vault monitoring image and the corresponding target pixel data and time difference are input into the prediction model to obtain the output label, and the corresponding pouring information is matched according to the output label; wherein the prediction model is constructed based on the recurrent neural network model.

9. A periscope vault monitoring system according to claim 8, characterized in that: The method for constructing the prediction model includes: The BP neural network model is trained by standard training data, and the trained BP neural network model is marked as a prediction model; wherein the standard training data consists of training input data and training output data, the training input data is the vault monitoring image and the corresponding target pixel data and time difference, and the training output data is the output label corresponding to the training input data.

10. The periscope vault monitoring system according to claim 1, characterized in that: The image recognition module is also used to: determine the availability of the depth of field monitoring image; and when the monitoring information is unavailable, adjust the monitoring parameters of the monitoring device and reacquire the monitoring information.

Images