Method and system for identifying construction procedures in tunnel based on video information

Through the in-tunnel construction process identification technology based on convolutional neural network and EnlightenGAN algorithm, the problem of poor real-time performance of the in-tunnel construction process identification technology is solved, and efficient and accurate in-tunnel construction monitoring is achieved, reducing maintenance costs and human resources waste.

CN119007078AActive Publication Date: 2024-11-22QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES) +1

Patent Information

Application Number
CN202411093098.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing construction process identification technology in the tunnel has poor real-time, slow response, strong human subjectivity, and high calculation and time complexity of detection algorithms based on deep learning, making it difficult to adapt to the real-time monitoring requirements in the tunnel.

Method used

Using the construction process recognition technology based on convolutional neural network, combined with the EnlightenGAN algorithm, the feature information of different scales is extracted through multi-scale expansion of attention blocks, and the feature information is fused to realize real-time monitoring in complex scenarios such as large changes in light and low visibility in the tunnel. , reduce the model volume and parameter quantity, and improve detection performance.

Benefits of technology

It realizes efficient and accurate monitoring of construction processes in the tunnel, reduces labor costs, improves inspection efficiency, ensures the rationality and standardization of construction processes, and reduces maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007078A_ABST
    Figure CN119007078A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of tunnel monitoring video analysis, and provides an in-tunnel construction process identification method and system based on video information, and the method comprises the steps: obtaining a construction process video, and carrying out the extraction of a construction process image; preprocessing the construction process image; based on the preprocessed construction process image, construction process identification is carried out; wherein the construction process identification based on the preprocessed construction process image comprises the following steps: unifying the size of the preprocessed construction process image to obtain a standard construction process image; based on the standard construction process image, using a multi-scale expansion attention block to extract construction process image feature information of different scales; fusing the construction process image feature information of different scales to obtain a fused construction process image; and performing construction process identification based on the fused construction process image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of tunnel monitoring video analysis, and in particular relates to a method and system for identifying construction procedures in a tunnel based on video information. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the current field of tunnel construction, construction management and monitoring are crucial aspects. However, traditional monitoring methods have certain limitations in real-time and accuracy. With the continuous development of technology, tunnel construction process recognition technology based on image processing and computer vision has become an effective means to improve the level of construction management. In the past, the monitoring of construction processes in tunnels usually relied on manual inspections and manual records. This method has problems such as high labor costs, untimely data collection, and prone to subjective errors. Therefore, existing technologies gradually tend to introduce automated process recognition technology, using computer algorithms and advanced sensor equipment to achieve real-time monitoring and accurate identification of construction processes in tunnels. This can not only improve the level of intelligent engineering management, but also effectively reduce human errors, providing reliable technical support for the efficient promotion of tunnel construction.

[0004] Therefore, developing an efficient and accurate tunnel construction process identification technology has become a research hotspot in the current field. Tunnel on-site construction process monitoring is a key link in tunnel construction and the basis for realizing information-based and dynamic construction. On-site monitoring aims to grasp the dynamic information in the tunnel, evaluate the rationality of the construction process, and then provide feedback to guide the design and construction of tunnel projects.

[0005] However, the existing tunnel construction process recognition technology has not been applied to the technology in the field of artificial intelligence, and often adopts the method of manual video remote monitoring, which has the disadvantages of poor real-time performance, slow response, and strong human subjectivity. The detection algorithm based on deep learning is represented by the one-stage detection algorithm (One-stage) and the two-stage detection algorithm (Two-stage). The two-stage algorithm first generates the target candidate frame, and then classifies and regresses the candidate frame. It is represented by the FasterRCNN series and Mask RCNN series algorithms. The detection accuracy of the target is high, but the detection speed is slow, the calculation amount and time complexity are high, and it is difficult to meet the real-time requirements of tunnel detection. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a method and system for identifying construction procedures in tunnels based on video information. The present invention proposes a construction procedure identification technology based on convolutional neural networks, combined with the EnlightenGAN algorithm, to achieve real-time monitoring of construction in complex scenes such as large lighting changes and low visibility in tunnels, while reducing the model volume, reducing the number of parameters, and reducing the training cost, while improving the detection performance and effectively alleviating the problem of low efficiency of real-time monitoring in tunnels.

[0007] According to some embodiments, a first solution of the present invention provides a method for identifying construction procedures in a tunnel based on video information, which adopts the following technical solutions:

[0008] A method for identifying construction procedures in a tunnel based on video information, comprising:

[0009] Obtain construction process videos to extract construction process images;

[0010] Preprocess the construction process images;

[0011] Based on the pre-processed construction process images, construction process recognition is performed;

[0012] The construction process recognition based on the pre-processed construction process image includes:

[0013] Unify the size of the preprocessed construction process images to obtain standard construction process images;

[0014] Based on the standard construction process images, the multi-scale dilated attention block is used to extract the feature information of construction process images at different scales.

[0015] The feature information of the construction process images at different scales is fused to obtain a fused construction process image;

[0016] Construction process recognition is performed based on the fused construction process images.

[0017] Furthermore, the acquisition of the construction process video to extract the construction process image is specifically as follows:

[0018] Obtain the construction process video, and perform color space conversion on the construction process video frames;

[0019] An average frame difference calculation is performed based on the converted construction process video frames to obtain an array of differences between construction process video frames;

[0020] Smoothing the construction process video frame difference array to obtain a smoothed frame difference array;

[0021] The index of the local maximum point in the smoothed inter-frame difference array is found to obtain the key frame of the construction process video and form a construction process image.

[0022] Furthermore, the preprocessing of the construction process image includes:

[0023] Performing data enhancement and expansion on the construction process image to obtain an expanded construction process image;

[0024] The EnlightenGAN algorithm is used to perform image enhancement on the expanded construction process image to obtain the preprocessed construction process image.

[0025] Furthermore, the pre-processed construction process images are resized to obtain standard construction process images, specifically:

[0026] Perform image enhancement based on the preprocessed construction process images;

[0027] Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for enhanced construction process images;

[0028] Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size to obtain a standard construction process image.

[0029] Furthermore, based on the standard construction process image, the multi-scale dilated attention block is used to extract the feature information of the construction process images of different scales, specifically:

[0030] Feature extraction is performed based on the standard construction process image to obtain the initial convolution feature;

[0031] Utilize multiple parallel dilated convolution operations with different dilation rates to capture information of different scales in the initial convolution features;

[0032] By passing the linear projection of query, key, and value into the dilated convolutional attention mechanism, the attention weight is calculated for each dilation rate, and local features of different scales are extracted to obtain the output features of multi-head dilated local attention;

[0033] The output features of the multi-head dilated local attention are mapped to the target dimension through linear projection to obtain the image feature information of construction processes at different scales.

[0034] Furthermore, the feature information of the construction process images of different scales is fused to obtain the fused construction process image, specifically:

[0035] The initial convolution features and the feature information of the construction process images at different scales are fused to obtain the fused features;

[0036] The fused features are normalized and then output through the fully connected layer to obtain the fused construction process image.

[0037] Furthermore, the multi-scale dilated attention block includes depthwise separable convolution, multi-scale sliding window dilated attention operation and rL.

[0038] According to some embodiments, a second solution of the present invention provides a tunnel construction process identification system based on video information, which adopts the following technical solutions:

[0039] A tunnel construction process identification system based on video information, comprising:

[0040] An image extraction module is configured to obtain a construction process video to extract a construction process image;

[0041] An image preprocessing module is configured to preprocess the construction process image;

[0042] A process identification module is configured to identify the construction process based on the preprocessed construction process image;

[0043] The construction process recognition based on the pre-processed construction process image includes:

[0044] Unify the size of the preprocessed construction process images to obtain standard construction process images;

[0045] Based on the standard construction process images, the multi-scale dilated attention block is used to extract the feature information of construction process images at different scales.

[0046] The feature information of the construction process images at different scales is fused to obtain a fused construction process image;

[0047] Construction process recognition is performed based on the fused construction process images.

[0048] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for identifying construction procedures in a tunnel based on video information as described in the first aspect above.

[0050] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0051] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method for identifying construction procedures in a tunnel based on video information as described in the first aspect above are implemented.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] This invention replaces the traditional manual inspection method. The application of artificial intelligence makes the monitoring in the tunnel more objective and accurate. Compared with the traditional manual inspection, it can accurately and comprehensively capture the dynamic behavior in the tunnel without being disturbed by subjective emotions. In the long run, automated intelligent monitoring can greatly reduce maintenance costs, reduce the waste of human resources, and save time and resources for tunnel managers. Intelligent process identification can detect with high precision and high speed whether the construction personnel in the tunnel are operating properly and whether the processes are efficiently connected, ensuring the rational use of construction materials, standardized operation of personnel, and on-time delivery of the project.

[0054] The present invention adopts an adaptive high-precision module to perform multi-scale feature extraction and enhancement processing on the feature map, thereby improving the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0056] Figure 1 This is a flow chart of a method for identifying construction procedures in a tunnel based on video information in an embodiment of the present invention;

[0057] Figure 2 is a feature extraction network structure diagram in an embodiment of the present invention;

[0058] Figure 3 is a structural diagram of SWDA in an embodiment of the present invention;

[0059] Figure 4 is a structural diagram of a convolutional neural network in an embodiment of the present invention;

[0060] Figure 5 is a schematic diagram of a sample data set in an embodiment of the present invention;

[0061] Figure 6 It is a schematic diagram of the final recognition result in the embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0063] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0064] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0065] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0066] Embodiment 1

[0067] like Figure 1 As shown, this embodiment provides a method for identifying construction procedures in a tunnel based on video information. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communications, middleware services, domain name services, security services CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In this embodiment, the method includes the following steps:

[0068] Step S1: Obtain construction process video to extract construction process images;

[0069] Step S2: preprocessing the construction process image;

[0070] Step S3: Perform construction process recognition based on the pre-processed construction process image.

[0071] With respect to step S1, the construction process video is obtained to extract the construction process image, specifically:

[0072] Step S1.1: Obtain a construction process video, and perform color space conversion on the construction process video frame;

[0073] Step S1.2: Calculate the average frame difference based on the converted construction process video frames to obtain an array of differences between construction process video frames;

[0074] Step S1.3: Smoothing the construction process video frame difference array to obtain a smoothed frame difference array;

[0075] Step S1.4: Find the index of the local maximum point in the smoothed inter-frame difference array, obtain the key frame of the construction process video, and form a construction process image.

[0076] Video analysis is usually based on video frames, but usually one second of video contains 24 frames of images. If the key frames in the video can be "captured", especially in tunnel environments and scenes where similar shots take a long time to take, deduplication of the images will help filter out most of the "noise" frames and extract the core content of the video to the greatest extent.

[0077] Average frame difference calculation:

[0078] In step S1.1, the read video is converted into a color space, and the frames are converted into a LUV color space, which represents brightness (L) and chrominance (UV), respectively, so as to more effectively capture the grayscale information of the image. Then, the difference calculation is performed on the converted frames, and the absolute difference is used here to measure the difference between adjacent frames.

[0079] In step S1.2, the pixel values ​​of the entire difference image are summed to obtain the comprehensive difference between frames. Finally, the sum of the difference between frames is divided by the number of pixels in the image to obtain the average difference between frames, which is saved in the corresponding array.

[0080] Smoothing:

[0081] In step S1.3, the "smooth" function is used to smooth the inter-frame difference array to reduce noise so that subsequent local maximum detection is more stable. First, the original inter-frame difference array is border-extended to handle the boundary situation and ensure that the convolution operation can be calculated correctly. This function also allows users to flexibly adjust the smoothing method according to specific needs and signal characteristics to achieve better smoothing effects. Then the data is averaged using the weights of the custom window through the convolution operation to reduce noise. Then use np.convolve to perform a convolution operation on the inter-frame difference array to finally achieve smooth data. Here, a normalized version of the convolution kernel w is used to ensure that the weight sum is 1.

[0082] In step S1.4, a function argrelextrema in the SciPy library is used to find the indices of the local maximum points in the smoothed inter-frame difference array. These indices correspond to the frames with larger differences in the video, namely the key frames. The indices of the local maximum points are traversed, and the indexes of the corresponding frames are used as key frames and saved as image files.

[0083] In step S2, the preprocessing of the construction process image includes:

[0084] Step S2.1: performing data enhancement and expansion on the construction process image to obtain an expanded construction process image;

[0085] Step S2.2: Use the EnlightenGAN algorithm to perform image enhancement on the expanded construction process image to obtain a preprocessed construction process image.

[0086] It should be noted that EnlightenGAN is the first efficient and unsupervised generative adversarial network that successfully introduces unpaired training into low-light image enhancement, eliminating the reliance on paired training data and can be trained using more scene images from different fields. Its structure adopts a global-local discriminator structure to ensure that all local areas of the enhanced image look like real natural light; the U-Net network is used as the backbone network of the generator, and the addition of an attention mechanism makes the enhancement of dark areas greater than that of bright areas. Therefore, compared with other currently advanced low-light image enhancement methods, EnlightenGAN not only successfully learns to enhance dark areas, retain texture details, and avoid overexposure artifacts, but also easily adapts to real low-noise low-light images and produces visually pleasing enhanced images. Through EnlightenGAN's preprocessing of images, irrelevant information in the image is eliminated, useful real information is restored, the detectability of relevant information is enhanced, and the data is simplified to the maximum extent, thereby improving the reliability of feature extraction and target recognition.

[0087] Among them, in step S3, the construction process recognition based on the pre-processed construction process image includes:

[0088] Step 3.1: Unify the size of the preprocessed construction process images to obtain standard construction process images;

[0089] Step 3.2: Based on the standard construction process image, use the multi-scale dilated attention block to extract the feature information of the construction process images at different scales;

[0090] Step 3.3: Fusing the feature information of the construction process images at different scales to obtain a fused construction process image;

[0091] Step 3.4: Perform construction process recognition based on the fused construction process images.

[0092] In step 3.1, the pre-processed construction process images are resized to obtain standard construction process images, specifically:

[0093] Perform image enhancement based on the preprocessed construction process images;

[0094] Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for enhanced construction process images;

[0095] Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size to obtain a standard construction process image.

[0096] In step 3.2, based on the standard construction process image, the multi-scale dilated attention block is used to extract the feature information of the construction process images of different scales, specifically:

[0097] Feature extraction is performed based on the standard construction process image to obtain the initial convolution feature;

[0098] Utilize multiple parallel dilated convolution operations with different dilation rates to capture information of different scales in the initial convolution features;

[0099] By passing the linear projection of query, key, and value into the dilated convolutional attention mechanism, the attention weight is calculated for each dilation rate, and local features of different scales are extracted to obtain the output features of multi-head dilated local attention;

[0100] The output features of the multi-head dilated local attention are mapped to the target dimension through linear projection to obtain the image feature information of construction processes at different scales.

[0101] In step 3.3, the feature information of the construction process images at different scales is fused to obtain the fused construction process image, specifically:

[0102] The initial convolution features and the feature information of the construction process images at different scales are fused to obtain the fused features;

[0103] The fused features are normalized and then output through the fully connected layer to obtain the fused construction process image.

[0104] In a specific embodiment, this embodiment proposes a method for identifying construction processes in tunnels based on video information, which is combined with the EnlightenGAN algorithm to achieve real-time monitoring of construction in complex scenarios such as large changes in lighting and low visibility in tunnels. While reducing the model volume, the number of parameters, and the training cost, it improves the detection performance and effectively alleviates the problem of low efficiency of real-time monitoring in tunnels.

[0105] The training phase of this method specifically includes:

[0106] (1) Obtain a dataset of tunnel construction processes, 519 tunnel construction site images, and annotate object detection frames and categories for 9 types of equipment.

[0107] (2) Use the roLabelImg tool to relabel some images.

[0108] (3) Perform data enhancement on the image data, expand the data set, and then divide it into training set and test set.

[0109] (4) The EnlightenGAN algorithm is used to perform image enhancement on the training set.

[0110] (5) Build a process identification model.

[0111] (6) Use the improved model to train and test the data set, and finally realize intelligent identification of the process, such as Figure 6 shown.

[0112] Dataset processing:

[0113] The data set of this embodiment is collected by on-site photography of a certain tunnel, and some of the image data are re-labeled using the roLabelImg tool. Among them, after the partial image data are re-labeled using the roLabelImg tool, the image data labeled using the roLabelImg tool is first saved with an .xml suffix, and then converted to a .txt suffix format. The labeled image data is enhanced and the data set is expanded to obtain a data set, and then the data set is divided into a training set and a test set. Among them, the data set is divided into a training set and a test set in a ratio of 4:1, and finally reaches 8,000 images for the training set and 2,000 images for the test set. Figure 5 As shown in Figure 2, the EnlightenGAN algorithm is used to enhance the image of the training set. EnlightenGAN adopts a global-local discriminator structure, uses the U-Net network as the backbone network of the generator, and adds an attention mechanism.

[0114] Model building and training

[0115] This embodiment constructs a convolutional neural network (CNN) model, namely a process recognition model, which is used to realize the intelligent recognition and classification of construction processes in tunnels. The network model mainly consists of four parts: input layer, backbone network layer, neck, and head. The input layer defines the input original image size as 640x640, the backbone network layer performs convolution operations to propose feature information of non-universal scales, the neck is responsible for fusing the image feature information, and the head performs non-maximum suppression processing on the predicted anchor frame coordinates, categories, and confidences and then outputs them.

[0116] The input layer mainly consists of Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling. Mosaic image enhancement can increase the diversity and difficulty of the training set, which helps to improve the robustness and generalization ability of the target detection model. At the same time, Mosaic image enhancement can also reduce the risk of overfitting and improve the training effect of the model. The idea of ​​adaptive anchor box is to automatically calculate the anchor box parameters that best suit the input image through learning without manual setting. This method can improve the accuracy and robustness of target detection. Adaptive image scaling is an image scaling method based on the target scale. It can adaptively scale the size of the input image to adapt to the detection of targets of different scales. This method can effectively solve the scale inconsistency problem in target detection and improve detection accuracy and robustness.

[0117] like Figure 2 As shown in the figure, this network uses the lightweight MobileOne network as the backbone network layer. The latency bottleneck is analyzed based on the network structure. The ReLU activation function has the lowest latency, so only the ReLU activation function is used in MobileOne. The key factors affecting latency for the Block structure are the memory access cost and the degree of model parallelism. If the network has fewer branches, this memory bottleneck can be avoided. Therefore, a structure without branches during inference is adopted to reduce the memory access cost. Therefore, only the SE block is used in the largest variant of MobileOne to improve accuracy. The structure of the MobileOne block is decomposed into depthwise layers and pointwise layers, and an over-parameterization branch is also introduced. The structure of the basic block is based on the design of MobileNet-V1, with a 3x3 depthwise convolution followed by a 1x1 pointwise convolution, and then a reparameterizable skip-connection branch is added, which specifically includes a batchnorm branch and a branch that can be replicated multiple times.

[0118] The backbone network layer divides the input feature map into two parts, one part is processed by the sub-network, and the other part is directly processed by the processing layer within the backbone network.

[0119] In the sub-network, a convolutional layer is used to compress the input feature map, followed by a series of convolution operations, and finally a convolutional layer is used for expansion. This can extract relatively few high-level features.

[0120] In the processing layer of the backbone network, the feature map processed by the sub-network is concatenated with the directly processed feature map, and then a series of convolution operations are performed. This can combine low-level detail features with high-level abstract features to improve the efficiency of feature extraction.

[0121] The Neck network layer refers to the intermediate feature extraction network added on the basis of the backbone network. It is mainly used to enhance the feature expression ability and receptive field of the model, and further improve the detection performance of the model.

[0122] SPP: A pyramid pooling structure that can pool feature maps of different sizes, thereby enhancing the model's ability to perceive objects of different scales.

[0123] Specifically, the SPP structure performs 1x1, 2x2, and 3x3 pooling operations on the input feature map, and concatenates the pooling results of different scales as the output of the SPP structure.

[0124] PAN: A multi-scale feature fusion structure that can effectively fuse features at different levels and avoid information loss. Specifically, the PAN structure first aligns the feature map scales of the higher level with the feature map scales of the lower level through upsampling, then fuses them through the feature fusion module, and finally outputs the fused feature map.

[0125] The SPP and PAN structures can be used in conjunction with each other to improve the detection performance of the model. The SPP structure can enhance the perception and scale invariance of the model, while the PAN structure can enhance the fusion capability of multi-scale features.

[0126] The head output layer is used to identify the construction process based on the fused construction process images.

[0127] The head, that is, the output layer, is mainly a prediction box. Each prediction box consists of the following information:

[0128] Confidence score: Indicates the probability of whether there is a target in the box, with a value range of 0 to 1.

[0129] Class probabilities: Indicates the probability that the target in the box belongs to each category, which is generally a predefined number of categories.

[0130] Bounding box coordinates: Indicates the location and size of the target, usually represented by a rectangular box.

[0131] The output layer predicts the bounding box position and size of the target by using the anchor box, and uses the softmax function to calculate the category probability for the prediction result corresponding to each anchor box.

[0132] A multi-scale hole attention (MSDA) module is added to the Upsample module in the Neck part of the model to effectively capture multi-scale semantic information and reduce the redundancy of the self-attention mechanism. This module adopts a multi-head design, divides the channels of the feature map into n different heads, and performs sliding window dilated attention (SWDA) with different hole rates in different heads.

[0133] like Figure 3 As shown in the figure, first, the channels of the feature map are divided into different heads. Then, in the window around the central query block, self-attention operations are performed between different heads using different dilation rates. In addition, the features of different heads are concatenated and then input into the linear layer. By default, we use a 3x 3 convolution kernel with dilation rates of 1, 2, and 3, and the sizes of the attended receptive fields in different heads are 3x 3, 5x 5, and 7x 7. This design aims to better capture the contextual information in the input image through the multi-head dilated attention mechanism and improve the model's perception of features of different scales.

[0134] like Figure 3 As shown in the figure, the SWDA mentioned therein first captures information of different scales in the image by applying multiple parallel dilated convolution operations on the input tensor with different dilation rates. Then, by passing the linear projection of the query (Q), key (K), and value (V) into the dilated convolution attention mechanism, the attention weights are calculated separately for each dilation rate, thereby effectively capturing local features of different scales. Finally, the output of the multi-head dilated local attention is mapped to the target dimension through linear projection, and the robustness of the model is improved through the dropout operation. In this way, semantic information can be aggregated at various scales within the attended receptive field, and the redundancy of the self-attention mechanism can be effectively reduced without complex operations and additional computational costs.

[0135] In summary, the overall architecture of the network is as follows: Figure 4 As shown in the figure, the top shows the proposed multi-scale dilated attention (MSDA) block, including DwConv, multi-scale sliding window dilated attention operation (SWDA) and rL. The bottom shows the overlapping word segmenter, overlapping downsampler, multi-scale dilated attention (MSDA) block and multi-head self-attention (MHSA) block. Overall, by using this hybrid method of multi-scale empty attention and multi-head self-attention, the long-distance dependency problem is successfully handled while maintaining computational efficiency and being able to adapt to inputs of different scales and resolutions.

[0136] Embodiment 2

[0137] This embodiment provides a system for identifying construction procedures in a tunnel based on video information, including:

[0138] An image extraction module is configured to obtain a construction process video to extract a construction process image;

[0139] An image preprocessing module is configured to preprocess the construction process image;

[0140] A process identification module is configured to identify the construction process based on the preprocessed construction process image;

[0141] The construction process recognition based on the pre-processed construction process image includes:

[0142] Unify the size of the preprocessed construction process images to obtain standard construction process images;

[0143] Based on the standard construction process images, the multi-scale dilated attention block is used to extract the feature information of construction process images at different scales.

[0144] The feature information of the construction process images at different scales is fused to obtain a fused construction process image;

[0145] Construction process recognition is performed based on the fused construction process images.

[0146] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0147] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0149] Embodiment 3

[0150] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for identifying construction procedures in a tunnel based on video information as described in the first embodiment above are implemented.

[0151] Embodiment 4

[0152] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a method for identifying construction procedures in a tunnel based on video information as described in the first embodiment above are implemented.

[0153] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0157] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0158] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for identifying construction procedures in a tunnel based on video information, characterized in that: include: Obtain construction process videos to extract construction process images; Preprocess the construction process images; Based on the pre-processed construction process images, construction process recognition is performed; The construction process recognition based on the pre-processed construction process image includes: Unify the size of the preprocessed construction process images to obtain standard construction process images; Based on the standard construction process images, the multi-scale dilated attention block is used to extract the feature information of construction process images at different scales. The feature information of the construction process images at different scales is fused to obtain a fused construction process image; Construction process recognition is performed based on the fused construction process images.

2. A method for identifying construction procedures in a tunnel based on video information as claimed in claim 1, characterized in that: The obtaining of the construction process video to extract the construction process image is specifically as follows: Obtain the construction process video, and perform color space conversion on the construction process video frames; An average frame difference calculation is performed based on the converted construction process video frames to obtain an array of differences between construction process video frames; Smoothing the construction process video frame difference array to obtain a smoothed frame difference array; The index of the local maximum point in the smoothed inter-frame difference array is found to obtain the key frame of the construction process video and form a construction process image.

3. A method for identifying construction procedures in a tunnel based on video information as claimed in claim 1, characterized in that: The preprocessing of the construction process image includes: Performing data enhancement and expansion on the construction process image to obtain an expanded construction process image; The EnlightenGAN algorithm is used to perform image enhancement on the expanded construction process image to obtain the preprocessed construction process image.

4. A method for identifying construction procedures in a tunnel based on video information as claimed in claim 1, characterized in that: The pre-processed construction process images are resized uniformly to obtain standard construction process images, specifically: Perform image enhancement based on the preprocessed construction process images; Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for enhanced construction process images; Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size to obtain a standard construction process image.

5. The method for identifying construction procedures in a tunnel based on video information according to claim 1, characterized in that: Based on the standard construction process image, the multi-scale dilated attention block is used to extract the feature information of the construction process images of different scales, specifically: Feature extraction is performed based on the standard construction process image to obtain the initial convolution feature; Utilize multiple parallel dilated convolution operations with different dilation rates to capture information of different scales in the initial convolution features; By passing the linear projection of query, key, and value into the dilated convolutional attention mechanism, the attention weight is calculated for each dilation rate, and local features of different scales are extracted to obtain the output features of multi-head dilated local attention; The output features of the multi-head dilated local attention are mapped to the target dimension through linear projection to obtain the image feature information of construction processes at different scales.

6. A method for identifying construction procedures in a tunnel based on video information as claimed in claim 5, characterized in that: The feature information of construction process images at different scales is fused to obtain the fused construction process image, which is as follows: The initial convolution features and the feature information of the construction process images at different scales are fused to obtain the fused features; The fused features are normalized and then output through the fully connected layer to obtain the fused construction process image.

7. A method for identifying construction procedures in a tunnel based on video information as claimed in claim 5, characterized in that: The multi-scale dilated attention block includes depthwise separable convolution, multi-scale sliding window dilated attention operation and rL.

8. A tunnel construction process identification system based on video information, characterized in that: include: An image extraction module is configured to obtain a construction process video to extract a construction process image; An image preprocessing module is configured to preprocess the construction process image; A process identification module is configured to identify the construction process based on the preprocessed construction process image; The construction process recognition based on the pre-processed construction process image includes: Unify the size of the preprocessed construction process images to obtain standard construction process images; Based on the standard construction process images, the multi-scale dilated attention block is used to extract the feature information of construction process images at different scales. The feature information of the construction process images at different scales is fused to obtain a fused construction process image; Construction process recognition is performed based on the fused construction process images.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of a method for identifying construction procedures in a tunnel based on video information as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a method for identifying construction procedures in a tunnel based on video information as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Real-time traffic sign detection method based on multi-scale pixel feature fusion

    CN112183203A

  • Target recognition network model training method, electronic equipment and program product

    CN115439693A

  • Real-time segmentation method for display screen defects under multiple backgrounds based on improved Mask R-CNN

    CN115456975A

  • Tunnel crack detection method based on multi-scale feature fusion

    CN117218504A

  • Super-resolution reconstruction method based on multi-scale attention expansion

    CN117333357A

Cited By

  • Three-dimensional reconstruction method based on improved TGS network

    CN119741429A

  • Method and system for positioning constructors in tunnel based on machine vision

    CN120912674A