A tunnel construction process identification method and system based on video information

Abstract

The application belongs to the field of tunnel monitoring video analysis, and provides a tunnel construction process identification method and system based on video information, which comprises the following steps: acquiring a construction process video to extract a construction process image; preprocessing the construction process image; and identifying the construction process based on the preprocessed construction process image; wherein the construction process identification based on the preprocessed construction process image comprises the following steps: uniformly sizing the preprocessed construction process image to obtain a standard construction process image; extracting construction process image feature information of different scales based on the standard construction process image by using a multi-scale expansion attention block; fusing the construction process image feature information of different scales to obtain a fused construction process image; and identifying the construction process based on the fused construction process image.

Description

Technical Field

[0001] This invention belongs to the field of tunnel monitoring video analysis technology, specifically relating to a method and system for identifying construction procedures inside tunnels based on video information. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the current field of tunnel construction, construction management and monitoring are crucial aspects. However, traditional monitoring methods have limitations in terms of real-time performance and accuracy. With the continuous development of technology, tunnel construction process identification technology based on image processing and computer vision has become an effective means to improve construction management. In the past, monitoring of tunnel construction processes typically relied on manual inspections and recording, which suffered from high labor costs, untimely data collection, and susceptibility to subjective errors. Therefore, existing technologies are gradually shifting towards introducing automated process identification technologies, using computer algorithms and advanced sensor equipment to achieve real-time monitoring and accurate identification of tunnel construction processes. This not only improves the intelligence level of project management but also effectively reduces human error, providing reliable technical support for the efficient advancement of tunnel construction.

[0004] Therefore, developing an efficient and accurate technology for identifying construction procedures within tunnels has become a research hotspot in the field. Monitoring construction procedures on-site is a crucial aspect of tunnel construction and the foundation for achieving information-based and dynamic construction. On-site monitoring aims to grasp dynamic information within the tunnel, thereby evaluating the rationality of construction procedures and providing feedback to guide the design and construction of tunnel projects.

[0005] However, existing tunnel construction process identification technologies have not yet incorporated artificial intelligence, often relying on manual video remote monitoring, which suffers from disadvantages such as poor real-time performance, slow response, and strong human subjectivity. Deep learning-based detection algorithms are represented by one-stage and two-stage algorithms. Two-stage algorithms first generate target candidate boxes, then classify and regress their locations. Algorithms like the Faster R-CNN and Mask R-CNN series offer high target detection accuracy but are slow, computationally intensive, and time-complex, making them unsuitable for the real-time requirements of tunnel detection. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a method and system for identifying construction procedures within tunnels based on video information. This invention presents a construction procedure identification technology based on convolutional neural networks, combined with the EnlightenGAN algorithm, to achieve real-time monitoring of construction in complex scenarios such as large changes in lighting and low visibility within tunnels. This approach reduces model size, parameter count, and training costs while improving detection performance and effectively alleviating problems such as low efficiency in real-time monitoring within tunnels.

[0007] According to some embodiments, the first aspect of the present invention provides a method for identifying construction procedures inside a tunnel based on video information, employing the following technical solution: A method for identifying construction procedures inside a tunnel based on video information includes: Obtain construction process videos and extract construction process images; Preprocess the images of the construction process; Based on the preprocessed construction process images, construction process identification is performed. The construction process identification based on the preprocessed construction process images includes: Standard construction process images are obtained by unifying the dimensions of the pre-processed construction process images. Based on standard construction process images, feature information of construction process images at different scales is extracted using multi-scale expanded attention blocks; The feature information of construction process images at different scales is fused to obtain a fused construction process image; Construction process identification is performed based on the fused construction process images.

[0008] Furthermore, the step of obtaining construction process videos and extracting construction process images specifically involves: Acquire construction process videos and perform color space conversion on the video frames. The average frame difference is calculated based on the converted construction process video frames to obtain the inter-frame difference array of the construction process video. The inter-frame difference array of the construction process video is smoothed to obtain a smoothed inter-frame difference array. Find the indices of local maxima in the smoothed inter-frame difference array to obtain keyframes of the construction process video, thus forming the construction process image.

[0009] Furthermore, the preprocessing of the construction process images includes: Data augmentation and expansion are performed on the construction process images to obtain expanded construction process images; The EnlightenGAN algorithm is used to enhance the expanded construction process images, resulting in preprocessed construction process images.

[0010] Furthermore, the process of standardizing the size of the preprocessed construction process images to obtain standard construction process images specifically involves: Image enhancement based on preprocessed construction process images; Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for the enhanced construction process images; Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size, resulting in a standard construction process image.

[0011] Furthermore, the step of extracting feature information of construction process images at different scales based on standard construction process images using multi-scale expanded attention blocks specifically involves: Initial convolutional features are obtained by extracting features from standard construction process images; By utilizing multiple parallel dilated convolution operations, information at different scales in the initial convolutional features is captured at different dilation rates; By passing the linear projection of the query, key, and value into the dilated convolutional attention mechanism, and calculating the attention weight for each dilation rate, local features at different scales are extracted to obtain the output features of multi-head dilated local attention. By mapping the output features of multi-head dilatational local attention to the target dimension through linear projection, image feature information of construction procedures at different scales can be obtained.

[0012] Furthermore, the feature information of construction process images at different scales is fused to obtain a fused construction process image, specifically: The initial convolutional features and the feature information of construction process images at different scales are fused to obtain fused features; The fused features are normalized and then output through a fully connected layer to obtain the fused construction process image.

[0013] Furthermore, the multi-scale expanded attention block includes depthwise separable convolution and multi-scale sliding window expanded attention operations.

[0014] According to some embodiments, the second aspect of the present invention provides a tunnel construction procedure identification system based on video information, employing the following technical solution: A tunnel construction process identification system based on video information includes: The image extraction module is configured to acquire construction process videos and extract construction process images. The image preprocessing module is configured to preprocess images of construction procedures; The process identification module is configured to identify construction processes based on preprocessed construction process images. The construction process identification based on the preprocessed construction process images includes: Standard construction process images are obtained by unifying the dimensions of the pre-processed construction process images. Based on standard construction process images, feature information of construction process images at different scales is extracted using multi-scale expanded attention blocks; The feature information of construction process images at different scales is fused to obtain a fused construction process image; Construction process identification is performed based on the fused construction process images.

[0015] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0016] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for identifying construction procedures in a tunnel based on video information as described in the first aspect above.

[0017] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0018] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a method for identifying tunnel construction procedures based on video information as described in the first aspect above.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention replaces the traditional manual inspection method. The application of artificial intelligence makes tunnel monitoring more objective and accurate. Compared with traditional manual inspection, it can accurately and comprehensively capture dynamic behavior in the tunnel without being affected by subjective emotions. In the long run, automated intelligent monitoring can greatly reduce maintenance costs, reduce the waste of human resources, and save time and resources for tunnel managers. Intelligent process recognition can detect with high precision and high speed whether the construction personnel in the tunnel are operating properly and whether the process is efficiently connected, ensuring the rational use of construction materials, standardized operation of personnel, and timely delivery of the project.

[0020] This invention employs an adaptive high-precision module to perform multi-scale feature extraction and enhancement processing on the feature map, thereby improving detection efficiency. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1This is a flowchart of a method for identifying construction procedures inside a tunnel based on video information, as described in an embodiment of the present invention. Figure 2 This is a diagram of the feature extraction network structure in an embodiment of the present invention; Figure 3 This is a structural diagram of SWDA in an embodiment of the present invention; Figure 4 This is a diagram of the convolutional neural network structure in an embodiment of the present invention; Figure 5 This is a schematic diagram of the sample dataset in an embodiment of the present invention; Figure 6 This is a schematic diagram of the final recognition result in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] Example 1 like Figure 1As shown, this embodiment provides a method for identifying construction procedures within a tunnel based on video information. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to a terminal, or to a system including a terminal, server, and system, and is implemented through interaction between the terminal and server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps: Step S1: Obtain construction process videos and extract construction process images; Step S2: Preprocess the construction process images; Step S3: Based on the preprocessed construction process images, perform construction process identification.

[0028] For step S1, the construction process video is acquired and the construction process images are extracted, specifically as follows: Step S1.1: Acquire construction process videos and perform color space conversion on the construction process video frames; Step S1.2: Calculate the average frame difference based on the converted construction process video frames to obtain the inter-frame difference array of the construction process video. Step S1.3: Smooth the inter-frame difference array of the construction process video to obtain the smoothed inter-frame difference array; Step S1.4: Find the index of the local maximum point in the smoothed inter-frame difference array to obtain the key frame of the construction process video and form the construction process image.

[0029] Video analysis is usually based on video frames. However, a video typically contains 24 frames per second. If key frames can be captured, especially in tunnel environments or scenes where similar shots are taken for a long time, deduplication of the images can help filter out most of the "noisy" frames and extract the core content of the video to the greatest extent.

[0030] Calculation of average inter-frame difference: In step S1.1, the read video is converted to a color space, converting these frames to the LUV color space. The LUV color space represents luminance (L) and chrominance (UV), respectively, to more effectively capture the grayscale information of the image. Next, difference calculation is performed on the converted frames, using absolute difference to measure the difference between adjacent frames.

[0031] In step S1.2, the pixel values ​​of the entire difference image are summed to obtain the overall inter-frame difference. Finally, the sum of the inter-frame differences is divided by the number of pixels in the image to obtain the average inter-frame difference, which is then stored in the corresponding array.

[0032] Smoothing: In step S1.3, the "smooth" function is used to smooth the inter-frame difference array to reduce noise, making subsequent local maximum detection more stable. First, the original inter-frame difference array is expanded at the boundaries to handle boundary conditions and ensure correct calculation of the convolution operation. This function also allows users to flexibly adjust the smoothing method according to specific needs and signal characteristics to achieve better smoothing results. Next, the data is averaged using weights from a custom window through a convolution operation to reduce noise. Then, np.convolve is used to perform a convolution operation on the inter-frame difference array to finally smooth the data. Here, a normalized version of the convolution kernel w is used to ensure that the weight sum is 1.

[0033] In step S1.4, the indexes of local maxima points in the smoothed inter-frame difference array are found using the function argrelextrema from the SciPy library. These indices correspond to frames with large differences in the video, i.e. keyframes. The indices of local maxima points are traversed, and the index of the corresponding frame is used as the keyframe and saved as an image file.

[0034] In step S2, the preprocessing of the construction process images includes: Step S2.1: Perform data augmentation and expansion on the construction process images to obtain expanded construction process images; Step S2.2: Use the EnlightenGAN algorithm to perform image enhancement on the expanded construction process image to obtain the preprocessed construction process image.

[0035] It's worth noting that EnlightenGAN is the first efficient unsupervised generative adversarial network to successfully introduce unpaired training into low-light image enhancement, eliminating the dependence on paired training data and allowing training with a wider range of scene images from different domains. Its architecture employs a global-local discriminator structure, ensuring that all local regions of the enhanced image appear like real natural light. Using a U-Net network as the backbone of the generator, an attention mechanism is incorporated to enhance dark areas more than bright areas. Therefore, compared to other state-of-the-art low-light image enhancement methods, EnlightenGAN not only successfully learns to enhance dark areas, preserve texture details, and avoid overexposure artifacts, but also easily adapts to realistic low-noise, low-light images, producing visually appealing enhanced images. Through image preprocessing, EnlightenGAN removes irrelevant information, recovers useful real-world information, enhances the detectability of relevant information, and simplifies the data to the maximum extent, thereby improving the reliability of feature extraction and object recognition.

[0036] In step S3, the construction process identification based on the preprocessed construction process image includes: Step 3.1: Standardize the dimensions of the pre-processed construction process images to obtain standard construction process images; Step 3.2: Based on standard construction process images, extract feature information of construction process images at different scales using multi-scale expanded attention blocks; Step 3.3: Fuse the feature information of construction process images at different scales to obtain the fused construction process image; Step 3.4: Recognize construction procedures based on the fused construction procedure images.

[0037] In step 3.1, the process of standardizing the size of the preprocessed construction process images to obtain standard construction process images specifically involves: Image enhancement based on preprocessed construction process images; Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for the enhanced construction process images; Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size, resulting in a standard construction process image.

[0038] In step 3.2, the step of extracting feature information of construction process images at different scales based on standard construction process images using multi-scale expanded attention blocks specifically involves: Initial convolutional features are obtained by extracting features from standard construction process images; By utilizing multiple parallel dilated convolution operations, information at different scales in the initial convolutional features is captured at different dilation rates; By passing the linear projection of the query, key, and value into the dilated convolutional attention mechanism, and calculating the attention weight for each dilation rate, local features at different scales are extracted to obtain the output features of multi-head dilated local attention. By mapping the output features of multi-head dilatational local attention to the target dimension through linear projection, image feature information of construction procedures at different scales can be obtained.

[0039] In step 3.3, the feature information of construction process images at different scales is fused to obtain a fused construction process image, specifically as follows: The initial convolutional features and the feature information of construction process images at different scales are fused to obtain fused features; The fused features are normalized and then output through a fully connected layer to obtain the fused construction process image.

[0040] In a specific embodiment, this implementation proposes a method for identifying construction procedures in tunnels based on video information. By combining the EnlightenGAN algorithm, it enables real-time monitoring of construction in complex scenarios such as large changes in lighting and low visibility in tunnels. This method improves detection performance and effectively alleviates problems such as low efficiency in real-time monitoring in tunnels while reducing model size, number of parameters, and training costs.

[0041] The training phase of this method specifically includes: (1) Obtain the data set of construction procedures in the tunnel, 519 pictures of the tunnel construction site, and label the object detection boxes and categories of 9 types of equipment.

[0042] (2) Use the roLabelImg tool to re-label some images.

[0043] (3) Perform data augmentation on the image data, expand the dataset, and then divide it into training set and test set.

[0044] (4) The EnlightenGAN algorithm is used to enhance the images of the training set.

[0045] (5) Build a process identification model.

[0046] (6) Use the improved model to train and test the dataset, and finally achieve intelligent recognition of the process, such as Figure 6 As shown.

[0047] Dataset processing: This embodiment uses images captured at a tunnel site, with some images re-annotated using the roLabelImg tool. The re-annotation process involves saving the annotated images with the roLabelImg tool as a .xml file extension, then converting it to a .txt file extension. The annotated image data is then augmented and expanded to obtain the dataset, which is then divided into training and testing sets. The dataset is divided into training and testing sets in a 4:1 ratio, ultimately resulting in 8000 images for the training set and 2000 images for the testing set. Figure 5 As shown, the EnlightenGAN algorithm is used to enhance images on the training set. EnlightenGAN employs a global-local discriminator structure, uses a U-Net network as the backbone network of the generator, and incorporates an attention mechanism.

[0048] Model building and training This embodiment constructs a convolutional neural network (CNN) model, namely a process recognition model, to achieve intelligent recognition and classification of construction processes within tunnels. The network model mainly consists of four parts: an input layer, a backbone layer, a neck layer, and a head layer. The input layer defines the original input image size as 640x640. The backbone layer performs convolutional operations to extract feature information at non-general scales. The neck layer is responsible for fusing image feature information, and the head layer outputs the predicted anchor box coordinates, category, and confidence level after non-maximum suppression processing.

[0049] The input layer mainly consists of Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling. Mosaic image enhancement increases the diversity and difficulty of the training set, helping to improve the robustness and generalization ability of the object detection model. Simultaneously, Mosaic image enhancement can reduce the risk of overfitting and improve the model's training effect. The idea behind adaptive anchor boxes is to automatically calculate the anchor box parameters best suited to the input image through learning, without requiring manual setting. This method can improve the accuracy and robustness of object detection. Adaptive image scaling is an image scaling method based on the target scale. It can adaptively scale the size of the input image to adapt to the detection of targets at different scales. This method can effectively solve the scale inconsistency problem in object detection, improving detection accuracy and robustness.

[0050] like Figure 2As shown, this network uses the lightweight MobileOne network as its backbone. Based on network structure analysis, latency bottlenecks were identified, with the ReLU activation function exhibiting the lowest latency; therefore, only the ReLU activation function is used in MobileOne. The key factors affecting latency in the block structure are memory access cost and model parallelism. Fewer branches in the network can avoid this memory bottleneck; therefore, a branchless inference structure is adopted to reduce memory access cost. Thus, the SE block is only used in the largest variant of MobileOne to improve accuracy. The MobileOne block structure is decomposed into depthwise and pointwise layers, and an over-parameterization branch is introduced. The basic block structure is based on the MobileNet-V1 design, with a 3x3 depthwise convolution followed by a 1x1 pointwise convolution, and then a reparameterizable skip-connection branch is added, specifically including a batchnorm branch and a branch that can be replicated multiple times.

[0051] The backbone network layer divides the input feature map into two parts: one part is processed by the sub-network, and the other part is directly processed by the processing layer within the backbone network.

[0052] In the sub-network, a convolutional layer is first used to compress the input feature map, followed by a series of convolutional operations, and finally a convolutional layer is used to expand it. This allows for the extraction of relatively few high-level features.

[0053] Within the backbone network's processing layers, the feature maps processed by the sub-networks are concatenated with the directly processed feature maps, followed by a series of convolutional operations. This combines low-level detailed features with high-level abstract features, improving the efficiency of feature extraction.

[0054] The Neck layer is an intermediate feature extraction network added on top of the backbone network. It is mainly used to enhance the model's feature representation ability and receptive field, thereby further improving the model's detection performance.

[0055] SPP: A pyramid pooling structure that can pool feature maps of different sizes, thereby enhancing the model's ability to perceive targets at different scales.

[0056] Specifically, the SPP structure performs 1x1, 2x2, and 3x3 pooling operations on the input feature map, and concatenates the pooling results at different scales as the output of the SPP structure.

[0057] PAN: A multi-scale feature fusion structure that can effectively fuse features at different levels and avoid information loss. Specifically, the PAN structure first aligns the scale of higher-level feature maps with that of lower-level feature maps through upsampling, then fuses them through a feature fusion module, and finally outputs the fused feature map.

[0058] SPP and PAN structures can be used together to improve the detection performance of the model. The SPP structure can enhance the model's perceptual ability and scale invariance, while the PAN structure can enhance the fusion ability of multi-scale features.

[0059] Construction process identification is performed based on the fused construction process images using the head output layer.

[0060] The header, or output layer, mainly consists of prediction boxes. Each prediction box is composed of the following information: Confidence score: indicates the probability that a target exists within the box, with a value ranging from 0 to 1.

[0061] Class probabilities: These represent the probability that an object within the bounding box belongs to each of the various classes. They are typically the number of classes defined in advance.

[0062] Bounding box coordinates: Represent the position and size of the target, usually represented by a rectangle.

[0063] The output layer uses anchor boxes to predict the location and size of the bounding box of the target, and uses the softmax function to calculate the class probability for the prediction result corresponding to each anchor box.

[0064] In the Neck part of the model, a multi-scale dilated attention (MSDA) module is added to the Upsample module to effectively capture multi-scale semantic information and reduce the redundancy of the self-attention mechanism. This module adopts a multi-head design, dividing the channels of the feature map into n different heads, and performing sliding window dilated attention (SWDA) with different dilation rates in different heads.

[0065] like Figure 3As shown, firstly, the channels of the feature map are divided into different heads. Then, within a window surrounding the central query block, self-attention operations are performed between the different heads using different dilation rates. Furthermore, the features from the different heads are concatenated and then fed into a linear layer. By default, we use 3x3 convolutional kernels with dilation rates of 1, 2, and 3, and the sizes of the receptive fields of attention in the different heads are 3x3, 5x5, and 7x7. This design aims to better capture contextual information in the input image through a multi-head dilation attention mechanism, improving the model's ability to perceive features at different scales.

[0066] like Figure 3 As shown, the SWDA mentioned first captures information at different scales in the image by applying multiple parallel dilated convolution operations to the input tensor at different dilation rates. Next, by feeding the linear projection of the query (Q), key (K), and value (V) into the dilated convolutional attention mechanism, attention weights are calculated for each dilation rate, thus achieving effective capture of local features at different scales. Finally, the output of the multi-head dilated local attention is mapped to the target dimension through linear projection, and the robustness of the model is improved through dropout. This allows semantic information to be aggregated at various scales within the receptive field of interest and effectively reduces the redundancy of the self-attention mechanism without complex operations and additional computational costs.

[0067] In summary, the overall architecture of this network is as follows: Figure 4 As shown, the top section displays the proposed Multi-Scale Expanded Attention (MSDA) block, including DwConv and the Multi-Scale Sliding Window Expanded Attention (SWDA) operation. The bottom section displays the overlapping tokenizer, overlapping downsampling unit, Multi-Scale Expanded Attention (MSDA) block, and Multi-Head Self-Attention (MHSA) block. Overall, this hybrid approach of using multi-scale dilated attention and multi-head self-attention successfully addresses long-range dependency problems while maintaining computational efficiency and adapting to inputs of different scales and resolutions.

[0068] Example 2 This embodiment provides a tunnel construction process identification system based on video information, including: The image extraction module is configured to acquire construction process videos and extract construction process images. The image preprocessing module is configured to preprocess images of construction procedures; The process identification module is configured to identify construction processes based on preprocessed construction process images. The construction process identification based on the preprocessed construction process images includes: Standard construction process images are obtained by unifying the dimensions of the pre-processed construction process images. Based on standard construction process images, feature information of construction process images at different scales is extracted using multi-scale expanded attention blocks; The feature information of construction process images at different scales is fused to obtain a fused construction process image; Construction process identification is performed based on the fused construction process images.

[0069] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0070] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0071] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0072] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the tunnel construction process identification method based on video information as described in Embodiment 1 above.

[0073] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the tunnel construction process identification method based on video information as described in Embodiment 1 above.

[0074] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0079] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for identifying construction procedures within a tunnel based on video information, characterized in that, include: Obtain construction process videos and extract construction process images; Preprocess the images of the construction process; Based on the preprocessed construction process images, construction process identification is performed. The step of identifying construction procedures based on preprocessed construction procedure images includes: Standard construction process images are obtained by unifying the dimensions of the pre-processed construction process images. Based on standard construction process images, multi-scale expanded attention blocks are used to extract feature information of construction process images at different scales, specifically: Initial convolutional features are obtained by extracting features from standard construction process images; By utilizing multiple parallel dilated convolution operations, information at different scales in the initial convolutional features is captured at different dilation rates; By passing the linear projection of the query, key, and value into the dilated convolutional attention mechanism, and calculating the attention weight for each dilation rate, local features at different scales are extracted to obtain the output features of multi-head dilated local attention. By mapping the output features of multi-head dilatational local attention to the target dimension through linear projection, we can obtain image feature information of construction procedures at different scales. The feature information of construction process images at different scales is fused to obtain a fused construction process image; Construction process identification is performed based on the fused construction process images.

2. The method for identifying tunnel construction procedures based on video information as described in claim 1, characterized in that, The process of obtaining construction process videos and extracting construction process images specifically involves: Acquire construction process videos and perform color space conversion on the video frames. The average frame difference is calculated based on the converted construction process video frames to obtain the inter-frame difference array of the construction process video. The inter-frame difference array of the construction process video is smoothed to obtain a smoothed inter-frame difference array. Find the indices of local maxima in the smoothed inter-frame difference array to obtain keyframes of the construction process video, thus forming the construction process image.

3. The method for identifying tunnel construction procedures based on video information as described in claim 1, characterized in that, The preprocessing of construction process images includes: Data augmentation and expansion are performed on the construction process images to obtain expanded construction process images; The EnlightenGAN algorithm is used to enhance the expanded construction process images, resulting in preprocessed construction process images.

4. The method for identifying tunnel construction procedures based on video information as described in claim 1, characterized in that, The process of unifying the dimensions of the preprocessed construction process images to obtain standard construction process images is as follows: Image enhancement based on preprocessed construction process images; Adaptive anchor frames are used to automatically calculate anchor frame parameters suitable for the enhanced construction process images; Based on the anchor frame parameters of the enhanced construction process image, adaptive image scaling is used to scale the image to a uniform size, resulting in a standard construction process image.

5. The method for identifying construction procedures inside a tunnel based on video information as described in claim 1, characterized in that, The feature information of construction process images at different scales is fused to obtain a fused construction process image, specifically: The initial convolutional features and the feature information of construction process images at different scales are fused to obtain fused features; The fused features are normalized and then output through a fully connected layer to obtain the fused construction process image.

6. The method for identifying tunnel construction procedures based on video information as described in claim 1, characterized in that, The multi-scale expanded attention block includes depthwise separable convolution and multi-scale sliding window expanded attention operations.

7. A tunnel construction procedure recognition system based on video information, characterized in that, The method for identifying tunnel construction procedures based on video information as described in any one of claims 1-6 includes: The image extraction module is configured to acquire construction process videos and extract construction process images. The image preprocessing module is configured to preprocess images of construction procedures; The process identification module is configured to identify construction processes based on preprocessed construction process images. The construction process identification based on the preprocessed construction process images includes: Standard construction process images are obtained by unifying the dimensions of the pre-processed construction process images. Based on standard construction process images, feature information of construction process images at different scales is extracted using multi-scale expanded attention blocks; The feature information of construction process images at different scales is fused to obtain a fused construction process image; Construction process identification is performed based on the fused construction process images.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the tunnel construction process identification method based on video information as described in any one of claims 1-6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the tunnel construction process identification method based on video information as described in any one of claims 1-6.

Images