Machine Vision-Based Method and System for Locating Construction Personnel Inside Tunnels

Abstract

This invention discloses a machine vision-based method and system for locating construction workers inside tunnels, belonging to the field of tunnel monitoring video analysis technology. The method includes: acquiring tunnel monitoring video of construction workers during their work; calculating and smoothing the average inter-frame difference of the video, extracting several key frames, and performing frame image enhancement processing; inputting the enhanced key frames into an improved target detection model, extracting multi-scale feature maps through a feature extraction layer, fusing features through a feature fusion layer, and outputting the identification and location results of the construction workers inside the tunnel through an output layer. The feature extraction layer includes multiple sequentially arranged feature extraction stages, each stage containing Shuffle Net units for downsampling and feature deepening, respectively. The feature fusion layer includes an attention module for guiding the generation of a saliency map and weighted feature fusion. This invention can effectively improve detection performance and location efficiency.

Description

Technical Field

[0001] This invention belongs to the field of video analysis technology for tunnel monitoring, and particularly relates to a method and system for locating construction personnel in tunnels based on machine vision. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the continuous development of technology, artificial intelligence has become an indispensable part of many fields. AI-based personnel positioning technology has great application potential in on-site construction operations. This technology can monitor the location and movement trajectory of workers in real time, provide early warnings of potential risks, prevent accidents, and improve personnel safety. Furthermore, by analyzing personnel movement trajectories and work habits, work processes can be further optimized, improving production efficiency. In addition, in emergency situations, quickly locating personnel can provide crucial information for rescue operations, offering significant assistance for rapid rescue efforts. Currently, AI-based personnel positioning mainly combines machine learning and big data analytics. It collects large amounts of personnel positioning data and uses this data to train machine learning models. For workers in real-world scenarios, the model analyzes and predicts based on its learned knowledge, identifying and accurately locating workers.

[0004] However, existing positioning schemes are difficult to apply directly and effectively to the positioning of construction workers in tunnels. The main reasons are: (1) Visual monitoring in tunnels has problems such as unstructured, weak light, occlusion, and complex background. As a result, the images obtained in low-light tunnel scenes are unevenly lit, blurry, have low contrast, and poor visibility. This will make it difficult for network models to extract features, increase the learning difficulty, and decrease the accuracy. This makes the existing neural network-based visual recognition methods perform poorly in this scenario; (2) In existing visual recognition models, the feature information extraction is not fully integrated, which affects the final recognition and positioning effect; (3) Construction operations in tunnels are tedious and complex. Construction workers often work in the same position for a long time, resulting in a large amount of visual monitoring data in tunnels with a lot of redundant information. This data characteristic will affect the training and positioning recognition efficiency of the model. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a machine vision-based method and system for locating construction workers in tunnels. By extracting key video frames from tunnel monitoring videos, enhancing these frames, and utilizing an improved lightweight target detection model, the method comprehensively aggregates low-level detail features and high-level semantic features of the key video frames. This enables accurate location of construction workers in complex construction scenarios such as large changes in lighting and low visibility within tunnels. It effectively improves detection performance and location efficiency while reducing model size and parameter count and lowering training costs.

[0006] In a first aspect, the present invention provides a method for locating construction workers inside tunnels based on machine vision.

[0007] A machine vision-based method for locating construction workers inside tunnels, comprising:

[0008] Obtain tunnel monitoring videos of construction workers during the construction process inside the tunnel;

[0009] The average inter-frame difference of the tunnel monitoring video is calculated and smoothed, several key frames are extracted, and frame image enhancement processing is performed.

[0010] Several enhanced keyframes are input into the improved target detection model. Multi-scale feature maps are extracted by the model's feature extraction layer, and then the features are fused by the feature fusion layer. Finally, the output layer outputs the identification and location results of the construction personnel inside the tunnel.

[0011] The feature extraction layer includes multiple feature extraction stages arranged sequentially. Each stage includes a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature enhancement. The feature fusion layer includes a content-guided attention module for guiding the generation of a saliency map and weighted fusion of features.

[0012] A further technical solution, the extraction of the keyframes, includes:

[0013] The acquired tunnel monitoring video is converted to a color space, and all video frames are converted to the LUV color space.

[0014] The inter-frame difference is calculated on the converted video frames to obtain a difference image. The pixel values ​​of the difference image are then summed to obtain the total inter-frame difference. The total inter-frame difference is divided by the number of pixels in the video frame to obtain the average inter-frame difference, forming an inter-frame difference array.

[0015] The inter-frame difference array is smoothed, and the indices of local maxima in the smoothed inter-frame difference array are traversed. The video frames corresponding to the indices are used as keyframes.

[0016] A further technical solution involves using the Enlighten GAN algorithm to perform image enhancement processing on each extracted keyframe.

[0017] In a further technical solution, in each stage of the feature extraction layer, the features of the original input image are first extracted by the ShuffleNet unit for downsampling, and then the features are deepened by stacked multiple ShuffleNet units for feature deepening in order to extract richer features.

[0018] The Shuffle Net unit used for downsampling includes two branches. The input image is processed by point-by-point grouping convolution, channel rearrangement, depthwise separable convolution and point-by-point grouping convolution to extract features in the main branch. At the same time, it is pooled by the average pooling layer set in the shortcut branch. The outputs of the two branches are then concatenated to output the downsampled features.

[0019] In the Shuffle Net unit used for feature enhancement, the input features are sequentially enhanced by pointwise grouped convolution, channel rearrangement, depthwise separable convolution, and pointwise grouped convolution, and then added to the original input features to output the enhanced features.

[0020] A further technical solution involves the input original image undergoing multiple feature extraction stages sequentially within the feature extraction layer to extract features at different scales. These multi-scale features are then input into a feature fusion layer for feature fusion, which results in the following feature fusion:

[0021] Multi-scale features include several deep features and shallow features. The deep features are upsampled to the same spatial resolution as the lowest shallow feature, and then all features are stitched together to obtain preliminary fused features.

[0022] The initial fusion features are input into the content-guided attention module to generate a channel-specific saliency map. The initial fusion features are then weighted using the saliency map to obtain the enhanced fusion features.

[0023] In a further technical solution, in the content-guided attention module, the input features are first subjected to global average pooling and global max pooling operations respectively, and the channel description vector and salient activation information of the input features are extracted respectively to generate two types of attention weights: channel weights and spatial weights. Among them, the channel weights focus on the overall importance of each channel, while the spatial weights reflect the important regions of spatial location in each channel.

[0024] Channel weights and spatial weights are combined through element-wise multiplication to form the final channel-space joint saliency map;

[0025] The original input features are weighted and enhanced using a channel-space joint saliency map, and then channel rearrangement is performed to finally output the enhanced fused features.

[0026] Secondly, the present invention provides a machine vision-based positioning system for construction workers inside tunnels.

[0027] A machine vision-based tunnel construction worker positioning system includes:

[0028] The video data acquisition module is used to acquire tunnel monitoring videos of construction workers during the construction process inside the tunnel.

[0029] The data preprocessing module is used to calculate and smooth the average inter-frame difference of the tunnel monitoring video, extract several key frames, and perform frame image enhancement processing.

[0030] The personnel identification and localization module is used to input several enhanced key frames into the improved target detection model. The model's feature extraction layer extracts multi-scale feature maps, and after feature fusion through the feature fusion layer, the output layer outputs the identification and localization results of construction personnel in the tunnel.

[0031] The feature extraction layer includes multiple feature extraction stages arranged sequentially. Each stage includes a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature enhancement. The feature fusion layer includes a content-guided attention module for guiding the generation of a saliency map and weighted fusion of features.

[0032] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the above-described machine vision-based method for locating construction workers in tunnels.

[0033] Fourthly, the present invention also provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-described machine vision-based method for locating construction workers in tunnels.

[0034] Fifthly, the present invention also provides a computer program product comprising executable instructions stored in a computer-readable storage medium; wherein, when the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned machine vision-based method for locating construction workers in tunnels is implemented.

[0035] The above one or more technical solutions have the following beneficial effects:

[0036] 1. This invention proposes a machine vision-based method and system for locating construction workers in tunnels. By extracting key video frames from tunnel monitoring videos and enhancing these frames, and utilizing an improved lightweight target detection model, the method comprehensively aggregates low-level detail features and high-level semantic features of the key video frames. This enables accurate location of construction workers in complex construction scenarios such as large changes in lighting and low visibility in tunnels. It can effectively improve detection performance and location efficiency while reducing model size and parameter quantity and lowering training costs.

[0037] 2. To address the issues of large size, numerous parameters, and difficult deployment associated with traditional CNN models, this invention employs a lightweight Shuffle Net as the backbone network. Through grouped convolution and channel rearrangement mechanisms, it significantly reduces model parameters, improves computational efficiency, makes it suitable for edge computing devices, lowers deployment difficulty, and adapts to construction site deployments. Furthermore, to address the problems of weak semantics in low-level features, low spatial resolution in high-level features, and insufficient information fusion, a Content-Guided Attention (CGA) module is introduced in the Neck layer. This module combines GAP and GMP to obtain a global description and generates a specific spatial saliency map for each channel for weighting. This enhances the effective information aggregation capability when fusing low-level detail features and high-level semantic features, highlights the target area of ​​construction workers, and improves the accuracy of final identification and localization.

[0038] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0040] Figure 1 This is an overall flowchart of the machine vision-based method for locating construction workers in tunnels, as described in an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the structure of the Shuffle Net unit in an embodiment of the present invention; wherein, (a) is the original residual structure, (b) is the Shuffle Net unit structure for feature enhancement, and (c) is the Shuffle Net unit structure for downsampling;

[0042] Figure 3 This is a schematic diagram of grouped convolution in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the point-by-point grouping convolution and channel rearrangement in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of the content-guided attention module in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the image after GridMask enhancement processing in an embodiment of the present invention. Detailed Implementation

[0046] It should be noted that the following detailed descriptions are exemplary and are intended only to describe specific embodiments and to provide further explanation of the invention, and are not intended to limit the scope of exemplary embodiments of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] Example 1

[0048] This embodiment provides a machine vision-based method for locating construction workers inside tunnels, such as... Figure 1 As shown, the specific steps include:

[0049] Step S1: Obtain tunnel monitoring video of the construction process by the construction personnel inside the tunnel.

[0050] Step S2: Calculate and smooth the average inter-frame difference of the tunnel monitoring video, extract several key frames, and perform frame image enhancement processing.

[0051] Specifically, in the tunnel monitoring video obtained above, 1 second of video contains 24 frames (i.e., video frame images, or simply video frames). Considering that construction workers in the tunnel often work in the same position for a long time and similar shots are taken for a long time, the amount of data in the tunnel monitoring video is too large and contains a lot of redundant information. If the key frames in the video are directly "captured", especially in the tunnel environment, a large number of video frames can be effectively deduplicated, which helps to filter out the vast majority of "noise" frames and extract the core content of the video to the greatest extent. This can ensure recognition accuracy while facilitating subsequent model processing and model training.

[0052] In this embodiment, step S2.1 involves calculating and smoothing the average inter-frame difference of all video frames in the acquired tunnel monitoring video, and extracting several key frames, including:

[0053] Step S2.1.1: Perform color space conversion on the acquired tunnel monitoring video, converting all video frames to the LUV color space. The LUV color space represents luminance (L) and chrominance (UV), which can more effectively capture the grayscale information of the image.

[0054] Step S2.1.2: Calculate the inter-frame difference for the converted video frames. For example, use absolute difference to measure the difference between adjacent frames to obtain a difference image. Then sum the pixel values ​​of the difference image to obtain the sum of the inter-frame differences. Divide the sum of the inter-frame differences by the number of pixels in the video frame to obtain the average inter-frame difference. Form an inter-frame difference array and store it in the corresponding array.

[0055] Step S2.1.3: Smooth the inter-frame difference array to reduce noise, so that subsequent local maximum detection is more stable. Then, traverse the indices of local maximum points in the smoothed inter-frame difference array and take the video frame corresponding to the index as the keyframe.

[0056] Specifically, the smoothing process using the smooth algorithm described above is as follows: First, the original inter-frame difference array is expanded to handle boundary conditions and ensure correct calculation of the convolution operation. Preferably, this function also allows users to flexibly adjust the smoothing method according to specific needs and signal characteristics to achieve better smoothing results. Next, the data is averaged using weights from a custom window through convolution operations to reduce noise. Then, the `np.convolve` function is used to perform a convolution operation on the inter-frame difference array to finally smooth the data. In this process, a normalized version of the convolution kernel `w` is used to ensure that the weight sum is 1. Finally, the `argrelextrema` function from the SciPy library is used to find the indices of local maxima in the smoothed inter-frame difference array. These indices correspond to frames with large differences in the video, i.e., keyframes. The indices of the local maxima are traversed, and the frames corresponding to the indices are used as keyframes and saved as image files.

[0057] Step S2.2: The Enlighten GAN algorithm is used to enhance the image of each extracted keyframe to improve the low-light image and solve the problem that insufficient lighting in the tunnel affects the accuracy of image feature extraction and recognition.

[0058] Step S3: Input several enhanced keyframes into the improved target detection model. Multi-scale feature maps are extracted by the model's feature extraction layer, and after feature fusion by the feature fusion layer, the identification and positioning results of construction personnel in the tunnel are output by the output layer.

[0059] In this embodiment, an improved target detection model based on a convolutional neural network (CNN) is constructed to identify and locate construction workers inside a tunnel. Specifically, the network model mainly consists of four parts: an input layer, a backbone network layer (i.e., a feature extraction layer), a neck layer (i.e., a feature fusion layer), and a head layer (i.e., an output layer). The input layer defines the original image size as 640×640. The backbone network layer performs convolution operations to extract feature information at different scales, i.e., extracting multi-scale feature maps. The neck layer is responsible for fusing image features, and the head layer is used to predict anchor box coordinates, category, and confidence level. Based on the prediction results, non-maximum suppression is performed, and finally, the final identification and localization results are output.

[0060] Specifically, for the feature extraction layer, this embodiment uses lightweight Shuffle Net units to construct the feature extraction network, such as... Figure 2 As shown, (a) is the original residual structure. To further enrich the content of feature extraction, this embodiment adopts the Shuffle Net unit structure shown in (b) and (c). Specifically, the feature extraction layer in this embodiment is designed to include multiple feature extraction stages set in sequence. Each stage has a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature deepening. The input of each Shuffle Net Unit is the feature map output by the previous Unit or the previous network layer (such as the initial convolutional layer of the original image).

[0061] Furthermore, (b) is the Shuffle Net unit used for feature enhancement. Compared to the original residual structure, this unit uses pointwise grouped convolution instead of pointwise convolution, and sets a channel shuffle operation afterward. Then, it passes through the original 3x3 DWConv, and outputs the final pointwise grouped convolution. The output features are then added to the original input features to output the extracted features. Unit (b) enhances semantic representation by repeating feature extraction on the same spatial scale, while maintaining the resolution of the extracted feature map. (c) is the Shuffle Net unit used for downsampling. Compared to unit (b), it uses a 3×3 average pooling operation with a stride of 2 in the shortcut branch, a stride of 2 in DWConv in the main branch, and Concat instead of add for feature fusion. This halves the input image size, reduces spatial resolution, increases the receptive field, and doubles the number of channels.

[0062] That is, each stage includes one (c) type downsampling unit and multiple stacked (b) type size-preserving units. This allows for downsampling and feature enhancement of the feature map at each stage, while maintaining the feature map scale within each stage, with only the number of channels changing. When transitioning from one stage to the next, for example, through convolution or pooling with a stride of 2, the spatial scale of the feature map is halved while the number of channels increases, thus extracting feature information at different scales. The entire backbone network extracts feature maps with different spatial resolutions and channel numbers progressively through multi-level concatenation. The extracted multi-scale feature maps are then output to the subsequent neck network for feature fusion.

[0063] Based on the above design, in each stage of the feature extraction layer, features of the original input image are first extracted through Shuffle Net units for downsampling, and then the features are further refined through stacked Shuffle Net units for feature deepening to extract richer features, specifically:

[0064] In the Shuffle Net unit used for downsampling, the size of its input feature map is... This unit comprises two branches. The input image is sequentially processed by pointwise grouped convolution, channel rearrangement, depthwise separable convolution, and pointwise grouped convolution to extract features in the main branch. Simultaneously, it is pooled by an average pooling layer in the shortcut branch. The outputs of the two branches are then concatenated to output a downsampled feature map. The size of this downsampled feature map is [size missing]. ,generally yes Twice as much;

[0065] In the Shuffle Net unit used for feature enhancement, the size of its input feature map is... The input features of this unit are sequentially processed through pointwise grouped convolution, channel rearrangement, depthwise separable convolution, and pointwise grouped convolution to deepen the features. These features are then added to the original input features to output the deepened features. The size of this deepened feature map is [size missing]. ,generally equal .

[0066] For example, the network model takes a 640×640 image as input, passes through a (c) type unit and a series of (b) type units, and gradually obtains feature maps of 320×320, 160×160, 80×80, 40×40, or even smaller scales. At the same time, the number of channels increases continuously, thereby extracting multi-scale feature maps.

[0067] Furthermore, pointwise group convolution and channel shuffle are fundamental operations in the Shuffle Net unit during the above process. Group convolution is as follows: Figure 3 As shown, it divides the original feature map into multiple groups and then performs convolution on each group separately. The number of parameters and computational cost of group convolution are both less than those of ordinary convolution. Based on group convolution, it is combined with pointwise convolution to form pointwise group convolution, which means a group convolution with a kernel size of 1×1. After pointwise group convolution, feature channels are rearranged, and the process is as follows. Figure 4 As shown.

[0068] Furthermore, for the feature fusion layer, a content-guided attention module is included to guide the generation of saliency maps and weighted fusion of features. Specifically, a content-guided attention (CGA) module is added to the upsampling module of the neck network in the model, and a coarse-to-fine approach is introduced to generate channel-specific saliency maps (SIMs). The CGA module guides the generation of SIMs by input features, assigning a unique SIM to each channel, allowing the model to focus on important regions within each channel. This emphasizes more useful information encoded in the features, thereby effectively improving performance.

[0069] Specifically, the input raw image undergoes multiple feature extraction stages sequentially set in the feature extraction layer to extract features at different scales. These multi-scale features are then input into the feature fusion layer for feature fusion. This feature fusion process includes:

[0070] First, feature extraction is performed. Multi-scale features from the backbone network output are obtained, including several deep features (or high-resolution features) and shallow features (or low-resolution features). The deep features are upsampled to the same spatial resolution as the lowest shallowest features, and then all features are concatenated to obtain preliminary fused features.

[0071] Secondly, the preliminary fusion features are input into the content-guided attention module to generate a channel-specific saliency map. The preliminary fusion features are then weighted using this saliency map to obtain enhanced fusion features, such as... Figure 5 As shown, it is:

[0072] (1) Generation of Channel Descriptors: The input features are first subjected to Global Average Pooling (GAP) and Global Max Pooling (GMP) operations to extract the channel descriptor vectors and salient activation information of the input features, generating two types of attention weights: channel weights Wc and spatial weights Ws. That is, two 1×1×C vectors are generated, representing the global maximum activation and global average activation of each channel, respectively. Among them, the channel weights focus on the overall importance of each channel, while the spatial weights reflect the important regions of spatial location in each channel;

[0073] (2) The channel weights and spatial weights are fused by element-wise multiplication to form the final channel-space joint saliency maps (SIMs);

[0074] (3) Feature weighting and output: The original input features are weighted and enhanced using the channel-space joint saliency map, and then the enhanced fusion features are finally output through channel rearrangement.

[0075] Unlike traditional channel attention mechanisms such as SE-Net, which generate only a global channel weight vector, CGA generates channel-specific SIMs and assigns a unique SIM to each channel. This process combines the learned channel weights with spatial information to enhance useful feature regions and suppress irrelevant regions.

[0076] By embedding the CGA module into the initial fusion path as described above, the CGA module will perform content-guided attention enhancement on the features after initial fusion, generating channel-specific saliency maps (SIMs). These SIMs will then perform channel-by-channel weighting or modulation on the fused features, thereby enhancing water-related features and suppressing background noise and interference.

[0077] Finally, the feature map refined by CGA is output to the output layer, which can focus on the multi-scale fusion features of the target with high quality and output more accurate recognition and localization results.

[0078] As another implementation method, the training process of the improved target detection model described above is as follows:

[0079] First, obtain tunnel monitoring videos of the construction workers inside the tunnel during the construction process.

[0080] Secondly, the average inter-frame difference of the tunnel monitoring video is calculated and smoothed, several key frames are extracted, and the location of construction personnel is marked on each key frame to construct a dataset. The dataset is divided into a training set and a test set, and the GridMask algorithm is used to enhance the images of the training set.

[0081] Specifically, in this embodiment, the dataset is obtained from on-site photography in the tunnel. Several keyframe images were extracted and re-annotated using the roLabelImg tool. After re-annotation, the annotated image data was first saved with a .xml extension, and then converted to a .txt extension. Further, the annotated image data was augmented and the dataset expanded to obtain a training set and a test set. The dataset was divided into training and test sets in a 4:1 ratio, ultimately resulting in 8000 images for the training set and 2000 images for the test set.

[0082] Preferably, the GridMask algorithm is used to enhance the images on the training set. The GridMask algorithm is an information deletion algorithm that randomly discards a region on the image, essentially adding a regularization term to the network to prevent overfitting. Augmentation is only required during data input, making it simple and convenient. Specifically, GridMask does not remove a large, continuous region, nor does it randomly select blocks for deletion. Instead, it deletes a set of spatially uniformly distributed blocks. In this structure, by controlling the density and size of the deleted regions, there is a statistically higher chance of achieving a good balance in various situations, such as... Figure 6 As shown, this allows for a significant improvement to the baseline CNN model with a very low computational budget.

[0083] Then, an improved object detection model is constructed, which includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer.

[0084] Finally, the improved target detection model is trained using the enhanced training set. The ReLU loss function is used during the training process, and the training is iterated until the set conditions are met (such as reaching the set number of iterations or minimizing the loss) to complete the training of the model. The trained model is then tested and evaluated using the test set to further verify the superiority of the proposed solution in this embodiment.

[0085] Example 2

[0086] This embodiment provides a machine vision-based tunnel construction worker positioning system, including:

[0087] The video data acquisition module is used to acquire tunnel monitoring videos of construction workers during the construction process inside the tunnel.

[0088] The data preprocessing module is used to calculate and smooth the average inter-frame difference of the tunnel monitoring video, extract several key frames, and perform frame image enhancement processing.

[0089] The personnel identification and localization module is used to input several enhanced key frames into the improved target detection model. The model's feature extraction layer extracts multi-scale feature maps, and after feature fusion through the feature fusion layer, the output layer outputs the identification and localization results of construction personnel in the tunnel.

[0090] The feature extraction layer includes multiple feature extraction stages arranged sequentially. Each stage includes a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature enhancement. The feature fusion layer includes a content-guided attention module for guiding the generation of a saliency map and weighted fusion of features.

[0091] Example 3

[0092] This embodiment provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided in this embodiment.

[0093] Example 4

[0094] This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, will cause the processor to execute the method described above in this embodiment.

[0095] Example 5

[0096] This embodiment provides a computer program product including executable instructions, which are computer instructions; the executable instructions are stored in a computer-readable storage medium. When the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the method described in this embodiment.

[0097] The steps and methods involved in Embodiments 2 to 5 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0098] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0099] The above description is only a preferred embodiment of the present invention. Although the specific implementation of the present invention has been described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A method for locating construction workers inside a tunnel based on machine vision, characterized in that, include: Obtain tunnel monitoring videos of construction workers during the construction process inside the tunnel; The average inter-frame difference of the tunnel monitoring video is calculated and smoothed, several key frames are extracted, and frame image enhancement processing is performed. Several enhanced keyframes are input into the improved target detection model. Multi-scale feature maps are extracted by the model's feature extraction layer, and then the features are fused by the feature fusion layer. Finally, the output layer outputs the identification and location results of the construction personnel inside the tunnel. The feature extraction layer includes multiple feature extraction stages arranged sequentially, each stage containing a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature enhancement; the feature fusion layer includes a content-guided attention module for guiding the generation of saliency maps and weighted fusion of features. In each stage of the feature extraction layer, features of the original input image are first extracted through Shuffle Net units for downsampling, and then the features are deepened through multiple stacked Shuffle Net units for feature deepening to extract richer features. The Shuffle Net unit used for downsampling includes two branches. The input image is processed by point-by-point grouping convolution, channel rearrangement, depthwise separable convolution and point-by-point grouping convolution to extract features in the main branch. At the same time, it is pooled by the average pooling layer set in the shortcut branch. The outputs of the two branches are then concatenated to output the downsampled features. In the Shuffle Net unit used for feature enhancement, the input features are sequentially enhanced by pointwise grouped convolution, channel rearrangement, depthwise separable convolution, and pointwise grouped convolution, and then added to the original input features to output the enhanced features; The input raw image undergoes multiple feature extraction stages sequentially within the feature extraction layer to extract features at different scales. These multi-scale features are then input into the feature fusion layer for feature fusion, resulting in: Multi-scale features include several deep features and shallow features. The deep features are upsampled to the same spatial resolution as the lowest shallow feature, and then all features are stitched together to obtain preliminary fused features. The initial fusion features are input into the content-guided attention module to generate a channel-specific saliency map. The initial fusion features are then weighted using the saliency map to obtain the enhanced fusion features. In the content-guided attention module, the input features are first subjected to global average pooling and global max pooling operations respectively, and the channel description vector and salient activation information of the input features are extracted respectively to generate two types of attention weights: channel weights and spatial weights. Among them, the channel weights focus on the overall importance of each channel, while the spatial weights reflect the important regions of spatial location in each channel. Channel weights and spatial weights are combined through element-wise multiplication to form the final channel-space joint saliency map; The original input features are weighted and enhanced using a channel-space joint saliency map, and then the channels are rearranged to finally output the enhanced fused features.

2. The method for locating construction workers in tunnels based on machine vision as described in claim 1, characterized in that, The extraction of the keyframes includes: The acquired tunnel monitoring video is converted to a color space, and all video frames are converted to the LUV color space. The inter-frame difference is calculated on the converted video frames to obtain a difference image. The pixel values ​​of the difference image are then summed to obtain the total inter-frame difference. The total inter-frame difference is divided by the number of pixels in the video frame to obtain the average inter-frame difference, forming an inter-frame difference array. The inter-frame difference array is smoothed, and the indices of local maxima in the smoothed inter-frame difference array are traversed. The video frames corresponding to the indices are used as keyframes.

3. The machine vision-based method for locating construction workers inside tunnels as described in claim 1, characterized in that, The Enlighten GAN algorithm is used to perform image enhancement processing on each extracted keyframe.

4. A machine vision-based tunnel construction worker positioning system, characterized in that, include: The video data acquisition module is used to acquire tunnel monitoring videos of construction workers during the construction process inside the tunnel. The data preprocessing module is used to calculate and smooth the average inter-frame difference of the tunnel monitoring video, extract several key frames, and perform frame image enhancement processing. The personnel identification and localization module is used to input several enhanced key frames into the improved target detection model. The model's feature extraction layer extracts multi-scale feature maps, and after feature fusion through the feature fusion layer, the output layer outputs the identification and localization results of construction personnel in the tunnel. The feature extraction layer includes multiple feature extraction stages arranged sequentially, each stage containing a Shuffle Net unit for downsampling and multiple stacked Shuffle Net units for feature enhancement; the feature fusion layer includes a content-guided attention module for guiding the generation of saliency maps and weighted fusion of features. In each stage of the feature extraction layer, features of the original input image are first extracted through Shuffle Net units for downsampling, and then the features are deepened through multiple stacked Shuffle Net units for feature deepening to extract richer features. The Shuffle Net unit used for downsampling includes two branches. The input image is processed by point-by-point grouping convolution, channel rearrangement, depthwise separable convolution and point-by-point grouping convolution to extract features in the main branch. At the same time, it is pooled by the average pooling layer set in the shortcut branch. The outputs of the two branches are then concatenated to output the downsampled features. In the Shuffle Net unit used for feature enhancement, the input features are sequentially enhanced by pointwise grouped convolution, channel rearrangement, depthwise separable convolution, and pointwise grouped convolution, and then added to the original input features to output the enhanced features; The input raw image undergoes multiple feature extraction stages sequentially within the feature extraction layer to extract features at different scales. These multi-scale features are then input into the feature fusion layer for feature fusion, resulting in: Multi-scale features include several deep features and shallow features. The deep features are upsampled to the same spatial resolution as the lowest shallow feature, and then all features are stitched together to obtain preliminary fused features. The initial fusion features are input into the content-guided attention module to generate a channel-specific saliency map. The initial fusion features are then weighted using the saliency map to obtain the enhanced fusion features. In the content-guided attention module, the input features are first subjected to global average pooling and global max pooling operations respectively, and the channel description vector and salient activation information of the input features are extracted respectively to generate two types of attention weights: channel weights and spatial weights. Among them, the channel weights focus on the overall importance of each channel, while the spatial weights reflect the important regions of spatial location in each channel. Channel weights and spatial weights are combined through element-wise multiplication to form the final channel-space joint saliency map; The original input features are weighted and enhanced using a channel-space joint saliency map, and then the channels are rearranged to finally output the enhanced fused features.

5. An electronic device, characterized in that, include: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the machine vision-based method for locating construction workers in tunnels as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The device stores executable instructions that, when executed by a processor, implement the machine vision-based method for locating construction workers in tunnels as described in any one of claims 1-3.

7. A computer program product, characterized in that, The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the machine vision-based method for locating construction workers in tunnels as described in any one of claims 1-3.

Images