A method and system for multi-scale segmentation of tunnels with multiple defects under low illumination conditions

Abstract

This invention relates to the field of computer vision technology, and particularly to a method and system for multi-scale segmentation of tunnels with multiple defects in low-light environments. The method includes: constructing a smoothing and sharpening module to perform noise reduction and sharpening processing on the input image; improving EfficientNetV2 and using it as the backbone network to extract features from the image; adding an adaptive image enhancement S-curve during progressive downsampling to adaptively enhance the dark areas of the feature map and reduce the highlights; feeding the feature map into an improved pyramid pooling module for multi-scale feature extraction, with each scale's feature map undergoing dimensionality reduction using a Hilbert curve and then performing self-attention mechanism calculations to obtain spatial information from the image; fusing the multi-scale feature maps through channel attention and upsampling to the original image size to obtain the defect segmentation map. This invention addresses the problem of low segmentation accuracy caused by limited image quality generated by drones in dark tunnels by designing an adaptive S-curve enhancement curve and adding it to the segmentation network.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for multi-scale segmentation of tunnels with multiple defects in low-light environments. Background Technology

[0002] Tunnels and other underground facilities are characterized by large spaces, enclosed environments, long service lives, and easily damaged structures, making drones a common method for detecting structural defects. However, insufficient lighting in these environments limits the quality of drone-captured images, which generally exhibit low illumination, low contrast, and low signal-to-noise ratios, significantly impacting detection and segmentation. Therefore, designing a lightweight, real-time segmentation network capable of operating in such environments is essential.

[0003] Segmentation techniques are rapidly evolving, and some mainstream network models, such as PSPNet, U-Net, and SETR, offer high segmentation accuracy. Other networks, like Enet and SeaFormer, are lightweight and have low hardware requirements. However, all of these networks are designed and trained under normal lighting conditions and cannot handle segmentation in low-light environments. Summary of the Invention

[0004] To address the issue that existing solutions, which are based on design and training under normal lighting conditions, cannot handle segmentation in low-light environments, this invention provides a method and system for multi-scale segmentation of tunnels with multiple defects in low-light environments. The technical solution is as follows: On the one hand, a method for multi-scale segmentation of tunnels with multiple defects under low illumination is provided, characterized in that the method includes: S1. Obtain the image to be processed; construct a smoothing and sharpening module to perform noise reduction and sharpening processing on the image to be processed, and obtain the input image; S2. Improve upon EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image and obtain feature maps. S3. During the stepwise downsampling process, an adaptive image enhancement S-curve is added. The adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the highlights of the feature map to obtain a feature map with a resolution of 16×16. S4. The feature map with a resolution of 16×16 is put into the improved pyramid pooling module for multi-scale feature extraction. The feature map at each scale is reduced in dimensionality using Hilbert curve and then the spatial information in the image is obtained by calculating the self-attention mechanism. S5. Multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map.

[0005] Optionally, in S1, the image to be processed is obtained; a smoothing and sharpening module is constructed to perform noise reduction and sharpening processing on the image to be processed, thereby obtaining the input image, including: Obtain the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the smoothed image is sharpened by edge detection using the Laplacian kernel. The resolution of the smoothed and sharpened image is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

[0006] Optionally, in S2, an improvement is made to EfficientNetV2, and the improved EfficientNetV2 is used as the backbone network to extract features from the input image to obtain feature maps, including: Input an image of size 3×512×512 into the EfficientNetV2 network; The input image is sampled 7 times to generate a feature map with a resolution of 16×16. The size of the feature map generated in each layer is as follows: 24×256×256, 48×128×128, 80×64×64, 160×32×32, 176×32×32, 304×16×16, and 512×16×16.

[0007] Optionally, in step S3, an adaptive image enhancement S-curve is added during the progressive downsampling process. This adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the bright areas, resulting in a feature map with a resolution of 16×16, including: An adaptive S-curve image enhancement was added during the progressive downsampling process, and S-curve illumination enhancement was performed before each feature extraction stage. Three of the seven feature extraction stages were selected for adaptive S-curve image illumination enhancement processing: before the first stage, the enhanced resolution was 3×512×512; before the second stage, the enhanced resolution was 24×256×256; and before the third stage, the enhanced resolution was 48×128×128. The formula for the curve is as follows: (1) (1) in, Value for each pixel; The weights are calculated for the neural network, and their values ​​range from (-1, 1). This represents the enhanced pixel value.

[0008] Optionally, in S4, the 16×16 resolution feature map is fed into the improved pyramid pooling module for multi-scale feature extraction. For each scale of feature map, dimensionality reduction using a Hilbert curve is performed, followed by self-attention mechanism calculation to obtain spatial information from the image, including: The feature map with a resolution of 512×16×16 was subjected to average pooling to generate feature maps with resolutions of 170×8×8, 170×4×4, and 170×2×2. Hilbert curve dimensionality reduction is performed on the three feature maps, and a spatial weight map is generated through the calculation of the self-attention mechanism. The spatial weight map is then multiplied with the corresponding feature map to generate three new feature maps.

[0009] Optionally, in S5, multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map, including: The three new feature maps are upsampled to make their resolution 16×16, and the three new feature maps with a resolution of 170×16×16 are concatenated with the original feature map with a resolution of 512×16×16 by channel dimension. Channel attention is calculated on the stitched feature map; The feature map is directly upsampled to a segmentation map of size C×512×512, where C is the type to be segmented, and the original image size is obtained to get the disease segmentation map.

[0010] On the other hand, a multi-scale segmentation system for tunnels with multiple defects under low illumination is provided. This system is applied to the multi-scale segmentation method for tunnels with multiple defects. The system includes: The image processing module is used to acquire the image to be processed; the smoothing and sharpening module is used to perform noise reduction and sharpening processing on the image to be processed to obtain the input image. The feature extraction module is used to improve EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image to obtain feature maps. The image enhancement module is used to add an adaptive image enhancement S-curve during the progressive downsampling process. The adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the highlights of the feature map to obtain a feature map with a resolution of 16×16. The pyramid pooling module is used to feed the 16×16 resolution feature map into the improved pyramid pooling module for multi-scale feature extraction. For each scale of feature map, the Hilbert curve is used to reduce the dimensionality and then the self-attention mechanism is used to calculate the spatial information in the image. The feature fusion module is used to fuse multi-scale feature maps through channel attention and upsample them to the original image size to obtain a disease segmentation map.

[0011] Optionally, an image processing module is used to acquire the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the smoothed image is sharpened by edge detection using the Laplacian kernel. The resolution of the smoothed and sharpened image is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

[0012] On the other hand, a tunnel multi-defect multi-scale segmentation device is provided, which includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the tunnel multi-defect multi-scale segmentation methods described above is implemented.

[0013] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, which is loaded and executed by a processor to implement any of the above-described methods for multi-scale segmentation of tunnel defects.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, the designed adaptive S-curve is a family of curve functions based on color grading experience. These functions are generated through deep learning. The curve functions adaptively increase pixel values ​​in dark areas and reduce overexposed highlights, which is beneficial for subsequent feature extraction from the feature map by the backbone network. For the environment involved in this invention, EfficientNetV2 is selected as the backbone network and improved upon. This improvement mainly focuses on information preservation in the downsampling part of the backbone network. This invention utilizes multi-scale feature fusion combined with spatial attention and channel attention mechanisms to improve the neural network's extraction of spatial information from images. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a method for multi-scale segmentation of tunnels with multiple defects in low-light environments, provided by an embodiment of the present invention. Figure 2 This is a diagram of the S-curve gradient generation model provided in an embodiment of the present invention; Figure 3These are graphs showing the performance of the S-curve under different gradients provided in the embodiments of the present invention; Figure 4 This is a spatial information calculation diagram in the feature pyramid provided in this embodiment of the invention; Figure 5 This is an enhanced visualization effect image of an adaptive S-shaped curve underground building structure provided by an embodiment of the present invention; Figure 6 This is a block diagram of a multi-scale segmentation system for tunnels with multiple defects in low-light environments, provided by an embodiment of the present invention. Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides a method for multi-scale segmentation of multiple defects in tunnels under low-light conditions. This method can be implemented using a multi-scale segmentation device for multiple defects in tunnels, which can be a terminal or a server. Figure 1 The flowchart shown is for a multi-scale segmentation method for multiple defects in tunnels. The processing flow of this method may include the following steps: S1. Obtain the image to be processed; construct a Smooth Sharpening Module (SSM) to perform noise reduction and sharpening processing on the image to be processed, and obtain the input image; In one feasible implementation, in step S1, the image to be processed is acquired; a smoothing and sharpening module is constructed to perform noise reduction and sharpening processing on the image to be processed to obtain an input image, including: Obtain the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the smoothed image is sharpened by edge detection using the Laplacian kernel. The resolution of the smoothed and sharpened image is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

[0023] In one feasible implementation, the image to be processed is a low-light tunnel photograph taken by a drone, and the image to be processed is put into a segmentation network.

[0024] In one feasible implementation, since low-light images contain a lot of noise, and most of the noise is Gaussian noise, the smoothing and sharpening module can initially perform noise reduction and sharpening processing on the input image, which makes it more advantageous for the subsequent backbone network to extract features from the image.

[0025] S2. Improve upon EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image and obtain feature maps. In one feasible implementation, in S2, an improvement is made to EfficientNetV2, and the improved EfficientNetV2 is used as the backbone network to extract features from the input image to obtain a feature map, including: Input an image of size 3×512×512 into the EfficientNetV2 network; The input image is sampled 7 times to generate a feature map with a resolution of 16×16. The size of the feature map generated in each layer is as follows: 24×256×256, 48×128×128, 80×64×64, 160×32×32, 176×32×32, 304×16×16, and 512×16×16.

[0026] In one feasible implementation, EfficientNetV2 is improved and used as the backbone network for image feature extraction. The improvement involves adding adaptive image enhancement S-curves before stages 1, 3, and 4 of the eight stages of EfficientNetV2. Specifically, before inputting into EfficientNetV2, a 3×512×512 feature map is adaptively enhanced and then fed into EfficientNetV2. After the second stage of EfficientNetV2 outputs a 24×256×256 feature map, another adaptive image enhancement is performed before it is fed into the third stage for feature extraction. The operations before the fourth stage are the same as before.

[0027] S3. During the stepwise downsampling process, an adaptive image enhancement S-curve is added. The adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the highlights of the feature map to obtain a feature map with a resolution of 16×16. In one feasible implementation, S3, during the progressive downsampling process, an adaptive image enhancement S-curve is added. This adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the bright areas, resulting in a feature map with a resolution of 16×16. This includes: An adaptive S-curve image enhancement was added during the progressive downsampling process, and S-curve illumination enhancement was performed before each feature extraction stage. Three of the seven feature extraction stages were selected for adaptive S-curve image illumination enhancement processing: before the first stage, the enhanced resolution was 3×512×512; before the second stage, the enhanced resolution was 24×256×256; and before the third stage, the enhanced resolution was 48×128×128. The formula for the curve is as follows: (1) (1) in, Value for each pixel; The weights are calculated for the neural network, and their values ​​range from (-1, 1). This represents the enhanced pixel value.

[0028] In one feasible implementation, the curve gradient generation module is as follows: Figure 2 As shown, different values ​​of the parameter α generated by the module affect the change of the curve as follows: Figure 3 As shown, and the enhancement effect obtained by its module can be achieved in Figure 5 It can be seen that the enhancement process improves the detailed information of some underground building structural defects, thus enhancing the recognition and segmentation effect of the subsequent network.

[0029] In one feasible implementation, the designed adaptive S-curve is a rough curve function based on color grading experience. Since most pixels in low-light images are distributed in dark tones, the gradient differences between pixels are small, making it difficult for neural networks to extract semantic information from the image. The adaptive S-curve increases the low pixel values ​​of the input, appropriately lowers high pixel values ​​(overexposed areas), and maintains the monotonicity of the overall pixel values. Because the pixel distribution varies between scenes but is roughly the same within a scene, the neural network can be trained on specific scenes to achieve the best curve gradient adjustment, thus enabling rapid image enhancement of the feature map. Adaptively increasing pixel values ​​in dark areas and reducing overexposed highlights facilitates feature extraction from the feature map by the subsequent backbone network.

[0030] In the environment described in this invention, the network is typically deployed on a terminal. Therefore, the selected network needs to possess good feature extraction capabilities while also being lightweight and real-time. Thus, EfficientNetV2 was chosen as the backbone network and improved upon. An adaptive S-curve module is added before stages 1, 3, and 4 of EfficientNetV2 to enhance the dark area features of the network output from the previous stage, thereby preserving dark area information in the feature map across multiple stages. This improvement primarily targets the information preservation of the downsampling portion in the backbone network.

[0031] S4. The feature map with a resolution of 16×16 is put into the improved pyramid pooling module for multi-scale feature extraction. The feature map at each scale is reduced in dimensionality using Hilbert curve and then the spatial information in the image is obtained by calculating the self-attention mechanism. In one feasible implementation, in step S4, a 16×16 resolution feature map is fed into an improved pyramid pooling module for multi-scale feature extraction. For each scale of feature map, dimensionality reduction using a Hilbert curve is performed, followed by self-attention mechanism calculation to obtain spatial information from the image, including: The feature map with a resolution of 512×16×16 was subjected to average pooling to generate feature maps with resolutions of 170×8×8, 170×4×4, and 170×2×2. Hilbert curve dimensionality reduction is performed on the three feature maps, and a spatial weight map is generated through the calculation of the self-attention mechanism. The spatial weight map is then multiplied with the corresponding feature map to generate three new feature maps.

[0032] In one feasible implementation, the pyramid pooling module is changed from the original four pooling methods to three: 2×2, 4×4, and 8×8. For each scale of feature map, dimensionality reduction is performed using a Hilbert curve, and spatial information in the image is obtained through self-attention mechanism calculation. Finally, the three feature maps are upsampled and concatenated with the original 16×16 feature map along the channel dimension. The implementation process is as follows: Figure 4 As shown.

[0033] In one feasible implementation, this invention utilizes multi-scale feature fusion combined with spatial attention and channel attention mechanisms to improve the extraction of spatial information from images by neural networks. Different feature information is obtained by multi-scale pooling of the 16×16 feature map, and better features are extracted through spatial attention and channel attention. Furthermore, dimensionality reduction of the low-resolution feature map using Hilbert curves enhances the computational power of the spatial attention mechanism, thereby increasing the network's focus on the target.

[0034] S5. Multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map.

[0035] In one feasible implementation, in step S5, multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map, including: The three new feature maps are upsampled to make their resolution 16×16, and the three new feature maps with a resolution of 170×16×16 are concatenated with the original feature map with a resolution of 512×16×16 by channel dimension. Channel attention is calculated on the stitched feature map; The feature map is directly upsampled to a segmentation map of size C×512×512, where C is the type to be segmented, and the original image size is obtained to get the disease segmentation map.

[0036] In this embodiment of the invention, the designed adaptive S-curve is a rough curve function based on color grading experience. It adaptively increases pixel values ​​in dark areas and reduces overexposed highlights, which is beneficial for subsequent feature extraction from the feature map by the backbone network. For the environment involved in this invention, EfficientNetV2 is selected as the backbone network and improved upon. This improvement mainly focuses on information preservation in the downsampling part of the backbone network. This invention utilizes multi-scale feature fusion combined with spatial attention and channel attention mechanisms to improve the neural network's extraction of spatial information from images.

[0037] Figure 6 This is a block diagram illustrating a multi-scale segmentation system for tunnels with multiple defects in a low-light environment, according to an exemplary embodiment. This system is used in a multi-scale segmentation method for tunnels with multiple defects. (Refer to...) Figure 6 The system includes an image processing module 310, a feature extraction module 320, an image enhancement module 330, a pyramid pooling module 340, and a feature fusion module 350. Among them: Image processing module 310 is used to acquire the image to be processed; and to construct a smoothing and sharpening module to perform noise reduction and sharpening processing on the image to be processed to obtain the input image. The feature extraction module 320 is used to improve EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image to obtain feature maps. Image enhancement module 330 is used to add an adaptive image enhancement S-curve during the progressive downsampling process. The adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the highlights of the feature map to obtain a feature map with a resolution of 16×16. The pyramid pooling module 340 is used to put the 16×16 resolution feature map into the improved pyramid pooling module for multi-scale feature extraction. For each scale feature map, the dimensionality is reduced by Hilbert curve and then the spatial information in the image is obtained by self-attention mechanism calculation. The feature fusion module 350 is used to fuse multi-scale feature maps through channel attention and upsample them to the original image size to obtain a disease segmentation map.

[0038] Optionally, an image processing module is used to acquire the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the smoothed image is sharpened by edge detection using the Laplacian kernel. The resolution of the smoothed and sharpened image is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

[0039] Optionally, the feature extraction module 320 is used to input an input image of size 3×512×512 into the EfficientNetV2 network; The input image is sampled 7 times to generate a feature map with a resolution of 16×16. The size of the feature map generated in each layer is as follows: 24×256×256, 48×128×128, 80×64×64, 160×32×32, 176×32×32, 304×16×16, and 512×16×16.

[0040] Optionally, the image enhancement module 330 is used to add an adaptive image enhancement S-curve during the progressive downsampling process, performing S-curve illumination enhancement before each feature extraction stage; three stages are selected from the seven feature extraction stages to perform adaptive S-curve image illumination enhancement processing, namely before the first stage with an enhancement resolution of 3×512×512; before the second stage with an enhancement resolution of 24×256×256; and before the third stage with an enhancement resolution of 48×128×128. The formula for the curve is as follows: (1) (1) in, Value for each pixel; The weights are calculated for the neural network, and their values ​​range from (-1, 1). This represents the enhanced pixel value.

[0041] Optionally, the pyramid pooling module 340 is used to perform average pooling on the feature map with a resolution of 512×16×16 to generate feature maps with resolutions of 170×8×8, 170×4×4, and 170×2×2. Hilbert curve dimensionality reduction is performed on the three feature maps, and a spatial weight map is generated through the calculation of the self-attention mechanism. The spatial weight map is then multiplied with the corresponding feature map to generate three new feature maps.

[0042] Optionally, the feature fusion module 350 is used to upsample the three new feature maps to make the resolution of the three new feature maps 16×16, and to stitch the three new feature maps with a resolution of 170×16×16 with the original feature map with a resolution of 512×16×16 along the channel dimension. Channel attention is calculated on the stitched feature map; The feature map is directly upsampled to a segmentation map of size C×512×512, where C is the type to be segmented, and the original image size is obtained to get the disease segmentation map.

[0043] In this embodiment of the invention, the designed adaptive S-curve is a rough curve function based on color grading experience. It adaptively increases pixel values ​​in dark areas and reduces overexposed highlights, which is beneficial for subsequent feature extraction from the feature map by the backbone network. For the environment involved in this invention, EfficientNetV2 is selected as the backbone network and improved upon. This improvement mainly focuses on information preservation in the downsampling part of the backbone network. This invention utilizes multi-scale feature fusion combined with spatial attention and channel attention mechanisms to improve the neural network's extraction of spatial information from images.

[0044] Figure 7 This is a structural schematic diagram of a tunnel multi-defect, multi-scale segmentation device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the tunnel multi-defect multi-scale segmentation equipment may include the above-mentioned Figure 6 The tunnel multi-defect multi-scale segmentation system shown. Optionally, the tunnel multi-defect multi-scale segmentation device 410 may include a first processor 2001.

[0045] Optionally, the tunnel multi-disease multi-scale segmentation device 410 may also include a memory 2002 and a transceiver 2003.

[0046] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0047] The following is combined Figure 7 A detailed introduction to each component of the tunnel multi-defect multi-scale segmentation equipment 410 is provided below: The first processor 2001 is the control center of the tunnel multi-defect multi-scale segmentation device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0048] Optionally, the first processor 2001 can perform various functions of the tunnel multi-disease multi-scale segmentation device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0049] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 7 CPU0 and CPU1 are shown in the diagram.

[0050] In a specific implementation, as one example, the tunnel multi-defect multi-scale segmentation device 410 may also include multiple processors, for example... Figure 7The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0051] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0052] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the tunnel multi-path multi-scale segmentation device 410. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0053] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0054] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 7 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0055] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently, and it can be connected to the interface circuit of the tunnel multi-disease multi-scale segmentation device 410. Figure 7 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0056] It should be noted that, Figure 7 The structure of the tunnel multi-disease multi-scale segmentation device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] Furthermore, the technical effects of the tunnel multi-disease multi-scale segmentation device 410 can be referred to the technical effects of the tunnel multi-disease multi-scale segmentation method described in the above method embodiments, and will not be repeated here.

[0058] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0059] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0060] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0061] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0062] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0063] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0064] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0065] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0066] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for multi-scale segmentation of tunnels with multiple defects under low illumination conditions, characterized in that, The method includes: S1. Obtain the image to be processed; construct a smoothing and sharpening module to perform noise reduction and sharpening processing on the image to be processed to obtain the input image; S2. Improve upon EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image to obtain a feature map. Input an image of size 3×512×512 into the EfficientNetV2 network; The input image is sampled 7 times to generate a feature map with a resolution of 16×16; the size of the feature map generated in each layer is as follows: 24×256×256, 48×128×128, 80×64×64, 160×32×32, 176×32×32, 304×16×16, 512×16×16; S3. During the stepwise downsampling process, an adaptive image enhancement S-curve is added to adaptively enhance the dark areas of the feature map and reduce the highlights of the feature map to obtain a feature map with a resolution of 16×16. An adaptive S-curve image enhancement was added during the seven sampling processes of the input image. S-curve illumination enhancement was performed before each feature extraction stage. Three stages were selected from the seven feature extraction stages for adaptive S-curve image illumination enhancement processing. The three stages were: before the first stage, with an enhancement resolution of 3×512×512; before the second stage, with an enhancement resolution of 24×256×256; and before the third stage, with an enhancement resolution of 48×128×128. The formula for the curve is as follows: (1) （1） in, Value for each pixel; The weights are calculated for the neural network, and their values ​​range from (-1, 1). This represents the enhanced pixel value; S4. The feature map with a resolution of 16×16 is put into the improved pyramid pooling module for multi-scale feature extraction. The feature map at each scale is reduced in dimensionality using Hilbert curve and then the spatial information in the image is obtained by calculating the self-attention mechanism. The feature map with a resolution of 512×16×16 is subjected to average pooling of three different sizes to finally generate feature maps with resolutions of 170×8×8, 170×4×4, and 170×2×2. Hilbert curve dimensionality reduction is performed on the three feature maps, a spatial weight map is generated through the calculation of the self-attention mechanism, and the spatial weight map is multiplied with the corresponding feature map to generate three new feature maps. S5. Multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map.

2. The method for multi-scale segmentation of tunnels under low illumination conditions according to claim 1, characterized in that, In step S1, the image to be processed is acquired; A smoothing and sharpening module is constructed to perform noise reduction and sharpening processing on the image to be processed to obtain the input image, including: Obtain the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the denoised image is sharpened by edge detection using the Laplacian kernel. The resolution of the image after noise reduction and sharpening is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

3. The method for multi-scale segmentation of tunnels under low illumination conditions according to claim 2, characterized in that, In step S5, multi-scale feature maps are fused through channel attention and upsampled to the original image size to obtain a disease segmentation map, including: The three new feature maps are upsampled to make their resolution 16×16, and the three new feature maps with a resolution of 170×16×16 are concatenated with the original feature map with a resolution of 512×16×16 by channel dimension. Channel attention is calculated on the stitched feature map; The feature map is directly upsampled to a segmentation map of size C×512×512, where C is the type to be segmented, and the original image size is obtained to get the disease segmentation map.

4. A multi-scale segmentation system for tunnels with multiple defects under low illumination conditions, wherein the multi-scale segmentation system for tunnels with multiple defects is used to implement the multi-scale segmentation method for tunnels with multiple defects as described in any one of claims 1-3, characterized in that, The system includes: An image processing module is used to acquire the image to be processed; a smoothing and sharpening module is used to perform noise reduction and sharpening processing on the image to be processed to obtain the input image; The feature extraction module is used to improve EfficientNetV2 and use the improved EfficientNetV2 as the backbone network to extract features from the input image to obtain a feature map. An image enhancement module is used to add an adaptive image enhancement S-curve during the progressive downsampling process. The adaptive image enhancement S-curve adaptively enhances the dark areas of the feature map and reduces the highlights of the feature map to obtain a feature map with a resolution of 16×16. The pyramid pooling module is used to feed the 16×16 resolution feature map into the improved pyramid pooling module for multi-scale feature extraction. For each scale of feature map, the Hilbert curve is used to reduce the dimensionality and then the self-attention mechanism is used to calculate the spatial information in the image. The feature fusion module is used to fuse multi-scale feature maps through channel attention and upsample them to the original image size to obtain a disease segmentation map.

5. The multi-scale segmentation system for tunnels in low-light environments according to claim 4, characterized in that, The image processing module is used to acquire the image to be processed; The image to be processed is denoised using Gaussian filtering, and then the smoothed image is sharpened by edge detection using the Laplacian kernel. The resolution of the smoothed and sharpened image is changed to a fixed size, with a fixed resolution of 512×512, to obtain the input image.

6. A multi-scale segmentation device for tunnels with multiple defects under low illumination conditions, characterized in that, The multi-scale segmentation equipment for tunnels in low-light environments includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 3.

Images