A visual-based non-contact monitoring method for wading bridge water level

Abstract

This application proposes a vision-based non-contact monitoring method for water level on bridges navigating watercourses, relating to the field of computer vision. The method includes: extracting features from a water level gauge image using a convolutional neural network to obtain an optimized feature map; performing feature extraction, feature enhancement, and image reconstruction on the optimized feature map using a ROI image reconstruction network to obtain an ROI image; performing HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image; performing multi-scale feature extraction, multi-scale feature fusion, and pixel segmentation on the optimized HSV image using a multi-scale pyramid network to obtain an initial water level gauge marking region image; performing dual range detection on the initial water level gauge marking region image to obtain an optimized water level gauge marking region image; removing reflections from the optimized water level gauge marking region image to obtain a final water level gauge marking region image; and retrieving the corresponding water level scale value from a database based on the final water level gauge marking region image.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a vision-based non-contact monitoring method for water level on bridges navigating waterways. Background Technology

[0002] Water level monitoring of bridges across waterways is a crucial aspect of bridge safety operation and flood control. Accurate water level data provides vital support for bridge structural risk assessment and emergency response. Currently, vision-based non-contact water level monitoring methods for bridges across waterways are gradually replacing traditional manual observation methods, but many technical bottlenecks still exist in practical applications.

[0003] Existing methods do not differentiate the importance of image regions when extracting features from water level gauge images. They treat the core water level gauge region, background region, and auxiliary region equally, resulting in key water level gauge features being interfered with by redundant background information, making it difficult to achieve effective focusing.

[0004] Meanwhile, existing methods mostly adopt a single-scale feature extraction mode, which either focuses on local details while ignoring the global water level distribution, or focuses on global positioning while losing fine scale information. This can easily cause water level mark segmentation breakage or missegmentation, affecting the accuracy of subsequent scale recognition.

[0005] In addition, water level gauge images are easily affected by environmental factors during acquisition, such as water surface reflection, water level gauge reflection, and changes in lighting. Existing technologies lack targeted interference removal solutions, making it difficult to effectively remove invalid information such as reflective areas and reflections, resulting in deviations in the identification of water level gauge marked areas. Summary of the Invention

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a vision-based non-contact monitoring method for water levels on bridges navigating watercourses, comprising: The water level gauge images of the target bridge in the water are acquired by a camera. A convolutional neural network is constructed, and features are extracted from the water level gauge images by the convolutional neural network to obtain an initial feature map. At the same time, the water level gauge images are divided into image blocks to obtain image blocks. The initial feature map is then optimized by using the image blocks to obtain an optimized feature map. A ROI image reconstruction network is constructed. The ROI image reconstruction network is used to extract features, enhance features, and reconstruct images from optimized feature maps to obtain ROI images. Perform HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image; A multi-scale pyramid network is constructed, and multi-scale feature extraction, multi-scale feature fusion and pixel segmentation are performed on the optimized HSV image to obtain the initial water level mark region image. Dual range detection is performed on the initial water level mark region image to obtain an optimized water level mark region image. Reflection removal is then performed on the optimized water level mark region image to obtain the final water level mark region image. Retrieve from the database, and obtain the corresponding water level scale value from the database based on the final water gauge marked area image.

[0007] Optionally, the step of extracting features from the water level indicator image using a convolutional neural network to obtain an initial feature map; simultaneously dividing the water level indicator image into image blocks to obtain image blocks; and optimizing the initial feature map using the image blocks to obtain an optimized feature map includes: The water level indicator image is input into a convolutional neural network for feature extraction and water level indicator recognition to obtain an initial feature map y. in and recognition box; The water level indicator image is divided into multiple image blocks, and a position index matrix is ​​constructed based on the position of the image blocks. Image blocks that are completely within the recognition box are labeled as ROI blocks, image blocks adjacent to ROI blocks are labeled as ROP blocks, and the remaining image blocks are labeled as RON blocks; In the location index matrix, the elements corresponding to the ROI block, ROP block, and RON block are assigned the first preset value, the second preset value, and the third preset value, respectively, to obtain the weight allocation matrix γ. e Where the first preset value > the second preset value > the third preset value; the weight allocation matrix γ is upsampled. e The size is increased to match the initial feature map y in The same applies, yielding the weight mask γ. m ; Weight mask γ m Transform into importance feature map γ f The importance feature map γ f With the initial feature map y in Add them together to obtain the optimized feature map.

[0008] Optionally, the ROI image reconstruction network includes a Self-Attention network, a Spatial-Attention network, and a feedforward neural network. The step of using the ROI image reconstruction network to extract features, enhance features, and reconstruct the image from the optimized feature map to obtain the ROI image includes: Extract ROI features from ROI blocks, ROP features from ROP blocks, and RON features from RON blocks from the optimized feature map. The ROP features corresponding to ROP blocks whose overlap with ROI blocks exceeds the first threshold are taken as candidate ROP features, and the RON features corresponding to RON blocks adjacent to ROP blocks are taken as candidate RON features. The ROI features and candidate ROP features are enhanced using a Self-Attention network to obtain the first enhanced feature set. The remaining ROP features and candidate RON features are enhanced using a Spatial-Attention network to obtain a second enhanced feature set. The first and second enhanced feature sets are concatenated to obtain the final feature map; The final feature map is input into a feedforward neural network for image reconstruction to obtain the ROI image.

[0009] Optionally, the step of performing HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image includes: The ROI image is converted from RGB space to HSV space to obtain the initial HSV image, which includes the initial H channel image, the initial S channel image and the initial V channel image. The initial S-channel image is subjected to bilateral filtering and contrast linear stretching to obtain an optimized S-channel image. Logarithmic transformation, adaptive gamma correction, illuminance map calculation, and illuminance map merging are performed on the initial V-channel image to obtain an optimized V-channel image. The initial HSV image, the optimized S-channel image, and the optimized V-channel image are merged to obtain the optimized HSV image.

[0010] Optionally, the multi-scale pyramid network includes high-resolution branch units, medium-resolution branch units, low-resolution branch units, attention units, and a segmentation network. The step of using the multi-scale pyramid network to perform multi-scale feature extraction, multi-scale feature fusion, and pixel segmentation on the optimized HSV image to obtain an initial water level gauge region image includes: Feature extraction of the optimized HSV image is performed using high-resolution branch units to obtain first-scale features; The optimized HSV image is downsampled by pooling once to obtain a pooled HSV image. The second-scale features are then extracted from the pooled HSV image using a medium-resolution branch unit. The optimized HSV image is downsampled twice by pooling to obtain a double-pooled HSV image. The feature of the double-pooled HSV image is extracted by low-resolution branch units to obtain the third-scale feature. The first-scale feature, the second-scale feature, and the third-scale feature are weighted and fused by an attention unit to obtain the fused feature. Based on the fusion features, the initial water level mark region image is obtained by segmenting the pixels of the water level mark in the water level mark image using a segmentation network.

[0011] Optionally, the step of performing dual range detection on the initial water level mark region image to obtain an optimized water level mark region image includes: Obtain the HSV value of each pixel in the initial water level mark area image, set a first HSV value range for light-toned marks, a second HSV value range for dark-toned marks, and a third HSV value range for reflective areas. Binarize the pixels whose HSV values ​​are within the first HSV value range to obtain the first mask; The pixels with HSV values ​​within the second HSV value range are binarized to obtain the second mask; Binarize the pixels whose HSV values ​​fall within the third HSV value range to obtain the third mask; Perform a logical OR operation on the first mask, the second mask, and the third mask to obtain an optimized mask, and then convert the optimized mask into an optimized water level mark region image.

[0012] Optionally, the step of removing reflections from the optimized water level mark region image to obtain the final water level mark region image includes: Obtain the bottom water level mark of the optimized water level mark region image. Using the bottom water level mark as the boundary, extend upwards by a preset number of pixels to divide the optimized water level mark region image into an upper image and a lower image. Obtain the average HSV value of the pixels in the upper image. Calculate the HSV distance of each pixel in the image below based on the average HSV value; Remove pixels in the lower image whose HSV distance is greater than the second threshold to obtain the lower image after removing the reflection; The top image and the bottom image after removing the reflection are stitched together to form the final water level mark area image.

[0013] Secondly, the present invention provides a vision-based non-contact monitoring device for water level on bridges navigating watercourses, used to implement the aforementioned vision-based non-contact monitoring method for water level on bridges navigating watercourses, the device comprising: The optimized feature map acquisition module is used to acquire water level images of the target bridge across the water using a camera, construct a convolutional neural network, extract features from the water level images using the convolutional neural network to obtain an initial feature map; at the same time, the water level images are divided into image blocks to obtain image blocks; and the initial feature map is optimized using image blocks to obtain an optimized feature map. The ROI image acquisition module is used to construct an ROI image reconstruction network. The ROI image reconstruction network performs feature extraction, feature enhancement, and image reconstruction on the optimized feature map to obtain the ROI image. The HSV image acquisition module is optimized to perform HSV spatial transformation and image optimization on ROI images to obtain optimized HSV images. The initial water level gauge marked region image acquisition module is used to construct a multi-scale pyramid network. Through the multi-scale pyramid network, multi-scale feature extraction, multi-scale feature fusion and pixel segmentation are performed on the optimized HSV image to obtain the initial water level gauge marked region image. The final water level mark region image acquisition module is used to perform dual range detection on the initial water level mark region image to obtain an optimized water level mark region image, and to remove the reflection from the optimized water level mark region image to obtain the final water level mark region image. The water gauge level value acquisition module is used to retrieve the database and obtain the corresponding water gauge level value from the database based on the final water gauge marked area image.

[0014] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned vision-based non-contact monitoring method for water level of bridges across water.

[0015] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned vision-based non-contact monitoring method for water level in water-crossing bridges.

[0016] The present invention has the following beneficial effects: 1. Feature map extraction, feature enhancement, and image reconstruction of water level gauge images are performed using convolutional neural networks and ROI image reconstruction networks to obtain ROI images. Key region features in the ROI images are enhanced, and background interference on water level gauge features is avoided. Multi-scale feature extraction of optimized HSV images is performed using a multi-scale pyramid network, combined with attention-weighted fusion, which can capture small auxiliary scales and avoid segmentation breaks caused by local noise, thus improving the accuracy of water level gauge marked area images. Dark and reflective areas in the water level gauge marked area images are removed by dual range detection, and reflections are also removed, further improving the accuracy of the water level gauge marked area images. Finally, the corresponding water level scale values ​​are obtained from the database using the final water level gauge marked area images, realizing high-precision monitoring of water level gauges on bridges crossing water.

[0017] 2. Divide the water level indicator image into ROI, ROP, and RON blocks. By assigning different weights to the ROI, ROP, and RON blocks, the ROI block, being the core area, is enhanced with a high weight, the ROP block, being the auxiliary area, is supplemented with a medium weight, and the RON block, being the irrelevant area, is suppressed with a low weight, thereby achieving targeted enhancement of key features.

[0018] 3. The multi-scale pyramid network includes high-resolution branch units, medium-resolution branch units, low-resolution branch units, attention units, and a segmentation network. The high-resolution branch units accurately capture fine scale details, the medium-resolution branch units balance the transition between details and semantics, and the low-resolution branch units enhance the global water level distribution localization. The attention units perform weighted fusion of first-scale, second-scale, and third-scale features to enhance effective features and suppress redundant information. The segmentation network restores the fused features into a segmentation map that matches the original ROI image, accurately locates water level marker pixels, eliminates background interference, and achieves high-precision water level marker segmentation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a structural diagram of the device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0022] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0023] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0024] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0025] Reference Figure 1 This invention provides a vision-based non-contact monitoring method for water levels on bridges navigating watercourses, comprising: 100. The system acquires water level images of the target bridge by means of a camera, constructs a convolutional neural network, extracts features from the water level images using the convolutional neural network to obtain an initial feature map, divides the water level images into image blocks to obtain image blocks, and optimizes the initial feature map using the image blocks to obtain an optimized feature map. In some embodiments, the step of extracting features from the water level indicator image using a convolutional neural network to obtain an initial feature map; simultaneously, dividing the water level indicator image into image blocks to obtain image blocks; and optimizing the initial feature map using the image blocks to obtain an optimized feature map includes: The water level indicator image is input into a convolutional neural network for feature extraction and water level indicator recognition to obtain an initial feature map y. in and recognition box; The water level indicator image is divided into multiple image blocks, and a position index matrix is ​​constructed based on the position of the image blocks. Image blocks that are completely within the recognition box are labeled as ROI blocks, image blocks adjacent to ROI blocks are labeled as ROP blocks, and the remaining image blocks are labeled as RON blocks; In the location index matrix, the elements corresponding to the ROI block, ROP block, and RON block are assigned the first preset value, the second preset value, and the third preset value, respectively, to obtain the weight allocation matrix γ. e Where the first preset value > the second preset value > the third preset value; the weight allocation matrix γ is upsampled. e The size is increased to match the initial feature map y in The same applies, yielding the weight mask γ. m ; Weight mask γ m Transform into importance feature map γ f The importance feature map γ f With the initial feature map y in Add them together to obtain the optimized feature map.

[0026] In some embodiments, the acquired water level gauge image is 1920×1080 pixels, containing red water level gauge markings and slight background clutter. A lightweight YOLOv8n is selected as the base convolutional neural network, with its detection head removed, retaining only the feature extraction backbone for feature extraction and coarse water level gauge recognition. After the water level gauge image is input into the network, it undergoes 5 convolutional downsampling iterations to output an initial feature map yin; simultaneously, a recognition bounding box is output through the network's built-in candidate box generation mechanism.

[0027] The image block size is set to 32×32 pixels. The 1920×1080 water level image is evenly divided into 60 columns and 33 rows of image blocks. A position index matrix M1 is constructed with a matrix dimension of 60×33. Each element corresponds to the position index of an image block. For example, M1[5]

[10] corresponds to the 32×32 image block in the 5th row and 10th column.

[0028] Traverse all image blocks and determine their positional relationship with the recognition box, including ROI (Region of Interest) blocks: image blocks completely located within the recognition box; ROP (Region of Possible) blocks: image blocks adjacent to ROI blocks; RON (Region of No-target) blocks: remaining image blocks that have not been marked as ROI or ROP blocks.

[0029] Set preset values: first preset value = 1.0, second preset value = 0.5, and third preset value = 0.1. Replace the elements corresponding to the ROI block, ROP block, and RON block in the position index matrix M1 with the above preset values ​​to obtain the weight allocation matrix γ. e The weight allocation matrix γ is calculated using bilinear interpolation. e Upsampling is performed, expanding its size from 60×33 to 60×33×256, and comparing it with the initial feature map y. in The dimensions are consistent, and the same matrix values ​​are reused for each feature channel to obtain the weight mask γ. m The dimensions are 60×33×256.

[0030] Weight mask γ m Transform into importance feature map γ f Then, the initial feature map yin and the importance feature map γ are... f Element-wise addition is performed, that is, the values ​​of each channel and each pixel position of the feature map are added together, and the final output is an optimized feature map with dimensions of 60×33×256. Its ROI region features are enhanced with a weight of 1.0, ROP region features are supplemented with a weight of 0.5, and RON region irrelevant features are suppressed with a weight of 0.1, which effectively improves the targeting of subsequent feature extraction.

[0031] 200 Construct a ROI image reconstruction network, and use the ROI image reconstruction network to perform feature extraction, feature enhancement and image reconstruction on the optimized feature map to obtain the ROI image; In some embodiments, the ROI image reconstruction network includes a Self-Attention network, a Spatial-Attention network, and a feedforward neural network. The step of using the ROI image reconstruction network to extract features, enhance features, and reconstruct the image from the optimized feature map to obtain the ROI image includes: Extract ROI features from ROI blocks, ROP features from ROP blocks, and RON features from RON blocks from the optimized feature map. The ROP features corresponding to ROP blocks whose overlap with ROI blocks exceeds the first threshold are taken as candidate ROP features, and the RON features corresponding to RON blocks adjacent to ROP blocks are taken as candidate RON features. The ROI features and candidate ROP features are enhanced using a Self-Attention network to obtain the first enhanced feature set. The remaining ROP features and candidate RON features are enhanced using a Spatial-Attention network to obtain a second enhanced feature set. The first and second enhanced feature sets are concatenated to obtain the final feature map; The final feature map is input into a feedforward neural network for image reconstruction to obtain the ROI image.

[0032] In some embodiments, considering that an excessively high overlap ratio with an ROI block indicates a critical auxiliary region, a first threshold is set to 50%, meaning the overlap area between a ROP block and any ROI block accounts for ≥50% of the total area of ​​the ROP block. All ROP blocks are traversed, and their overlap ratio with ROI blocks is calculated. For example, if the overlap area between the ROP block in column 17, row 9 and the ROI block in column 17, row 10 accounts for 62.5% of the ROP block's area, it is determined to be an associated ROP block. Finally, the ROP features of the ROP blocks that meet the criteria are selected to form a candidate ROP feature set; the features of the remaining ROP blocks are denoted as the remaining ROP features.

[0033] Adjacency is defined as vertical, horizontal, left-right, and diagonal adjacency. RON blocks adjacent to any ROP block are then filtered. For example, the RON block in column 15, row 9 is adjacent to the ROP block in column 16, row 9, and is therefore considered an associated RON block. Finally, the RON features of RON blocks that meet the criteria are filtered to form a candidate RON feature set.

[0034] A multi-head Self-Attention network is employed to focus on enhancing the semantic association between ROI features and candidate ROP features. The input feature matrix is ​​mapped to query (Q), key (K), and value (V) matrices, each with a dimension of 108×256. The Q, K, and V matrices are split into eight 32-dimensional sub-matrices by head, and the attention weight of each head is calculated. The output of each head is equal to the weight multiplied by the value. The outputs of the eight heads are concatenated and mapped back to 256 dimensions through a linear layer to obtain the first enhanced feature set, which includes the enhanced ROI features and the enhanced candidate ROP features.

[0035] The Spatial-Attention network is a spatial attention network that focuses on enhancing the spatial correlation between the remaining ROP features and the candidate RON features. The remaining ROP features and the candidate RON features are concatenated in the sample dimension to obtain the first feature matrix. At the same time, to reflect spatial location information, the spatial coordinates of the optimized feature map corresponding to each feature sample are encoded into a 2-dimensional vector and concatenated with the first feature matrix to obtain the second feature matrix. The spatial weights are normalized to 0-1 using the Sigmoid function and multiplied element-wise with the second feature matrix to output the second enhanced feature set, which contains the enhanced remaining ROP features and the enhanced candidate RON features.

[0036] The spatial location of all features in the first and second enhanced feature sets is related to the optimized feature. Figure 1 To ensure spatial consistency between feature stitching and image reconstruction, a row-stacked approach is used to sequentially stitch the features into a final feature map. Finally, a feedforward neural network is used to reconstruct the ROI image from the final feature map. The feedforward neural network is trained using a loss function, L(x,x'), which is expressed as follows: L(x,x')=d(x,x')+α×d(x_ROI,x'_ROI)+β×d(x_ROP,x'_ROP) Where x represents the original image; x' represents the reconstructed image; d() represents the mean squared error, used to calculate the difference in pixel values ​​between the two sets of images; x_ROI represents the ROI block in the original image; x'_ROI represents the ROI block in the reconstructed image; x_ROP represents the ROP block in the original image; x'_ROP represents the ROP block in the reconstructed image; α and β are hyperparameters that satisfy 0 < β < α < 1, so that the model will prioritize the quality of the ROI during training, then focus on the ROP, and finally the overall image.

[0037] 300 ROI images were subjected to HSV spatial transformation and image optimization to obtain optimized HSV images; In some embodiments, performing HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image includes: The ROI image is converted from RGB space to HSV space to obtain the initial HSV image, which includes the initial H channel image, the initial S channel image and the initial V channel image. The initial S-channel image is subjected to bilateral filtering and contrast linear stretching to obtain an optimized S-channel image. Logarithmic transformation, adaptive gamma correction, illuminance map calculation, and illuminance map merging are performed on the initial V-channel image to obtain an optimized V-channel image. The initial HSV image, the optimized S-channel image, and the optimized V-channel image are merged to obtain the optimized HSV image.

[0038] In some embodiments, the ROI image undergoes HSV (Hue, Saturation, Value) spatial transformation, and a likelihood-based bilateral filtering method is used to perform bilateral filtering on the initial S-channel image. Noise denoising and edge preservation of the S-channel image are achieved through a joint weight of spatial distance and likelihood difference. The joint weight w... i The formula is: Where G() is the spatial Gaussian kernel, used to measure the spatial distance weight between neighboring pixels and the center pixel; |ij| is the Euclidean distance between the neighboring pixel and the center pixel (i,j); α1 is the standard deviation of the spatial Gaussian kernel, with a value of 1.0~2.0; H() is the range Gaussian kernel, used to measure the likelihood difference weight between neighboring pixels and the center pixel; α2 is the standard deviation of the range Gaussian kernel, with a value of 0.5~1.0; The baseline integral representing the horizontal coordinate i of the center pixel; The baseline integral representing the ordinate j of the center pixel; y j Represents a given measured value of the center pixel's ordinate j; This represents the conditional likelihood function, used to characterize likelihood similarity.

[0039] The filtered S-channel image is linearly stretched to extend its pixel value distribution across the entire dynamic range, thereby further enhancing color contrast.

[0040] The initial V channel image contains the main brightness information of the image. Logarithmic transformation is used to stretch the pixel values, increasing image brightness. Following the principle of using small gamma values ​​to enhance brightness in dark areas and large gamma values ​​to suppress brightness in bright areas, the gamma values ​​are adjusted in blocks. First, the V channel image is evenly divided into multiple pixel blocks; then, the gamma value is calculated for each block, and finally, the correction is completed using the gamma values. Multi-scale illuminance maps are generated through Gaussian filtering at different scales to capture the light distribution in different ranges, generating illuminance maps I1 and I2. Illumination map I3; for example, after filtering a background pixel at three scales, I1=0.35, I2=0.38, and I3=0.40, corresponding to different ranges of illumination estimates; according to the principle of high weight for fine-grained scales and low weight for coarse-grained scales, the weights w1=0.5, w2=0.3, and w3=0.2 are set. The illumination map is merged by w1×I1+w2×I2+w3×I3, which not only preserves fine-grained details but also conforms to the global illumination trend, and finally obtains the optimized V channel image.

[0041] 400. A multi-scale pyramid network is constructed. Through the multi-scale pyramid network, multi-scale feature extraction, multi-scale feature fusion and pixel segmentation are performed on the optimized HSV image to obtain the initial water level mark region image. In some embodiments, the multi-scale pyramid network includes high-resolution branch units, medium-resolution branch units, low-resolution branch units, attention units, and a segmentation network. The step of using the multi-scale pyramid network to perform multi-scale feature extraction, multi-scale feature fusion, and pixel segmentation on the optimized HSV image to obtain an initial water level marker region image includes: Feature extraction of the optimized HSV image is performed using high-resolution branch units to obtain first-scale features; The optimized HSV image is downsampled by pooling once to obtain a pooled HSV image. The second-scale features are then extracted from the pooled HSV image using a medium-resolution branch unit. The optimized HSV image is downsampled twice by pooling to obtain a double-pooled HSV image. The feature of the double-pooled HSV image is extracted by low-resolution branch units to obtain the third-scale feature. The first-scale feature, the second-scale feature, and the third-scale feature are weighted and fused by an attention unit to obtain the fused feature. Based on the fusion features, the initial water level mark region image is obtained by segmenting the pixels of the water level mark in the water level mark image using a segmentation network.

[0042] In some embodiments, the input to the high-resolution branch unit is an optimized HSV image with a resolution of 1920×1080, which is assigned to a lightweight detection model with minimal parameters, such as the YOLOv8n model, which focuses on extracting local detail features of the water level markers.

[0043] The input to the medium-resolution branch unit is a medium-resolution, single-pooled HSV image obtained by downsampling the optimized HSV image once, with a resolution of 960×540. This image is assigned to a moderate detection model with a suitable number of parameters, such as the YOLOv8m model, to balance detail preservation and semantic extraction.

[0044] The low-resolution branch unit input is a low-resolution double-pooled HSV image obtained by double-pooling downsampling the optimized HSV image. The resolution is 480×270. This image is assigned to a detection model with a large number of parameters, such as the YOLOv8L model, which uses its powerful semantic understanding capabilities to extract global contextual information.

[0045] The attention unit calculates the importance weights of each channel through squeezing and activation operations. Squeezing refers to average pooling of the channel dimension of each feature map to obtain the channel mean, which reflects the global information of each channel. Activation refers to mapping the channel mean to a weight value of 0-1 through a two-layer fully connected network to strengthen important channels and suppress redundant channels. The three-scale features are weighted channel by channel according to the channel weight × feature value to obtain the fused features.

[0046] The segmentation network adopts an upsampling + convolution structure to restore the fused features into a pixel-level segmentation map that matches the original ROI image. The goal is to accurately separate the red water level marker from the background. After upsampling, the water level marker feature area in the fused features forms a high response value in the output image, while the background area has a lower response value. The output image is binarized, with a pixel value of 1 representing the water level marker and 0 representing the background, to obtain the initial water level marker area image.

[0047] Double range detection is performed on 500 pairs of initial water level mark region images to obtain optimized water level mark region images. Reflection removal is then performed on the optimized water level mark region images to obtain the final water level mark region images. In some embodiments, performing dual range detection on the initial water level mark region image to obtain an optimized water level mark region image includes: Obtain the HSV value of each pixel in the initial water level mark area image, set a first HSV value range for light-toned marks, a second HSV value range for dark-toned marks, and a third HSV value range for reflective areas. Binarize the pixels whose HSV values ​​are within the first HSV value range to obtain the first mask; The pixels with HSV values ​​within the second HSV value range are binarized to obtain the second mask; Binarize the pixels whose HSV values ​​fall within the third HSV value range to obtain the third mask; Perform a logical OR operation on the first mask, the second mask, and the third mask to obtain an optimized mask, and then convert the optimized mask into an optimized water level mark region image.

[0048] In some embodiments, the water level markers are either red or black, thus requiring dual-range detection of both red and black to adapt to variations in the red markings under different lighting conditions and improve detection robustness.

[0049] Taking red as an example, for bright red markings, a first HSV value range is set, including H value ∈ [0,10], S value ∈ [100,255], and V value ∈ [50,255]. For dark red reflective areas, a second HSV value range is set, including H value ∈ [0,15], S value ∈ [80,200], and V value ∈ [10,80]. For reflective areas, a third HSV value range is set, including H value ∈ [0,180], S value ∈ [0,120], and V value ∈ [180,255].

[0050] Traverse all coordinates of pixel value = 1 in the initial marked area, and determine whether its HSV value is within the first HSV value range. If it is, set it to 1; otherwise, set it to 0. Set all values ​​outside the initial marked area to 0 to generate the first mask.

[0051] Iterate through all coordinates of pixel value = 1 in the initial marked area, and determine whether its HSV value is within the second HSV value range. If it is, set it to 1; otherwise, set it to 0. Set all values ​​outside the initial marked area to 0 to generate the second mask.

[0052] Traverse all coordinates of pixel value = 1 in the initial marked area, and determine whether its HSV value is within the range of the third HSV value. If it is, set it to 1; otherwise, set it to 0. Set all values ​​outside the initial marked area to 0 to generate the third mask.

[0053] Perform a logical OR operation on the pixels at the same coordinate of the first mask, second mask, and third mask. If any mask pixel value is 1, the corresponding pixel value of the optimized mask is 1; if both masks are 0, then the value is 0. The binary image of the optimized mask is directly used as the optimized water level mark region image, and the pixel definition is the same as the initial image, with 1 being a valid mark and 0 being the background.

[0054] In some embodiments, the step of removing reflections from the optimized water level mark region image to obtain the final water level mark region image includes: Obtain the bottom water level mark of the optimized water level mark region image. Using the bottom water level mark as the boundary, extend upwards by a preset number of pixels to divide the optimized water level mark region image into an upper image and a lower image. Obtain the average HSV value of the pixels in the upper image. Calculate the HSV distance of each pixel in the image below based on the average HSV value; Remove pixels in the lower image whose HSV distance is greater than the second threshold to obtain the lower image after removing the reflection; The top image and the bottom image after removing the reflection are stitched together to form the final water level mark area image.

[0055] In some embodiments, the HSV information of the reflection area differs from the HSV information of the water gauge under natural lighting. Therefore, a reference area can be established using the HSV information under natural lighting: with the bottom edge of the detected bottom water gauge mark extended upwards by 10 pixels as the boundary, and the image above the boundary as the reference benchmark, its average HSV value is calculated, including the average H value, average S value and average V value.

[0056] The area below the boundary is where watermarks and reflections coexist. Reflection detection is performed in this area. For each pixel in the image below, its HSV distance to the reference baseline is calculated. The expression for the HSV distance D is: in, , and The H, S, and V values ​​of the pixel to be measured are... , and The average HSV value includes the average H value, average S value, and average V value; the second threshold is set to 0.3, and when the HSV distance exceeds 0.3, it is determined to be a reflection pixel.

[0057] 600. Retrieve the database and obtain the corresponding water level scale value from the database based on the final water gauge marked area image.

[0058] The database is the foundation for scale value matching. It is built using MySQL, and the core data table is the water level calibration table, which is specifically used to store the image coordinates, actual physical scale, color characteristics and other related data of the water gauge mark. The data needs to be calibrated through on-site measurement and synchronous image acquisition to ensure the accurate correspondence between the image and the physical quantity.

[0059] In some embodiments, the regions with all pixel values ​​of 1 in the final water level gauge marking area image are traversed, and the main scale is filtered according to its characteristics. Only the marks with three consecutive horizontal pixels of 1 and a normalized Euclidean distance of HSV value to calibration value less than 0.1 are retained as the main marks, and auxiliary scales with a width of 1 pixel and possible residual noise are excluded. The average HSV value of the main marks is calculated, and the scale values ​​with the mark type of main marks and a normalized Euclidean distance of HSV value to average HSV value less than 0.1 are selected from the database as the water level gauge scale values.

[0060] Reference Figure 2This invention provides a vision-based non-contact water level monitoring device 20 for bridges navigating waterways, used to implement a vision-based non-contact water level monitoring method for bridges navigating waterways. The device includes: The optimized feature map acquisition module 21 is used to acquire water level images of the target water-crossing bridge through a camera, construct a convolutional neural network, extract features from the water level images through the convolutional neural network to obtain an initial feature map; at the same time, the water level images are divided into image blocks to obtain image blocks; and the initial feature map is optimized through the image blocks to obtain an optimized feature map. The ROI image acquisition module 22 is used to construct an ROI image reconstruction network, which performs feature extraction, feature enhancement and image reconstruction on the optimized feature map to obtain the ROI image. The optimized HSV image acquisition module 23 is used to perform HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image; The initial water level gauge marked region image acquisition module 24 is used to construct a multi-scale pyramid network, and to perform multi-scale feature extraction, multi-scale feature fusion and pixel segmentation on the optimized HSV image through the multi-scale pyramid network to obtain the initial water level gauge marked region image. The final water level mark region image acquisition module 25 is used to perform dual range detection on the initial water level mark region image to obtain an optimized water level mark region image, and to remove the reflection from the optimized water level mark region image to obtain the final water level mark region image. The water level gauge scale value acquisition module 26 is used to acquire the database and obtain the corresponding water level gauge scale value from the database based on the final water level gauge marked area image.

[0061] This application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements a vision-based non-contact monitoring method for water level of bridges across waterways, as described above.

[0062] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0063] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0064] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the vision-based non-contact monitoring method for water levels on bridges navigating waterways, as described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not incorporated into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0065] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0066] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vision-based non-contact monitoring method for water levels on bridges navigating watercourses, characterized in that, include: The water level gauge images of the target bridge in the water are acquired by a camera. A convolutional neural network is constructed, and features are extracted from the water level gauge images by the convolutional neural network to obtain an initial feature map. At the same time, the water level gauge images are divided into image blocks to obtain image blocks. The initial feature map is then optimized by using the image blocks to obtain an optimized feature map. A ROI image reconstruction network is constructed. The ROI image reconstruction network is used to extract features, enhance features, and reconstruct images from optimized feature maps to obtain ROI images. Perform HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image; A multi-scale pyramid network is constructed, and multi-scale feature extraction, multi-scale feature fusion and pixel segmentation are performed on the optimized HSV image to obtain the initial water level mark region image. Dual range detection is performed on the initial water level mark region image to obtain an optimized water level mark region image. Reflection removal is then performed on the optimized water level mark region image to obtain the final water level mark region image. Retrieve from the database, and obtain the corresponding water level scale value from the database based on the final water gauge marked area image.

2. The vision-based non-contact monitoring method for water level on bridges crossing waterways according to claim 1, characterized in that, The process involves extracting features from the water level indicator image using a convolutional neural network to obtain an initial feature map; simultaneously, dividing the water level indicator image into image blocks to obtain image blocks; and optimizing the initial feature map using these image blocks to obtain an optimized feature map, including: The water level indicator image is input into a convolutional neural network for feature extraction and water level indicator recognition to obtain an initial feature map y. in and recognition box; The water level indicator image is divided into multiple image blocks, and a position index matrix is ​​constructed based on the position of the image blocks. Image blocks that are completely within the recognition box are labeled as ROI blocks, image blocks adjacent to ROI blocks are labeled as ROP blocks, and the remaining image blocks are labeled as RON blocks; In the location index matrix, the elements corresponding to the ROI block, ROP block, and RON block are assigned the first preset value, the second preset value, and the third preset value, respectively, to obtain the weight allocation matrix γ. e Where the first preset value > the second preset value > the third preset value; the weight allocation matrix γ is upsampled. e The size is increased to match the initial feature map y in The same applies, yielding the weight mask γ. m ; Weight mask γ m Transform into importance feature map γ f The importance feature map γ f With the initial feature map y in Add them together to obtain the optimized feature map.

3. The vision-based non-contact monitoring method for water level on bridges crossing waterways according to claim 2, characterized in that, The ROI image reconstruction network includes a Self-Attention network, a Spatial-Attention network, and a feedforward neural network. The process of extracting features, enhancing features, and reconstructing the image from the optimized feature map using the ROI image reconstruction network to obtain the ROI image includes: Extract ROI features from ROI blocks, ROP features from ROP blocks, and RON features from RON blocks from the optimized feature map. The ROP features corresponding to ROP blocks whose overlap with ROI blocks exceeds the first threshold are taken as candidate ROP features, and the RON features corresponding to RON blocks adjacent to ROP blocks are taken as candidate RON features. The ROI features and candidate ROP features are enhanced using a Self-Attention network to obtain the first enhanced feature set. The remaining ROP features and candidate RON features are enhanced using a Spatial-Attention network to obtain a second enhanced feature set. The first and second enhanced feature sets are concatenated to obtain the final feature map; The final feature map is input into a feedforward neural network for image reconstruction to obtain the ROI image.

4. The vision-based non-contact monitoring method for water level on bridges navigating waterways according to claim 1, characterized in that, The process of performing HSV spatial transformation and image optimization on the ROI image to obtain an optimized HSV image includes: The ROI image is converted from RGB space to HSV space to obtain the initial HSV image, which includes the initial H channel image, the initial S channel image and the initial V channel image. The initial S-channel image is subjected to bilateral filtering and contrast linear stretching to obtain an optimized S-channel image. Logarithmic transformation, adaptive gamma correction, illuminance map calculation, and illuminance map merging are performed on the initial V-channel image to obtain an optimized V-channel image. The initial HSV image, the optimized S-channel image, and the optimized V-channel image are merged to obtain the optimized HSV image.

5. The vision-based non-contact monitoring method for water level on bridges navigating waterways according to claim 1, characterized in that, The multi-scale pyramid network includes high-resolution branch units, medium-resolution branch units, low-resolution branch units, attention units, and a segmentation network. The process of using the multi-scale pyramid network to perform multi-scale feature extraction, multi-scale feature fusion, and pixel segmentation on the optimized HSV image to obtain an initial water level gauge marker region image includes: Feature extraction of the optimized HSV image is performed using high-resolution branch units to obtain first-scale features; The optimized HSV image is downsampled by pooling once to obtain a pooled HSV image. The second-scale features are then extracted from the pooled HSV image using a medium-resolution branch unit. The optimized HSV image is downsampled twice by pooling to obtain a double-pooled HSV image. The feature of the double-pooled HSV image is extracted by low-resolution branch units to obtain the third-scale feature. The first-scale feature, the second-scale feature, and the third-scale feature are weighted and fused by an attention unit to obtain the fused feature. Based on the fusion features, the initial water level mark region image is obtained by segmenting the pixels of the water level mark in the water level mark image using a segmentation network.

6. The vision-based non-contact monitoring method for water level on bridges navigating waterways according to claim 1, characterized in that, The step of performing dual range detection on the initial water level mark region image to obtain an optimized water level mark region image includes: Obtain the HSV value of each pixel in the initial water level mark area image, set a first HSV value range for light-toned marks, a second HSV value range for dark-toned marks, and a third HSV value range for reflective areas. Binarize the pixels whose HSV values ​​are within the first HSV value range to obtain the first mask; The pixels with HSV values ​​within the second HSV value range are binarized to obtain the second mask; Binarize the pixels whose HSV values ​​fall within the third HSV value range to obtain the third mask; Perform a logical OR operation on the first mask, the second mask, and the third mask to obtain an optimized mask, and then convert the optimized mask into an optimized water level mark region image.

7. The vision-based non-contact monitoring method for water level on bridges crossing waterways according to claim 1, characterized in that, The step of removing reflections from the optimized water level mark region image to obtain the final water level mark region image includes: Obtain the bottom water level mark of the optimized water level mark region image. Using the bottom water level mark as the boundary, extend upwards by a preset number of pixels to divide the optimized water level mark region image into an upper image and a lower image. Obtain the average HSV value of the pixels in the upper image. Calculate the HSV distance of each pixel in the image below based on the average HSV value; Remove pixels in the lower image whose HSV distance is greater than the second threshold to obtain the lower image after removing the reflection; The top image and the bottom image after removing the reflection are stitched together to form the final water level mark area image.

8. A vision-based non-contact water level monitoring device for bridges navigating waterways, used to implement the vision-based non-contact water level monitoring method for bridges navigating waterways as described in any one of claims 1 to 7, characterized in that, The device includes: The optimized feature map acquisition module is used to acquire water level images of the target bridge across the water using a camera, construct a convolutional neural network, extract features from the water level images using the convolutional neural network to obtain an initial feature map; at the same time, the water level images are divided into image blocks to obtain image blocks; and the initial feature map is optimized using image blocks to obtain an optimized feature map. The ROI image acquisition module is used to construct an ROI image reconstruction network. The ROI image reconstruction network performs feature extraction, feature enhancement, and image reconstruction on the optimized feature map to obtain the ROI image. The HSV image acquisition module is optimized to perform HSV spatial transformation and image optimization on ROI images to obtain optimized HSV images. The initial water level gauge marked region image acquisition module is used to construct a multi-scale pyramid network. Through the multi-scale pyramid network, multi-scale feature extraction, multi-scale feature fusion and pixel segmentation are performed on the optimized HSV image to obtain the initial water level gauge marked region image. The final water level mark region image acquisition module is used to perform dual range detection on the initial water level mark region image to obtain an optimized water level mark region image, and to remove the reflection from the optimized water level mark region image to obtain the final water level mark region image. The water level gauge value acquisition module is used to retrieve the corresponding water level gauge value from the database based on the final water level gauge marked area image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the vision-based non-contact monitoring method for water level of bridges across water as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the vision-based non-contact monitoring method for water level of bridges across water as described in any one of claims 1 to 7.

Images