An ice slurry identification method based on a double-branch visual semantic segmentation network

Abstract

The application discloses an ice slurry identification method based on a double-branch visual semantic segmentation network, relates to the technical field of civil and water conservancy safety early warning, and aims at the low accuracy of the existing ice slurry identification method. The main segmentation head and the edge detection head respectively output semantic segmentation prediction and edge prediction consistent with the resolution of an input image, the result of the main detection branch is taken as the final ice slurry identification result, and the result of the edge detection branch is taken as auxiliary training, so that the model is forced to learn the target boundary details, the problem of boundary blurring is effectively relieved, and the accuracy of ice slurry identification is improved.

Description

Technical Field

[0001] This application relates to the field of civil engineering and water conservancy safety early warning technology, specifically to an ice shard recognition method based on a dual-branch visual semantic segmentation network. Background Technology

[0002] Embankment engineering is a crucial construction project for protecting water-adjacent roads and even vital border defense engineering. However, due to the cold winters in northern regions, rivers are prone to freezing, and large amounts of ice often form during the freezing and thawing periods, impacting embankments and compromising their structural stability and functionality. Therefore, timely and effective monitoring of ice is of great significance. However, existing methods cannot accurately identify different stages of ice conditions, such as no ice, little ice, dense floating ice, localized ice jams, and severe blockages, resulting in low accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide an ice crystal recognition method based on a dual-branch visual semantic segmentation network, addressing the problem of low accuracy in existing ice crystal recognition methods.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] An ice crystal recognition method based on a dual-branch visual semantic segmentation network includes:

[0006] The image of the ice crystal to be identified is acquired and input into the ConvNeXt–FPN network to obtain the segmentation result. The ConvNeXt–FPN network specifically performs the following steps:

[0007] The input image is first processed by the backbone encoder to extract multi-scale features, and then fused by the feature pyramid decoder. The fused features are simultaneously fed into two detection branches, which contain the main segmentation head and the edge detection head respectively. The main segmentation head and the edge detection head are each processed by convolution and upsampling to output semantic segmentation prediction and edge prediction with the same resolution as the input image.

[0008] The edge detection head uses the real edge map extracted by the Sobel operator from the real mask as supervision to assist the network learning during the training process.

[0009] Furthermore, the fused features are obtained through the following steps:

[0010] The input image is first processed by Stage 1 for feature extraction to obtain feature map A;

[0011] Feature map A is input into Stage 2 for feature extraction, resulting in feature map B;

[0012] Feature map B is input into Stage 3 for feature extraction, resulting in feature map C;

[0013] Feature map C is input into Stage 4 for feature extraction, resulting in feature map D;

[0014] Feature map A is convolved to obtain feature map P4;

[0015] Feature map P4 is upsampled by 2 times to obtain feature map PA;

[0016] Feature map B is convolved and then fused with feature map PA to obtain feature map P3;

[0017] Feature map P3 is upsampled by 2 times to obtain feature map PB;

[0018] After convolution, feature map C is fused with feature map B and feature map P4B to obtain feature map P2;

[0019] Feature map P2 is upsampled by 2 times to obtain feature map PC;

[0020] After feature map P4 is fused with feature map D, it is then convolved to obtain feature map PD;

[0021] Feature map PD is fused with feature map PC and feature map A to obtain an enhanced feature map.

[0022] Furthermore, the main segmentation head includes a convolutional block, a Dropout layer, and a convolutional layer, wherein the convolutional block includes a 3x3 convolution, batch normalization, and a GELU activation function;

[0023] The main segmentation head extracts features using convolutional blocks, then uses Dropout layers to enhance the model's generalization ability and prevent overfitting. The number of feature channels is compressed to the number of target categories through convolutional layers, outputting a Logits map. Finally, it is restored to the original resolution through 4× bilinear upsampling, outputting pixel-by-pixel semantic segmentation predictions.

[0024] Furthermore, the edge detection head specifically performs the following steps:

[0025] First, obtain the edge Logits, represented as:

[0026] ,

[0027] in, This indicates a 3x3 convolution operation with 128 output channels. This represents a 1x1 convolution operation with an output channel of 1. For batch normalization, For GELU activation function, The output edge logits are then restored to the original resolution through 4× bilinear upsampling, and the pixel-by-pixel edge prediction is output.

[0028] Furthermore, the edge detection head uses the real edge map extracted from the real mask by the Sobel operator as supervision, and its supervision label... It is obtained through the following steps:

[0029] Step 1: Obtain the segmentation truth mask The mask pixels satisfy:

[0030] ,

[0031] Where H is the image height, W is the image width, and B is the batch size. For mask pixel values, For batch indexing, =1 is the channel index. These are pixel space coordinates;

[0032] Step 2: Construct the horizontal gradient kernel and vertical gradient kernel , represented as:

[0033] ,

[0034] ,

[0035] in, This represents the grayscale variation of a pixel in the horizontal direction. This represents the grayscale variation of a pixel in the vertical direction.

[0036] Step 3: Based on horizontal gradient kernel and vertical gradient kernel By performing a two-dimensional convolution operation between the Sobel kernel and the segmentation ground truth mask M, the horizontal gradient map and the vertical gradient map are obtained, as follows:

[0037] ,

[0038] ,

[0039] in, For horizontal gradient plots in Gradient value at position, For the vertical gradient map in Gradient value at position, This represents the offset in the vertical direction. This represents the horizontal offset.

[0040] Step 4: Merge the horizontal gradient map and the vertical gradient map to obtain the edge intensity of the pixel. , is represented as:

[0041] ,

[0042] in, It is the minimum value;

[0043] Step 5: Based on the threshold T, Convert to binary marginal truth value , is represented as:

[0044] .

[0045] Furthermore, the loss function of the ConvNeXt–FPN network is expressed as:

[0046]

[0047] in, The total loss of the ConvNeXt–FPN network, , and The weights of each loss term, Main segmentation cross-entropy loss, For edge binary cross-entropy loss, The main segmentation is Dice loss.

[0048] Furthermore, the main segmentation cross-entropy loss Represented as:

[0049] ,

[0050] in, Total number of pixels For the number of categories, For the first 1 pixel, The total number of categories, For the first Categories The i-th pixel belongs to the category The true label, After the main segmentation header is output and normalized by Softmax, the... Each pixel belongs to the category The predicted probability.

[0051] Furthermore, the main segmentation Dice loss Represented as:

[0052] ,

[0053] in, After the main segmentation header is output and normalized by Softmax, the... The predicted probability that a pixel belongs to category 1. For the first Each pixel belongs to the true label of category 1. This is a smoothing term.

[0054] Furthermore, the edge binary cross-entropy loss Represented as:

[0055] ,

[0056] in, For the first 1 pixel in the edge label The values ​​in the table are 1 for edge and 0 for non-edge. Output feature map for edge head In the The original Logit value at each pixel. This is the Sigmoid function.

[0057] Furthermore, the backbone encoder is a pre-trained ConvNeXt-Tiny.

[0058] The beneficial effects of this invention are:

[0059] This application outputs semantic segmentation predictions and edge predictions with the same resolution as the input image through a main segmentation head and an edge detection head, respectively. The result of the main detection branch is used as the final result of ice crystal recognition, while the result of the edge detection branch is used as auxiliary training. This forces the model to learn the details of the target boundary, effectively alleviates the problem of boundary blurring, and thus improves the accuracy of ice crystal recognition. Attached Figure Description

[0060] Figure 1 This is the overall flowchart of this application;

[0061] Figure 2 This is a structural diagram of the model in this application;

[0062] Figure 3 This is a schematic diagram illustrating the changes in the loss function values ​​of the three components during the training phase in an embodiment of this application.

[0063] Figure 4 This is a schematic diagram of the ice crystal recognition results in an embodiment of this application. Detailed Implementation

[0064] It should be noted that, where there is no conflict, the various embodiments disclosed in this application can be combined with each other.

[0065] Specific Implementation Method 1: The ice crystal recognition method based on a dual-branch visual semantic segmentation network described in this implementation method includes:

[0066] The image of the ice crystal to be identified is acquired and input into the ConvNeXt–FPN network to obtain the segmentation result. The ConvNeXt–FPN network specifically performs the following steps:

[0067] The input image is first processed by the backbone encoder to extract multi-scale features, and then fused by the feature pyramid decoder. The fused features are simultaneously fed into two detection branches, which contain the main segmentation head and the edge detection head respectively. The main segmentation head and the edge detection head are each processed by convolution and upsampling to output semantic segmentation prediction and edge prediction with the same resolution as the input image.

[0068] The edge detection head uses the real edge map extracted by the Sobel operator from the real mask as supervision to assist the network learning during the training process.

[0069] The ConvNeXt–FPN network is trained through the following steps:

[0070] Step 1: Collect and preprocess video data to construct a semantic segmentation dataset;

[0071] Step 2: Construct a dual-branch visual semantic segmentation network based on ConvNeXt–FPN;

[0072] Step 3: Train and validate the visual semantic segmentation model;

[0073] Collect raw visual data and preprocess it to construct a semantic segmentation dataset;

[0074] First, key locations along the target embankment project should be monitored by deploying surveillance cameras or conducting regular manual inspections to collect image or video data covering multiple time periods, weather conditions, and lighting conditions. This will yield data on ice conditions under various scenarios, including strong reflections in clear weather, low contrast in cloudy weather, backlighting at dawn and dusk, and low visibility due to snow and fog. To ensure data representativeness, it is best to include different ice condition stages, such as no ice, little ice, dense floating ice, localized ice jams, and severe blockages, while also preserving the actual distribution of targets at different scales, such as small ice fragments, large ice floes, and continuous ice belts.

[0075] Secondly, the collected video and image data are cleaned and randomly sampled to obtain high-quality and clear image information. Then, a semantic segmentation training set is constructed using pixel-level annotation, that is, labelme is used to label the data, focusing on labeling the "ice" category, but background categories such as "water" and "stone" can also be labeled, thereby obtaining feature point files in JSON format; then, based on these JSON files, an integer mask image of the original image is generated, thus completing the initial construction of the dataset.

[0076] Finally, data augmentation and normalization are performed on the original image and the corresponding mask image. Each image undergoes random horizontal flipping, random vertical flipping, random rotation, random brightness adjustment, and size unification. For random rotation, bilinear interpolation is used for the original image to ensure visual continuity; nearest neighbor interpolation is used for the corresponding mask image to ensure that class values ​​are not corrupted by interpolation. Random brightness adjustment is only applied to the original image. These processes enhance the model's robustness to natural disturbances, enabling the network to adapt to slight changes in camera viewpoint and texture disturbances caused by wind and waves, thus improving its adaptability to the complex environment of real river channels from a data perspective.

[0077] Construct a dual-branch visual semantic segmentation model based on ConvNeXt–FPN;

[0078] This application proposes a dual-branch semantic segmentation model based on ConvNeXt–FPN. The network architecture of this model adopts an efficient encoder-decoder design and incorporates a multi-task learning mechanism to improve the performance of binary semantic segmentation. The input image is first processed by the backbone encoder to extract multi-scale features, which are then fused by the feature pyramid decoder. The fused high-resolution features are simultaneously fed into two parallel detection branches—one containing the main segmentation head and the other containing the auxiliary edge detection head. Finally, the output is upsampled to restore the original image size.

[0079] The feature extraction part of the network uses a pre-trained ConvNeXt-Tiny as the backbone encoder. This encoder consists of four stages, as shown in the figure (Stage 1 to Stage 4), which are responsible for progressively transforming the input image into an abstract feature representation. As the network depth increases, the spatial resolution of the feature maps decreases step by step, downsampling from H / 4 in Stage 1 to H / 32 in Stage 4, while the channel dimension increases accordingly, expanding from 96 to 768. This process effectively captures multi-scale information from low-level texture to high-level semantics, providing rich basic features for the subsequent decoding process.

[0080] To effectively fuse multi-scale features generated by the encoder and recover spatial details, the network integrates a Feature Pyramid Network (FPN) as a decoder. The FPN first uses 1x1 convolutions for lateral connections, uniformly mapping feature maps from different stages of the backbone network to a 256-dimensional channel space. Then, it employs a top-down path, performing a 2x bilinear upsampling of the high-level feature maps at layer P4 and fusing them element-wise with the laterally processed feature maps from the previous stage. This mechanism successfully transfers strong semantic information from higher layers to high-resolution features at lower layers. Finally, the FPN outputs a semantically rich P1 layer feature map with H / 4 spatial resolution for use by subsequent prediction heads.

[0081] The main segmentation prediction head directly receives features from the highest-resolution P1 layer of the FPN, undertaking the primary semantic segmentation task. This module's design begins with a convolutional block consisting of 3x3 convolutions, batch normalization (BN), and the GELU activation function for further feature extraction. A Dropout layer is then introduced to enhance the model's generalization ability and prevent overfitting. Finally, a 1x1 convolutional layer compresses the number of feature channels to the number of target classes, outputting a Logits map for final classification. Then, 4×bilinear upsampling restores the original resolution, outputting pixel-by-pixel semantic segmentation predictions. During training, this branch is supervised by the ground truth mask (GT Mask) and employs a combination of cross-entropy and Dice loss to optimize classification accuracy while improving region overlap and mitigating the imbalanced sample problem.

[0082] An auxiliary edge detection head is introduced in parallel in the architecture. Let the high-resolution feature map output by the FPN decoder be... The output of the edge detection head The calculation process is as follows:

[0083] ,

[0084] In the formula, This indicates a 3x3 convolution operation with 128 output channels; Indicates batch normalization; This represents the GELU activation function; This represents a 1x1 convolution operation with an output channel of 1; This is the edge Logits of the output.

[0085] This module also operates based on the P1 layer features of FPN. It includes an independent convolutional block that reduces the number of channels to 128 and a final 1x1 convolutional layer, outputting a single-channel edge probability Logits map. Then, it restores the original resolution through 4×bilinear upsampling, outputting pixel-by-pixel edge predictions. During training, this branch is supervised by a ground truth edge map dynamically generated from the ground truth mask using the Sobel operator. It employs binary cross-entropy loss, explicitly learning fine-grained features of the target boundary while providing geometric constraints for the main segmentation task, thus improving the edge sharpness of the segmentation results.

[0086] The main segmentation head is constrained by the cross-entropy loss (CELoss) and Dice loss calculated based on the true mask (GTMask), where the main segmentation cross-entropy loss... :

[0087] ,

[0088] In the formula, N is the total number of pixels involved in the calculation; M is the number of classification categories, which is 2 in this application; For the first Each pixel belongs to the category The real label is 0 or 1, where 0 represents the background and 1 represents icicles; After the main segmentation header is output and normalized by Softmax, the... Each pixel belongs to the category The predicted probability.

[0089] Main segmentation Dice loss :

[0090] ,

[0091] In the formula, N is the total number of pixels involved in the calculation;

[0092] Meanwhile, in order to supervise the edge header, the system uses the Sobel operator to extract edge labels from GTMas in real time during training and calculates the binary cross-entropy loss (Edge BCE Loss). After the main segmentation header is output and normalized by Softmax, the... The predicted probability that a pixel belongs to category 1; For the first Each pixel belongs to the true label of category 1; To smooth out the terms and prevent numerical instability caused by a denominator of 0, the value in this application is 0.000001;

[0093] The improvements to the decoder-encoder architecture mainly consist of three parts:

[0094] First, a cross-stage high-resolution detail injection strategy is introduced. The output features of Backbone Stage 1, with a spatial resolution of H / 4 and 96 channels, are first aligned to 256 channels using a 1×1 convolution, making them consistent with the channel number of Stage 4 after convolution and the P2 features. Then, a three-branch element-wise summation operation is performed together with the features of Stage 4 after a 1×1 convolution (which needs to be upsampled to H / 4 by a 1×1 convolution to match the spatial resolution) and the features of P2 after a 2x upsampling, finally generating an enhanced version of the P1 feature map. This operation, while preserving the strong semantic information at the top level, directly injects high-resolution textures and edge details from the bottom level, achieving deep fusion of semantic information and spatial details at the highest level.

[0095] Meanwhile, to address the issue that P4 in the original structure only serves as the upstream semantic input generated by P3, a semantic residual connection with the same resolution is performed. Leveraging the characteristic that the output features of BackboneStage4 have the same spatial resolution as the feature map of P4, after P4 is generated by a 1×1 convolution, an element-wise residual summation operation is performed between it and the original output features of Stage4. To ensure channel dimension matching, the 768-dimensional channels of the Stage4 output are first aligned to the number of channels in P4 using a 1×1 convolution. This operation constructs a direct information pathway from the highest semantic layer to the top layer of the FPN, strengthening the expression of global contextual features. Simultaneously, the residual form alleviates the information dilution problem during convolution and upsampling, improving the robustness of high-level semantic features.

[0096] Finally, a mesoscale local structure guidance strategy is introduced. The output features of BackboneStage2 (spatial resolution H / 8, 192 channels) are first convolved with 1×1 to match the number of channels after convolution with Stage3 and upsampling with P3. Then, a three-branch element-wise summation operation is performed on the features from Stage3 after 1×1 convolution and the features from P3 after 2x upsampling to generate an enhanced P2 feature map. This operation directly introduces the medium receptive field information from Stage2 at the mesoscale feature level, enhancing the modeling ability of medium-sized target contours and local structures. It fills the gap in the interaction between mesoscale details and semantic information in the original FPN, providing richer structural priors for subsequent P1 generation and edge heads.

[0097] To supervise edge detection, the system extracts edge labels from GTMas in real-time during training using the Sobel operator and calculates the binary cross-entropy loss (Edge BCE Loss). The Sobel operator calculates the gradient values ​​of image pixels in the horizontal and vertical directions through convolution operations; the magnitude of the gradient reflects the edge strength at the pixel. The input is a ground-value mask for segmentation provided by the dataset. (B is the batch size, H / W is the image width, and the number of channels is 1), the mask pixel values ​​satisfy:

[0098] ,

[0099] in For batch indexing, c=1 is for channel indexing. These are pixel space coordinates.

[0100] Constructing horizontal gradient kernels and vertical gradient kernel :

[0101] ,

[0102] In the formula, Used to detect vertical edges, that is, the grayscale changes of pixels in the horizontal direction. Used to detect horizontal edges, that is, the grayscale changes of pixels in the vertical direction.

[0103] Perform a two-dimensional convolution operation between the Sobel kernel and the segmentation ground truth mask M to obtain the horizontal gradient map. and vertical gradient plot The convolution process is as follows:

[0104] ,

[0105] ,

[0106] In the formula, where For batch indexing, =1 is the channel index. These are pixel space coordinates; For horizontal gradient plots in Gradient value at position, For the vertical gradient map in Gradient value at position; final output It is exactly the same size as the input mask. Represents the offset in the vertical direction (height dimension). Represents the offset in the horizontal direction (width dimension), and both values ​​range from {-1, 0, +1}, corresponding to the 9 pixel positions of the 3×3 Sobel convolution kernel.

[0107] Then, the horizontal and vertical gradients are merged, and the edge intensity of each pixel is calculated.

[0108] ,

[0109] in To minimize the value, avoid numerical calculation errors caused by the square root being 0; This is a gradient magnitude map; the larger the value, the higher the probability that the pixel is an edge.

[0110] Convert continuous gradient magnitude maps into binary edge ground truth maps. This is achieved by setting a threshold T=0.8:

[0111] ,

[0112] final Only 1 at the foreground boundary, 0 at all other locations, exactly the same size as the segmentation ground truth mask, serving as the supervision label for EdgeHead. Then The supervision labels used as inputs to EdgeHead are fed into the Edge BCE Loss module, and the loss is calculated together with the Final Edge Prediction (edge ​​probability map) output by EdgeHead.

[0113] Edge binary cross-entropy loss :

[0114] ,

[0115] In the formula, N is the total number of pixels involved in the calculation; For the i-th pixel at the edge label The values ​​in the range are 1 for edge and 0 for non-edge; Output feature map for edge head The original Logit value at the i-th pixel; This is the Sigmoid function.

[0116] The edge loss is then weighted and fused with the CE+DiceLoss of the segmentation branch to form the total loss. The parameters of Backbone, FPN, segmentation head and edge head are updated synchronously through backpropagation. While optimizing the region segmentation accuracy, the model is forced to learn the details of the target boundary, which effectively alleviates the problem of blurred boundaries in traditional segmentation networks and improves the edge sharpness and localization accuracy of the segmentation results.

[0117] The total loss function is then:

[0118] ,

[0119] In the formula, , and As for the weights of each loss term, considering that the ice image itself has the characteristics of uneven distribution and blurred edges, it is necessary to prioritize ensuring the recall rate of small targets in the ice to alleviate class imbalance, then ensure the overall classification accuracy to avoid background misjudgment, and finally enhance the edge details of the ice to make the outline clearer. Therefore, the above three undetermined parameters are set to 1, 0.6, and 0.3 respectively.

[0120] This combined loss function ensures that the network can accurately fit the target contour while optimizing the overall region segmentation.

[0121] Training and validation of the visual semantic segmentation model;

[0122] A multi-objective joint optimization training strategy is adopted, enabling the model to simultaneously learn ice region segmentation and boundary localization. The training loss consists of three parts: cross-entropy loss for pixel-level classification supervision, Dice loss to improve the overlap quality of foreground regions and alleviate the class imbalance problem caused by fluctuations in foreground proportion, and BCE loss to constrain edge prediction in the boundary branch. The boundary supervision signal is automatically generated from the real mask using the Sobel operator, thus obtaining stable contour supervision without the need for additional labeled edge data, which is beneficial for quickly building a training system usable in engineering.

[0123] To ensure training stability and reproducibility, this method employs the AdamW optimizer and sets appropriate weight decay to suppress overfitting. Reproducibility is further enhanced by fixing the random seed and limiting the nondeterminism of cuDNN. A phased training strategy is recommended, initially using a smaller learning rate to stabilize convergence, followed by gradual fine-tuning of the backbone to adapt to the river scene's domain characteristics. During the validation phase, metrics such as mIoU, F1, Precision, Recall, and pixel accuracy are continuously output. The confusion matrix is ​​then used to diagnose the main sources of false positives (FP) and false negatives (FN), providing a basis for setting the credibility threshold for subsequent warning logic.

[0124] Model hardware deployment and river ice monitoring;

[0125] On-site, key locations need to be selected to deploy fixed cameras, edge computing boxes, and communication and power supply equipment. The front-end cameras should be low-temperature capable and provide stable nighttime imaging, installed on the embankment slope with lenses pointing towards the main river channel. The edge computing boxes should be installed near the power distribution box or solar panel, using an industrial-grade sealed enclosure, integrating a GPU inference module and local storage for all-weather real-time inference and offline caching. The communication link should use a 4G / 5G industrial router directly connected to the edge computing boxes. Solar power should ideally be used on the power supply side to ensure continuous monitoring data delivery.

[0126] The system continuously receives camera video via RTSP streaming, with frames extracted at a fixed frequency from the edge into the inference pipeline. Each frame undergoes ROI cropping, retaining only the effective water surface area of ​​the river channel while removing the sky, bridge structures, and irrelevant bank slopes, reducing false detection probability and improving inference efficiency from the source. The resulting image is then fed into the trained network to obtain an ice segmentation map. The system further converts the segmentation results into structured monitoring indicators, including ice coverage and maximum connected component area, forming a continuous time-series curve. A summary result is output for each time window, uploaded to the backend platform in real-time via the network, and simultaneously saved locally to ensure traceability and replayability.

[0127] Ice storm severity classification and ice storm warning.

[0128] First, the ROI of the monitoring section is defined as the effective water surface area of ​​the river channel. For each frame segmentation result, the ice coverage rate R(t) is calculated, which is the proportion of ice pixels to the total pixels of the ROI, used to characterize the overall ice intrusion intensity. Simultaneously, the maximum connected component proportion Amax(t) is calculated, which is the proportion of the area of ​​the largest ice-connected region in the current frame to the ROI, used to characterize whether large ice masses or accumulation trends are occurring. Further, a fixed window area is set in the key region, and the key region coverage rate Rpier(t) is calculated to capture the most critical risks of ice jamming, ice accumulation, and buildup near the key region. All three indicators can be calculated in real time at the edge and output as sliding statistics in minutes, thus ensuring that the indicators are insensitive to instantaneous noise and possess engineering stability.

[0129] Based on the above monitoring indicators, the severity of ice storms can be classified into different levels, as shown below:

[0130]

[0131] In the formula, a~l are all percentages, and .

[0132] When the edge computing box reaches Level II or above, it automatically generates early warning event information, including: the original image of the current frame, the segmented overlay image, the boundary response image, and four core indicators and level results. This event information is then uploaded to the management platform in real time to form an event log. The platform displays the early warning level change curve in a timeline format and pushes early warning information when the level escalates. This allows managers to quickly understand the source and severity of the risk without reviewing the original video, thereby improving response efficiency and reducing misjudgments.

[0133] Example

[0134] Ice crystal recognition dataset was created by collecting three consecutive days of video data on ice formations from a dike project in a region of Heilongjiang Province, followed by data annotation and preprocessing. A two-branch semantic segmentation network based on ConvNeXt–FPN was constructed and trained, with the results shown below. Figure 3 , 4 As shown.

[0135] Based on the severity of ice damage in the local area, the ice disaster situation has been classified into levels, and the results are as follows:

[0136]

[0137] The result obtained by the edge computing box in one instance was "1 camera monitoring point, ice coverage R(t)=38%, maximum connected domain ratio Amax(t)=14%, risk level III, showing a continuous upward trend". Then, the event information was uploaded to the management platform in real time through the 5G industrial router to form an event log. After viewing the information, the managers quickly understood the source and severity of the risk, communicated with the local area and carried out corresponding engineering measures to reduce the harm of ice.

[0138] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solution of the present invention and should not be used to limit the scope of protection. Any modifications made in accordance with the claims and specification that are only partial should still fall within the protection scope of the present invention.

Claims

1. An ice crystal recognition method based on a dual-branch visual semantic segmentation network, characterized in that... include: The image of the ice crystal to be identified is acquired and input into the ConvNeXt–FPN network to obtain the segmentation result. The ConvNeXt–FPN network specifically performs the following steps: The input image is first processed by the backbone encoder to extract multi-scale features, and then fused by the feature pyramid decoder. The fused features are simultaneously fed into two detection branches, which contain the main segmentation head and the edge detection head respectively. The main segmentation head and the edge detection head are each processed by convolution and upsampling to output semantic segmentation prediction and edge prediction with the same resolution as the input image. The edge detection head uses the real edge map extracted by the Sobel operator from the real mask as supervision to assist the network learning during the training process.

2. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 1, characterized in that... The fused features are obtained through the following steps: The input image is first processed by Stage 1 for feature extraction to obtain feature map A; Feature map A is input into Stage 2 for feature extraction, resulting in feature map B; Feature map B is input into Stage 3 for feature extraction, resulting in feature map C; Feature map C is input into Stage 4 for feature extraction, resulting in feature map D; Feature map A is convolved to obtain feature map P4; Feature map P4 is upsampled by 2 times to obtain feature map PA; Feature map B is convolved and then fused with feature map PA to obtain feature map P3; Feature map P3 is upsampled by 2 times to obtain feature map PB; After convolution, feature map C is fused with feature map B and feature map P4B to obtain feature map P2; Feature map P2 is upsampled by 2 times to obtain feature map PC; After feature map P4 is fused with feature map D, it is then convolved to obtain feature map PD; Feature map PD is fused with feature map PC and feature map A to obtain an enhanced feature map.

3. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 2, characterized in that... The main segmentation head includes a convolutional block, a Dropout layer, and a convolutional layer. The convolutional block includes a 3x3 convolution, batch normalization, and a GELU activation function. The main segmentation head extracts features using convolutional blocks, then uses Dropout layers to enhance the model's generalization ability and prevent overfitting. The number of feature channels is compressed to the number of target categories through convolutional layers, outputting a Logits map. Finally, it is restored to the original resolution through 4× bilinear upsampling, outputting pixel-by-pixel semantic segmentation predictions.

4. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 3, characterized in that... The edge detection head specifically performs the following steps: First, obtain the edge Logits, represented as: ， in, This indicates a 3x3 convolution operation with 128 output channels. This represents a 1x1 convolution operation with an output channel of 1. For batch normalization, For GELU activation function, The output edge logits are then restored to the original resolution through 4× bilinear upsampling, and the pixel-by-pixel edge prediction is output.

5. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 4, characterized in that... The edge detection head uses the real edge map extracted by the Sobel operator from the real mask as supervision, and its supervision label is... It is obtained through the following steps: Step 1: Obtain the segmentation truth mask The mask pixels satisfy: ， Where H is the image height, W is the image width, and B is the batch size. For mask pixel values, For batch indexing, =1 is the channel index. These are pixel space coordinates; Step 2: Construct the horizontal gradient kernel and vertical gradient kernel , is represented as: ， ， in, This represents the grayscale variation of a pixel in the horizontal direction. This represents the grayscale variation of a pixel in the vertical direction. Step 3: Based on horizontal gradient kernel and vertical gradient kernel By performing a two-dimensional convolution operation between the Sobel kernel and the segmentation ground truth mask M, the horizontal gradient map and the vertical gradient map are obtained, as follows: ， ， in, For horizontal gradient plots in Gradient value at position, For the vertical gradient map in Gradient value at position, This represents the offset in the vertical direction. This represents the horizontal offset. Step 4: Merge the horizontal gradient map and the vertical gradient map to obtain the edge intensity of the pixel. , is represented as: ， in, It is the minimum value; Step 5: Based on the threshold T, Convert to binary marginal truth value , is represented as: 。 6. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 5, characterized in that... The loss function of the ConvNeXt–FPN network is expressed as: in, The total loss of the ConvNeXt–FPN network, , and The weights of each loss term, Main segmentation cross-entropy loss, For edge binary cross-entropy loss, The main segmentation is Dice loss.

7. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 6, characterized in that... The main segmentation cross-entropy loss Represented as: ， in, Total number of pixels For the number of categories, For the first 1 pixel, The total number of categories, For the first Categories The i-th pixel belongs to the category The true label, After the main segmentation header is output and normalized by Softmax, the... Each pixel belongs to the category The predicted probability.

8. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 7, characterized in that... The main segmentation Dice loss Represented as: ， in, After the main segmentation header is output and normalized by Softmax, the... The predicted probability that a pixel belongs to category 1. For the first Each pixel belongs to the true label of category 1. This is a smoothing term.

9. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 8, characterized in that... The edge binary cross-entropy loss Represented as: ， in, For the first 1 pixel in the edge label The values ​​in the table are 1 for edge and 0 for non-edge. Output feature map for edge head In the The original Logit value at each pixel. This is the Sigmoid function.

10. The ice crystal recognition method based on a dual-branch visual semantic segmentation network according to claim 1, characterized in that... The backbone encoder is a pre-trained ConvNeXt-Tiny.

Images