A method and system for identifying exposed features of a termite in a water conservancy project

Abstract

This invention relates to the field of water conservancy project safety inspection and termite control technology, and particularly to a method and system for identifying termite exposure features in water conservancy projects. The method includes the following steps: acquiring RGB and multispectral images of the area to be detected, and performing multimodal alignment and background suppression processing to obtain a background-suppressed RGB image; using a small target detection network to perform hierarchical detection on the background-suppressed RGB image to obtain small target bounding boxes; using a semantic segmentation network to segment the verified and filtered small target bounding boxes to obtain a small target segmentation map; using a temporal correlation method to verify the small target segmentation map; and determining the hazard level based on the feature type, affected area, and number of affected locations of the termite exposure features, generating a detection report containing the segmentation results and the hazard level.

Description

Technical Field

[0001] This invention relates to the field of safety inspection and termite control technology for water conservancy projects, and in particular to a method and system for identifying exposed termite features in water conservancy projects. Background Technology

[0002] In water conservancy projects, termite exposure features (such as swarming holes, mud lines, and termite umbrellas) are mostly small-target features, attached to various carriers such as tree bark, dead branches, and bare soil. Accurate identification of these features is crucial for assessing termite activity and preventing dam breaches. However, existing termite exposure feature identification technologies have the following drawbacks: low accuracy in small-target detection and segmentation. Existing technologies often employ single semantic segmentation or single-target detection, failing to establish a collaborative "detection-segmentation" logic, resulting in low accuracy in small-target identification; a disconnect between multimodal and small-target technologies. While some technologies introduce multimodal data such as RGB+multispectral data, they are not optimized for the characteristics of small targets and the differences in carrier environments, and they do not employ independent multi-lens layouts, making it difficult to meet the needs of mid- to long-distance perception; and insufficient small-target feature learning, failing to utilize prior category information and a dynamic weight fusion mechanism of "color-shape-texture" to assist in small-target segmentation. Furthermore, deep networks are prone to losing edge features of small targets, and shallow features lack sufficient semantic information, making it impossible to simultaneously address "small target localization" and "edge detail segmentation." Environmental adaptability is mismatched with the needs of small targets; interference such as fluctuating outdoor lighting and swaying branches further reduces the recognizability of small target features. Existing fixed inference threshold schemes exhibit low stability in small target recognition under complex environments. Handheld terminal adaptability is poor: a lightweight technology system has not been designed to meet the portability requirements of field inspections, and a hierarchical recognition mechanism is lacking, making it difficult to achieve rapid and robust detection at medium to long distances, and failing to meet the real-time inference and efficient operation requirements of handheld terminals.

[0003] Traditional termite identification methods primarily rely on manual inspections, which are not only inefficient but also prone to missed detections. With the development of computer vision and deep learning technologies, more and more researchers are applying intelligent identification techniques to termite detection. Automated detection methods based on image processing can significantly improve detection efficiency and reduce labor costs, providing a new solution for termite control in water conservancy projects. However, due to the small size, diverse morphology, and complex and variable environment of exposed termites, existing identification technologies still have shortcomings in terms of accuracy and robustness, making it difficult to meet the needs of practical engineering applications.

[0004] Chinese patent CN114626445A discloses a video-based termite identification method for dams, based on optical flow networks and Gaussian background modeling. This method identifies termite targets in surveillance videos by constructing an optical flow estimation network and a Gaussian mixture background model. Specifically, the method processes the video sequence frame by frame, estimates optical flow information using the FlowNet2 network structure, extracts moving targets using a Gaussian mixture model, and finally weights and fuses the results of the two methods to achieve automatic termite identification. This method improves the accuracy and recall of termite identification to a certain extent, showing better performance compared to using optical flow or Gaussian background modeling methods alone. However, this method mainly targets moving termites in the video sequence and has limited detection capability for static termite exposed features (such as swarming holes, mud lines, etc.), and it does not fully consider the special needs of small target detection and adaptability issues in complex environments. Summary of the Invention

[0005] In view of this, the present invention proposes a method and system for identifying exposed termite features in water conservancy projects, in order to solve the technical problems of low accuracy, high false detection rate and poor adaptability of intelligent identification of exposed termite features in water conservancy projects.

[0006] The technical solution of this invention is implemented as follows: A method for identifying the exposed features of termites in water conservancy projects, comprising the following steps:

[0007] S1. Acquire RGB and multispectral images of the area to be detected, generate a background mask based on the multispectral image, and map the background mask to the RGB image coordinate system for pixel-level alignment. Perform background suppression processing on the RGB image through mask subtraction to obtain the background-suppressed RGB image.

[0008] S2. A small target detection network is used to perform hierarchical detection on the RGB image after background suppression. First, the attachment carrier with exposed termite features is detected to obtain the carrier bounding box. Then, the exposed termite features are detected within the carrier bounding box and verified and filtered to obtain the small target bounding box.

[0009] S3. Use a semantic segmentation network to segment the bounding boxes of small targets after verification and filtering. Input the category information of the small targets as prior information into the segmentation network to obtain the segmentation map of the small targets.

[0010] S4. Adaptively adjust the inference parameters for detection and segmentation based on environmental parameters, and use the temporal correlation method to track and verify the segmentation map of small targets in consecutive frames, filter out false detection targets, and obtain an effective segmentation map;

[0011] S5. Based on the effective segmentation map, calculate the area of ​​the hazardous area and the number of hazardous sites, determine the hazard level, and generate a detection report.

[0012] Based on the above scheme, preferably, step S1 specifically includes:

[0013] S11. Simultaneously acquire RGB and multispectral images of the area to be detected;

[0014] S12. Calculate the vegetation index for the multispectral image and generate a background mask based on the vegetation index threshold.

[0015] S13. Use the feature point matching method to extract feature points from the multispectral image and the RGB image and calculate the transformation matrix. Map the background mask to the RGB image coordinate system to achieve pixel-level alignment.

[0016] S14. Apply the background mask to the RGB image using mask subtraction to subtract the vegetation background area and obtain the RGB image after background suppression.

[0017] Based on the above scheme, preferably, step S2 specifically includes:

[0018] S21. A carrier detection network is used to perform primary detection on the background-suppressed RGB image to identify the attached carrier and obtain the carrier bounding box and carrier category.

[0019] S22. A termite exposure feature detection network is used to perform secondary detection within the carrier bounding box to identify termite exposure features and obtain the small target bounding box and the small target detection confidence.

[0020] S23. Extract image features within the bounding box of the small target, dynamically adjust the feature weights according to the carrier category and the small target detection confidence, calculate the feature matching degree, and filter the effective small target bounding boxes according to the matching degree and overlap rate to obtain the filtered small target bounding boxes.

[0021] Based on the above scheme, preferably, the carrier detection network and the termite exposure feature detection network respectively use clustering methods to generate exclusive anchor frames. The anchor frame size of the carrier detection network is determined based on the width and height of the carrier samples through clustering, and the anchor frame size of the termite exposure feature detection network is determined based on the width and height of the termite exposure feature samples through clustering. The carrier includes tree bark, dead branches, and bare soil, and the termite exposure features include swarming holes, mud lines, and termite umbrellas.

[0022] Based on the above scheme, preferably, in step S23, the extracted image features include color features, shape features, and texture features. When the carrier category is bark or dead branches, the weight of texture features is increased; when the carrier category is bare soil, the weight of color features is increased. An adaptive adjustment factor is set according to the small target detection confidence to dynamically correct the feature weights. The weighted feature matching degree is calculated. When the feature matching degree is ≥ the preset matching degree threshold and the overlap rate between the small target bounding box and the carrier bounding box is ≥ the preset overlap rate threshold, it is determined to be a valid small target bounding box.

[0023] Based on the above scheme, preferably, step S3 specifically includes:

[0024] A dictionary is constructed based on the termite exposure feature categories. The small target category information after verification and screening is mapped to the encoding vector. A fusion vector is generated based on the detection confidence and expanded into a prior category matrix. The prior category matrix is ​​concatenated with the deep feature map of the semantic segmentation network to perform pixel-level segmentation of the image region within the small target bounding box, and the local segmentation map corresponding to the small target bounding box is output. The local segmentation map corresponding to the small target bounding box is compared and corrected with the global segmentation map of the RGB image after background suppression to obtain the corrected small target segmentation map.

[0025] Based on the above scheme, preferably, the semantic segmentation network adopts an improved DeepLab network. After the encoder extracts deep features, the improved DeepLab network concatenates the prior category matrix with the deep feature map through channels. In the decoder, shallow features and deep features are fused through skip connections. Furthermore, a channel attention module is introduced in the decoder to enhance the features of the edge regions of the small target segmentation map.

[0026] Based on the above scheme, preferably, step S4 specifically includes:

[0027] S41. Obtain the light intensity of the current environment and the contrast of the small target region in the RGB image after background suppression. When the light intensity is lower than the preset light threshold or the contrast of the small target region is lower than the preset contrast threshold, reduce the confidence threshold of small target detection and increase the texture feature weight of the segmentation network.

[0028] S42. Use Kalman filtering to predict the position of small targets in consecutive frames, calculate the intersection-over-union ratio of small target bounding boxes in adjacent frames for inter-frame correlation, count the number of frames in consecutive frames whose segmentation confidence meets the conditions, filter false detection targets based on the consecutive frame verification results, and obtain an effective segmentation map.

[0029] Based on the above scheme, preferably, step S5 specifically includes:

[0030] Based on the effective segmentation map, the feature types and number of termite exposures within the detection area are statistically analyzed. According to the detection area type, the number of exposures, and the feature type, the degree of hazard is divided into Level I, Level II, and Level III according to the hazard level judgment criteria. A detection report containing small target segmentation mask, location information, carrier information, and hazard level is generated.

[0031] The present invention also provides a system for identifying the exposed features of termites in water conservancy projects, the system being used to perform the method described above, including:

[0032] The multimodal sensing unit, including an RGB camera, a multispectral camera, and a laser ranging module, is used to simultaneously acquire RGB and multispectral images of the area to be detected and obtain shooting distance information.

[0033] The data preprocessing module is used to generate a background mask based on the multispectral image, map the background mask to the RGB image coordinate system for pixel-level alignment, and perform background suppression processing on the RGB image through mask subtraction.

[0034] The hierarchical detection module includes a carrier detection network and a termite exposure feature detection network, which are used to perform hierarchical detection on the RGB image after background suppression. First, it detects the attached carrier with termite exposure features, then it detects the termite exposure features within the carrier bounding box, and uses a dynamic feature fusion method for verification and screening.

[0035] The semantic segmentation module includes an improved semantic segmentation network, which takes the category information of small targets as prior information and inputs it into the segmentation network to segment the bounding boxes of small targets after verification and filtering.

[0036] The adaptive optimization module is used to adaptively adjust the inference parameters of detection and segmentation according to environmental parameters, and to use the temporal correlation method to track and verify small targets in consecutive frames;

[0037] The hazard assessment module is used to determine the hazard level based on the characteristic type of termite exposure, the affected area, and the number of affected sites, and to generate a detection report that includes the segmentation results and the hazard level.

[0038] The main control unit is used to control the coordinated operation of each module and perform local real-time inference.

[0039] The interactive transmission unit includes a touch screen display module and a wireless transmission module, used to display detection results and transmit data.

[0040] The method and system for identifying exposed termite features in water conservancy projects of the present invention have the following advantages over the prior art:

[0041] (1) By integrating multimodal collaborative perception, hierarchical detection network, semantic segmentation network guided by prior information, environmental adaptive reasoning, and temporal correlation verification, a termite exposure feature recognition technology system of "multimodal perception-hierarchical detection-fine segmentation-adaptive optimization" was constructed. This invention addresses the characteristics of termite exposure features in field inspection scenarios of water conservancy projects, such as small size, diverse attachment carriers, and complex environmental conditions. It achieves background suppression and detail capture through the collaboration of RGB images and multispectral images, improves the positioning accuracy of small targets through hierarchical strategies of carrier detection and small target detection, improves the segmentation effect through prior category information assistance and skip-layer connection mechanism, and enhances the robustness of the system through adaptive adjustment of environmental parameters and temporal verification. Finally, it achieves accurate identification of termite exposure features and standardized determination of hazard level, providing a reliable intelligent detection method for termite control in water conservancy projects.

[0042] (2) A background mask is generated by calculating the vegetation index using multispectral images and then mapped to the RGB image coordinate system for pixel-level alignment. The vegetation background area is then subtracted by mask subtraction. Utilizing the high reflectivity of vegetation in the near-infrared band and the low reflectivity of vegetation in the red band, the exposed features of green vegetation and their attachment carriers are effectively distinguished, reducing the impact of vegetation interference on small target detection. Precise alignment of multimodal data is achieved through feature point matching and homography matrix transformation, ensuring the effectiveness of background suppression processing and providing clear input data for subsequent small target detection.

[0043] (3) A hierarchical detection strategy is adopted. First, the attachment carriers of exposed termite features are detected, and then the exposed termite features are detected within the carrier bounding boxes. The dynamic feature fusion method is used for verification and screening. The hierarchical detection strategy takes into account the characteristics of termite exposure features attached to different carriers such as tree bark, dead branches, and bare soil. It generates dedicated anchor boxes for the carrier detection network and the termite exposure feature detection network respectively through clustering, which improves the detection accuracy of targets at different scales. By extracting color, shape, and texture features and dynamically adjusting the feature weights according to the carrier category and detection confidence, the accuracy of feature matching for small targets is enhanced. By calculating the feature matching degree and overlap rate to screen valid targets, false detection targets not attached to the carrier are effectively eliminated, reducing the false detection rate.

[0044] (4) An improved DeepLab network is used for semantic segmentation of small targets. The category information obtained in the detection stage is converted into a prior feature map, which guides the segmentation network to focus on small targets of a specific category, thereby enhancing the category semantic information. By introducing a channel attention module in the decoder, the feature channels and spatial regions related to the target are strengthened, improving the network's ability to perceive small targets. By using a skip connection mechanism to supplement the edge detail information of shallow features, the defect of edge feature loss during the upsampling process of deep networks is compensated for, thereby improving the accuracy of small target segmentation and the ability to express edge details. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the method for identifying the exposed features of termites in water conservancy projects according to the present invention;

[0047] Figure 2 This is a schematic diagram of the semantic segmentation network of the present invention. Detailed Implementation

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0049] like Figure 1 As shown, this invention provides a method for identifying exposed termite features in water conservancy projects, comprising the following steps:

[0050] S1. Acquire RGB and multispectral images of the area to be detected, generate a background mask based on the multispectral image, and map the background mask to the RGB image coordinate system for pixel-level alignment. Perform background suppression processing on the RGB image through mask subtraction to obtain the background-suppressed RGB image.

[0051] S2. A small target detection network is used to perform hierarchical detection on the RGB image after background suppression. First, the attachment carrier with exposed termite features is detected to obtain the carrier bounding box. Then, the exposed termite features are detected within the carrier bounding box and verified and filtered to obtain the small target bounding box.

[0052] S3. Use a semantic segmentation network to segment the bounding boxes of small targets after verification and filtering. Input the category information of the small targets as prior information into the segmentation network to obtain the segmentation map of the small targets.

[0053] S4. Adaptively adjust the inference parameters for detection and segmentation based on environmental parameters, use the temporal correlation method to track and verify small targets in consecutive frames, filter out false detection targets, and obtain an effective segmentation map;

[0054] S5. Based on the effective segmentation map, calculate the area of ​​the hazardous area and the number of hazardous sites, determine the hazard level, and generate a detection report.

[0055] In one embodiment of the present invention, step S1 specifically includes:

[0056] S11. Simultaneously acquire RGB and multispectral images of the area to be detected. A high-resolution RGB camera and a multispectral camera are used to simultaneously acquire images of areas in the water conservancy project that may be infested by termites. The RGB camera has 50 million effective pixels and is equipped with a 25mm fixed-focus lens. The multispectral camera covers the 450nm blue light band, 550nm green light band, 650nm red light band, and 800nm ​​near-infrared band, with a half-bandwidth of no more than 15nm. Synchronous acquisition is achieved through hardware trigger signals or timestamps, ensuring that the time difference between the two modalities is less than 10ms.

[0057] S12. Calculate the vegetation index for the multispectral image and generate a background mask based on the vegetation index threshold.

[0058] Specifically, the normalized vegetation index (NVI) and the soil-regulated vegetation index (SRI) are used to jointly characterize vegetation features. The calculation formulas for the NVI and SRI are as follows:

[0059] ;

[0060] ;

[0061] in, Represents the normalized vegetation index. The soil-modified vegetation index (NDVI) is defined as follows: NIR represents the reflectance in the 800nm ​​near-infrared band, R represents the reflectance in the 650nm red band, and L is the soil modifier coefficient, with a value of 0.5. Based on statistical analysis of multispectral data collected at the water conservancy project site, the index range for green vegetation areas was determined to be NDVI ≥ 0.5 and SAVI ≥ 0.35. Pixel-by-pixel analysis was performed on the multispectral image; if a pixel met the vegetation index criteria, it was marked as vegetation background (pixel value M). B =0), otherwise it is marked as the target region (pixel value M). B =255). Perform a 5×5 morphological opening operation on the binarized result to eliminate small-area noise and obtain the background mask.

[0062] S13. Feature points of the multispectral image and RGB image are extracted using the feature point matching method and the transformation matrix is ​​calculated. The background mask is then mapped to the RGB image coordinate system to achieve pixel-level alignment.

[0063] Specifically, a checkerboard calibration board is first used, and the camera intrinsic parameter matrix M and extrinsic parameter matrices (rotation matrix R and translation vector t) are solved using the Zhang Zhengyou calibration method. Fifteen sets of calibration images covering ±15° rotation and ±10° pitch are captured. Corner points are extracted through corner detection and sub-pixel optimization. The parameters are then solved based on the core projection formula of the Zhang Zhengyou calibration method. The projection formula is as follows:

[0064] ;

[0065] Where s is the scale factor; u and v are the pixel coordinates of the corner points; x, y, and z are the world coordinates of the corner points; and M is the intrinsic parameter matrix. This is the extrinsic parameter matrix.

[0066] The calibration board was placed horizontally on the simulated dam plane. Ten calibration distances were selected, ranging from 5m to 50m with 5m intervals. Distance D was recorded using a laser rangefinder. For each distance, automatic exposure lock was enabled, and five images were captured. For each set of images, the homography matrix H was solved using corner detection and matching. The geometric properties of the homography matrix were then utilized (…). The actual shooting distance is the perpendicular distance from the camera's optical center to the z=0 plane. The actual distance D of any point P=(x, y, z) in the plane can be calculated as follows:

[0067] ;

[0068] Among them, 5 sets of verification distances that were not included in the calibration were selected, and the images were taken to solve for H. The predicted distance Dp was then calculated by substituting it into the mapping model. The distances Dp and the measured distances Da were compared to ensure that the absolute error was less than 0.1m and the relative error was less than 3%. If the error did not meet the standard, the calibration samples of the corresponding distance segments were encrypted.

[0069] During field inspections, pre-calibrated mapping models may experience alignment errors due to camera attitude fluctuations and variations in carrier surface flatness. To correct these alignment errors in real time, SIFT feature matching is used for fine-tuning. Feature points are extracted from both RGB and multispectral images, and the FLANN matching algorithm is used for coarse matching. Based on the matching pairs, the homography matrix for the current scene is recalculated. By comprehensively considering the pre-calibration matrix and the correction matrix, the corrected homography matrix is ​​obtained. :

[0070] ;

[0071] ;

[0072] ;

[0073] in, Represents the coordinates of feature points in an RGB image. Represents the coordinates of feature points in a multispectral image. Indicates the correction parameters. This represents the average error between the coordinates of feature points in an RGB image and the coordinates of feature points in a multispectral image. The upper limit of the error, the correction parameter The default value is 0.5. The corrected homography matrix is ​​used. The background mask is mapped from the multispectral image coordinate system to the RGB image coordinate system to achieve pixel-level alignment.

[0074] S14. Apply the background mask to the RGB image using mask subtraction to subtract the vegetation background area and obtain the RGB image after background suppression.

[0075] Specifically, the aligned background mask is compared pixel-by-pixel with the RGB image using the following formula: ,in For the original RGB image, M B As background mask, This is the RGB image after background suppression. Before mask subtraction, Gaussian filtering and bilateral filtering are applied to the RGB image for noise reduction to avoid noise interference with the edges of small targets and carriers. Simultaneously, adaptive histogram equalization is used to improve the contrast between small targets, attached carriers, and the background. This step uses multispectral background masking to eliminate interference from green vegetation, preserving suspected termite exposure features and their attached carrier areas, providing clear input data for subsequent small target detection.

[0076] In one embodiment of the present invention, step S2 specifically includes:

[0077] S21. A carrier detection network is used to perform primary detection on the background-suppressed RGB image to identify the attached carrier and obtain the carrier bounding box and carrier category.

[0078] Specifically, the carrier detection network adopts a lightweight small-target detection network architecture. Targeting the attachment characteristics of termite-exposed carriers in hydraulic engineering projects, it uses K-means clustering to cluster the width and height of carrier training samples, generating carrier-specific anchor frames with sizes including 50×100mm, 100×200mm, and 200×300mm. Based on the shooting distance D obtained from the laser ranging module, the focal length f of the camera's intrinsic parameters, and the sensor pixel size s, the pixel equivalent is calculated. The physical dimensions of the anchor frame Convert to pixel size The carrier detection network identifies three types of attached carriers in an image: bark, dead branches, and bare soil, and outputs the carrier bounding boxes. and carrier category The detection confidence threshold was set to 0.7.

[0079] S22. A termite exposure feature detection network is used within the carrier bounding box to perform secondary detection, identify termite exposure features, and obtain the small target bounding box and the small target detection confidence.

[0080] Specifically, the termite exposure feature detection network is also based on a lightweight small target detection network architecture. It uses K-means clustering to cluster the width and height of the termite exposure feature training samples, generating anchor boxes specific to small targets. The anchor box sizes include 2×3mm, 3×5mm, and 5×8mm, which are then converted to pixel size according to the method in step S21. A single detection network and a feature pyramid hierarchical adaptation strategy are employed to simultaneously meet the target detection needs of different scales. Through adaptive receptive field adjustment, the lower-level network uses small convolutional kernels (3×3) to improve the small target capture ability, while the higher-level network uses large convolutional kernels (5×5) to expand the carrier coverage. Within the carrier bounding box... The system performs targeted detection to identify three types of exposed termite features: swarming holes, mud lines, and termite umbrellas, and outputs small target bounding boxes. Small goal categories and confidence level of small target detection .

[0081] S23. Extract image features within the bounding box of the small target, dynamically adjust the feature weights according to the carrier category and the small target detection confidence, calculate the feature matching degree, and filter the effective small target bounding boxes according to the matching degree and overlap rate to obtain the filtered small target bounding boxes.

[0082] Specifically, in the bounding box of the small target Extract three types of image features from the corresponding image sub-regions:

[0083] Color characteristics A histogram is constructed using the HSV color space, with the H channel divided into 18 intervals, the S channel into 32 intervals, and the V channel into 32 intervals, forming an 82-dimensional feature vector. The robustness of the HSV space to changes in illumination is utilized to improve the stability of color features.

[0084] Shape features The target contour is extracted by adaptive threshold binarization, a 7-dimensional Hu moment is calculated and two features of aspect ratio and roundness are added to form a 9-dimensional shape feature vector, which is suitable for shape recognition in small target blurry scenes.

[0085] Texture features A 256-dimensional histogram is constructed using the local binary pattern of a 3×3 neighborhood. Combined with the gray-level co-occurrence matrix, four types of features—contrast, energy, entropy, and correlation—are calculated in the four directions of 0°, 45°, 90°, and 135°, forming a 272-dimensional texture feature vector. Through multi-directional texture extraction, it is adapted to the texture characteristics of different carrier surfaces.

[0086] Based on a database of termite exposure feature images, a color-shape-texture feature template library for different targets was constructed, and the average value was used to obtain standard features. Cosine similarity is used to measure the features to be detected. With standard features Matching degree S The calculation formula is:

[0087] ;

[0088] According to carrier type Dynamically adjust the weights of color, shape, and texture features. The weights of bark carriers are enhanced by texture, the weights of bare soil carriers are enhanced by color, and the weights of dead branches and dead wood carriers are balanced among the three categories.

[0089] The initial weights for bark carriers were 0.2, 0.3, and 0.5, respectively; the initial weights for bare soil carriers were 0.5, 0.3, and 0.2, respectively; and the initial weight for dead branch carriers was 1 / 3 for all. The confidence level for small target detection was then used. Set the adaptive adjustment factor:

[0090] ;

[0091] when When ≥0.8, For values ​​∈ [0, 0.1], the higher the confidence level, the more significant the weight bias.

[0092] For bark carriers, the weight adjustment should enhance texture features, and the calculation formula is as follows:

[0093] , , ;

[0094] For bare soil carriers, the weight adjustment should emphasize color characteristics, and the calculation formula is as follows:

[0095] , , ;

[0096] The weights of the three categories of dead branches and dead wood carriers are balanced, and the feature weights are not adjusted.

[0097] The matching scores of different features are weighted and summed based on dynamically adjusted weights to obtain the feature fusion matching score SF. The calculation formula is as follows:

[0098] ;

[0099] Calculate the bounding box of the small target With carrier bounding box Overlapping box area and overlap rate Retain overlap rate ≥ 0.6, detection confidence level. Small target bounding boxes with a feature fusion matching degree F≥0.5 and a feature fusion matching degree F≥0.5 are used to exclude falsely detected targets that are not attached to the carrier. This step, through hierarchical detection and dynamic feature fusion, effectively improves the accuracy and robustness of small target detection and adapts to the differences in different carrier environments.

[0100] In one embodiment of the present invention, step S3 specifically includes: constructing a dictionary according to the termite exposure feature categories; mapping the verified and screened small target category information into an encoding vector; generating a fusion vector based on the detection confidence and expanding it into a prior category matrix; concatenating the prior category matrix with the deep feature map of the semantic segmentation network; performing pixel-level segmentation on the image region within the small target bounding box; and outputting the local segmentation map corresponding to the small target bounding box; comparing and correcting the local segmentation map corresponding to the small target bounding box with the global segmentation map of the RGB image after background suppression to obtain the corrected small target segmentation map.

[0101] First, prior category coding is performed. A dictionary is constructed based on the exposed features of K-type termites, such as mud lines, swarming holes, and termite umbrellas, to validate the small target category information. The mapping is done as a K-dimensional encoded vector. A fusion vector is generated based on the detection confidence scores for different categories. ,in This represents the detection confidence score for the exposed features of the i-th type of termite. It is expanded into a prior feature map by the image size 1×H×W (H and W are the image height and width). (K×H×W), where the values ​​at each spatial location in the figure are all... This prior category matrix transforms the category semantic information acquired during the detection phase into a tensor form with the same dimension as the image features, providing category guidance for the segmentation network.

[0102] Then, semantic segmentation is performed. The semantic segmentation network uses an improved DeepLab network, which concatenates the prior class matrix with the deep feature map through channels after deep feature extraction from the encoder. For example... Figure 2 As shown, the specific process is as follows: The input small target image patch undergoes multi-scale feature extraction via an encoder. The encoder includes 1×1 convolutional layers, 3×3 convolutional layers, and dilated convolutional layers. Multi-scale receptive fields are achieved through dilated convolutions with different dilation rates (rate=6, rate=12, rate=18). Simultaneously, pooling operations are performed on the input image to form global features. These multi-scale features are then concatenated through channels to form a deep feature map. This deep feature map possesses rich semantic information but has low spatial resolution. The prior feature map is then... By concatenating deep feature maps with semantic segmentation networks, category semantic guidance is enhanced, enabling the network to perform accurate segmentation by combining prior category information from the detection phase.

[0103] The concatenated features are input into the CBAM channel attention module to enhance key information. The CBAM module consists of two sub-modules: channel attention and spatial attention. The channel attention module extracts the interdependencies between channels through global average pooling and global max pooling, calculates channel attention weights through a multilayer perceptron, and weights the feature map accordingly. The spatial attention module extracts the importance of spatial locations through pooling operations along the channel axes, calculates spatial attention weights through convolutional layers, and weights the feature map accordingly. The features processed by the CBAM module not only enhance the feature channels related to the target category but also highlight the spatial region where the target is located, improving the network's ability to perceive small targets.

[0104] The decoder restores the spatial resolution of the feature map through a 4x upsampling. A skip-layer connection mechanism is introduced during decoding, where the shallow features from the encoder are dimensionality-reduced using a 1×1 convolution kernel and then concatenated with the deep feature map from the decoder. Shallow features retain more edge details and spatial location information, and their fusion with deep features compensates for the edge feature loss during upsampling in deep networks, enhancing the feature representation of small target edge regions. After concatenation, features are fused using a 3×3 convolution and then upsampled again by 4x. The final output is a segmentation result of the same size as the input image, where the value of each pixel represents the probability distribution of its category.

[0105] Finally, segmentation correction is performed. This is based on the valid small target bounding boxes detected by the small target detection team. Cropping an RGB image yields small target image patches. Small target image patches Multispectral background mask, attachment carrier type and the category of exposed termite characteristics The improved DeepLab network, which receives information from various sources, dynamically adjusts segmentation parameters to adapt to differences in the carrier environment and outputs segmentation maps of small targets. Let the initial segmentation of the overall RGB image be M0, and then... For each non-background pixel, if its label does not match the label at the corresponding position of M0, use... Replace the marker at the corresponding position of M0 with the marker to obtain the corrected final segmentation image. This segmentation correction process corrects the global segmentation result by using local fine-grained segmentation results, avoiding the problem of small targets being disturbed or missed when directly segmenting the entire image. At the same time, it reduces computational complexity and improves segmentation accuracy.

[0106] The semantic segmentation network, by introducing prior category information, CBAM channel attention modules, and skip-layer connection mechanisms, compensates for the shortcomings of deep networks in edge feature loss and shallow networks in semantics, significantly improving the accuracy of small target segmentation and the ability to express edge details. The segmentation correction process further ensures the accuracy of small target region segmentation, adapting to the precise identification needs of termite exposure features in complex environments of water conservancy projects.

[0107] In one embodiment of the present invention, step S4 specifically includes:

[0108] S41. Obtain the current ambient light intensity and small target contrast. When the light intensity is lower than the preset light threshold or the small target contrast is lower than the preset contrast threshold, reduce the detection confidence threshold and increase the texture feature weights of the segmentation network.

[0109] Illumination intensity L is obtained through a light intensity sensor, and the contrast ratio C of small targets is calculated using RGB images. When L < 2000 lux, it is determined to be a low-light environment. The detection confidence threshold of the small target detection network is lowered by 15%, and the texture feature weight of the DeepLab segmentation network is increased by 20%, enhancing the edge recognition capability of small targets. When C < 0.2, it is determined to be a low-contrast environment. The multispectral background mask threshold is relaxed by 20%, while the weight of the small target edge detection branch is strengthened to prevent small targets from being obscured by the background.

[0110] S42. Use Kalman filtering to predict the position of small targets in consecutive frames, calculate the intersection-over-union ratio of small target bounding boxes in adjacent frames for inter-frame correlation, count the number of frames in consecutive frames whose segmentation confidence meets the conditions, filter false detection targets based on the consecutive frame verification results, and obtain an effective segmentation map.

[0111] Specifically, the state vector of the Kalman filter includes the center coordinates, width, height, and rate of change of the small target bounding box, predicting the position of the small target in the next frame. The Intersection over Union (IOU) of the small target bounding boxes in adjacent frames is calculated; if two consecutive frames have an IOU ≥ 0.5, they are identified as the same small target. The number of frames with a segmentation confidence ≥ 0.75 in consecutive frames is counted. Small targets appearing only in a single frame or not associated in three consecutive frames are identified as false detections and deleted; small targets associated in five consecutive frames with a mean segmentation confidence ≥ 0.75 are retained. This step, through environmental adaptive parameter adjustment and temporal optimization verification, effectively improves the stability and accuracy of small target recognition in complex environments.

[0112] In one embodiment of the present invention, step S5 specifically includes: based on the effective segmentation map, statistically analyzing the feature types and number of termite exposures within the detection area; according to the detection area type, number of exposures, and feature types, classifying the degree of hazard into Level I, Level II, and Level III according to the hazard level determination criteria; and generating a detection report containing a small target segmentation mask, location information, carrier information, and hazard level.

[0113] Specifically, according to the "Technical Specification for Termite Control in Water Conservancy Projects" (SL / T836-2024), the degree of termite damage is divided into Level I, Level II and Level III, which correspond to mild damage, moderate damage and severe damage, respectively.

[0114] (a) If a unit is found to be infested with termites and meets one of the following conditions, the termite infestation level of the unit shall be assessed as Level I: ① The termite infestation area has an average of 1000 m² / day of termite infestation. 2 ① One or more exposed features such as mud blankets, mud lines, and ant nest umbrellas were found; ② On average, every 1000m² of the ant source area... 2 Three or more exposed features such as mud blankets, mud lines, and ant nest umbrellas were found.

[0115] (b) If a unit is found to be infested with termites and meets one of the following conditions, the termite infestation level of the unit shall be assessed as Level II: ① The termite infestation area has an average of 1000 m² / day of termite infestation. 2 ① More than 5 or more exposed features such as mud blankets, mud lines, and ant nest umbrellas were found; ② The average number of ant nests per 1000m² in the ant source area was [missing information]. 2 ① 15 or more exposed features such as mud blankets, mud lines, and ant nest umbrellas were found; ③ Swarming holes were found in the ant source area;

[0116] (c) If a unit is found to have termite infestation and meets one of the following conditions, the termite infestation level of the unit should be assessed as Level III: ① The termite infestation area has an average of 1000 m² / day of termite infestation. 2 ① More than 10 locations show exposed features such as mud blankets, mud lines, and ant nest umbrellas; ② Swarming holes are found in the ant-infested area.

[0117] The hazard level is determined according to the hazard level assessment criteria, and a detection report is generated. The report includes the visualization results of the small target segmentation mask, GPS positioning information, engineering station number, carrier type, type of termite exposure characteristics, area of ​​the affected area, number of affected locations, and hazard level.

[0118] The present invention also provides a system for identifying the exposed features of termites in water conservancy projects, the system being used to perform the method described above, including:

[0119] The multimodal sensing unit includes an RGB camera, a multispectral camera, a laser ranging module, and a temperature and humidity sensor, which are used to simultaneously acquire RGB and multispectral images of the area to be detected, and obtain shooting distance information and shooting environment temperature and humidity.

[0120] The multimodal sensing unit adopts an "RGB + multispectral multi-lens independent layout," integrating multiple core sensing components: a high-resolution RGB camera with 50 million effective pixels and a 25mm fixed-focus lens (aperture F / 2.8) ensures that a single pixel corresponds to a millimeter-level physical size at medium to long distances, meeting the needs for small target detail acquisition; a multispectral camera covers the 450nm (blue), 550nm (green), 650nm (red), and 800nm ​​(near-infrared) bands, with a half-bandwidth ≤15nm, distinguishing between vegetation and muddy small targets through the near-infrared band and generating a background mask to assist in small target detection; a laser ranging module can measure the actual distance between the small target and the lens, correcting the imaging ratio to adapt to the segmentation threshold for medium to long distance small target detection; and temperature and humidity sensors measure ranges of -20~60℃ (accuracy ±0.5℃) and 0-100%RH (accuracy ±3%RH), providing environmental data support for adaptive adjustment of the environment for small target detection and segmentation. Inspection personnel use handheld terminals to adjust RGB / multispectral cameras around water conservancy project pile numbers (such as the withered willow tree at K1+500) to aim at areas with high termite infestation. By locating and recording coordinates, data is automatically collected in the form of video streams or manually triggered for collection.

[0121] The data preprocessing module is used to generate a background mask based on the multispectral image, map the background mask to the RGB image coordinate system for pixel-level alignment, and perform background suppression processing on the RGB image through mask subtraction.

[0122] The hierarchical detection module includes a carrier detection network and a termite exposure feature detection network. It is used to perform hierarchical detection on the background-suppressed RGB image. First, it detects the attached carrier with termite exposure features, then it detects the termite exposure features within the carrier bounding box, and uses a dynamic feature fusion method for verification and screening.

[0123] The semantic segmentation module includes an improved semantic segmentation network, which uses the category information of small targets as prior information to input the segmentation network and performs segmentation processing on the bounding boxes of small targets after verification and filtering.

[0124] The adaptive optimization module is used to adaptively adjust the inference parameters of detection and segmentation based on environmental parameters, and to use a temporal correlation method to track and verify small targets in consecutive frames.

[0125] The hazard assessment module is used to determine the hazard level based on the characteristic type of termite exposure, the affected area, and the number of affected sites, and to generate a detection report that includes the segmentation results and the hazard level.

[0126] The main control unit is used to control the coordinated operation of each module and perform local real-time inference. This unit supports local real-time inference for small object detection and DeepLab semantic segmentation, adapting to the edge intelligence needs of handheld terminals.

[0127] The storage unit can cache at least 10,000 small object detection and segmentation data, including RGB images, multispectral masks, segmentation results, environmental parameters, etc.

[0128] The interactive transmission unit, comprising an interactive module and a wireless transmission module, is used to display detection results and transmit data. This unit has a single-point positioning accuracy of ≤1m and a differential positioning accuracy of 0.1m, and can accurately mark the location of small targets by associating them with engineering station numbers (e.g., "30cm from the base of the tree bark on the water-facing side of K1+500"). The wireless transmission module supports encrypted transmission of small target detection and segmentation results, and can cache data locally in offline mode. The interactive module is equipped with a 7-inch touchscreen, capable of displaying the small target bounding box, segmentation heatmap, and hazard level in real time, and supports data export, adapting to the efficient operational needs of field inspections.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying exposed termite features in water conservancy projects, characterized in that: Includes the following steps: S1. Simultaneously acquire RGB and multispectral images of the area to be detected, calculate the vegetation index of the multispectral image, and generate a background mask based on the vegetation index threshold; use the feature point matching method to extract feature points of the multispectral image and RGB image and calculate the transformation matrix, and map the background mask to the RGB image coordinate system to achieve pixel-level alignment. The background mask is applied to the RGB image using mask subtraction, and the vegetation background area is subtracted to obtain the RGB image after background suppression; The method of extracting feature points from multispectral and RGB images and calculating the transformation matrix using feature point matching specifically includes: fine-tuning using SIFT feature matching to extract feature points from both RGB and multispectral images; coarse matching using the FLANN matching algorithm; and recalculating the homography matrix for the current scene based on the matching pairs. By comprehensively considering the pre-calibration matrix and the correction matrix, the corrected homography matrix is ​​obtained. : ； ； ； Where H is the pre-calibrated homography matrix, Represents the coordinates of feature points in an RGB image. Represents the coordinates of feature points in a multispectral image. Indicates the correction parameters. This represents the average error between the coordinates of feature points in an RGB image and the coordinates of feature points in a multispectral image. This is the upper limit of the error; S2. A small target detection network is used to perform hierarchical detection on the RGB image after background suppression. First, the attachment carrier with exposed termite features is detected to obtain the carrier bounding box. Then, the exposed termite features are detected within the carrier bounding box and verified and filtered to obtain the small target bounding box. S3. Use a semantic segmentation network to segment the bounding boxes of small targets. Input the category information of the small targets as prior information into the semantic segmentation network to obtain the segmentation map of the small targets. A dictionary was constructed based on the exposed characteristics of K-type termites, including mud cover, mud lines, swarming holes, and termite umbrellas. This dictionary will be used to verify the filtered sub-target category information. The mapping is performed as a K-dimensional encoded vector, and a fusion vector is generated based on the detection confidence of different categories. ,in This represents the detection confidence score of the exposed features of the i-th type of termite; a fusion vector is generated based on the detection confidence score and expanded into a prior class matrix, which is then used to fusion small target image patches. Multispectral background mask, attachment carrier type and small target category information The improved DeepLab network is input, and the segmentation parameters are dynamically adjusted to adapt to the differences in the carrier environment. The local segmentation map corresponding to the bounding box of the small target is output. The local segmentation map corresponding to the bounding box of the small target is compared and corrected with the global segmentation map of the RGB image after background suppression to obtain the corrected small target segmentation map. S4. Adaptively adjust the inference parameters for detection and segmentation based on environmental parameters, and use the temporal correlation method to track and verify the segmentation map of small targets in consecutive frames, filter out false detection targets, and obtain an effective segmentation map; S5. Based on the effective segmentation map, calculate the area of ​​the hazardous area and the number of hazardous sites, determine the hazard level, and generate a detection report; Step S2 specifically includes: S21. A carrier detection network is used to perform primary detection on the background-suppressed RGB image to identify attached carriers, including bark, dead branches and bare soil, and to obtain carrier bounding boxes and carrier categories. S22. A termite exposure feature detection network is used to perform secondary detection within the carrier bounding box to identify termite exposure features and obtain the small target bounding box and the small target detection confidence. S23. Extract image features within the bounding box of the small target, dynamically adjust feature weights based on the carrier category and the small target detection confidence, calculate feature matching degree, and filter valid small target bounding boxes based on matching degree and overlap rate to obtain the filtered small target bounding boxes. The extracted image features include color features, shape features, and texture features. When the carrier category is bark or dead branches, increase the weight of texture features; when the carrier category is bare soil, increase the weight of color features. Dynamically correct the feature weights based on the small target detection confidence by setting an adaptive adjustment factor, calculate the weighted feature matching degree, and determine a valid small target bounding box when the feature matching degree is ≥ the preset matching degree threshold and the overlap rate between the small target bounding box and the carrier bounding box is ≥ the preset overlap rate threshold.

2. The method for identifying exposed termite features in water conservancy projects as described in claim 1, characterized in that: The carrier detection network and the termite exposure feature detection network each use a clustering method to generate a dedicated anchor frame. The anchor frame size of the carrier detection network is determined by clustering the width and height of the carrier samples, and the anchor frame size of the termite exposure feature detection network is determined by clustering the width and height of the termite exposure feature samples. The termite exposure features include swarming holes, mud lines, and termite umbrellas.

3. The method for identifying exposed termite features in water conservancy projects as described in claim 1, characterized in that: The semantic segmentation network adopts an improved DeepLab network. After the encoder extracts deep features, the improved DeepLab network concatenates the prior category matrix with the deep feature map through channels. In the decoder, shallow features and deep features are fused through skip connections. Furthermore, a channel attention module is introduced in the decoder to enhance the features of the edge regions of the small target segmentation map.

4. The method for identifying exposed termite features in water conservancy projects as described in claim 1, characterized in that: Step S4 specifically includes: S41. Obtain the light intensity of the current environment and the contrast of the small target region in the RGB image after background suppression. When the light intensity is lower than the preset light threshold or the contrast of the small target region is lower than the preset contrast threshold, reduce the confidence threshold of small target detection and increase the texture feature weight of the segmentation network. S42. Use Kalman filtering to predict the position of small targets in consecutive frames, calculate the intersection-over-union ratio of small target bounding boxes in adjacent frames for inter-frame correlation, count the number of frames in consecutive frames whose segmentation confidence meets the conditions, filter false detection targets based on the consecutive frame verification results, and obtain an effective segmentation map.

5. The method for identifying exposed termite features in water conservancy projects as described in claim 1, characterized in that: Step S5 specifically includes: Based on the effective segmentation map, the feature types and number of termite exposures within the detection area are statistically analyzed. According to the detection area type, the number of exposures, and the feature type, the degree of hazard is divided into Level I, Level II, and Level III according to the hazard level judgment criteria. A detection report containing small target segmentation mask, location information, carrier information, and hazard level is generated.

6. A system for identifying exposed termite features in water conservancy projects, characterized in that: The system is used to perform the method as described in any one of claims 1-5, including: The multimodal sensing unit, including an RGB camera, a multispectral camera, and a laser ranging module, is used to simultaneously acquire RGB and multispectral images of the area to be detected and obtain shooting distance information. The data preprocessing module is used to generate a background mask based on the multispectral image, map the background mask to the RGB image coordinate system for pixel-level alignment, and perform background suppression processing on the RGB image through mask subtraction. The hierarchical detection module includes a carrier detection network and a termite exposure feature detection network, which are used to perform hierarchical detection on the RGB image after background suppression. First, it detects the attached carrier with termite exposure features, then it detects the termite exposure features within the carrier bounding box, and uses a dynamic feature fusion method for verification and screening. The semantic segmentation module includes an improved semantic segmentation network, which takes the category information of small targets as prior information and inputs it into the segmentation network to segment the bounding boxes of small targets after verification and filtering. The adaptive optimization module is used to adaptively adjust the inference parameters of detection and segmentation according to environmental parameters, and to use the temporal correlation method to track and verify the segmentation map of small targets in consecutive frames; The hazard assessment module is used to determine the hazard level based on the characteristic type of termite exposure, the affected area, and the number of affected sites, and to generate a detection report that includes the segmentation results and the hazard level. The main control unit is used to control the coordinated operation of each module and perform local real-time inference. The interactive transmission unit includes an interactive module and a wireless transmission module, used to display detection results and transmit data.

Images