A dike danger identification method and system based on visible light and electronic equipment

Abstract

The application provides a dike danger identification method and system based on visible light, and an electronic device, and relates to the technical field of dike inspection. The method creates a general scene data set, a general danger data set, a real scene data set and a real danger data set; based on the general danger data set and the real scene data set, fusion enhancement processing is performed to generate an enhanced danger data set; based on the general scene data set and the real scene data set, a scene detection model is constructed; and based on the general danger data set, the real danger data set and the enhanced danger data set, a danger identification model of the dike is constructed. The application fuses the general data set and the danger data set, adopts a data enhancement method, increases danger data samples, makes the danger identification model have stronger generalization ability, and improves the accuracy of identification.

Description

Technical Field

[0001] This application relates to the field of dike inspection technology, and in particular to a method, system and electronic device for identifying dike hazards based on visible light. Background Technology

[0002] Currently, identifying potential hazards on dikes mainly relies on manual inspections. Traditional manual inspections depend heavily on experience and observation of the dike surface, placing a high emphasis on the personal experience and sense of responsibility of the inspectors. Novices or those with insufficient experience are prone to overlooking or misjudging hazards, and the process is slow and labor-intensive. Existing inspection methods both domestically and internationally utilize drones equipped with high-definition cameras, primarily relying on images transmitted by the drones for identification. However, due to the limited number of hazard samples and the lack of data samples with specific backgrounds, identification accuracy is low when relying solely on general datasets. Furthermore, drone-captured images are large in scale, making hazards appear as small pixels within the overall image, which can easily lead to missed detections. Therefore, it is urgent to solve this technical problem. Summary of the Invention

[0003] In view of the above problems, this application is made to provide a visible light-based method, system, and electronic device for identifying dike hazards, which overcomes or at least partially solves the above problems. The technical solution is as follows: Firstly, a method for identifying dike hazards based on visible light is provided, the method comprising: Create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset; among them, the general scenario dataset includes data from multiple scenarios, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes data from multiple real levee scenarios, and the real hazard dataset includes hazard data from multiple real levee scenarios. Based on a general hazard dataset and a real-world scenario dataset, a fusion and enhancement process is performed to generate an enhanced hazard dataset. A scene detection model is built based on general scene datasets and real scene datasets; A hazard identification model for dikes is constructed based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. The visible light image to be identified is input into the scene detection model, the scene feature region is cropped out, and then the cropped scene feature region is input into the hazard identification model to identify the levee hazard in the visible light image to be identified.

[0004] In one possible implementation, a realistic scenario dataset and a realistic hazard dataset are created, including: By flying drones above the dikes, data on the dikes is collected, generating a real-world dataset. Using annotation tools, we annotate the dangerous situations in the dike data of real-world scenario datasets to establish a real-world dangerous situation dataset.

[0005] In one possible implementation, an enhanced hazard dataset is generated by fusing and enhancing a general hazard dataset and a real-world scenario dataset, including: The hazard targets of general samples in a general hazard dataset are transferred to real samples in a real scene dataset to achieve fusion enhancement processing and generate an enhanced hazard dataset.

[0006] In one possible implementation, the hazard targets of general samples in a general hazard dataset are transferred to real samples in a real-world scenario dataset, including: Read the general sample image img1 and the embankment background image img2 from the real sample; The hazardous area is cropped from the target label in the general sample image img1 and denoted as mask; The mask is randomly placed on the background image img2 of the dike. The specific process is as follows: a point (x,y) is randomly selected on img2, the center point of the mask is overlapped with (x,y), the brightness h1 of the mask area and the brightness h2 of the area of ​​img2 covered by the mask are calculated respectively, the mask is adjusted to h2, and the data on img2 is replaced with the data of the mask. Feathering is performed on the connection between the mask and img2.

[0007] In one possible implementation, a levee hazard identification model is constructed based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, including: Using the YOLO v11 framework, an attention mechanism is added to build the initial model; The initial model is trained using a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset to construct a hazard identification model for dikes.

[0008] In one possible implementation, the initial model includes a backbone network, a neck network, and a detection head; the backbone network includes a feature extraction branch, a first feature fusion module, a second feature fusion module, a spatial pyramid pooling module, and a channel and position sensitive attention module; the feature extraction branch includes an initial convolutional layer and multiple convolutional blocks connected in sequence; the first feature fusion module is connected to the first preset layer convolutional block of the feature extraction branch; the second feature fusion module is connected to the second preset layer convolutional block of the feature extraction branch; the spatial pyramid pooling module is connected to the last layer convolutional block of the feature extraction branch; and the channel and position sensitive attention module is connected to the spatial pyramid pooling module. The neck network includes a first upsampling module, a fourth feature fusion module, a first feature extraction module, a second upsampling module, a fifth feature fusion module, a second feature extraction module, a first convolutional layer, a sixth feature fusion module, a third feature extraction module, a second convolutional layer, a seventh feature fusion module, and a fourth feature extraction module connected in sequence. The second upsampling module is connected to the sixth feature fusion module. The detection head includes a first attention module, a second attention module, a third attention module, a first dynamic detection head, a second dynamic detection head, and a third dynamic detection head. The first attention module is connected to the second feature extraction module, the second attention module is connected to the third feature extraction module, and the third attention module is connected to the fourth feature extraction module.

[0009] In one possible implementation, when training the initial model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, the loss function is as follows: Loss=a1×Lcls+a2×Lciou+a3×Ldfl Where a1, a2, and a3 are weight coefficients; Lcls is the target classification loss value, which uses a sigmoid classifier to perform binary classification on each category, and is the cross-entropy loss between the predicted result pred_scores and the true label target_scores; Lciou is the target localization loss value, which is the sum of the overlap ratio between the predicted bounding box and the true bounding box, the center constraint, and the aspect ratio; Ldfl is the distribution focus loss value, which is calculated by converting the true bounding box coordinates into distances relative to anchor points and limiting them to the range of 0 to reg_max-1, where reg_max=16. Then, the cross-entropy loss is calculated between the network prediction output and the true label. Here, the network prediction output is the probability distribution after softmax processing, and the true label is the bounding box distance after integer processing.

[0010] Secondly, a visible light-based levee hazard identification system is provided, the system comprising: The creation unit is used to create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset. The general scenario dataset includes data from multiple scenarios, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes data from multiple real levee scenarios, and the real hazard dataset includes hazard data from multiple real levee scenarios. The enhancement unit is used to perform fusion enhancement processing based on a general hazard dataset and a real-world scenario dataset to generate an enhanced hazard dataset. The first building unit is used to construct a scene detection model based on a general scene dataset and a real scene dataset; The second building unit is used to construct a levee hazard identification model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. The recognition unit is used to input the visible light image to be recognized into the scene detection model, crop out the scene feature region, and then input the cropped scene feature region into the hazard recognition model to identify the levee hazard in the visible light image to be recognized.

[0011] Thirdly, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the visible light-based levee hazard identification method described in any of the preceding claims.

[0012] Using the above technical solutions, the visible light-based method, system, and electronic device for identifying dike hazards provided in this application create a general scene dataset, a general hazard dataset, a real scene dataset, and a real hazard dataset; perform fusion enhancement processing on the general hazard dataset and the real scene dataset to generate an enhanced hazard dataset; construct a scene detection model based on the general scene dataset and the real scene dataset; construct a dike hazard identification model based on the general hazard dataset, the real hazard dataset, and the enhanced hazard dataset; input the visible light image to be identified into the scene detection model, crop out the scene feature region, and then input the cropped scene feature region into the hazard identification model to identify the dike hazards in the visible light image to be identified. As can be seen, this application integrates a general dataset and a hazard dataset, and uses data augmentation methods to increase the number of hazard data samples, making the hazard identification model more generalizable and improving the accuracy of identification. Furthermore, it identifies levee hazards in visible light images based on hazard targets and background targets. After identification, background detection is used to eliminate targets outside the levee area, quickly identifying hazards of different sizes and achieving real-time identification and early warning. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0014] Figure 1a A flowchart of the visible light-based levee hazard identification method provided in an embodiment of this application is shown; Figure 1b An architecture diagram of the initial model provided in an embodiment of this application is shown; Figure 2 The diagram shows the structure of a visible light-based levee hazard identification system provided in an embodiment of this application. Figure 3A structural diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0015] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0017] To address the aforementioned technical problems, embodiments of this application provide a method for identifying dike hazards based on visible light, such as... Figure 1a As shown, the visible light-based method for identifying dike hazards may include the following steps S101 to S105: Step S101: Create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset. The general scenario dataset includes data from multiple scenarios, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes data from multiple real levee scenarios, and the real hazard dataset includes hazard data from multiple real levee scenarios. Here, the multiple scenario data can be riverbanks, dams, river channels, levees, etc.; the multiple hazard data can be hazard data such as collapses and cracks in riverbanks, dams, river channels, and levees; the multiple real levee scenario data can be actual, collected data from real levees; and the multiple real levee scenario hazard data can be actual, collected data such as collapses and cracks in real levees. Step S102: Based on the general hazard dataset and the real scene dataset, perform fusion enhancement processing to generate an enhanced hazard dataset; Step S103: Construct a scene detection model based on a general scene dataset and a real scene dataset; Step S104: Based on the general hazard dataset, the real hazard dataset, and the enhanced hazard dataset, construct a hazard identification model for the dike; Step S105: Input the visible light image to be identified into the scene detection model, crop out the scene feature region, and then input the cropped scene feature region into the hazard identification model to identify the levee hazard in the visible light image to be identified.

[0018] This application integrates a general dataset and a hazard dataset, and uses data augmentation methods to increase the number of hazard data samples, making the hazard identification model more generalizable and improving the accuracy of identification. Furthermore, it identifies levee hazards in visible light images based on hazard targets and background targets. After identification, background detection is used to remove targets outside the levee area, enabling rapid identification of hazards of different sizes and achieving real-time identification and early warning.

[0019] This application embodiment provides a possible implementation method. Step S101, which involves creating a real scene dataset and a real danger dataset, may specifically include the following steps A1 and A2: Step A1: Collect levee data by flying a drone above the levee to generate a real-world dataset. Step A2: Use annotation tools to annotate the dangerous situations in the dike data of the real scene dataset to establish a real dangerous situation dataset.

[0020] This embodiment uses drones for data collection, which directly solves the problems of low efficiency, high risk and limited field of vision faced by traditional manual inspections; in addition, it uses annotation tools to annotate the dangerous situations in the dike data of real scene datasets, establishes a real dangerous situation dataset, and provides a data foundation for training models.

[0021] This application embodiment provides a possible implementation method. Step S102 is based on a general hazard dataset and a real scene dataset, and performs fusion enhancement processing to generate an enhanced hazard dataset. Specifically, it may include the following step B1: Step B1 involves transferring the hazard targets from the general samples in the general hazard dataset to the real samples in the real scene dataset, thereby achieving fusion enhancement processing and generating an enhanced hazard dataset.

[0022] This embodiment enhances data diversity through conventional methods and then precisely increases the sample size of key features through target transfer techniques. The two work together to provide rich data for training a high-performance levee hazard identification model.

[0023] This application provides a possible implementation method in which step B1 transfers the hazard targets of general samples in a general hazard dataset to real samples in a real scene dataset. Specifically, this may include the following steps B1-1 to B1-3: Step B1-1: Read the general sample image img1 and the embankment background image img2 from the real sample; Step B1-2: Based on the target label in the general sample image img1, crop out the dangerous area and record it as mask; Steps B1-3 involve randomly placing the mask onto the background image img2 of the embankment. Specifically, a point (x, y) is randomly selected on img2, and the center point of the mask is overlapped with (x, y). The brightness h1 of the mask area and the brightness h2 of the area of ​​img2 covered by the mask are calculated respectively. The mask is then adjusted to h2, and the data on img2 is replaced with the data of the mask. Feathering is then applied to the connection between the mask and img2.

[0024] This embodiment creates a large number of diverse and realistic training samples, enabling the model to learn the essential characteristics of dike hazards, rather than memorizing what they look like under certain backgrounds or lighting conditions. This allows the model to maintain a high recognition accuracy when encountering real dike scenarios that have never appeared in the training set.

[0025] This application embodiment provides a possible implementation method. Step S104 constructs a levee hazard identification model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. Specifically, it may include the following steps C1 and C2: Step C1: Using the YOLO v11 framework, add an attention mechanism to the YOLO v11 framework to build the initial model; Step C2: Train the initial model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset to construct a hazard identification model for the dike.

[0026] YOLO stands for You Only Look Once. YOLO v11 is the 11th version of YOLO, a real-time object detection model that includes: The backbone network employs an improved CSPDarknet architecture, introducing the C3k2 and C2PSA modules. The C3k2 module replaces the traditional 3×3 convolutions with two consecutive 2×2 convolutions, reducing computation by 11% while maintaining the same receptive field. C2PSA (convolutional blocks with parallel spatial attention) enhances the model's ability to focus on key regions through a spatial attention mechanism. Here, CSPDarknet is a high-performance backbone feature extraction network widely used in YOLO series object detection models. By introducing a CSP (Cross Stage Partial) structure, it significantly improves the model's training efficiency and inference speed while maintaining high accuracy.

[0027] The Neck Network uses an improved PAN-FPN (Path Aggregation Network-Feature Pyramid Network) structure to support bidirectional multi-scale feature fusion. This design can better handle targets of different scales and is very effective for identifying dangerous objects of different sizes that may coexist in hazard identification.

[0028] The head (detection head) adopts a decoupled detection head design, separating the classification and regression tasks, reducing feature competition and improving accuracy; at the same time, depthwise separable convolution (DWConv) is used in the classification branch, which greatly reduces the number of parameters without significantly affecting performance.

[0029] This application provides one possible implementation method, such as... Figure 1b As shown, the initial model constructed includes a backbone network, a neck network, and a detection head. The backbone network includes a feature extraction branch, a first feature fusion module, a second feature fusion module, a spatial pyramid pooling module, and a channel and position sensitive attention module. The feature extraction branch includes an initial convolutional layer and multiple convolutional blocks connected in sequence. The first feature fusion module is connected to the first preset layer convolutional block of the feature extraction branch. The second feature fusion module is connected to the second preset layer convolutional block of the feature extraction branch. The spatial pyramid pooling module is connected to the last layer convolutional block of the feature extraction branch. The channel and position sensitive attention module is connected to the spatial pyramid pooling module. The neck network includes a first upsampling module, a fourth feature fusion module, a first feature extraction module, a second upsampling module, a fifth feature fusion module, a second feature extraction module, a first convolutional layer, a sixth feature fusion module, a third feature extraction module, a second convolutional layer, a seventh feature fusion module, and a fourth feature extraction module connected in sequence. The second upsampling module is connected to the sixth feature fusion module. The detection head includes a first attention module, a second attention module, a third attention module, a first dynamic detection head, a second dynamic detection head, and a third dynamic detection head. The first attention module is connected to the second feature extraction module, the second attention module is connected to the third feature extraction module, and the third attention module is connected to the fourth feature extraction module.

[0030] exist Figure 1bIn this architecture, there are four convolutional blocks. The initial convolutional layer, the first convolutional layer, and the second convolutional layer are all Conv. The convolutional blocks are Conv connected by C3K2 (C3k2 stands for Cross Stage Partial 3 with k=2). The first feature fusion module, the second feature fusion module, the fourth feature fusion module, the fifth feature fusion module, the sixth feature fusion module, and the seventh feature fusion module are all Concat. The spatial pyramid pooling module is Spatial Pyramid Pooling-Fast (SPPF). The channel and position-sensitive attention module is Cross-Stage Partial with Position-Sensitive Attention (C2PSA). The first upsampling module and the second upsampling module are both Upsample. The first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module are all C3K2.

[0031] Based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, visible light images of the hazard data are input into the feature extraction branch. The first preset convolutional block is the second convolutional block from top to bottom, the second preset convolutional block is the third convolutional block from top to bottom, and the last convolutional block is the last convolutional block from top to bottom. As the network depth increases, the downsampling factor increases layer by layer, and the scales of the first, second, and third visible light features gradually increase. The scale of the feature after first-scale fusion is smaller than that after second-scale fusion, and the scale of the feature after second-scale fusion is smaller than that of the weighted enhanced feature. The feature after first-scale fusion is rich in detail and suitable for small hazards such as small cracks and point-like piping. The feature after second-scale fusion balances detail and semantics and is suitable for medium-sized hazards such as medium-sized cracks and localized leaks. The weighted enhanced feature has prominent semantics and is suitable for large hazards such as large collapses and large-scale leaks. The three work together to achieve accurate identification of hazards in all scenarios.

[0032] The spatial pyramid pooling module is a fast spatial pyramid pooling method that achieves multi-scale spatial feature enhancement processing in a lightweight way. It performs parallel spatial sampling of the third visible light feature through multi-scale pooling kernels, extracts global and local spatial features of different receptive fields, and then outputs multi-scale spatial enhancement features through feature splicing and fusion.

[0033] The channel and location-sensitive attention module is a cross-stage component – ​​location-sensitive attention. It addresses the issues of channel redundancy, severe background interference, and ambiguous location information in multi-scale spatial enhancement features by performing channel and location-weighted enhancement processing. Through the channel attention mechanism, it assigns high weights to feature channels related to the hazard and suppresses redundant background channels. Through the location-sensitive attention mechanism, it focuses on the spatial location of the levee hazard and suppresses irrelevant background locations. In short, it highlights the key features of the hazard and filters out useless background interference.

[0034] By using the spatial pyramid pooling module to supplement multi-scale spatial information, and the channel and location-sensitive attention modules to highlight key features of hazards and filter out useless background interference, the third-scale fused features are optimized in a coordinated manner to improve the accuracy of hazard identification.

[0035] The neck network is used for cross-scale fusion. Specifically, it performs feature fusion by upsampling the weighted enhanced features of the large scale layer by layer with the features fused at the second scale of the medium scale and the features fused at the first scale of the small scale, thus supplementing the shallow feature semantics. Then, it supplements the deep feature details by downsampling. After multiple rounds of feature extraction and convolution purification, the second feature extraction module outputs the small-scale features, which are input into the first attention module of the detection head; the third feature extraction module outputs the medium-scale features, which are input into the second attention module of the detection head; and the fourth feature extraction module outputs the large-scale features, which are input into the third attention module of the detection head.

[0036] Correspondingly, the first scale target output by the first dynamic detection head after passing through the first attention module is a small-scale target, such as a small crack or point-like piping. The first recognition result is the first class probability and first bounding box coordinates of the small-scale target in the 256×256 feature map pixel coordinate system. After passing through the second attention module, the second scale target output by the second dynamic detection head is a medium-scale target, such as a medium-sized crack or localized leakage. The second recognition result is the second class probability and second bounding box coordinates of the medium-scale target in the 192×192 feature map pixel coordinate system. After passing through the third attention module, the third scale target of the third dynamic detection head is a large-scale target, such as a large collapse or large-scale leakage. The third recognition result is the third class probability and third bounding box coordinates of the large-scale target in the 128×128 feature map pixel coordinate system, covering targets of all scales and improving the recognition accuracy in complex scenes.

[0037] The total loss function value is calculated based on the first identification result, the second identification result, the third identification result and the hazard labeling information, including: calculating the target classification loss value, the distribution focus loss value and the target location loss value based on the first identification result, the second identification result, the third identification result and the hazard labeling information; and weighting and summing the target classification loss value, the distribution focus loss value and the target location loss value to obtain the total loss function value.

[0038] After obtaining the total loss function value, compare the total loss function value with the preset threshold. If the total loss function value is less than or equal to the preset threshold, the initial hazard identification model converges and the trained hazard identification model is obtained. If the total loss function value is greater than the preset threshold, the initial hazard identification model does not converge and the model parameters of the hazard identification model need to be adjusted.

[0039] This embodiment adds a dedicated attention mechanism to YOLO v11, which can perform specific optimization for specific hazard data. Unlike general object detection, hazard identification often needs to focus on specific types of features and patterns. The attention mechanism allows the model to dynamically learn which areas are most relevant to the hazard.

[0040] This application provides a possible implementation method. When training the initial model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, the loss function is as follows: Loss=a1×Lcls+a2×Lciou+a3×Ldfl Where a1, a2, and a3 are weight coefficients; Lcls is the target classification loss value, which uses a sigmoid classifier to perform binary classification for each category, and is the cross-entropy loss between the predicted result pred_scores and the ground truth label target_scores (after one-hot encoding); Lciou is the target localization loss value, which is the sum of the overlap ratio between the predicted bounding box and the ground truth bounding box, the center constraint, and the aspect ratio; Ldfl is the distribution focus loss value, which is calculated by converting the ground truth bounding box coordinates into distances relative to anchor points and limiting them to the range of 0 to reg_max-1, where reg_max=16. Then, the cross-entropy loss is calculated between the network prediction output and the ground truth label. Here, the network prediction output is the probability distribution after softmax processing, and the ground truth label is the bounding box distance after integer processing.

[0041] For example, a1 is 0.5, a2 is 8.5, and a3 is 1.5. It should be noted that the examples listed here are only illustrative and do not limit this embodiment.

[0042] This application provides a possible implementation method in which, during the training process, every 10 batches, a size is randomly selected from the next size, as follows: [320,384,448,512,640,704,768,832,896,960]; Change the input image size to this size for training.

[0043] It should be noted that the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In practical applications, all the above possible implementation methods can be arbitrarily combined in a combined manner to form possible embodiments of this application, which will not be described in detail here.

[0044] Based on the visible light-based dike hazard identification method provided in the above embodiments, and based on the same inventive concept, this application also provides a visible light-based dike hazard identification system.

[0045] Figure 2 This is a structural diagram of the visible light-based levee hazard identification system provided in an embodiment of this application. Figure 2 As shown, the visible light-based levee hazard identification system may specifically include a creation unit 210, an enhancement unit 220, a first construction unit 230, a second construction unit 240, and an identification unit 250.

[0046] Creation unit 210 is used to create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset; wherein, the general scenario dataset includes multiple scenario data, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes multiple real levee scenario data, and the real hazard dataset includes hazard data from multiple real levee scenarios. Enhancement unit 220 is used to perform fusion enhancement processing based on a general hazard dataset and a real scene dataset to generate an enhanced hazard dataset; The first building unit 230 is used to build a scene detection model based on a general scene dataset and a real scene dataset; The second building unit 240 is used to build a levee hazard identification model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. The recognition unit 250 is used to input the visible light image to be recognized into the scene detection model, crop out the scene feature region, and then input the cropped scene feature region into the hazard recognition model to identify the levee hazard in the visible light image to be recognized.

[0047] This application embodiment provides a possible implementation, wherein the creation unit 210 is further configured to: By flying drones above the dikes, data on the dikes is collected, generating a real-world dataset. Using annotation tools, we annotate the dangerous situations in the dike data of real-world scenario datasets to establish a real-world dangerous situation dataset.

[0048] This application embodiment provides a possible implementation, wherein the enhancement unit 220 is further configured to: The hazard targets of general samples in a general hazard dataset are transferred to real samples in a real scene dataset to achieve fusion enhancement processing and generate an enhanced hazard dataset.

[0049] This application embodiment provides a possible implementation, wherein the enhancement unit 220 is further configured to: Read the general sample image img1 and the embankment background image img2 from the real sample; The hazardous area is cropped from the target label in the general sample image img1 and denoted as mask; The mask is randomly placed on the background image img2 of the dike. The specific process is as follows: a point (x,y) is randomly selected on img2, the center point of the mask is overlapped with (x,y), the brightness h1 of the mask area and the brightness h2 of the area of ​​img2 covered by the mask are calculated respectively, the mask is adjusted to h2, and the data on img2 is replaced with the data of the mask. Feathering is performed on the connection between the mask and img2.

[0050] This application embodiment provides a possible implementation, wherein the second building unit 240 is further configured to: Using the YOLO v11 framework, an attention mechanism is added to build the initial model; The initial model is trained using a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset to construct a hazard identification model for dikes.

[0051] This application provides a possible implementation method. The constructed initial model includes a backbone network, a neck network, and a detection head. The backbone network includes a feature extraction branch, a first feature fusion module, a second feature fusion module, a spatial pyramid pooling module, and a channel and position sensitive attention module. The feature extraction branch includes an initial convolutional layer and multiple convolutional blocks connected in sequence. The first feature fusion module is connected to the first preset layer convolutional block of the feature extraction branch. The second feature fusion module is connected to the second preset layer convolutional block of the feature extraction branch. The spatial pyramid pooling module is connected to the last layer convolutional block of the feature extraction branch. The channel and position sensitive attention module is connected to the spatial pyramid pooling module. The neck network includes a first upsampling module, a fourth feature fusion module, a first feature extraction module, a second upsampling module, a fifth feature fusion module, a second feature extraction module, a first convolutional layer, a sixth feature fusion module, a third feature extraction module, a second convolutional layer, a seventh feature fusion module, and a fourth feature extraction module connected in sequence. The second upsampling module is connected to the sixth feature fusion module. The detection head includes a first attention module, a second attention module, a third attention module, a first dynamic detection head, a second dynamic detection head, and a third dynamic detection head. The first attention module is connected to the second feature extraction module, the second attention module is connected to the third feature extraction module, and the third attention module is connected to the fourth feature extraction module.

[0052] This application provides a possible implementation method. When training the initial model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, the loss function is as follows: Loss=a1×Lcls+a2×Lciou+a3×Ldfl Where a1, a2, and a3 are weight coefficients; Lcls is the target classification loss value, which uses a sigmoid classifier to perform binary classification on each category, and is the cross-entropy loss between the predicted result pred_scores and the true label target_scores; Lciou is the target localization loss value, which is the sum of the overlap ratio between the predicted bounding box and the true bounding box, the center constraint, and the aspect ratio; Ldfl is the distribution focus loss value, which is calculated by converting the true bounding box coordinates into distances relative to anchor points and limiting them to the range of 0 to reg_max-1, where reg_max=16. Then, the cross-entropy loss is calculated between the network prediction output and the true label. Here, the network prediction output is the probability distribution after softmax processing, and the true label is the bounding box distance after integer processing.

[0053] Based on the same inventive concept, this application also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the visible light-based dike hazard identification method of any of the above embodiments.

[0054] In an exemplary embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3The illustrated electronic device 300 includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this electronic device 300 does not constitute a limitation on the embodiments of this application.

[0055] Processor 301 may be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0056] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0057] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0058] The memory 303 stores computer program code that executes the scheme of this application, and its execution is controlled by the processor 301. The processor 301 executes the computer program code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0059] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0060] Those skilled in the art will clearly understand that the specific working process of the systems, devices, and modules described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0061] Those skilled in the art will understand that the technical solution of this application, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several program instructions to cause an electronic device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of this application when running the program instructions. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0062] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as electronic devices like personal computers, servers, or network devices) associated with program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of this application.

[0063] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of this application, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to leave the protection scope of this application.

Claims

1. A method for identifying dike hazards based on visible light, characterized in that, The method includes: Create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset; among them, the general scenario dataset includes data from multiple scenarios, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes data from multiple real levee scenarios, and the real hazard dataset includes hazard data from multiple real levee scenarios. Based on a general hazard dataset and a real-world scenario dataset, a fusion and enhancement process is performed to generate an enhanced hazard dataset. A scene detection model is built based on general scene datasets and real scene datasets; A hazard identification model for dikes is constructed based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. The visible light image to be identified is input into the scene detection model, the scene feature region is cropped out, and then the cropped scene feature region is input into the hazard identification model to identify the levee hazard in the visible light image to be identified.

2. The method according to claim 1, characterized in that, Create realistic scenario datasets and realistic hazard datasets, including: By flying drones above the dikes, data on the dikes is collected, generating a real-world dataset. Using annotation tools, we annotate the dangerous situations in the dike data of real-world scenario datasets to establish a real-world dangerous situation dataset.

3. The method according to claim 2, characterized in that, Based on a general hazard dataset and a real-world scenario dataset, a fusion and enhancement process is performed to generate an enhanced hazard dataset, including: The hazard targets of general samples in a general hazard dataset are transferred to real samples in a real scene dataset to achieve fusion enhancement processing and generate an enhanced hazard dataset.

4. The method according to claim 3, characterized in that, Transferring hazard targets from general samples in a general hazard dataset to real samples in a real-world scenario dataset includes: Read the general sample image img1 and the embankment background image img2 from the real sample; The hazardous area is cropped from the target label in the general sample image img1 and denoted as mask; The mask is randomly placed on the background image img2 of the dike. The specific process is as follows: a point (x,y) is randomly selected on img2, the center point of the mask is overlapped with (x,y), the brightness h1 of the mask area and the brightness h2 of the area of ​​img2 covered by the mask are calculated respectively, the mask is adjusted to h2, and the data on img2 is replaced with the data of the mask. Feathering is performed on the connection between the mask and img2.

5. The method according to any one of claims 1 to 4, characterized in that, Based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, a hazard identification model for dikes is constructed, including: Using the YOLO v11 framework, an attention mechanism is added to build the initial model; The initial model is trained using a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset to construct a hazard identification model for dikes.

6. The method according to claim 5, characterized in that, The initial model consists of a backbone network, a neck network, and a detection head. The backbone network includes a feature extraction branch, a first feature fusion module, a second feature fusion module, a spatial pyramid pooling module, and a channel and position-sensitive attention module. The feature extraction branch includes an initial convolutional layer and multiple convolutional blocks connected in sequence. The first feature fusion module is connected to the first preset layer convolutional block of the feature extraction branch. The second feature fusion module is connected to the second preset layer convolutional block of the feature extraction branch. The spatial pyramid pooling module is connected to the last layer convolutional block of the feature extraction branch. The channel and position-sensitive attention module is connected to the spatial pyramid pooling module. The neck network includes a first upsampling module, a fourth feature fusion module, a first feature extraction module, a second upsampling module, a fifth feature fusion module, a second feature extraction module, a first convolutional layer, a sixth feature fusion module, a third feature extraction module, a second convolutional layer, a seventh feature fusion module, and a fourth feature extraction module connected in sequence. The second upsampling module is connected to the sixth feature fusion module. The detection head includes a first attention module, a second attention module, a third attention module, a first dynamic detection head, a second dynamic detection head, and a third dynamic detection head. The first attention module is connected to the second feature extraction module, the second attention module is connected to the third feature extraction module, and the third attention module is connected to the fourth feature extraction module.

7. The method according to claim 5, characterized in that, When training the initial model using a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset, the loss function is as follows: Loss=a1×Lcls+a2×Lciou+a3×Ldfl Where a1, a2, and a3 are weight coefficients; Lcls is the target classification loss value, which uses a sigmoid classifier to perform binary classification on each category, and is the cross-entropy loss between the predicted result pred_scores and the true label target_scores; Lciou is the target localization loss value, which is the sum of the overlap ratio between the predicted bounding box and the true bounding box, the center constraint, and the aspect ratio; Ldfl is the distribution focus loss value, which is calculated by converting the true bounding box coordinates into distances relative to anchor points and limiting them to the range of 0 to reg_max-1, where reg_max=16. Then, the cross-entropy loss is calculated between the network prediction output and the true label. Here, the network prediction output is the probability distribution after softmax processing, and the true label is the bounding box distance after integer processing.

8. A visible light-based levee hazard identification system, characterized in that, The system includes: The creation unit is used to create a general scenario dataset, a general hazard dataset, a real scenario dataset, and a real hazard dataset. The general scenario dataset includes data from multiple scenarios, the general hazard dataset includes hazard data from multiple scenarios, the real scenario dataset includes data from multiple real levee scenarios, and the real hazard dataset includes hazard data from multiple real levee scenarios. The enhancement unit is used to perform fusion enhancement processing based on a general hazard dataset and a real-world scenario dataset to generate an enhanced hazard dataset. The first building unit is used to construct a scene detection model based on a general scene dataset and a real scene dataset; The second building unit is used to construct a levee hazard identification model based on a general hazard dataset, a real hazard dataset, and an enhanced hazard dataset. The recognition unit is used to input the visible light image to be recognized into the scene detection model, crop out the scene feature region, and then input the cropped scene feature region into the hazard recognition model to identify the levee hazard in the visible light image to be recognized.

9. The system according to claim 8, characterized in that, The creation unit is also used for: By flying drones above the dikes, data on the dikes is collected, generating a real-world dataset. Using annotation tools, we annotate the dangerous situations in the dike data of real-world scenario datasets to establish a real-world dangerous situation dataset.

10. An electronic device, characterized in that, The method includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the visible light-based method for identifying dike hazards as described in any one of claims 1 to 7.

Images