A double-layer height prior fusion detection method for unmanned aerial vehicle maritime search and rescue

Abstract

The application belongs to the technical field of unmanned aerial vehicle (UAV) maritime search and rescue, and discloses a double-layer height prior fusion detection method for UAV maritime search and rescue. The application expands the height information from a single global parameter to a spatial distribution prior by constructing a height image and introducing a longitudinal position feature. Meanwhile, the application improves the utilization depth and stability of height information in the target detection training and reasoning process by combining a GT box target level height description quantity, a height perception weighted detection loss and an auxiliary height supervision loss. The application also proposes a height-guided multi-scale detection network adapted to double-layer height prior. Based on a YOLO model, the application increases a P2 small target detection branch, removes a P5 detection output branch, and combines an EMA feature enhancement module and a height-guided multi-scale fusion module, so that the detection resources are more concentrated on maritime search and rescue small targets such as fallen personnel, and the network's adaptability to complex sea surface backgrounds, wave reflections and scale changes is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of UAV maritime search and rescue technology, and specifically relates to a two-layer height prior fusion detection method for UAV maritime search and rescue. Background Technology

[0002] In maritime search and rescue scenarios, drones need to identify and locate targets such as people in the water, lifebuoys, buoys, and small boats in real time. Compared to general scenarios, maritime target detection faces challenges such as repetitive backgrounds, strong reflections, and variable lighting and weather, resulting in low contrast and blurred boundaries between targets and backgrounds. Furthermore, due to the influence of flight altitude, pitch angle, and field of view, targets occupy a small proportion of the image, exhibiting small size, weak texture, and drastic scale variations. Therefore, maritime target detection is highly challenging.

[0003] To address the aforementioned issues, existing technologies commonly employ YOLO-based target detection methods for maritime search and rescue. While these methods can improve sensitivity to small targets, they do not utilize prior information such as flight altitude, attitude, and gimbal angle. When the UAV's altitude or the target imaging scale changes, the YOLO model can only passively rely on the statistical patterns of the training data, making it difficult to actively adjust the focus of feature detection based on flight conditions. This results in insufficient detection stability under complex sea conditions and large-scale altitude variations. To alleviate these problems, metadata such as flight conditions and imaging parameters can be introduced as auxiliary information into the target detection network.

[0004] However, existing methods have a coarse modulation granularity, making it difficult to finely model the local scale differences of different targets within the same image. Especially during target detection training, widely used data augmentation methods such as Mosaic stitch multiple images together into a single training sample. Mosaic combines four training images into one to improve generalization capabilities regarding target scale, displacement, and scene diversity. Based on this mechanism, it can be inferred that if the fusion method of "a single global height value corresponding to the entire training image" is still used, after Mosaic stitching, unclear correspondences or semantic distortions will occur between target instances in images from different sources and the single global height prior, thus affecting the consistency between height information and visual features.

[0005] In addition, existing technologies include a class of multimodal fusion target detection schemes that typically fuse data from heterogeneous sensors such as infrared, visible light, and radar with visual information to improve target perception capabilities in complex environments. In contrast, flight altitude is a low-dimensional platform state parameter, and its data form differs significantly from image features. It lacks the spatial distribution structure of infrared images and does not naturally correspond to the fixed pixel pattern of the entire image. Therefore, directly applying traditional multimodal image fusion approaches often leads to problems such as redundant fusion paths, insufficient interpretability, and poor coupling with the detection enhancement process.

[0006] In summary, existing methods for detecting maritime search and rescue targets based on the YOLO model have the following shortcomings: 1. Insufficient utilization of altitude prior. Existing YOLO-like methods mainly rely on pure visual feature enhancement to improve small target detection performance, lacking effective utilization of the external prior of UAV flight altitude. This results in the model only being able to passively adapt to changes in target scale under different altitude conditions, making it difficult to maintain stable detection capabilities when flight altitude changes significantly.

[0007] 2. Existing methods for utilizing height information are too coarse-grained. Most existing solutions treat height as a conditional input at the whole-image level. While this provides some prior information, it is difficult to directly apply it to specific target instances. Especially under Mosaic enhancement conditions, if a single global height value is still used, it will cause semantic mismatch between targets in images from different sources and the height prior information.

[0008] 3. There is a modal difference between height information and high-dimensional visual features, and a complete training and carrying mechanism is lacking. If height is simply used as a scalar and directly concatenated or weighted, it is often difficult to establish a stable and effective mapping relationship.

[0009] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art. Summary of the Invention

[0010] The purpose of this invention is to propose a two-layer height prior fusion detection method for UAV maritime search and rescue. This method improves the model's ability to detect small and medium-sized targets at sea under different flight altitude conditions and enhances its scene adaptability by proposing a two-layer height prior fusion detection method based on image-level height images and ground truth box-level height supervision.

[0011] To achieve the above objectives, the present invention adopts the following technical solution: A two-layer height prior fusion detection method for UAV maritime search and rescue includes the following steps: Step 1. First, acquire images of the UAV-based maritime search and rescue target as the raw input images; The original input image is sequentially processed by flight altitude normalization, constructing image vertical position features, and constructing image-level altitude image preprocessing. The image-level altitude image is then used as an additional input and synchronously enhanced with the original input image. Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision map is constructed. Step 2. Build a UAV maritime search and rescue target detection model based on the improved YOLO model, where the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; The model is based on the YOLOv8n structure, with the addition of the P2 detection branch and the removal of the P5 detection output branch. It also introduces the EMA feature enhancement module and adds the highly guided multi-scale fusion module HGMF at key fusion locations. HGMF enhances the responsiveness of target-related regions by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information to weight and modulate visual fusion features at various scales. Step 3. Train the UAV maritime search and rescue target detection model using the training dataset constructed from the synchronously enhanced images, and use the trained model to realize maritime search and rescue target detection; During training, the total loss function consists of highly perceptive weighted detection loss and auxiliary highly supervised loss.

[0012] Furthermore, based on the aforementioned two-layer height prior fusion detection method for UAV maritime search and rescue, this invention also proposes a corresponding two-layer height prior fusion detection system for UAV maritime search and rescue, the technical solution of which is as follows: A dual-layer altitude prior fusion detection system for unmanned aerial vehicle (UAV) maritime search and rescue includes the following modules: The preprocessing module is used to first acquire images of the UAV maritime search and rescue target as the raw input image; The original input image is sequentially processed by flight altitude normalization, constructing image vertical position features, and constructing image-level altitude image preprocessing. The image-level altitude image is then used as an additional input and synchronously enhanced with the original input image. Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision map is constructed. And a prediction module, used to build a UAV maritime search and rescue target detection model based on the improved YOLO model, wherein the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; The model is based on the YOLOv8n structure, with the addition of the P2 detection branch and the removal of the P5 detection output branch. It also introduces the EMA feature enhancement module and adds the highly guided multi-scale fusion module HGMF at key fusion locations. HGMF enhances the responsiveness of target-related regions by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information to weight and modulate visual fusion features at various scales. The training dataset constructed using images with synchronous enhancement processing was used to train the UAV maritime search and rescue target detection model, and the trained model was used to realize maritime search and rescue target detection. During training, the total loss function consists of highly perceptive weighted detection loss and auxiliary highly supervised loss.

[0013] Furthermore, based on the aforementioned dual-layer height prior fusion detection method for UAV maritime search and rescue, this invention also proposes a computer device, which includes a memory and one or more processors.

[0014] The executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the aforementioned two-layer altitude prior fusion detection method for UAV-based maritime search and rescue.

[0015] Furthermore, based on the aforementioned two-layer height prior fusion detection method for UAV maritime search and rescue, this invention also proposes a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the aforementioned two-layer height prior fusion detection method for UAV maritime search and rescue.

[0016] The present invention has the following advantages: As described above, this invention discloses a two-layer altitude prior fusion detection method for UAV maritime search and rescue. By constructing an image-level altitude image and introducing vertical position features, the altitude information is expanded from a single global parameter to a spatially distributed prior. Furthermore, it combines ground truth bounding box target-level altitude descriptors, altitude-aware weighted detection loss, and auxiliary altitude supervision loss, enabling altitude information to not only play a role in the input layer and enhancement stage but also directly participate in the target-level training process. This solves the problems of coarse-grained utilization of traditional altitude information, difficulty in compatibility with Mosaic enhancement, and difficulty in applying it to target instance supervision, thus improving the depth and stability of altitude information utilization in target detection training and inference processes. Compared with existing schemes that mainly use a single altitude scalar or whole-image-level conditional assistance, this invention can more fully and stably utilize UAV flight altitude prior information. Furthermore, this invention presents a height-guided multi-scale detection network adapted to a two-layer height prior. Based on the YOLOv8 detection network, it adds a P2 small target detection branch, removes the P5 detection output branch, and combines an EMA feature enhancement module and a height-guided multi-scale fusion module. This allows detection resources to be more focused on small and medium-sized targets in maritime search and rescue scenarios, such as people in the water, lifebuoys, and buoys. It also enhances the network's adaptability to complex sea surface backgrounds, wave reflections, and scale variations. Compared to existing schemes that mainly rely on pure visual feature enhancement or simply introduce height conditions, this invention improves the matching degree between the small and medium-sized target detection structure and task requirements in maritime search and rescue scenarios, while avoiding the reliance on large-scale network expansion for performance improvement. Therefore, it has better training compatibility, ease of implementation, and application promotion value. This invention improves the model's detection capability and scene adaptability for small and medium-sized targets at sea under different flight altitude conditions, enhancing the adaptability of small and medium-sized target detection in maritime search and rescue scenarios. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of the dual-layer height prior fusion detection method for UAV maritime search and rescue in this embodiment of the invention; Figure 2 This is a schematic diagram of image-level height image generation in an embodiment of the present invention; Figure 3 This is a network structure diagram of the UAV maritime search and rescue target detection model in an embodiment of the present invention; Figure 4 This is a network structure diagram of the highly guided multi-scale fusion module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the GT box-level height supervision construction in an embodiment of the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 As shown, this invention addresses the problems of existing UAV-based maritime search and rescue target detection methods, such as the difficulty in effectively utilizing flight altitude priors, the difficulty in balancing Mosaic enhancement compatibility with target-level supervision effectiveness, and the difficulty in stably fusing altitude information with visual features. To solve these problems, this invention proposes a dual-layer altitude prior fusion detection method for UAV-based maritime search and rescue. This method takes maritime search and rescue images collected by the UAV and their corresponding flight altitude information as input. Based on a YOLO-like target detection network, it first constructs a global-target dual-layer altitude prior by building image-level altitude images and ground truth bounding box target-level altitude descriptions, and further constructs an altitude-aware training loss. Then, it adds a multi-scale detection branch adapted to the dual-layer altitude prior, an altitude-guided fusion module, and a feature enhancement module to the detection network. This improves the network's ability to detect small and medium-sized targets such as people in the water, lifebuoys, and buoys in maritime search and rescue scenarios, while ensuring that altitude information can continue to participate in common data augmentation processes such as Mosaic. Specifically, this invention does not append flight altitude as a single scalar to the network after image enhancement. Instead, it first generates an image-level height image with the same spatial dimensions as the original image based on the flight altitude and the image's vertical position. This height image is then used as an additional channel, forming a four-channel input along with the original RGB image. During Mosaic enhancement, the original RGB image, the height image, and the corresponding ground truth bounding box annotations are synchronously transformed using the same stitching, scaling, cropping, and translation parameters. This ensures that each local region in the enhanced image retains its prior height information from the source image. This avoids the semantic mismatch problem that occurs between traditional whole-image-level height scalars and local target regions after multi-image stitching, guaranteeing that height information maintains spatial consistency with image content and target annotations throughout common data enhancement processes such as Mosaic.

[0019] like Figure 1 As shown, the two-layer height prior fusion detection method for UAV maritime search and rescue includes the following steps: Step 1. First, acquire images of the UAV-based maritime search and rescue target as the raw input images; The original input image is sequentially processed by flight altitude normalization, construction of vertical position features, and construction of image-level altitude image preprocessing. The image-level altitude image is then used as an additional input for synchronous enhancement processing with the original input image.

[0020] Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision graph is constructed.

[0021] This invention proposes a two-layer height prior fusion detection method based on image-level height images and GT box-level height supervision, which helps to improve the model's ability to detect small and medium-sized targets at sea under different flight altitude conditions and its scene adaptability.

[0022] This invention first constructs a two-layer altitude prior training method, which involves normalizing the flight altitude, introducing the longitudinal position feature y, and generating an altitude image that can participate in Mosaic enhancement synchronously with the original image.

[0023] Meanwhile, target-level height descriptors are further extracted from the ground truth boxes, and height-aware weighted detection loss and auxiliary height supervision loss are constructed, so that height information can participate in training at the image level and directly affect the supervision process at the target level.

[0024] This embodiment uses the SeaDronesSee dataset to validate the proposed dual-layer height prior fusion detection method for UAV maritime search and rescue. The SeaDroneSeev2 dataset contains targets in the following categories: ignore, swimmer, boat, jetski, life-saving equipment, and buoy.

[0025] To improve the standardization of categories and the generalization ability of the model, this invention renames the "swimmer" category to the more general name "person"; ignores the "ignore" category; and merges "boat" and "jetski" into a unified category "boat" because the two have high similarity in visual features. Merging them can reduce category redundancy and improve detection performance.

[0026] The training set contains a total of 8,930 images, of which 5,800 images have complete height information, complete motion state information, and complete pose information, and 5,800 images have all three types of complete information.

[0027] In the validation set, there are a total of 1547 images, of which 1030 images have complete height information, complete motion state information, and complete pose information, and 1030 images have all three types of complete information.

[0028] Based on the above statistical results, a data subset DataB was constructed using samples that simultaneously possess all three types of complete information. The training set contains 5800 images, and the validation set contains 1030 images. The number of cleaned and recombined target instances in the four categories are as follows: person (22737), boat (11695), life_saving (767), and buoy (3206).

[0029] Let the input image be .in, This represents the original input image, i.e., the original RGB image; Indicates the height of the input image; Indicates the width of the input image; This represents a three-channel image space composed of real numbers.

[0030] Let the image annotation set be: ;in, Indicates the coordinates of the target bounding box; Indicates the target category label; This represents the set of target annotations for the current image.

[0031] This invention uses YOLOv8n as the basic detection network, initializes it with yolov8n.pt pre-trained weights, sets the input image size to 640×640, the batch size to 8, and the training epochs to 100.

[0032] The training process employs the conventional data augmentation strategy of Mosaic.

[0033] Next, a dual-layer height prior is constructed. This invention first constructs an image-level height image, and then further constructs a ground truth bounding box object-level height descriptor based on this, so that height information participates in training at both the image level and the object level.

[0034] Step 1 of this paper mainly addresses the problem that existing methods of utilizing height information are primarily limited to the whole-image level and are difficult to directly apply to target instances. It proposes a global-target two-layer height prior fusion training method, in which: By constructing a height image, the height information can be synchronized with the original image to participate in enhancement processes such as Mosaic.

[0035] By extracting highly relevant descriptors for each ground truth (GT) box and incorporating them into loss weighting and auxiliary supervision, the height prior is further applied to the target-level training process, thereby improving the actual constraint of height information on the detection model.

[0036] like Figure 2 As shown, the image-level height image generation process in this embodiment is as follows: Step 1.1. Flight altitude normalization.

[0037] Let the flight altitude corresponding to the input image be... First, the flight altitude is normalized using the following formula: ; in and These are the minimum and maximum flight altitudes in the training dataset samples; This is the normalized height value.

[0038] Step 1.2. Construct the vertical position features of the image.

[0039] Considering that the target scale in UAV images is related not only to the flight altitude but also to the target's vertical position in the image, a vertical position feature of the image is further introduced. Let the input image size be... .

[0040] For any pixel position in the input image Its normalized vertical position feature is defined as: ; in This represents the vertical coordinate of a pixel in the image. This indicates the vertical positional features of the image.

[0041] This yields a vertical position feature map with the same size as the input image. : ; Step 1.3. Construct an image-level height image.

[0042] Obtaining normalized height values and vertical position feature map Then, construct an image-level height image. , means as follows: ; in, The vertical position feature weighting coefficient, Used to control the degree to which the vertical position of an image affects the height image. This represents a truncation function used to restrict the result to a specific range. Within the range.

[0043] Represents image-level height image The position of the middle pixel is The pixel value at that location.

[0044] This image-level height image It is not a simple constant graph, but a two-dimensional prior image generated jointly by flight altitude and longitudinal position. Of course, the representation of the altitude image in this embodiment can also be replaced with the following form: The height image is constructed using a mapping function, the formula of which is as follows: ; in The height image mapping function can be a linear mapping function, a piecewise function, or a lightweight perceptron mapping function.

[0045] Step 1.4. Convert the image-level height image As an additional input, it is enhanced synchronously with the original input image.

[0046] To ensure that the height information is compatible with the Mosaic target detection enhancement workflow, the generated image-level height image is directly used. As an auxiliary input corresponding to the original image space; let the original input image be: The superscript 3 indicates the number of channels.

[0047] Then the image-level height image Expanded into a single-channel auxiliary image and combined with the original input image to obtain the expanded input. : ; in, This indicates a channel-based stitching operation, resulting in the combined input image. Four-channel input: ; in This represents a four-channel image space composed of real numbers.

[0048] Suppose four four-channel input images are respectively Then its Mosaic result is expressed as: ; in This represents the four-channel input image enhanced by Mosaic, which consists of the enhanced RGB image and the enhanced image-level height image. This indicates a multi-image stitching enhancement operation.

[0049] Therefore, since the original image and the image-level height image use the same stitching, scaling and translation parameters, each local region in the enhanced image still corresponds to the height prior information of its original source image.

[0050] This approach avoids the semantic mismatch problem that arises when a traditional single global height value is used under Mosaic enhancement conditions.

[0051] Based on the image-level height image, this step further constructs a ground truth bounding box target-level height descriptor.

[0052] Step 1.5. Construct the target-level height description of the GT box.

[0053] In addition to the image-level height image, this invention extracts a target-level height descriptor for each ground truth (GT) box.

[0054] Let the first target bounding box coordinates for: ; in , They represent the first The top-left and bottom-right coordinates of the target bounding box are pixel coordinates. The image coordinate system has the top-left corner of the image as the origin, and the horizontal axis is to the right. The positive direction of the axis, with the longitudinal axis pointing downwards. Positive direction of the axis.

[0055] Definition of the first Vertical position of the normalization center of each target for: ; in Represents the image height, defining the first... Normalized area of ​​each target for: ; Image-level height image Above, for the first The average height response of the target bounding boxes is obtained by averaging the values ​​of each bounding box region. : ; in, Indicates the first The pixel area covered by each GT box, This indicates the number of pixels in the corresponding area.

[0056] Further construct the joint height description vector of the target. : .

[0057] The joint height description vector of the target is used as the target-level height description.

[0058] Step 1.6. Construct a GT-level box-level height supervised graph.

[0059] Based on the height response of each GT frame Constructing a GT-level box-level height supervised graph : ; Indicates pixel position The GT box-level height supervision value at the location; where For indicator functions, when pixel Falling into The value is 1 if there is a GT box region, otherwise it is 0; GT box-level height supervision map Sampling to each detection scale separately ,get: ; in This indicates a downsampling operation at the corresponding scale; These correspond to the P2, P3, and P4 detection branches, respectively.

[0060] In this embodiment, the image-level height image and the GT box-level height supervision in the dual-layer height prior are used together.

[0061] Among them, the image-level height image is used to ensure that the flight altitude prior can persist in the input layer and the enhancement stage, and maintain spatial alignment with the image content; the ground truth box target-level height description, the height-aware weighted detection loss, and the auxiliary height supervision map are used to further transfer the height information from the whole image level to the target-level training process.

[0062] The detection network adopts a three-scale detection output structure of P2, P3, and P4. The P2 branch enhances the perception of small and medium-sized targets at sea. The P5 detection output is removed, but its feature layer still participates in feature fusion. The EMA module is used to strengthen the features of key small targets. The highly guided multi-scale fusion module is used to improve the network's ability to utilize high-level priors.

[0063] Step 2. Build a UAV maritime search and rescue target detection model based on the improved YOLO model, where the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; After completing the two-layer height prior construction and the joint training design for height perception, this step further constructs a detection network adapted to the aforementioned height prior. The model is based on the YOLOv8n architecture, and the input image is set as follows. .

[0064] in and These represent the height and width of the input image, respectively. Traditional YOLOv8n has... , , Three detection branches.

[0065] The input image is fed into the backbone feature extraction network to obtain feature maps at different levels: ; in This is a shallow, high-resolution feature map. This is a deep, low-resolution feature map.

[0066] In the original YOLO detection structure, the detection head typically operates on three scale features: ; in, , , These are feature maps of three detection scales obtained through the feature fusion network.

[0067] Based on this, the present invention achieves high-level prior fusion and detection network improvement through the following two steps.

[0068] The model is based on the YOLOv8n architecture, adding a P2 detection branch and removing the P5 detection output branch. It also introduces an EMA feature enhancement module and incorporates a height-guided multi-scale fusion module (HGMF) at key fusion locations to improve the practical utilization of height information for multi-scale detection features. The improved network structure is as follows: Figure 3 As shown.

[0069] HGMF, by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information, performs weight guidance and feature modulation on visual fusion features at various scales to enhance the responsiveness of target-related regions.

[0070] To address the issues of small target size, weak texture, and easy missed detection in maritime search and rescue scenarios, this invention adds a P2 detection branch to the original YOLOv8n model detection structure, introducing shallow high-resolution features into the detection output.

[0071] Specifically, the shallow features output by the backbone network The P2 feature map is constructed by fusing the upsampling results of the previous level detection features: .

[0072] In a preferred embodiment, it can be further expressed as: ;in, Indicates an upsampling operation. Indicates feature splicing, This indicates a convolution fusion operation.

[0073] After adding the P2 detection branch, the detection output set changes from the original three-scale output to a four-scale output: .

[0074] After adding the P2 detection branch, the P5 detection output branch is further removed to reduce redundant detection of large targets.

[0075] It should be noted that this step is not deletion. Instead of the feature layer itself, only its detection output is removed. It still participates in the multi-scale feature fusion process, for example: .

[0076] but The data will no longer be sent to the final detection head for independent prediction. Therefore, the adjusted set of detection output heads becomes: .

[0077] After completing P2 enhancement and P5 output removal, to address the issues of strong interference from sea surface reflection, wave texture, and weak target texture, an EMA feature enhancement module was added to the key detection scale feature path to improve the target feature representation capability.

[0078] like Figure 3 As shown, the EMA feature enhancement module is set in the neck network of the YOLOv8n model and is located before the output head corresponding to the P2 detection branch. The processing flow of the EMA feature enhancement module is as follows: Let the feature map input to the EMA module be... First, the feature map Perform channel statistics to obtain channel description vectors. : ; in This represents global average pooling; then, channel weights are generated through a mapping function. : ; in and For learnable parameter matrix, Represents a non-linear activation function. This represents the Sigmoid function.

[0079] Further spatial attention enhancement is applied to the feature map to obtain the spatial weight map. : ; in, and These represent average pooling and max pooling along the channel dimension, respectively.

[0080] Finally, channel and spatial joint enhancement is performed on the input feature map to obtain the enhanced feature map. : .

[0081] To enable the height image and GT box-level height supervision information constructed in step 1 to play a more significant role in the network feature fusion process, a height-guided multi-scale fusion module is introduced after the concatenation of features at the P2, P3, and P4 scales and before the C2f module.

[0082] like Figure 4 As shown, this height-guided multi-scale fusion module simultaneously introduces height feature maps of the same scale and GT box-level height supervision information to perform weight guidance and feature modulation on visual fusion features at each scale, thereby enhancing the response capability of target-related regions and increasing the utilization depth of height priors in the multi-scale detection process.

[0083] The Highly Guided Multi-Scale Fusion (HGMF) module is set in the neck network of the YOLOv8n model; specifically, the HGMF module is introduced after the concatenation of features at the P2, P3, and P4 scales and before the C2f module.

[0084] The processing flow of the HGMF module is as follows: Let the first The visual fusion feature map after feature stitching at each scale is as follows: ;in, Indicates the first A visual fusion feature map of each scale after feature stitching in the neck network; Indicates the detection scale level.

[0085] The image-level height image generated in step 1 Downsampling to the corresponding scale yields a height feature map of the same scale: ;in, Indicates the first Height feature map after downsampling at each scale; This indicates a downsampling operation at the corresponding scale; Represents a height image. Indicates the detection scale level.

[0086] Visual fusion feature maps and height feature map of the same scale The concatenation is performed, and the first generation is generated through convolutional mapping and activation functions. Initial height guided weight map at each scale : ; in Indicates the activation function; Represents a convolution mapping; This indicates a feature splicing operation.

[0087] Supervised graph of GT box level height constructed in step 1 Performing downsampling at the same scale yields: ;in, Indicates the first GT-level bounding box height supervision map after downsampling at each scale; This represents the original GT box-level height supervision map; This indicates the downsampling operation at the corresponding scale.

[0088] Obtaining the initial height guide weight map and the GT-level frame height supervision map at various scales Then, the initial height guiding weights are corrected to obtain the corrected height guiding weight map with the ground truth bounding boxes participating in the process. : ; in, The adjustment factor represents the level of supervision at the GT (Gross Target) level. This indicates element-wise multiplication.

[0089] The visual fusion features are modulated using the corrected height-guided weight map to obtain the height-guided feature map: ; in, Indicates the first Feature maps at various scales after height guidance; This indicates element-wise multiplication.

[0090] The height-guided feature map is then fed into the subsequent C2f module to obtain the detection features at the corresponding scale: ; in, Indicates the first Detection feature maps at various scales; This indicates the feature extraction module.

[0091] Through the above approach, this step enables the continuous spatial height prior generated from the image-level height image and the GT bounding box-level height supervision information to simultaneously participate in the multi-scale feature fusion process. Specifically, the height image primarily provides continuous height features at the same scale, while the GT bounding box-level height supervision mainly corrects and constrains the height-guided weights. Both work together at the three detection scales P2, P3, and P4, thereby improving the depth of utilization of height information within the detection network and the representational ability of target-related regions.

[0092] Therefore, the detection network used in this invention not only completes the small target-friendly detection scale reconstruction, but also enables the image-level height image to play a role in the feature fusion process at each scale through the height-guided multi-scale fusion module, thus forming a complete two-layer height prior detection method together with the aforementioned GT box-level height supervision.

[0093] This invention addresses the problem that existing pure vision detection structures cannot effectively support height priors. It constructs a multi-scale detection network adapted to a two-layer height prior. This network adds a P2 small target detection branch to the basic detection network YOLOv8 to enhance the high-resolution representation of small targets in maritime search and rescue. The P5 detection output branch is removed, retaining only the P5 feature layer for multi-scale feature fusion to reduce large-scale redundant detection. An EMA feature enhancement module is introduced in the key small target branch to strengthen target-related feature responses. Simultaneously, a height-guided multi-scale fusion module is introduced at key fusion locations at scales P2, P3, and P4, jointly modulating the height features after height image downsampling with visual features, allowing the height prior to continue playing a role in the feature fusion process. Through the technical means of "P2 enhancement + P5 output removal + EMA enhancement + height-guided fusion," a detection network implementation path matching the two-layer height prior training method is formed.

[0094] Step 3. Train the UAV maritime search and rescue target detection model using the training dataset constructed from the synchronously enhanced images, and then use the trained model to detect maritime search and rescue targets. In this embodiment, yolov8n.pt is used as the pre-training weights, the input image size is uniformly set to 640×640, the batch size is set to 8, and the training epochs are set to 100.

[0095] The training process employs a Mosaic augmentation strategy, and the training samples use the previously cleaned subset of Data B.

[0096] In terms of fusion training, this invention addresses the problem that existing methods of utilizing height information mostly remain at the whole-image level and are difficult to directly apply to target instance training, and proposes a two-layer height prior fusion training method.

[0097] First, the flight altitude information is normalized, and a height image with the same spatial resolution as the original image is constructed by combining the vertical position feature y of the image. This allows the altitude information to participate in the Mosaic enhancement operation synchronously with the original image in image form. Second, target-level altitude descriptions, such as the center vertical position, normalized area, and average altitude response of the region, are extracted for each ground truth (GT) bounding box (see step 1 for details). Based on these descriptions, an altitude-aware weighted detection loss and an auxiliary altitude supervision loss are constructed, allowing altitude information to be further transferred from the whole image level to the target-level training process. This invention achieves the coordinated utilization of altitude priors at the image and target levels through the technical means of "altitude image generation + GT bounding box-level altitude description + altitude-aware weighted detection loss".

[0098] The total loss function is designed as the sum of highly perceptive weighted detection loss and auxiliary highly supervised loss.

[0099] First, a height-based weighted detection loss is used based on the ground truth bounding box. To incorporate the target-level height description into the training process, based on the joint height description vector... Generate target-level height weights : ; in This represents a highly weighted generation function. Indicates the first The height perception weight corresponding to each target.

[0100] Based on this, the original detection loss is rewritten as a highly perceptual weighted detection loss. : ; in Indicates the target quantity; , and They represent the first The bounding box regression loss, classification loss, and distribution focus loss for each target; , and This is the loss balance coefficient.

[0101] Then, a GT box-level height supervision graph is constructed, and an auxiliary height supervision loss is generated.

[0102] Specifically, based on the height response of each GT box Constructing a GT-level box-level height supervised graph : ; in Indicates the first The height response corresponding to each GT box; Indicates the first One GT frame area; For indicator functions, when pixel Falling into The value is 1 if there is a GT box region, otherwise it is 0.

[0103] GT box-level height supervision map Sampling to each detection scale separately ,get: ;in Indicates the first GT box-level height supervision map at various scales This indicates the downsampling operation at the corresponding scale. These correspond to the P2, P3, and P4 detection branches, respectively.

[0104] Suppose that the height-guided multi-scale fusion module HGMF is at the 1st... Each scale generates a height-guided weight map. This is used as the height response map at that scale, and the downsampled GT box-level height supervision map is used for constraint.

[0105] Define the auxiliary height supervision loss as: ; in This indicates the loss due to high level of supervision. Indicates the first The height-guided weight map generated by the HGMF module at each scale; Indicates the first GT box-level height supervision map at various scales; This represents the L1 norm.

[0106] Ultimately, the total training loss is: .

[0107] in Indicates total training loss. This represents the weighting coefficient of the auxiliary highly supervised loss.

[0108] In a preferred embodiment, The range of values ​​is , , , and All are positive numbers.

[0109] It should be noted that the ground truth bounding box-level height supervision map proposed in this embodiment is constructed only from real labeled bounding boxes during the training phase and is used to constrain the height guidance weight map generated by the HGMF module. During the actual inference phase of the UAV maritime search and rescue target detection model, only the original RGB image and its corresponding height image are received to output the maritime search and rescue target detection results.

[0110] Example 2 This embodiment 2 describes a two-layer height prior fusion detection system for UAV maritime search and rescue, which is based on the same inventive concept as the two-layer height prior fusion detection method for UAV maritime search and rescue in embodiment 1.

[0111] This system for dual-layer altitude prior fusion detection for UAV maritime search and rescue includes the following modules: The preprocessing module is used to first acquire images of the UAV maritime search and rescue target as the raw input image; The original input image is sequentially processed by flight altitude normalization, constructing image vertical position features, and constructing image-level altitude image preprocessing. The image-level altitude image is then used as an additional input and synchronously enhanced with the original input image. Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision map is constructed. And a prediction module, used to build a UAV maritime search and rescue target detection model based on the improved YOLO model, wherein the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; The model is based on the YOLOv8n structure, with the addition of the P2 detection branch and the removal of the P5 detection output branch. It also introduces the EMA feature enhancement module and adds the highly guided multi-scale fusion module HGMF at key fusion locations. HGMF enhances the responsiveness of target-related regions by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information to weight and modulate visual fusion features at various scales. The training dataset constructed using images with synchronous enhancement processing was used to train the UAV maritime search and rescue target detection model, and the trained model was used to realize maritime search and rescue target detection. During training, the total loss function is designed to be the sum of highly perceptive weighted detection loss and auxiliary highly supervised loss.

[0112] It should be noted that any content not mentioned in the above-described functional modules of the system described in Embodiment 2 can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be repeated in detail here.

[0113] Example 3 This embodiment 3 describes a computer device including a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the two-layer altitude prior fusion detection method for UAV maritime search and rescue described in embodiment 1 above.

[0114] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0115] Example 4 This embodiment 4 describes a computer-readable storage medium storing a program that, when executed by a processor, is used to implement the steps of the dual-layer height prior fusion detection method for UAV maritime search and rescue described in embodiment 1 above.

[0116] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0117] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A two-layer height prior fusion detection method for UAV maritime search and rescue, characterized in that, Includes the following steps: Step 1. First, acquire images of the UAV-based maritime search and rescue target as the raw input images; The original input image is sequentially processed by flight altitude normalization, constructing image vertical position features, and constructing image-level altitude image preprocessing. The image-level altitude image is then used as an additional input and synchronously enhanced with the original input image. Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision map is constructed. Step 2. Build a UAV maritime search and rescue target detection model based on the improved YOLO model, where the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; The model is based on the YOLOv8n structure, with the addition of the P2 detection branch and the removal of the P5 detection output branch. It also introduces the EMA feature enhancement module and adds the highly guided multi-scale fusion module HGMF at key fusion locations. HGMF enhances the responsiveness of target-related regions by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information to weight and modulate visual fusion features at various scales. Step 3. Train the UAV maritime search and rescue target detection model using the training dataset constructed from the synchronously enhanced images, and use the trained model to realize maritime search and rescue target detection; During training, the total loss function consists of highly perceptive weighted detection loss and auxiliary highly supervised loss.

2. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 1, characterized in that, Step 1 specifically involves: Let the flight altitude corresponding to the input image be... The input image size is , and Indicates the image height and width; Step 1.

1. Normalize flight altitude; First, the flight altitude is normalized using the following formula: ； in and These are the minimum and maximum flight altitudes in the training dataset samples; Normalized height value; Step 1.

2. Construct the vertical position features of the image; For any pixel position in the input image Its normalized vertical position feature is defined as: ； in This represents the vertical coordinate of a pixel in the image. Indicates the vertical positional features of the image; This yields a vertical position feature map with the same size as the input image. : ； Step 1.

3. Construct an image-level height image; Obtaining normalized height values and vertical position feature map Then, construct an image-level height image. , means as follows: ； in, These are the vertical position feature weighting coefficients, used to control the degree to which the vertical position of the image affects the height image. This represents a truncation function used to restrict the result to a specific range. Within the range; Represents image-level height image The position of the middle pixel is Pixel value at; Step 1.

4. Convert the image-level height image As an additional input, and enhanced synchronously with the original input image; The generated image-level height image As an auxiliary input corresponding to the original image space; Let the original input image be: The superscript 3 indicates the number of channels. Image-level height image Expanded into a single-channel auxiliary image and combined with the original input image to obtain the expanded input. : ； in This indicates a channel-based stitching operation, resulting in the combined input image. Four-channel input: ； Suppose four four-channel input images are respectively Then its Mosaic result is expressed as: ； in This represents the four-channel input image after Mosaic enhancement, which consists of the enhanced RGB image and the enhanced image-level height image. This indicates a multi-image stitching enhancement operation.

3. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 2, characterized in that, In step 1.3, the image-level height image is replaced by a mapping function, the formula of which is as follows: ； in The height image mapping function can be a linear mapping function, a piecewise function, or a lightweight perceptron mapping function.

4. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 2, characterized in that, In step 1, the process of extracting the target-level height descriptor and constructing the ground truth (GT) box-level height supervision map is as follows: Step 1.

5. In addition to the image-level height image, extract the target-level height descriptor for each ground truth bounding box; Let the first target bounding box coordinates for: ； in , They represent the first The coordinates of the top left and bottom right positions of the target bounding box; Definition of the first Vertical position of the normalization center of each target for: ； in Represents the image height, defining the first... Normalized area of ​​each target for: ； in Indicates the image width; Image-level height image Above, for the first The average height response of the target bounding boxes is obtained by averaging the values ​​of each bounding box region. : ； in, Indicates the first The pixel area covered by each GT box, Indicates the number of pixels in the corresponding area; Further construct the joint height description vector of the target. : ； Joint height description vector of the target As the first The target-level height description of each GT box; Step 1.

6. Construct a GT-level box-level height supervised graph; Based on the height response of each GT frame Constructing a GT-level box-level height supervised graph : ； in Indicates pixel position The GT box-level height supervision value at the location; For indicator functions, when pixel Falling into The value is 1 if there is a GT box region, otherwise it is 0; GT box-level height supervision map Sampling to each detection scale separately ,get: ； in This indicates a downsampling operation at the corresponding scale; These correspond to the P2, P3, and P4 detection branches, respectively.

5. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 1, characterized in that, The EMA feature enhancement module is set in the neck network of the YOLOv8n model; The EMA feature enhancement module is located before the output head corresponding to the P2 detection branch, and its processing flow is as follows: Let the feature map input to the EMA module be... First, the feature map Perform channel statistics to obtain channel description vectors. : ； in This represents global average pooling; then channel weights are generated through a mapping function. : ； in and For learnable parameter matrix, Represents a non-linear activation function. Represents the Sigmoid function; Further spatial attention enhancement is applied to the feature map to obtain the spatial weight map. : ； in, and These represent average pooling and max pooling along the channel dimension, respectively; Finally, channel and spatial joint enhancement is performed on the input feature map to obtain the enhanced feature map. : 。 6. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 1, characterized in that, The height-guided multi-scale fusion module HGMF is set in the neck network of the YOLOv8n model; Specifically, the HGMF module is introduced after the features of the three scales P2, P3, and P4 are concatenated and before the C2f module; The processing flow of the HGMF module is as follows: Let the first The visual fusion feature map after feature stitching at each scale is as follows: ;in, Indicates the first A visual fusion feature map of each scale after feature stitching in the neck network; Indicates the detection scale level; The image-level height image generated in step 1 Downsampling to the corresponding scale yields a height feature map of the same scale: ;in Indicates the first Height feature map after downsampling at each scale; This indicates a downsampling operation at the corresponding scale; Indicates the detection scale level; Visual fusion feature maps and height feature map of the same scale The concatenation is performed, and the first generation is generated through convolutional mapping and activation functions. Initial height guided weight map at each scale : ; in Indicates the activation function; Represents a convolution mapping; Indicates feature concatenation operation; Supervised graph of GT box level height constructed in step 1 Performing downsampling at the same scale yields: ;in Indicates the first GT-level bounding box height supervision map after downsampling at each scale; This represents the original GT box-level height supervision map; This indicates a downsampling operation at the corresponding scale; Obtaining the initial height guide weight map and the GT-level frame height supervision map at various scales Then, the initial height guiding weights are corrected to obtain the corrected height guiding weight map with the ground truth bounding boxes participating in the process. : ； in, The adjustment factor represents the level of supervision at the GT (Gross Target) level. This represents element-wise multiplication; The visual fusion features are modulated using the corrected height-guided weight map to obtain the height-guided feature map: ； in, Indicates the first Feature maps at various scales after height guidance; This represents element-wise multiplication; The height-guided feature map is then fed into the subsequent C2f module to obtain the detection features at the corresponding scale: ； in, Indicates the first Detection feature maps at various scales; This indicates the feature extraction module.

7. The dual-layer height prior fusion detection method for UAV maritime search and rescue according to claim 4, characterized in that, In step 3, the design process of the total loss function during training is as follows: First, a height-based weighted detection loss is used based on the ground truth bounding box. Specifically, based on the joint height description vector Generate target-level height weights : ； in This represents a highly weighted generation function. Indicates the first The height perception weight corresponding to each target; Based on this, the original detection loss is rewritten as a highly perceptual weighted detection loss. : ； in Indicates the target quantity; , and They represent the first The bounding box regression loss, classification loss, and distribution focus loss for each target; , and This is the loss balance coefficient; Then, based on the constructed GT box-level height supervision map, an auxiliary height supervision loss is generated; Specifically, based on the height response of each GT box Constructing a GT-level box-level height supervised graph : ； in Indicates the first The height response corresponding to each GT box; Indicates the first One GT frame area; For indicator functions, when pixel Falling into The value is 1 if there is a GT box region, otherwise it is 0; GT box-level height supervision map Sampling to each detection scale separately ,get: ;in Indicates the first GT box-level height supervision map at various scales This indicates the downsampling operation at the corresponding scale. These correspond to detection branches P2, P3, and P4, respectively. Suppose that the height-guided multi-scale fusion module HGMF is at the 1st... Each scale generates a height-guided weight map. This is used as the height response map at that scale, and the downsampled GT box-level height supervision map is used for constraints. Define the auxiliary height supervision loss as: ； in This indicates the loss due to high level of supervision. Indicates the first The height-guided weight map generated by the HGMF module at each scale; Indicates the first GT box-level height supervision map at various scales; Represents the L1 norm; The final total training loss is: ; in Indicates total training loss. This represents the weighting coefficient of the auxiliary highly supervised loss.

8. A dual-layer altitude prior fusion detection system for unmanned aerial vehicle (UAV) maritime search and rescue, characterized in that, Includes the following modules: The preprocessing module is used to first acquire images of the UAV maritime search and rescue target as the raw input image; The original input image is sequentially processed by flight altitude normalization, constructing image vertical position features, and constructing image-level altitude image preprocessing. The image-level altitude image is then used as an additional input and synchronously enhanced with the original input image. Furthermore, target-level height descriptors are extracted for each ground truth (GT) box, and a GT box-level height supervision map is constructed. And a prediction module, used to build a UAV maritime search and rescue target detection model based on the improved YOLO model, wherein the input of the model is the image after synchronous enhancement processing, and the output of the model is the maritime search and rescue target detection result; The model is based on the YOLOv8n structure, with the addition of the P2 detection branch and the removal of the P5 detection output branch. It also introduces the EMA feature enhancement module and adds the highly guided multi-scale fusion module HGMF at key fusion locations. HGMF enhances the responsiveness of target-related regions by simultaneously introducing height feature maps of the same scale and ground truth box-level height supervision information to weight and modulate visual fusion features at various scales. The training dataset constructed using images with synchronous enhancement processing was used to train the UAV maritime search and rescue target detection model, and the trained model was used to realize maritime search and rescue target detection. During training, the total loss function consists of highly perceptive weighted detection loss and auxiliary highly supervised loss.

9. A computer device, comprising a memory and one or more processors; characterized in that, The memory stores executable code, which, when executed by the processor, is used to implement the dual-layer height prior fusion detection method for UAV maritime search and rescue as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon; characterized in that, When executed by the processor, the program is used to implement the dual-layer height prior fusion detection method for UAV maritime search and rescue as described in any one of claims 1 to 7.

Images