Unmanned aerial vehicle swarm sea area target identification method and device

Abstract

The embodiment of the application provides a kind of unmanned aerial vehicle group sea area target identification method and device, through innovatively designing data synchronization system, through calibration optimization and coordinate conversion, realize the effective fusion of data.Construct feature extraction mechanism, combined with depth completion and attention optimization, establish reliable feature expression.Introduce multiscale detection, through loss combination and position optimization, ensure the accuracy of identification.The method effectively solves the shortcomings of traditional technology in data synchronization, feature extraction and target detection, etc., provides technical support for sea area monitoring.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a method and apparatus for identifying marine targets in a swarm of unmanned aerial vehicles (UAVs). Background Technology

[0002] Existing methods for identifying maritime targets in UAV swarms have significant shortcomings. Traditional systems perform poorly in data acquisition and time synchronization, failing to effectively integrate information from multiple sensor sources, thus affecting identification accuracy.

[0003] Furthermore, existing technologies suffer from bottlenecks in feature extraction and depth completion. Most systems lack robust image processing mechanisms and depth optimization strategies, resulting in incomplete feature representation.

[0004] Existing systems have technical shortcomings in target detection. They lack in-depth analysis of multi-scale features, making it difficult to achieve accurate target localization through loss optimization, thus affecting detection performance. Solving these problems is of great significance for improving maritime surveillance capabilities. Summary of the Invention

[0005] To address the problems in existing technologies, this application provides a method and apparatus for identifying marine targets in a swarm of unmanned aerial vehicles (UAVs), which can effectively solve the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, and provide technical support for marine monitoring.

[0006] To solve at least one of the above problems, this application provides the following technical solution:

[0007] Firstly, this application provides a method for identifying maritime targets in a swarm of unmanned aerial vehicles (UAVs), including:

[0008] A swarm of drones is deployed to perform maritime reconnaissance missions, collecting image data from visible light cameras and point cloud data from lidar. The sampling times of the visible light cameras and lidar are recorded, and a timestamp based on a host computer time reference is established. The image data and point cloud data are indexed and matched according to the most recent time. The intrinsic parameter matrix of the visible light camera is calculated using the Zhang calibration method. The extrinsic parameter transformation matrix is ​​calculated by selecting the common viewpoint of the image and point cloud. Based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix, the point cloud data is projected onto the coordinate system of the image data.

[0009] The image data and the point cloud data are input into the marine target preprocessing convolutional network to extract the feature vector and depth feature vector of the ship image on the sea surface, respectively. The depth feature vector is completed by the conditional random field network to construct the marine depth weight matrix. The depth weight matrix is ​​optimized by the attention mechanism to generate dense depth features.

[0010] The image feature vector and the dense depth feature are input into the marine target feature pyramid network to generate multi-scale fusion features. The overlap between the ship target bounding box and the ground truth bounding box is calculated to generate a localization loss value. The cross-entropy loss value of ship category and confidence is calculated. The localization loss value and the cross-entropy loss value are weighted and combined to determine the type and location information of the target ship.

[0011] Furthermore, it also includes: deploying a swarm of drones in the target area of ​​the sea, controlling the first batch of drones equipped with visible light cameras to collect image information, controlling the second batch of drones equipped with lidar to collect point cloud information, and generating collection parameters including collection location, collection time, and data scale;

[0012] The sampling time information of the visible light camera and lidar is recorded. The host computer time base is used as a reference standard to correct the sampling time information, calculate the hardware transmission delay deviation, and generate a unified timestamp based on the host computer.

[0013] Furthermore, it also includes: sorting the image data and point cloud data according to the sampling time, calculating the time interval between adjacent data frames, selecting the image and point cloud data pair with the most recent time based on the time interval, generating a temporal matching matrix, and establishing an index association between the image data and point cloud data in the temporal matching matrix;

[0014] The intrinsic parameter matrix of the visible light camera is calculated based on Zhang's calibration method. Commonly viewed feature points of the image and point cloud are selected, and a three-dimensional coordinate mapping relationship of the feature points is constructed. The extrinsic parameter transformation matrix from the laser radar to the visible light camera is calculated. The point cloud data is then projected onto the image coordinate system through the intrinsic parameter matrix and the extrinsic parameter transformation matrix.

[0015] Furthermore, it also includes: inputting image data into a first preprocessing convolutional layer, setting the number of output channels to forty-eight, inputting point cloud data into a second preprocessing convolutional layer, setting the number of output channels to sixteen, superimposing the output feature maps of the two preprocessing convolutional layers to generate a fused feature map, and inputting the fused feature map into a third preprocessing convolutional layer, setting the number of output channels to sixty-four.

[0016] The fused feature map is subjected to fuzz enhancement processing. The enhanced feature map is then input into a feature extraction network. The image feature vector of the ship on the sea surface is calculated based on the neural network. The depth feature vector is calculated based on sparse convolution. The depth feature vector is then normalized.

[0017] Furthermore, it also includes: inputting the depth feature vector into the conditional random field network, calculating the unary potential function of each node in the fully connected graph, calculating the pairwise potential function between node pairs based on image features, generating a node weight matrix, normalizing the node weight matrix, and constructing a sea area depth weight matrix.

[0018] Feature patches are extracted from the depth weight matrix, the dot product of the query vector and the key vector is calculated, weight coefficients between the patches are generated, the weight coefficients are combined with the relative position encoding information, and the depth weight matrix is ​​optimized based on the attention mechanism to generate dense depth features.

[0019] Furthermore, it also includes: inputting the image feature vector and dense depth features into the feature pyramid network respectively, performing downsampling processing on the feature maps at different levels, enhancing the depth features through spatial attention mechanism and channel attention mechanism, and superimposing the enhanced depth features with the image features to generate multi-scale fusion features;

[0020] Based on the multi-scale fusion features, the target bounding box is predicted, the intersection area of ​​the target bounding box and the ground truth bounding box is calculated, the union area of ​​the target bounding box and the ground truth bounding box is calculated, the intersection area is divided by the union area to obtain the overlap, and the localization loss value is generated based on the overlap.

[0021] Furthermore, it also includes: normalizing the predicted ship category probability, comparing the normalized probability with the true label, calculating the cross-entropy loss value of the category prediction, normalizing the predicted target confidence, comparing the normalized confidence with the true label, and calculating the cross-entropy loss value of the confidence.

[0022] The positioning loss value and the cross-entropy loss value are linearly combined, and the weight coefficients are dynamically adjusted according to the training phase. The comprehensive scores are calculated for large ships, medium-sized civilian ships, and small fishing boats respectively, generating the target location coordinates and type probability in the sea area, and determining the type and location information of the target vessel.

[0023] Secondly, this application provides a device for identifying unmanned aerial vehicle (UAV) swarm targets in a sea area, comprising:

[0024] The marine point cloud data processing module is used to deploy a swarm of drones to perform marine reconnaissance missions. It collects image data acquired by a visible light camera and point cloud data acquired by a lidar, records the sampling time of the visible light camera and the lidar, establishes a timestamp based on the host computer's time reference, indexes and matches the image data and the point cloud data according to the most recent time, calculates the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, selects the common viewpoint of the image and point cloud to calculate the extrinsic parameter transformation matrix, and projects the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix.

[0025] The model feature vector processing module is used to input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, and optimize the depth weight matrix through the attention mechanism to generate dense depth features.

[0026] The marine target ship identification module is used to input the image feature vector and the dense depth feature into the marine target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate a localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0027] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the UAV swarm maritime target identification method.

[0028] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the UAV swarm maritime target identification method.

[0029] Fifthly, this application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the aforementioned UAV swarm maritime target identification method.

[0030] As described above, this application provides a method and apparatus for identifying maritime targets in a swarm of unmanned aerial vehicles (UAVs). Through an innovative data synchronization system design, and by employing calibration optimization and coordinate transformation, it achieves effective data fusion. A feature extraction mechanism is constructed, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and through loss combination and position optimization, the accuracy of identification is ensured. This method effectively addresses the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1This is a flowchart illustrating the method for identifying unmanned aerial vehicle (UAV) swarm targets in the sea area according to an embodiment of this application.

[0033] Figure 2 This is a structural diagram of the UAV swarm marine target identification device in the embodiments of this application;

[0034] Figure 3 This is a schematic diagram of the structure of the electronic device in the embodiments of this application.

[0035] Figure label:

[0036] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The acquisition, storage, use, and processing of data in this application all comply with relevant laws and regulations.

[0039] To address the shortcomings of existing technologies, this application provides a method and apparatus for identifying maritime targets by unmanned aerial vehicle (UAV) swarms. Through an innovative data synchronization system design, effective data fusion is achieved via calibration optimization and coordinate transformation. A feature extraction mechanism is constructed, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and loss combination and position optimization ensure the accuracy of identification. This method effectively solves the deficiencies of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0040] To effectively address the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, and to provide technical support for maritime surveillance, this application provides an embodiment of a method for identifying maritime targets using a swarm of unmanned aerial vehicles (UAVs). See [link to embodiment]. Figure 1 The method for identifying marine targets in a drone swarm specifically includes the following:

[0041] Step S101: Deploy a swarm of drones to perform a maritime reconnaissance mission, collect image data acquired by a visible light camera and point cloud data acquired by a lidar, record the sampling time of the visible light camera and the lidar, establish a timestamp based on the host computer time reference, index and match the image data and the point cloud data according to the most recent time, calculate the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, select the common viewpoint of the image and the point cloud to calculate the extrinsic parameter transformation matrix, and project the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix;

[0042] This embodiment deploys a swarm of UAVs consisting of several fixed-wing and multi-rotor platforms for reconnaissance in a typical near-shore waterway scenario. To ensure the fusion of multimodal data within a unified time domain, a unified time reference is first set on the host computer, using PPS or PTP as the hardware reference. The first batch of UAVs equipped with visible light cameras records the exposure start time via external triggering, while the second batch of UAVs equipped with lidar aligns the start and end times of each frame scan with the host computer's heartbeat. Considering the unstable latency in link and storage writes, this embodiment measures the transmission and buffer queuing time at the receiving end and subtracts it from the device's local time to generate a corrected sampling time. Subsequent software time synchronization does not simply align frame numbers but retrieves the most recent image frame for each point cloud frame, constructing a time-series index table. When sea conditions cause rapid changes in platform attitude, the most recent time matching can better approximate the same event cross-section, reducing spatial extrapolation errors.

[0043] To ensure spatial coordinate uniformity, this embodiment employs Zhang's calibration of the camera using a distorted planar calibration plate. Multi-pose sampling is performed on a deck or platform to calculate focal length, principal point, and radial and tangential distortion parameters, forming an intrinsic parameter matrix and distortion coefficients. The extrinsic parameter conversion from lidar to the camera is not a one-time static indoor process used long-term. Instead, after installation at sea, markers with highly reflective materials are deployed to extract common-view feature points from both the image and point cloud. A least-squares objective function is established to estimate rotation and translation, minimizing the positional error of the 3D point projection onto the image plane. Considering the minor deflection caused by wind loads, this embodiment saves the extrinsic parameter version and timestamp, allowing for retrospective adjustments or fine-tuning in case of subsequent systematic projection shifts.

[0044] After completing spatiotemporal calibration, the cross-modal mapping stage begins. Point cloud frames are located to their corresponding image frames based on a unified timestamp. Extrinsic parameters are used to transform the points from the laser coordinate system to the camera coordinate system, and then intrinsic parameters and distortion parameters are used to project them to pixel coordinates. In actual operation, outliers may appear due to specular reflection from the sea surface or splashing water mist. This embodiment applies frustum constraints and effective imaging area masking to the projected pixels, removing samples that fall outside the image boundaries or are located in severely distorted corners. For segments with rapid pitch changes in the rotorcraft, IMU short-window attitude assistance is also superimposed to ensure that the projection remains within an acceptable range even when there is a millisecond-level error in the most recent matching. After these processes, the point cloud is assigned pixel indices, creating conditions for the subsequent network to aggregate depth information according to the target bounding box.

[0045] At the data management level, the temporal index table records the image frame ID, point cloud frame ID, the sampling difference between the two, and delay correction parameters, facilitating batch processing and training set construction. Considering the drastic changes in illumination and wave surface texture in the sea scene, this embodiment introduces a quality score: for the image, it is based on the exposure histogram and motion blur estimation; for the point cloud, it is based on the echo intensity distribution and the proportion of effective points. If the score is lower than the threshold, the index pair is marked as low confidence and only participates in the regularization constraints of depth completion, without directly entering the target localization training. This process avoids introducing noise into the geometric-semantic alignment link, maintaining the stability of the input statistics of the subsequent fusion network.

[0046] To address the spatial drift risk brought about by multi-platform collaboration, a rolling self-check step was also implemented. Within a specific flight segment, stable edges of fixed sea surface structures or large ship hulls are randomly selected, and their relative positional deviations in the image and projected point cloud are calculated using a sliding window method. Once the deviation exceeds a threshold, extrinsic parameter fine-tuning is triggered. Fine-tuning optimizes only a small number of parameters to prevent overfitting of instantaneous noise. In practice, changes in flight path or temperature variations can easily induce minute structural deformations; this self-check-fine-tuning link ensures projection consistency across hourly scales.

[0047] At the target level, this embodiment does not directly use the entire projected point cloud for detection. Instead, it first completes index matching and projection, and then aggregates depth according to candidate target regions in subsequent steps. Here, it is emphasized that the purpose of S101 is to provide a highly reliable spatiotemporal alignment and projection foundation. Since visible light is difficult to denoise under foggy or backlighting conditions, the sparse structure of the point cloud is particularly critical for determining the scale and depth of the bounding box; while the point cloud is extremely sparse at long distances, it is necessary to rely on the texture and contour information of the image to determine the semantic category. At this stage, the two establish a one-to-one correspondence through dual temporal and spatial calibration to ensure that features from different modalities in the subsequent network share the same event reference surface.

[0048] To facilitate review and traceability, this embodiment records key quantities for each match in the metadata, including camera intrinsic parameter version, extrinsic parameter matrix, timestamp pair, attitude estimation source, and confidence level. In the event of sudden changes in sea state, data from the same time period can be retrieved in offline analysis to verify whether downstream identification deviations are caused by time drift or extrinsic parameter mismatch. Overall, S101 constrains the sampling process of the physical world to a unified coordinate and time domain, transforming sensor differences into computable geometric relationships. This ensures that subsequent depth completion and feature fusion are based on reliable alignment, thereby maintaining stable perception of target type and location under complex sea conditions.

[0049] Step S102: Input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, optimize the depth weight matrix through the attention mechanism, and generate dense depth features.

[0050] This embodiment uses the spatiotemporally aligned data obtained in S101 as input to construct a marine target preprocessing convolutional network for multimodal representation. Based on the imaging differences in marine scenes, RGB images are densely pixelated and contain texture, contours, and color semantics, while the sparse depth derived from point clouds is only effective at echo locations. The information density and physical meaning of the two differ significantly. If they are directly input into the same backbone, the network will allocate a large number of channels to the end with richer texture, diluting the geometric cues extracted by the depth branches, making subsequent detection of target scale and depth instability. This embodiment sets up a three-layer 3×3 convolutional preprocessing module before the backbone to extract early features separately for RGB and sparse depth. The output channels are set to OutchannelRGB=48 and OutchannelDepth=16, and then after concatenation, a 64-channel convolution is used for unified compression. Since reflections caused by sea conditions and the high brightness of wave crests can easily disrupt local consistency, a fuzzy joint enhancement is also introduced in the preprocessing stage. RGB and depth are simultaneously cropped, scaled, and tiled. In addition, annotation quality filtering logic is introduced. After cropping, annotations with too small a visible area are discarded to avoid mis-teaching the network to focus on incomplete targets during training.

[0051] The sparsity and irregular noise of depth features can hinder cross-modal fusion. In this embodiment, the preprocessed depth features are fed into a Neural Conditional Random Field (FC-CRF) for completion. This model extracts color and texture cues from the shared feature map derived from the image branches, establishing pixel dependencies in a fully connected graph. The core idea is to allow spatially similar and visually similar pixel pairs to share similar depths, naturally conforming to the continuity between maritime targets and the sea surface background. To balance long-range relationships and local geometry, the pairwise potential in the CRF is calculated using multi-head attention. A query and key vector are generated for each patch within the window, and the latent weights are obtained through dot product. These are then superimposed with relative position encoding to mitigate false connections caused by color similarity alone. In actual inference, the texture abruptness between the ship's hull and the shadow transition area is strong, so the weights of the pairwise potential are reduced to avoid "pulling" the background depth into the hull.

[0052] The intermediate results from depth completion are not directly used as the final input to the network. Instead, they are used to construct a sea area depth weight matrix to describe the reliability of each location's depth estimation and its contribution to the detection task. In this embodiment, the confidence information output by the CRF is fused with several indicators, including echo intensity, neighborhood consistency, and projection residual, to generate a weight map. Neighborhood consistency measures the stability of the depth difference between a pixel and its surrounding patches, echo intensity reflects the quality of point cloud observations, and projection residual comes from the error measure of geometric projection in S101. These three metrics have high coherence in open sea areas, but in the spray zone, a co-occurrence pattern of low consistency and low echo often appears. The weight matrix physically expresses an intuitive judgment of "where the depth is reliable and where it should be deweighted," which aligns with the need to find stable boundaries and scale estimation in target detection.

[0053] To ensure the correct application of this weighting in downstream fusion, this embodiment employs an attention mechanism to perform secondary filtering and amplification of the depth weight matrix. Spatial attention focuses on contiguous areas where targets may exist, while channel attention characterizes the value of different depth cues (e.g., planar continuity, edge transitions) for features at different scales. In specific scenarios, distant small fishing boats occupy only a few pixels in the image but present a relatively prominent foreground layer in terms of depth. In this case, channel attention increases the weights of depth sub-channels related to edges and contours, while spatial attention shrinks to the neighborhood of the hull, thus avoiding interference from large areas of sea surface texture. The resulting dense depth features contain both geometrically continuous structures and task-relevant saliency, making them more suitable for integration into the subsequent feature pyramid along with image features.

[0054] The network training process is structured around the above-described input and output parameters. The input consists of an aligned RGB image and its corresponding sparse depth map. The output is divided into two branches: one is a dense depth map or its high-dimensional feature tensor, and the other is a fusion feature used by the detection branch. Supervision for depth completion employs scale-invariant logarithmic loss. Considering the systematic bias caused by sea surface reflections and shadows, an additional mask loss is calculated within the labeled target area, emphasizing depth consistency in key areas such as the hull and deck. The loss weight is larger in the early stages of training to first stabilize the geometric structure before allowing the detection head to learn the category and confidence level. Because the logarithmic difference in the loss term has a monotonic relationship with physical distance, the parameters learned by the network naturally correspond to actual depth changes, avoiding unreasonable biases introduced by linear changes in brightness.

[0055] During the inference phase, the image features generated by preprocessing convolution emphasize texture and contours. Sparse depth features are transformed into dense depth features after CRF completion and attention filtering, and the two are fused at various pyramid scales in subsequent operations. Taking a backlit wake as an example, the sea surface reflection at the camera end is strong, and the boundaries are unclear in pure vision. However, the depth weight matrix still maintains continuous protrusions at the leading edge of the hull, and the network forms a high fusion response in this area. In a drizzly environment, the point cloud echo is sparse, and the weight matrix gives low confidence in large areas. During fusion, more reliance is placed on image branches to avoid over-relying on incomplete geometric signals. It can be seen that dense depth features are not a simple interpolation product, but a task-oriented representation obtained by combining observation quality, geometric consistency, and contextual semantics. It is directly related to the stability of subsequent bounding box scale and depth judgment.

[0056] To accommodate different platform and sensor combinations, this embodiment allows for fine-tuning of the number of preprocessing channels in engineering. For example, point clouds on fixed-wing high-altitude flight paths are sparser, so the OutchannelDepth can be appropriately increased to expand the receptive field; during near-shore multi-rotor low-altitude inspections, RGB texture details are dominant, so the number of image branch channels can be maintained or slightly reduced to suppress overfitting. Regardless of parameter adjustments, the overall logic remains unchanged: first, branch features are extracted; then, depth weights are constructed and purified; finally, dense and reliable depth features are output, providing a solid geometric foundation for subsequent multi-scale fusion and loss calculation.

[0057] Step S103: Input the image feature vector and the dense depth feature into the sea area target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate the localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0058] This embodiment inherits the two types of representations output from S102: one is an image feature vector emphasizing texture and contour, and the other is dense depth features after CRF completion and attention filtering. These two feature paths are fed in parallel into the marine target feature pyramid network. First, top-down and bottom-up cross-scale convergence is performed in their respective branches, and then step-by-step fusion is performed at the corresponding scales. Considering the large size range of marine targets—large ships occupying a wide area in fixed-wing routes, while small fishing boats in near-shore low-altitude scenes are only a few pixels—this embodiment introduces a combination of spatial and channel attention enhancements to the depth branches at each pyramid level. This allows regions with stable geometric continuity to obtain higher responses and reduces interference from the specular reflection texture of the sea surface. The fusion operation adopts a concatenated structure of superposition and lightweight convolution. The superposition part quickly aligns semantics, and the convolution part updates cross-modal weights, avoiding a single modality "suppressing" another modality at a certain scale.

[0059] For bounding box regression, the network head generates candidate bounding boxes, offsets, and object presence scores at each scale. Depth features primarily affect the consistency between scale and depth, while image features provide category clues for the ship's outline and superstructure. To measure localization error, the overlap between predicted and ground truth boxes is calculated, and a localization loss is generated accordingly. Considering aspect ratio deviations caused by sea state disturbances, this embodiment uses a generalized overlap term that includes a bounding rectangle constraint, ensuring effective gradients even when there is no overlap, and preventing convergence from stalling at "zero IoU". In actual training, distant small targets are represented by only a few activation units at certain scales. The pyramid's multi-scale prediction mechanism provides redundant representation paths, and shallow textures and deep semantics jointly participate in boundary determination, avoiding single-resolution mismatch.

[0060] The classification and confidence learning rely on the same set of fused features, but with different focuses. The classification branch extracts morphological semantics that can distinguish large ships, civilian vessels, and fishing boats, such as the texture combination of deck layout, smokestack, and mast shapes. The confidence branch expresses the overall grasp of whether a target exists within the candidate box, and is jointly influenced by image sharpness and depth consistency. Both branches use cross-entropy loss, and a sampling balance strategy is applied to the difference in the number of samples of different categories during training, so that negative samples of the sea background are not overly dominant. To avoid the model misclassifying highly reflective waves as small boats, this embodiment introduces a density depth stability threshold in the construction of positive and negative samples. Low-confidence candidates are downweighted in confidence supervision to ensure that the learned signal conforms to the natural imaging law.

[0061] A progressively weighted loss strategy is adopted. The total network loss is a linear superposition of the localization loss and two cross-entropy losses, with the weights dynamically adjusted during training: in the early stages, the localization term is strengthened to quickly establish a stable mapping between "pixels, geometry, and scale"; in the mid-to-late stages, the category and confidence terms are gradually increased to make the boundary between semantic discrimination and target existence clearer. This scheduling addresses the inherent difficulty of maritime data—unstable scale estimation directly hinders category learning; stabilizing geometry before refining semantics is more in line with the convergence path. For ease of description, the overlap ratio (IoU) is defined as the intersection-union ratio of the predicted and ground truth boxes. The generalized term introduces a penalty based on the area of ​​the minimum bounding rectangle of the two boxes when there is no overlap, making the gradient point closer to the ground truth box, which is a geometrically reasonable correction.

[0062] During the inference phase, candidate boxes output from each layer of the pyramid undergo non-maximum suppression to generate the final detection results. Instead of applying a fixed threshold, an adaptive selection process is used, incorporating multimodal consistency: when dense depth presents a continuous foreground within the candidate box and the projection residual is small, lower image confidence is allowed to pass; conversely, in conditions of sparse echoes or rain / fog, semantic evidence from the image branches is emphasized. For example, in backlit night navigation, the highlights of the ship's hull have blurred boundaries in the image, and the stability of the depth foreground layer ensures that the long side of the box and distance estimation remain convergent. Similarly, in a cluttered near-shore harbor background, the complex textures of loading / unloading equipment and the shoreline result in multi-tiered depth levels, leading the fused classification branch to favor large or medium-sized categories, thus avoiding misclassification of shoreline structures as ships.

[0063] From an engineering perspective, the fusion parameters of the feature pyramid can be adjusted based on the platform and flight path. In high-altitude sparse point cloud scenarios, the proportion of depth branches in the high-level semantic layer is increased to strengthen scale prior; in low-altitude fine-grained inspection, the weight of shallow image texture is increased to enhance the boundary resolution of small targets. Regardless of the value changes, the causal relationship of the information flow remains consistent: dense depth features provide geometric clues consistent with physical space, and image features provide category clues consistent with appearance semantics. The two are coupled in a multi-scale structure, ultimately outputting the type probability of the target vessel and its position parameters in the image coordinate system. Combined with the projection relationship of S101, the pose or distance in actual space can be deduced, meeting the positioning and identification needs of maritime reconnaissance.

[0064] As described above, the UAV swarm maritime target identification method provided in this application can achieve effective data fusion through an innovatively designed data synchronization system, calibration optimization, and coordinate transformation. It constructs a feature extraction mechanism, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and loss combination and position optimization are used to ensure identification accuracy. This method effectively addresses the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0065] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0066] Step S201: Deploy a swarm of drones in the target area of ​​the sea, control the first batch of drones equipped with visible light cameras to collect image information, control the second batch of drones equipped with lidar to collect point cloud information, and generate collection parameters including collection location, collection time, and data scale;

[0067] Step S202: Record the sampling time information of the visible light camera and the lidar, use the host computer time reference as a reference standard, correct the sampling time information, calculate the hardware transmission delay deviation, and generate a unified timestamp based on the host computer.

[0068] This embodiment selects the complex waters where the near-shore channel meets the harbor basin as the target area. After dividing the airspace into a grid according to the flight path, two types of platform mission orders are issued: the first batch consists of multi-rotor aircraft equipped with large-area visible light cameras, responsible for low-altitude fine imaging; the second batch consists of fixed-wing or endurance multi-rotor aircraft equipped with medium-range lidar, responsible for sparse but wide-coverage spatial ranging. During the mission planning phase, based on tidal windows, no-fly zones, and wind field forecasts, latitude and longitude waypoints, camera pitch angles, and laser scanning modes are assigned to each aircraft, and sampling frequency and exposure / scanning cycle are set. To form traceable acquisition parameters, this embodiment generates parameter blocks by the host computer before takeoff. The parameters include acquisition location (waypoint sequence and flight altitude), acquisition time (start and end times, cycle), data scale (estimated number of frames, point cloud packet size), etc., and are bound to the airborne equipment serial number to ensure consistency between subsequent data transmission and annotation.

[0069] In actual data acquisition, the first batch of UAVs used external hard triggers to control camera exposure, recording the local start and end times of exposures and frame counts, while simultaneously transmitting the attitude and GPS time provided by the flight control system. The second batch of UAVs used lidar to scan in a linear or rotating manner, recording the start and end times of each frame's point cloud scan, the intra-frame angle increment, and the echo intensity distribution. The sea surface environment is prone to strong reflections and wave crest obstruction; relying solely on the device's local clock makes it difficult to align the same event segment between the two data streams. Therefore, this embodiment uses a unified time reference from the host computer as a global reference, distributing the reference to each platform via PPS or PTP. Hardware synchronization of the onboard clock is completed before flight, and drift monitoring is maintained during flight using heartbeat packets.

[0070] Time synchronization alone is insufficient; link transmission and storage writing introduce asymmetric delays, leading to systematic deviations in the sampling records. This embodiment estimates the end-to-end delay for each data stream in real time at the receiving end, breaking it down into three sub-items: wireless transmission, decoding buffer, and disk writing. A sliding window is used to filter out jitter, obtaining the delay deviation between the camera and radar. This deviation is written back to the original sampling record to correct the sampling time. The corrected time is aligned with the host computer reference, generating a unified timestamp, which becomes the basic index key for the "most recent time index matching" in subsequent S101. Due to the high speed of fixed-wing aircraft, a difference of tens of milliseconds between a video frame and a point cloud frame can correspond to different wave peaks. This embodiment sets a threshold and backoff logic in the indexing strategy: if the minimum time difference exceeds the threshold, it is marked as low confidence, and the subsequent depth completion branch in S102 only performs regularization and does not participate in positioning supervision.

[0071] After establishing the data acquisition parameters and timestamp system, differences in data quality need to be recorded for downstream training sampling. This embodiment adds a quality summary to the parameter block: at the image end, the peak shift of the exposure histogram and motion blur estimation are calculated; at the depth end, a score is given based on the effective echo ratio and point cloud density. These indicators directly correspond to marine physical phenomena: in backlight, the histogram peak shift and blur increase; in light rain, the effective echo ratio decreases. During training, the network adjusts sample weights accordingly to avoid allowing noisy samples to dominate the gradient, conforming to the causal chain of "deteriorating observation conditions—reduced confidence—weakened learning signal" under natural rules.

[0072] To address the differences between heterogeneous platforms in a cluster of aircraft, this embodiment also dynamically adjusts the sampling configuration during mission execution. If enhanced wind field causes multirotor attitude disturbances, the system reduces camera exposure time and increases frame rate, sacrificing some dynamic range for clearer boundaries. If sea fog causes sparse radar echoes, the scanning speed is appropriately reduced to increase the point density per unit angle, and the effective time period of this configuration change is recorded in the acquisition parameters. All these changes are written to the parameter version number on the host computer, ensuring that timestamps and data scales can be interpreted correctly with the correct device status during subsequent playback.

[0073] The generation of a unified timestamp also involves attitude alignment. This embodiment applies the same delay correction to the IMU time attached to each frame of data, ensuring that the attitude and image / point cloud are referenced at the same time. This way, in the projection stage of S101, if there is a sub-10 millisecond error in the most recent time-match, attitude interpolation can be used to correct the parallax. In engineering implementation, the timestamp, attitude, waypoints, equipment temperature, and other key metadata are encapsulated in the same record, simplifying the workload of offline annotation and algorithm replay.

[0074] Regarding the above process, S201 and S202 address the fundamental questions of "what to collect, when to collect, and how to ensure different sources point to the same event." Acquisition parameters solidify spatial and planned actions, time correction pulls observations back from their respective local clocks to the same reference, and delay estimation eliminates biases introduced by the data links, ultimately forming a computable and traceable unified timestamp. Based on this infrastructure, subsequent steps can combine the semantics of the image and the geometry of the point cloud within a stable reference plane, thereby maintaining the reliability of target location and category determination in scenarios with frequently changing sea states.

[0075] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0076] Step S301: Sort the image data and point cloud data according to the sampling time, calculate the time interval between adjacent data frames, select the image and point cloud data pair with the most recent time based on the time interval, generate a time-series matching matrix, and establish an index association between the image data and point cloud data in the time-series matching matrix.

[0077] Step S302: Calculate the intrinsic parameter matrix of the visible light camera based on Zhang's calibration method, select the co-view feature points of the image and the point cloud, construct the three-dimensional coordinate mapping relationship of the feature points, calculate the extrinsic parameter transformation matrix from the laser radar to the visible light camera, and project the point cloud data to the image coordinate system through the intrinsic parameter matrix and the extrinsic parameter transformation matrix.

[0078] This embodiment uses the unified timestamps generated in S201 and S202 as a premise to organize images and point clouds from multiple platforms according to the time dimension. First, image frames and point cloud frames are sorted in ascending order by timestamp in the host computer's cache pool. The time interval between adjacent frames is calculated, and jitter and missing frames are statistically analyzed. Sorting is not the goal; the key is the reliability assessment of the subsequent matching strategy. During maritime flight, the sampling periods of different payloads often exhibit non-integer ratios. In this embodiment, for each point cloud frame, a binary search is performed in the image sequence to obtain the nearest upper / lower neighbor frames. The time difference is compared and scored. If the minimum time difference is less than a threshold, it is confirmed as a valid match and written into the temporal matching matrix. The record items include {image ID, point cloud ID, Δt, data quality label}. For segments exceeding the threshold but still requiring continuity, a low-confidence label is assigned for reference in subsequent deep completion branches but not for localization supervision. To avoid bias caused by unidirectional matching, mutual lookup from image to point cloud is also performed. Finally, the side with bidirectional consistency or smaller error is selected to form a robust index association.

[0079] Considering the rapid parallax caused by UAV formation changes and flight path turns, this embodiment introduces attitude interpolation verification in temporal matching. Specifically, the IMU attitude of candidate matching pairs is read at the corresponding time, and the camera line-of-view difference is estimated using linear interpolation with small time differences. If the estimated field-of-view overlap is too low, the match is rejected. This eliminates pairs that are "close in time but have deviated in view" under windy and wave conditions, reducing subsequent projection errors from the source. The generated temporal matching matrix not only serves online fusion but is also used for dataset construction. During the training phase, synchronized samples can be directly loaded in batches based on this matrix, ensuring that the network sees the same event cross-section.

[0080] In the spatial calibration stage, this embodiment employs Zhang's calibration method to solve the camera's intrinsic parameters. A calibration board with a high-contrast checkerboard or circular dot matrix is ​​selected. Images are acquired under multiple poses and distances to estimate the focal length, principal point coordinates, and radial and tangential distortion coefficients, forming the camera's intrinsic parameter matrix and distortion parameter set. Since temperature differences and load / unloading during offshore operations can alter the imaging geometry, the intrinsic parameters are managed using version numbers, and the acquisition temperature range is recorded. In case of systematic imaging drift, backtracking can be performed by version. The extrinsic parameter solution relies on the co-existing feature points of the image and point cloud. This embodiment deploys highly reflective markers on the deck or shoreline, or selects the sharp geometric edges of large ship hulls as natural targets. Candidate correspondences are obtained through image edge / corner detection and point cloud clustering. The correspondence set is then filtered by RANSAC and enters nonlinear least squares optimization to solve for the rotation matrix and translation vector of the laser radar to the camera, minimizing the pixel error of the 3D points projected onto the imaging model.

[0081] The projection stage strictly follows the temporal matching matrix for pairwise processing. Point cloud coordinates are first transformed from the laser coordinate system to the camera coordinate system using extrinsic parameters, and then the image plane coordinates are calculated using camera intrinsic parameters and distortion parameters. For wide-angle lens edge distortion, this embodiment prioritizes projection using distortion-corrected normalized coordinates before mapping back to the pixel domain, avoiding the accumulation of pixel errors at the corners. In a marine environment, echoes generated by waves and fog droplets can form outliers. After projection, these outliers are removed using the effective field-of-view mask and the quality indicators recorded in stage S201, reducing the interference of outliers on subsequent feature learning. To improve engineering robustness, the system saves the average reprojection error and maximum error for each projection. When the error exceeds a threshold, an extrinsic parameter fine-tuning process is triggered, updating only a small number of parameters in small steps to prevent overfitting of transient noise.

[0082] Spatiotemporal alignment and projection are not isolated steps; they directly affect depth completion and weight matrix construction in S102. If the matching error is too large, CRF will over-rely on color similarity to propagate depth in the missing region, causing the sea surface highlight area to be incorrectly flattened. Therefore, in this embodiment, Δt and attitude difference are retained in the temporal matching matrix and read by S102 as completion confidence priors. The influence of large Δt or large attitude difference on depth propagation is suppressed. This coupling method conforms to the natural imaging law: the greater the spatiotemporal distance, the less reliable the correspondence, and the lower the propagation weight should be.

[0083] To illustrate the practical significance of extrinsic parameter optimization for downstream detection, consider a clip from a near-shore berth scene. Fixed markers and moored vessels appear simultaneously in the image and point cloud. If there's a slight deviation in the extrinsic parameter tilt angle, the projected point cloud will shift to one side, causing the dense depth transitions at the vessel's edge to misalign with the image contour, resulting in the bounding box after pyramid fusion leaning towards the background. Through the aforementioned extrinsic parameter optimization, the reprojection error decreases, the depth steps align with the image's strong gradients, the localization loss converges faster during training, and the subsequent S103 bounding box regression shows more stable judgments regarding scale and depth.

[0084] In terms of data organization, this embodiment incorporates the temporal matching matrix, intrinsic and extrinsic parameter versions, and reprojection error statistics into the metadata, forming a traceable chain. During offline playback or problem tracing, it can quickly pinpoint whether the false detection is caused by time drift, attitude anomalies, or aging extrinsic parameters. In engineering deployment, different routes, sensor combinations, and sea conditions will change the matching threshold and sampling rhythm, but the above process remains unchanged: first, sorting is performed using a unified time reference and bidirectional nearest neighbor matching is performed; then, a physically interpretable imaging model is used to map 3D points to 2D images; and finally, error statistics are used to maintain the calibration status in a closed loop. Through this temporal-spatial collaborative mechanism, the image and point cloud share the same event reference surface before entering the neural network, providing reliable geometric support for the fusion of depth and semantics.

[0085] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0086] Step S401: Input image data into the first preprocessing convolutional layer, set the number of output channels to forty-eight, input point cloud data into the second preprocessing convolutional layer, set the number of output channels to sixteen, superimpose the output feature maps of the two preprocessing convolutional layers to generate a fused feature map, and input the fused feature map into the third preprocessing convolutional layer, set the number of output channels to sixty-four.

[0087] Step S402: Perform fuzz enhancement processing on the fused feature map, input the enhanced feature map into the feature extraction network, calculate the image feature vector of the ship on the sea surface based on the neural network, calculate the depth feature vector based on sparse convolution, and normalize the depth feature vector.

[0088] This embodiment follows the temporal alignment completed in S201-S202 and the spatial projection results in S101. After the airborne data enters the algorithm, it first enters the preprocessing channel configuration stage. Considering the objective fact that the sea surface image has rich texture but the point cloud is sparse at a distance, if the channel quota and receptive field are not differentiated, the network will be overly biased towards the RGB branch, the depth geometric cues will be submerged, and the subsequent scale and distance judgments will be unstable. Therefore, the aligned RGB image is input into the first preprocessing convolutional layer, with a 3×3 kernel, a stride of 1, and a fixed number of output channels of 48; the sparse depth map is projected from the point cloud and inserted with holes as placeholders before being input into the second preprocessing convolutional layer, with a 3×3 convolution, a stride of 1, and a number of output channels of 16. This proportional arrangement reflects the difference in information density: the RGB three channels and the local texture are dense, requiring more channels to capture diverse edges and shapes; the depth is a single channel and the effective pixels are sparse, requiring a larger receptive field rather than stacking too many channels. The feature maps output by the two branches are concatenated in the channel dimension to generate a fused feature map, which is then fed into the third preprocessing convolutional layer and uniformly compressed to sixty-four channels using 3×3 convolution to eliminate statistical differences between branches and form a "post-aligned shallow semantic" that can be shared downstream.

[0089] In marine scenes, the spatial arrangement of ship hulls, waves, and reflections creates numerous locally similar or dissimilar objects. Single-scale training can easily fall into short-texture memory and neglect cross-scale consistency. This embodiment employs blur enhancement on the fused feature map. Four frames from the same time period are synchronously scaled, cropped, and stitched together onto a unified canvas, preserving the relationship of multi-scale targets appearing in the same batch. Annotation quality filtering is introduced during enhancement: if the visible area of ​​a target after cropping is lower than the original area threshold, the annotation is discarded to avoid injecting incomplete contours into the network. The enhanced fused feature map is then fed into the backbone of the feature extraction network. The backbone uses Transformer-type or hierarchical convolutional structures that capture global relationships, emphasizing texture edges in shallow layers and aggregating semantic layout in higher layers. To address the sparsity of the depth channel, this embodiment uses sparse convolution or sub-Manhattan convolution on the depth branches, skipping invalid pixel calculations and emphasizing the causal chain of "effective point neighborhood—ship shape—depth jump." Deep features are normalized before output (e.g., mean and variance standardization of feature channels). This not only offsets the differences in point cloud intensity and distance distribution across different flight segments, but also allows the subsequent cross-modal fusion weights to be learned based on structure rather than dimensions.

[0090] The key to the above structure lies in solidifying the engineering principles of "who should be allocated more expressive power" and "which regions deserve a larger receptive field." Through a 48 / 16 / 64 channel arrangement and three layers of 3×3 convolutions, this embodiment assigns the RGB branch to detail rendering and the depth branch to geometric contours and hierarchical relationships. The fused third convolution layer serves to "digest differences and align statistics," preventing interference between different modalities during BatchNorm or attention after direct stitching. Taking a backlit sea route as an example, the RGB edges are contaminated by highlights, while the depth branch still forms continuous jumps at the leading edge of the hull. In light rain sea conditions, depth is sparse and noise increases; the variance of the normalized depth feature channels decreases, and the network naturally allocates weights to more reliable image texture evidence during the pyramid fusion stage—an adjustment consistent with natural imaging principles.

[0091] The input and output configurations of the feature extraction network are tailored to the requirements of subsequent steps S102 and S103. The input is a fused feature map after blur enhancement, and the output is divided into two categories: image feature vectors emphasize appearance semantics, providing clues such as superstructure, deck lines, and opening structures needed for ship type differentiation; depth feature vectors provide scale and depth clues, determining the stability of bounding box regression. To make the depth representation more closely resemble the true distance, this embodiment introduces a scale-invariant logarithmic error regularization term into the depth branch during training. Its core quantity Δd is the logarithmic difference between the predicted depth and the true depth, conforming to the visual geometry principle that "greater tolerance for errors at distant locations and greater accuracy required for near locations." In the construction of detection samples, weight attenuation is applied to low-quality echo regions to prevent the model from learning accidental associations in unreliable regions.

[0092] At the engineering level, platform differences and task switching are also considered. In the fixed-wing high-altitude coverage stage, targets are generally small and point clouds are extremely sparse. The porosity of the second preprocessing convolution can be increased to expand the receptive field, keeping the number of channels unchanged but increasing the stride receptive range. In the near-shore multi-rotor fine scanning stage, the proportion of targets is larger and the image details are richer. The first preprocessing layer can maintain 48 channels but reduce the stride and dilation rate of the first two layers to allow for finer-grained expression of fine textures. This type of parameter tuning does not change the overall process, which is still a chain structure of "branch feature extraction - stitching - unified compression - enhancement - backbone refinement - sparse convolution depth extraction - normalized output".

[0093] From a data flow perspective, the 64-channel fused shallow features output by S401 serve as a bridge between the preceding and following layers: they carry both the geometric traces after temporal-spatial alignment and retain the edges and textures of visible light. S402 establishes a multi-scale context through fuzzing, and through feature extraction and sparse convolution, it upscales the two types of evidence into "image feature vectors" and "depth feature vectors" respectively, and then eliminates inter-batch drift through normalization. When these two features subsequently enter the feature pyramid network, they can find their respective stress points at each scale, forming a stable basis for scale estimation and class separation, maintaining the repeatability and interpretability of the detection link in complex sea conditions and changing light environments.

[0094] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0095] Step S501: Input the depth feature vector into the conditional random field network, calculate the unary potential function of each node in the fully connected graph, calculate the pairwise potential function between node pairs based on image features, generate the node weight matrix, normalize the node weight matrix, and construct the sea area depth weight matrix.

[0096] Step S502: Extract feature patches from the depth weight matrix, calculate the dot product of the query vector and the key vector, generate weight coefficients between patches, combine the weight coefficients with the relative position encoding information, optimize the depth weight matrix based on the attention mechanism, and generate dense depth features.

[0097] This embodiment uses the depth feature vector output by S102 and the shared image semantic features as input. Addressing the sparse depth information and unstable noise points in the sea surface environment, a conditional random field model is constructed to structurally complete the depth. First, a fully connected graph is defined on the image coordinate system, where each pixel (or downsampled grid cell) is treated as a node, and the node state represents the depth estimate at that location. To avoid interference from local reflections during direct regression, the univariate potential is jointly provided by the sparse depth and the initial value of the feedforward network, expressing the node's "self-observation preference." The pairwise potential is used to characterize the preference for whether the depths of any two nodes should be similar, specifically depending on the color, texture, and edge strength of the image branch, as well as the geometric prior of the depth branch. Regions with similar colors and weak edges tend to propagate more smoothly, while coupling is weakened when crossing strong edges. Random highlights caused by sea surface splashes and wave crests present a fragmented high-frequency texture on the image; the pairwise potential is suppressed in these regions to prevent the background depth from propagating into the ship's interior. The node weight matrix is ​​obtained by minimizing the energy, with the values ​​representing the relative contribution of each node to global consistency. Considering the differences in observation quality among different segments, this embodiment uses the echo intensity and effective point ratio recorded in S201, the matching Δt and attitude difference in S301 as confidence priors, and integrates them into the weighted terms of unary potential or paired potential to ensure that physically more reliable observations have greater influence on the map.

[0098] To ensure the weight matrix possesses scalability and statistical stability usable downstream, normalization is necessary. Normalization is not a simple linear stretching, but rather a soft normalization based on the energy distribution within the window, preventing excessively sharp gradients near strong edges and avoiding gradient bursts during training. The normalized matrix can be interpreted as a "sea depth weight map," encoding two types of knowledge: a reliability distribution determined by observation reliability, and structural consistency determined by image semantics and geometric priors. The former decreases overall in foggy, backlit, or distant conditions, while the latter forms banded structures near the ship's outline, deck edges, and waterline. This distribution aligns with real physical processes: depth should not be over-relied when observation conditions worsen; depth discontinuities at object boundaries are geometrically inevitable.

[0099] Building upon this foundation, this embodiment further introduces an attention-based context optimization mechanism, explicitly parameterizing the long-range dependencies of CRF as selective interactions between patches. Specifically, it divides the weight matrix into fixed-size or adaptive-size feature patches, uses shallow convolutions to map each patch to the query vector and key vector space, and measures the correlation between any pair of patches using dot products. Pairs with high correlation are more likely to share a deep structure. The propagation of deep structure is not only based on appearance similarity but also on geometric relative position. For example, regular ripples on the sea surface can cause a large area of ​​similar color but without the same depth level. This embodiment incorporates relative position encoding into the attention weights, allowing distance and direction to suppress or enhance the weights. Closest neighbors in the same direction receive higher weights, while distant neighbors across directions experience attenuation, reducing false completions caused by spurious correlations.

[0100] To align the attention output with the task objective, this embodiment uses the edge intensity map of the image branch and the salience map of the hull candidate region as modulation signals applied to the attention distribution. Large ships have continuous metal plate structures with clear edges, making these areas more attractive for high-coupling attention. Small fishing boats, though occupying only a few pixels at a distance, form a prominent foreground layer in terms of depth. Channel-level attention increases the weights of sub-channels related to edges and contours, enabling the network to maintain correct step relationships even in areas with weak texture. After attention filtering and reweighting, the resulting depth features possess both density and task relevance, not only filling the gaps in the sparse point cloud but also focusing structural information on areas more critical for detection.

[0101] During training, the outputs of the CRF and attention module are constrained by both depth and detection supervision. Depth supervision employs scale-invariant logarithmic loss to mitigate the dimensionality effects of different distance segments, focusing the network on relative errors. Considering the systematic bias in highly reflective sea surface areas, additional losses are calculated and weighted within the labeled target mask areas, emphasizing the geometric consistency of the hull and deck. Detection supervision originates from the localization and classification branches of S103, with backpropagation gradients flowing back to dense depth features at the fusion point, forming a training dynamic of "geometry-driven, semantically corrected." The correspondence between loss and physical quantities is clear: the logarithmic depth difference is monotonically related to the actual distance, IoU aligns with geometric alignment, and network parameter updates follow interpretable error sources rather than arbitrary correlations.

[0102] In the inference phase, the ocean depth weight matrix acts as an adaptive gating mechanism. For example, under backlighting conditions, image contrast decreases, weakening the smoothing effect of color guidance by the CRF. Attention relies more on positional encoding and a small number of high-confidence echo neighborhood propagations. When light rain causes sparse point clouds, the weight matrix is ​​adjusted downwards overall to avoid spreading unreliable depth over a large area, and the detection branch tends to make judgments based on image features. Taking berthing operations as an example, shore-side loading and unloading facilities are similar in shape to ships. Without constraints, shore-side structures can easily be mistakenly added as foreground depth. The relative position and saliency map modulation in attention suppress long-distance connections across water areas, creating a clear discontinuity in dense depth near the waterline. This aligns with real geometry, making subsequent bounding box regression more stable in both depth and scale.

[0103] From an engineering adaptation perspective, this embodiment allows for adjustments to patch size and the number of attention heads based on the flight path and platform. For high-altitude sparse point clouds, the patch receptive field is increased to capture long-range consistency; for low-altitude detailed tasks, the patch size is reduced for more precise propagation along the ship's outline. Regardless of parameter tuning, the core pipeline remains consistent: a global constraint is constructed using a CRF, integrating observation confidence and semantic boundaries, and a sea area depth weight matrix is ​​obtained after normalization. Then, an attention mechanism is used for selective propagation based on similarity and relative position, ultimately outputting dense depth features for use by the feature pyramid and detection head. For traceability, the system records the reprojection error statistics, weight matrix histogram, and attention entropy for each inference. If the entropy is too low, indicating over-focusing, the channel weight can be adjusted back or the position penalty increased to maintain stable representation capabilities under different sea conditions.

[0104] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0105] Step S601: Input the image feature vector and dense depth features into the feature pyramid network respectively, perform downsampling processing on the feature maps of different levels, enhance the depth features through spatial attention mechanism and channel attention mechanism, and superimpose the enhanced depth features with the image features to generate multi-scale fusion features;

[0106] Step S602: Based on the multi-scale fusion features, predict the target bounding box, calculate the intersection area of ​​the target bounding box and the real bounding box, calculate the union area of ​​the target bounding box and the real bounding box, divide the intersection area by the union area to obtain the overlap, and generate a localization loss value based on the overlap.

[0107] In this embodiment, the image feature vector output from the previous stage and the dense depth features filtered by CRF and attention are used as input to perform cross-scale representation integration in the marine target feature pyramid structure. Considering the scale span and line-of-sight variations of marine targets, scale sequences are constructed for the two feature paths in the pyramid's bidirectional paths from top to bottom and bottom to top: the image branch preserves texture and morphological semantics, while the depth branch preserves foreground steps and planar continuity. Each level first performs downsampling and lightweight convolution for alignment to avoid statistical interference between features from different sources. For the depth branch, a concatenated enhancement of channel attention and spatial attention is applied to each layer. Channel attention is used to select several geometric sub-channels such as "edge transitions," "planar continuity," and "local concavity and convexity," while spatial attention forms a high response in the candidate foreground area of ​​the image to weaken large areas of sea surface texture. After enhancement, the features are superimposed with the image features of the same layer and then integrated with 1×1 convolution to obtain multi-scale fused features. The logic of this processing is to ensure that the geometric credibility is clearly expressed at each scale and then aligned with semantic texture, preventing a single modality from suppressing another modality at certain scales.

[0108] Multi-scale fusion features are delivered to the detection head for bounding box and confidence prediction. Backlighting at sea and highlights from waves reduce image boundary sharpness, while depth-level step structures are more stable, thus retaining higher depth weights on large-scale feature maps. Small targets near the shore at low altitudes are more easily distinguished in image texture, and image channel weights are higher at shallow scales. In training samples, distant small fishing boats often have only a few activation units; the multi-scale redundancy provided by the pyramid avoids missed detections due to single-resolution mismatch. In practical engineering, the statistics after feature fusion are corrected through batch normalization or layer normalization to ensure that the numerical ranges at each scale can be interpreted with a unified threshold, facilitating subsequent candidate selection and non-maximum suppression.

[0109] In terms of localization supervision, this embodiment uses multi-scale fusion features to predict bounding box parameters and existence scores, and measures regression quality through geometric overlap. For any predicted box and ground truth box, the intersection area and union area are first calculated to obtain the intersection-union ratio (IU) as the overlap index. The natural relationship between this index and geometric alignment is clear: when the box position and size approach the ground truth contour, the intersection increases, the union variation is limited, and the overlap increases; conversely, it decreases rapidly when there is misalignment or scale deviation. The common "half-loss of contour" situation in the sea surface scene will cause the box to only partially overlap with the ground truth. The overlap can continuously characterize the error magnitude, and the training gradient direction points to a geometric configuration closer to the target, which is beneficial for stable convergence under strong interference. To avoid gradient dilution caused by the huge union in the training of large targets, the network allocates layer weights when summing the loss at different scales, so that the deep features covering large targets participate in more regression signals, while shallow features dominate for small targets.

[0110] The attention enhancement in this embodiment is not isolated, but coupled with the ocean depth weight matrix generated by upstream S502. The weight matrix acts as a priori gated depth channels at each scale, suppressing channels in noisy areas and amplifying channels in structural areas. Spatial attention only forms a concentrated response in areas with high weights, consistent with marine physical processes: when water mist and splashes cause unstable echoes, geometric evidence should be downweighted and filled by image texture; when the hull and deck edges present stable steps, geometric evidence should dominate scale estimation. Taking a backlit route at night as an example, image branches struggle to lock boundaries at highlight bands, but the continuity of depth branches in the foreground layer remains intact, resulting in a more realistic long side of the fused bounding box. In berthing operations, shore-side equipment is mixed with ship structures; position encoding and spatial attention suppress mismatched connections across waterways, ensuring the bounding box does not cross obvious waterline discontinuities.

[0111] In terms of training configuration, the input consists of multi-scale fused features, and the output includes bounding box offsets, width and height, presence, and feature channels required for subsequent classification at each scale. The localization loss is constructed around overlap, and the gradient is backpropagated to the fusion nodes at each scale through a pyramid, promoting the division of labor between depth and image channels at their respective scales. Since there is less pixel evidence for distant targets, the network tends to rely on geometric continuity to determine the bounding box size; while for near targets, there is rich texture detail, and the network relies more on the degree of image edge closure. This decision logic is consistent with natural imaging conditions, rather than arbitrary statistical correlations.

[0112] During the inference phase, the network performs non-maximum suppression after merging candidate boxes from various scales. To reduce false detections at sea surface spots, this embodiment introduces a threshold adjustment related to depth weights: if the candidate region has dense depth continuity and low projection residuals, a lower image presence score is allowed to pass; conversely, if the depth weight is low and the image texture is clear, the threshold weight of the image branch is increased. This gating strategy reflects causal consistency—the side with higher observation reliability has a greater influence on the fusion decision. The final output bounding box position parameters are expressed in the image coordinate system. Combined with the projection relationship of S301 to S302, the target's scale or distance in the camera coordinate system can be calculated, providing a stable geometric input for path planning and subsequent tracking in maritime reconnaissance.

[0113] From an engineering adaptation perspective, the number of pyramid layers and the channel ratio per layer can be adjusted according to the platform. For long-haul high-altitude routes, the upper-level channels can be increased to cover the distant outlines of large ships, while for low-altitude near-shore inspections, the shallow layers can be enhanced to improve the boundary representation of small targets. Regardless of the adjustments, the main principles of S601 and S602 remain unchanged: first, allow geometry to be seen and re-weighted at multiple scales, then align it with semantic textures at the same layer, and finally drive the geometric convergence of the bounding boxes with overlap, so that the network can still stably output candidate targets with reasonable positions and scales under fluctuating sea conditions and complex lighting.

[0114] In one embodiment of the UAV swarm maritime target identification method of this application, it may further include the following:

[0115] Step S701: Normalize the predicted ship category probability, compare the normalized probability with the true label, calculate the cross-entropy loss value of the category prediction, normalize the predicted target confidence, compare the normalized confidence with the true label, and calculate the cross-entropy loss value of the confidence.

[0116] Step S702: Linearly combine the positioning loss value and the cross-entropy loss value, dynamically adjust the weight coefficients according to the training phase, calculate the comprehensive score for large ships, medium-sized civilian ships and small fishing boats respectively, generate the target location coordinates and type probability in the sea area, and determine the type and location information of the target vessel.

[0117] After completing the multi-scale fusion and localization supervision in steps S601-S602, this embodiment enters the classification and confidence learning stage, and weights various losses according to the training progress. The input consists of fused features from each scale of the feature pyramid. The detector head outputs the category probability vectors for three ship types and a confidence scalar for target existence at each candidate box location. The numerical ranges of both are first normalized. The category probabilities are soft-normalized to ensure the sum of each category is 1; the confidence is normalized using Sigmoid, reflecting the single-label judgment of "whether any ship exists within the box". Maritime data exhibits a significant long-tail distribution. Small fishing boats are numerous and varied in appearance at long distances, while large ships appear less frequently but have a wide scale range. If the learning of category probabilities and confidence is not separated, the network will confuse easily distinguishable categories with easily occurring backgrounds, distorting the training signal.

[0118] In terms of loss construction, the category branch compares the normalized class probability with the true class label sample by sample, using cross-entropy as the metric; the confidence branch compares the normalized confidence score with a binary label indicating whether the target is present, also using cross-entropy. The two branches focus on different aspects: the category term drives the network to find textures and geometric combinations that distinguish ship types from fused features, such as deck layout, funnel shape, and bow lines; the confidence term emphasizes whether the candidate box covers the overall evidence of the foreground, relying on the continuity of depth steps and the closure of image edges. Changes in sea conditions alter the hierarchy of evidence; for example, light rain leads to sparse point clouds, causing confidence learning to absorb more information from the image branch; backlighting blurs boundaries, causing confidence learning to rely more on geometric continuity. This causal relationship aligns with natural imaging, preventing the model from being led astray by erroneous gradients when noise dominates.

[0119] The overall goal is to unify localization, classification, and confidence. Localization loss comes from the overlap metric of S602, while classification and confidence losses come from the cross-entropy term in this step. This embodiment linearly combines these three and introduces dynamic weights. In the early stages of training, the weight of the localization term is increased, allowing the network to learn to geometrically align the bounding boxes. In the mid-stage, the weights of confidence and classification are gradually increased to make the foreground / background and type boundaries clearer. In the later stages, fine-tuning is performed based on the convergence status of the validation set to avoid overfitting caused by any one factor dominating for a long time. The adjustment of dynamic weights follows the temporal progression of actual difficulty: geometric alignment is a prerequisite for classification; if the bounding box deviation is large in the early stages, the category supervision signal may mislead feature learning. After geometric convergence, semantic boundaries are refined, resulting in a more stable training path.

[0120] In the reasoning and evaluation phases, candidate results are uniformly ranked based on a "comprehensive score." The comprehensive score is derived from a linear combination of localization quality, confidence level, and class probability, weighted according to their respective scales. These weights maintain the same scale as the training weights to ensure end-to-end consistency. For the three target categories—large ships, medium-sized civilian vessels, and small fishing boats—the category component of the comprehensive score uses the normalized probability of the corresponding category, the localization component uses the overlap estimate of the pyramid layer that best matches the scale of the bounding box, and the confidence component reflects the overall credibility of the foreground within the candidate box. In maritime applications, the visibility of the three target categories differs significantly: large ships have clear boundaries at long distances but sparse texture details, so the proportion of localization and confidence in the comprehensive score should be slightly higher; small fishing boats have complex textures near the shore but weak geometric transitions, making class probability and shallow confidence more valuable. This allocation aligns with the natural principle of prioritizing geometric reliability when it is geometrically reliable and semantic reliability when it is semantically reliable.

[0121] To avoid bias caused by class imbalance during training, this embodiment introduces sample reweighting or hard sample mining strategies for class cross-entropy. The sea surface background easily generates a large number of negative samples. If the confidence branch is simply averaged, it will be dominated by negative samples, causing the network to tend to raise the threshold and miss weak targets. By weighting hard examples (low IoU but high confidence, or high IoU but low confidence), the gradient is more distributed to real challenges such as blurred boundaries, strong reflections, and sparse echoes, aligning the learning direction with physical difficulties rather than statistical randomness. The same applies to class branches. Small fishing boats at a distance have poor texture and are easily misclassified as background. Sample reweighting creates a stronger class discrimination in this area, gradually establishing the combined semantics of "short, thin bow + shallow draft".

[0122] In terms of output organization, this embodiment outputs position parameters, a comprehensive score, and the probabilities of three categories for each valid candidate, and provides the center point and dimensions in the image coordinate system. The position parameters can be derived back to the spatial scale of the camera coordinate system using the projection transformation of S302, for subsequent ranging or path planning. The comprehensive score is not filtered by a fixed threshold under different sea conditions, but rather by introducing a slight adaptive approach: when the depth weight is high and the positioning overlap is good, a lower category probability is allowed to pass to avoid missed detections under backlight conditions; when the depth weight is low and the image boundaries are clear, the threshold for category probability is increased to reduce false alarms of wave highlights. This gating is based on the upstream real observation quality and geometric consistency, and has a clear causal chain.

[0123] From an engineering perspective, the dynamic weighting timeline is coordinated with the learning rate scheduling. In the early stages, a smaller learning rate is used with a higher localization weight to prevent bounding box regression oscillations. In the mid-to-late stages, as the bounding boxes stabilize, the weights of the category and confidence are increased, and the cosine warming magnitude of the learning rate is slightly increased, allowing the network to extract semantic details while maintaining geometric convergence. For different platforms and tasks, the prior distributions of the three target classes can be injected into the category item of the comprehensive score. For example, in the long-range maritime patrol phase, the prior distributions of large ships are higher, and the classification threshold is slightly lowered to ensure that key targets are not missed; in the near-shore fishing port patrol phase, the category weights of small fishing boats are increased to meet on-site needs. The entire process starts with S701 generating clear and interpretable category and confidence learning signals, which are then fused into an integrated indicator for decision-making by S702. The output type and location information is consistent with physical observation conditions and geometric relationships, facilitating stable integration with subsequent tracking and control systems.

[0124] To effectively address the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, and to provide technical support for maritime surveillance, this application provides an embodiment of a UAV swarm maritime target identification device for implementing all or part of the aforementioned UAV swarm maritime target identification method. See [link to embodiment]. Figure 2 The aforementioned unmanned aerial vehicle (UAV) swarm maritime target identification device specifically includes the following components:

[0125] The marine point cloud data processing module 10 is used to deploy a swarm of drones to perform marine reconnaissance missions, collect image data acquired by a visible light camera and point cloud data acquired by a lidar, record the sampling time of the visible light camera and the lidar, establish a timestamp based on the host computer time reference, index and match the image data and the point cloud data according to the most recent time, calculate the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, select the common viewpoint of the image and the point cloud to calculate the extrinsic parameter transformation matrix, and project the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix.

[0126] The model feature vector processing module 20 is used to input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, and optimize the depth weight matrix through the attention mechanism to generate dense depth features.

[0127] The sea area target ship identification module 30 is used to input the image feature vector and the dense depth feature into the sea area target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate a localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0128] As described above, the UAV swarm maritime target identification device provided in this application can achieve effective data fusion through an innovatively designed data synchronization system, calibration optimization, and coordinate transformation. It constructs a feature extraction mechanism, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and loss combination and position optimization are used to ensure identification accuracy. This method effectively solves the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0129] From a hardware perspective, in order to effectively address the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, and to provide technical support for maritime surveillance, this application provides an embodiment of an electronic device for implementing all or part of the aforementioned UAV swarm maritime target identification method. The electronic device specifically includes the following components:

[0130] The system comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between the UAV swarm maritime target identification device and core business systems, user terminals, and related databases and other related devices; the logic controller can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the logic controller can be implemented with reference to the embodiments of the UAV swarm maritime target identification method and the UAV swarm maritime target identification device in the embodiments, the contents of which are incorporated herein, and repeated details will not be described again.

[0131] It is understood that the user terminal may include smartphones, tablet computers, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), in-vehicle devices, smart wearable devices, etc. Among these, the smart wearable devices may include smart glasses, smartwatches, smart bracelets, etc.

[0132] In practical applications, some parts of the UAV swarm maritime target identification method can be executed on the electronic device side as described above, or all operations can be completed in the client device. The choice can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are completed in the client device, the client device may further include a processor.

[0133] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0134] Figure 3 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 3 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 3 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0135] In one embodiment, the function of the UAV swarm maritime target identification method can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0136] Step S101: Deploy a swarm of drones to perform a maritime reconnaissance mission, collect image data acquired by a visible light camera and point cloud data acquired by a lidar, record the sampling time of the visible light camera and the lidar, establish a timestamp based on the host computer time reference, index and match the image data and the point cloud data according to the most recent time, calculate the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, select the common viewpoint of the image and the point cloud to calculate the extrinsic parameter transformation matrix, and project the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix;

[0137] Step S102: Input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, optimize the depth weight matrix through the attention mechanism, and generate dense depth features.

[0138] Step S103: Input the image feature vector and the dense depth feature into the sea area target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate the localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0139] As described above, the electronic device provided in this application, through an innovatively designed data synchronization system, achieves effective data fusion through calibration optimization and coordinate transformation. A feature extraction mechanism is constructed, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and through loss combination and position optimization, the accuracy of recognition is ensured. This method effectively solves the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0140] In another embodiment, the UAV swarm marine target identification device can be configured separately from the central processing unit 9100. For example, the UAV swarm marine target identification device can be configured as a chip connected to the central processing unit 9100, and the function of the UAV swarm marine target identification method can be realized through the control of the central processing unit.

[0141] like Figure 3 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 3 All components shown; in addition, the electronic device 9600 may also include Figure 3 For components not shown, please refer to existing technology.

[0142] like Figure 3 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0143] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0144] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0145] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0146] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0147] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processing unit 9100 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0148] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored audio via the speaker 9131.

[0149] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the UAV swarm maritime target identification method with a server or client as the execution subject in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the UAV swarm maritime target identification method with a server or client as the execution subject in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0150] Step S101: Deploy a swarm of drones to perform a maritime reconnaissance mission, collect image data acquired by a visible light camera and point cloud data acquired by a lidar, record the sampling time of the visible light camera and the lidar, establish a timestamp based on the host computer time reference, index and match the image data and the point cloud data according to the most recent time, calculate the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, select the common viewpoint of the image and the point cloud to calculate the extrinsic parameter transformation matrix, and project the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix;

[0151] Step S102: Input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, optimize the depth weight matrix through the attention mechanism, and generate dense depth features.

[0152] Step S103: Input the image feature vector and the dense depth feature into the sea area target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate the localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0153] As described above, the computer-readable storage medium provided in this application embodiment achieves effective data fusion through an innovatively designed data synchronization system, calibration optimization, and coordinate transformation. A feature extraction mechanism is constructed, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and loss combination and position optimization are used to ensure accurate identification. This method effectively addresses the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0154] Embodiments of this application also provide a computer program product capable of implementing all steps in the UAV swarm maritime target identification method with the execution subject being a server or client as described in the above embodiments. When executed by a processor, this computer program / instruction implements the steps of the UAV swarm maritime target identification method. For example, the computer program / instruction implements the following steps:

[0155] Step S101: Deploy a swarm of drones to perform a maritime reconnaissance mission, collect image data acquired by a visible light camera and point cloud data acquired by a lidar, record the sampling time of the visible light camera and the lidar, establish a timestamp based on the host computer time reference, index and match the image data and the point cloud data according to the most recent time, calculate the intrinsic parameter matrix of the visible light camera using the Zhang calibration method, select the common viewpoint of the image and the point cloud to calculate the extrinsic parameter transformation matrix, and project the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix;

[0156] Step S102: Input the image data and the point cloud data into the sea area target preprocessing convolutional network, extract the sea surface ship image feature vector and depth feature vector respectively, complete the depth feature vector through the conditional random field network, construct the sea area depth weight matrix, optimize the depth weight matrix through the attention mechanism, and generate dense depth features.

[0157] Step S103: Input the image feature vector and the dense depth feature into the sea area target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate the localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

[0158] As described above, the computer program product provided in this application, through an innovatively designed data synchronization system, achieves effective data fusion through calibration optimization and coordinate transformation. It constructs a feature extraction mechanism, combining depth completion and attention optimization to establish reliable feature representations. Multi-scale detection is introduced, and through loss combination and position optimization, the accuracy of recognition is ensured. This method effectively solves the shortcomings of traditional technologies in data synchronization, feature extraction, and target detection, providing technical support for maritime monitoring.

[0159] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0160] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0161] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0163] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for identifying maritime targets in a swarm of unmanned aerial vehicles (UAVs), characterized in that, The method includes: Deploying a swarm of drones to perform maritime reconnaissance missions, collecting image data from visible light cameras and point cloud data from lidar, recording the sampling times of the visible light cameras and lidar, and establishing a timestamp based on a host computer time reference, includes: deploying the drone swarm in the target maritime area, controlling the first batch of drones equipped with visible light cameras to collect image information, controlling the second batch of drones equipped with lidar to collect point cloud information, generating acquisition parameters including acquisition location, acquisition time, and data scale; recording the sampling time information of the visible light cameras and lidar, using the host computer time reference as a reference standard to correct the sampling time information, calculating the hardware transmission delay deviation, and generating a unified timestamp based on the host computer; The image data and the point cloud data are indexed and matched according to the most recent time. The intrinsic parameter matrix of the visible light camera is calculated by Zhang's calibration method. The extrinsic parameter transformation matrix is ​​calculated by selecting the common viewpoint of the image and the point cloud. Based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix, the point cloud data is projected onto the coordinate system of the image data. The image data and point cloud data are input into a marine target preprocessing convolutional network to extract surface ship image feature vectors and depth feature vectors, respectively. The depth feature vectors are then completed using a conditional random field network to construct a marine depth weight matrix. This depth weight matrix is ​​then optimized using an attention mechanism to generate dense depth features. The process includes: inputting the depth feature vectors into the conditional random field network; calculating the unary potential function of each node in the fully connected graph; calculating the pairwise potential function between node pairs based on image features to generate a node weight matrix; normalizing the node weight matrix to construct the marine depth weight matrix; extracting feature patches from the depth weight matrix; calculating the dot product of the query vector and the key vector to generate weight coefficients between patches; combining the weight coefficients with relative position encoding information; and optimizing the depth weight matrix using an attention mechanism to generate dense depth features. The image feature vector and the dense depth feature are input into the marine target feature pyramid network to generate multi-scale fusion features. The overlap between the ship target bounding box and the ground truth bounding box is calculated to generate a localization loss value. The cross-entropy loss value of ship category and confidence is calculated. The localization loss value and the cross-entropy loss value are weighted and combined to determine the type and location information of the target ship.

2. The method for identifying marine targets in a swarm of unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The steps include: indexing and matching the image data and the point cloud data according to the most recent time; calculating the intrinsic parameter matrix of the visible light camera using the Zhang calibration method; selecting common viewpoints of the image and point cloud to calculate the extrinsic parameter transformation matrix; and projecting the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix. The image data and point cloud data are sorted according to the sampling time, the time interval between adjacent data frames is calculated, the image and point cloud data pairs with the most recent time are selected based on the time interval, a temporal matching matrix is ​​generated, and an index association is established between the image data and point cloud data in the temporal matching matrix. The intrinsic parameter matrix of the visible light camera is calculated based on Zhang's calibration method. Commonly viewed feature points of the image and point cloud are selected, and a three-dimensional coordinate mapping relationship of the feature points is constructed. The extrinsic parameter transformation matrix from the laser radar to the visible light camera is calculated. The point cloud data is then projected onto the image coordinate system through the intrinsic parameter matrix and the extrinsic parameter transformation matrix.

3. The method for identifying marine targets in a swarm of unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The step of inputting the image data and the point cloud data into a sea area target preprocessing convolutional network to extract feature vectors and depth feature vectors of the sea surface ship image includes: Image data is input into the first preprocessing convolutional layer, and the number of output channels is set to 48. Point cloud data is input into the second preprocessing convolutional layer, and the number of output channels is set to 16. The output feature maps of the two preprocessing convolutional layers are superimposed to generate a fused feature map. The fused feature map is input into the third preprocessing convolutional layer, and the number of output channels is set to 64. The fused feature map is subjected to fuzz enhancement processing. The enhanced feature map is then input into a feature extraction network. The image feature vector of the ship on the sea surface is calculated based on the neural network. The depth feature vector is calculated based on sparse convolution. The depth feature vector is then normalized.

4. The method for identifying marine targets in a swarm of unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The process of inputting the image feature vector and the dense depth feature into a marine target feature pyramid network to generate multi-scale fusion features, calculating the overlap between the ship target bounding box and the ground truth bounding box, and generating a localization loss value includes: Image feature vectors and dense depth features are input into a feature pyramid network, and feature maps at different levels are downsampled. The depth features are enhanced through spatial attention and channel attention mechanisms. The enhanced depth features are then superimposed with image features to generate multi-scale fusion features. Based on the multi-scale fusion features, the target bounding box is predicted, the intersection area of ​​the target bounding box and the ground truth bounding box is calculated, the union area of ​​the target bounding box and the ground truth bounding box is calculated, the intersection area is divided by the union area to obtain the overlap, and the localization loss value is generated based on the overlap.

5. The method for identifying maritime targets in a swarm of unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The calculation of the cross-entropy loss value between ship category and confidence level, and the weighted combination of the positioning loss value and the cross-entropy loss value to determine the type and location information of the target ship, includes: The predicted ship category probabilities are normalized, the normalized probabilities are compared with the true labels, and the cross-entropy loss value of the category prediction is calculated. The predicted target confidence is normalized, the normalized confidence is compared with the true labels, and the cross-entropy loss value of the confidence is calculated. The positioning loss value and the cross-entropy loss value are linearly combined, and the weight coefficients are dynamically adjusted according to the training phase. The comprehensive scores are calculated for large ships, medium-sized civilian ships, and small fishing boats respectively, generating the target location coordinates and type probability in the sea area, and determining the type and location information of the target vessel.

6. A device for identifying maritime targets in a swarm of unmanned aerial vehicles (UAVs), characterized in that, The device includes: The marine point cloud data processing module is used to deploy a swarm of drones to perform marine reconnaissance missions, collect image data from visible light cameras and point cloud data from lidar, record the sampling times of the visible light cameras and lidar, and establish a timestamp based on a host computer time reference. This includes: deploying a swarm of drones in a target marine area; controlling the first batch of drones equipped with visible light cameras to collect image information; controlling the second batch of drones equipped with lidar to collect point cloud information; generating acquisition parameters including acquisition location, acquisition time, and data scale; recording the sampling time information of the visible light cameras and lidar; using the host computer time reference as a standard to correct the sampling time information; calculating the hardware transmission delay deviation; generating a unified timestamp based on the host computer; indexing and matching the image data and the point cloud data according to the most recent time; calculating the intrinsic parameter matrix of the visible light camera using the Zhang calibration method; selecting common viewpoints of the image and point cloud to calculate the extrinsic parameter transformation matrix; and projecting the point cloud data onto the coordinate system of the image data based on the intrinsic parameter matrix and the extrinsic parameter transformation matrix. The model feature vector processing module is used to input the image data and the point cloud data into a marine target preprocessing convolutional network, extract the feature vectors and depth feature vectors of the ship image on the sea surface, respectively, complete the depth feature vectors through a conditional random field network, construct a marine depth weight matrix, and optimize the depth weight matrix through an attention mechanism to generate dense depth features. This includes: inputting the depth feature vectors into a conditional random field network, calculating the unary potential function of each node in the fully connected graph, calculating the pairwise potential function between node pairs based on image features, generating a node weight matrix, normalizing the node weight matrix, constructing a marine depth weight matrix; extracting feature patches from the depth weight matrix, calculating the dot product of the query vector and the key vector, generating weight coefficients between patches, combining the weight coefficients with relative position encoding information, and optimizing the depth weight matrix based on an attention mechanism to generate dense depth features. The marine target ship identification module is used to input the image feature vector and the dense depth feature into the marine target feature pyramid network to generate multi-scale fusion features, calculate the overlap between the ship target bounding box and the real bounding box, generate a localization loss value, calculate the cross-entropy loss value of ship category and confidence, and weight and combine the localization loss value and the cross-entropy loss value to determine the type and location information of the target ship.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for identifying unmanned aerial vehicle (UAV) swarm targets in the sea area as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method for identifying unmanned aerial vehicle (UAV) swarm targets in the sea area as described in any one of claims 1 to 5.

Images