Method and system for searching for a person fallen into the sea by combining multi-modal and reinforcement learning

Abstract

The application discloses a multimodal and reinforcement learning combined offshore fallen personnel searching method and system. The method comprises the following steps: collecting sensor data including visible light image data, infrared image data and radar point cloud data, and performing high-precision space-time alignment; extracting image features of each sensor data, and splicing and then fusing into a unified encoding feature vector through an encoder; collecting environmental data such as rain and fog grades and light intensity and encoding into environmental codes; inputting the encoding feature vector and the environmental code into a combined strategy network driven by an LSTM and a PPO algorithm, and dynamically calculating the weight of each sensor data; adaptively weighting and fusing the image features based on the weight, and finally outputting a searching result through a target detection algorithm. The application realizes intelligent scheduling of sensor resources and deep fusion of multi-sensor data in complex sea conditions, and effectively improves the searching precision, response speed and system energy efficiency.

Description

Technical Field

[0001] This application relates to the field of multimodal reinforcement learning technology, and more specifically, to a method and system for searching for people who have fallen into the water at sea that combines multimodal learning and reinforcement learning. Background Technology

[0002] Maritime search and rescue, as a crucial safeguard for global maritime security, demands extremely high efficiency and accuracy. However, the marine environment is highly dynamic and complex: the reflective interference caused by undulating waves, the visibility limitations of sea fog and salt spray, the drastic changes in day and night illumination, and the similarity between life-saving equipment and the sea surface background all greatly increase the difficulty of target identification and seriously affect search and rescue efficiency.

[0003] Existing maritime search and rescue drones mostly employ single or simple sensor configurations, primarily relying on visible light cameras for target searching. While visible light cameras can provide high spatial resolution of 0.3-1.2m under ideal conditions, their performance is severely limited in complex maritime environments: strong sea surface reflections cause the signal-to-noise ratio to drop below 15dB, rendering them almost inoperable at night or in foggy weather with visibility less than 50m; infrared sensors can detect human thermal radiation characteristics (8-14μm wavelength) in all weather conditions, but are limited by sea surface temperature interference and low spatial resolution of 3-5m / 100m, resulting in a 45% increase in false detection rate in tropical waters where seawater temperatures are close to human body temperature; although lidar has centimeter-level ranging accuracy (±3cm) and can accurately depict sea surface undulations, point cloud density attenuation exceeds 60% in foggy environments, and its high power consumption of 25W makes it difficult to support long-term maritime search missions. More critically, traditional multi-sensor fusion methods use fixed weight strategies, failing to adapt to changes in environmental factors such as sea state, weather, and time, leading to poor performance in the face of complex and ever-changing maritime environments.

[0004] Existing technologies have the following main problems in maritime search and rescue scenarios:

[0005] Heterogeneous Fusion Conflicts of Multimodal Features at Sea: Maritime search and rescue targets (such as people in the water, life rafts, and floating objects) exhibit drastically different features under different sensors: visible light images primarily show color and texture features, infrared images show thermal radiation intensity distribution, and lidar shows three-dimensional geometric structure information. Existing fusion methods often employ simple feature-level stitching or decision-level voting. However, due to the lack of an effective cross-modal feature alignment mechanism, heterogeneous features acquired from different sensors frequently conflict during the fusion process, leading to the mutual cancellation of useful information. Especially against the complex background of the sea surface, the semantic correlation between optical texture features and thermal radiation features is weak; direct fusion often results in feature space chaos, reducing the accuracy of target identification. Summary of the Invention

[0006] The main objective of this application is to provide a multimodal and reinforcement learning-based method and system for searching for people who have fallen into the water at sea, so as to at least solve the problems mentioned in the background art and improve the detection accuracy, precision and response speed of search and rescue targets.

[0007] To achieve the above objectives, a multimodal and reinforcement learning-based method and system for searching for people who have fallen into the water at sea is provided, as detailed below.

[0008] Firstly, this application provides a multimodal and reinforcement learning-based method for searching for people who have fallen overboard at sea, including:

[0009] Multiple sensors deployed in the target area are used to collect multiple sensor data in the target area. Spatiotemporal alignment processing is performed on the multiple sensor data to obtain multiple sensor aligned data of the target area.

[0010] Image features from multiple sensor aligned data are extracted separately, multiple image features are concatenated into a feature vector, and an encoder is used to convert the feature vector into an encoded feature vector;

[0011] Multiple environmental data points of the target area are collected, converted into environmental vectors, and then encoded using a fully connected network to obtain environmental codes.

[0012] The weights of each sensor are calculated using a combined policy network based on the encoded feature vector and the environment encoding. This includes: calculating the total encoding using the encoded feature vector and the environment encoding in the fully connected layer of the combined policy network.

[0013] The formula for calculating the total code is as follows:

[0014] ;

[0015] in, Indicates the total code. Indicates environment encoding. Indicates the parameters for weight calculation. Offset calculation parameters, Represents the encoded feature vector;

[0016] In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the context encoding, and the total encoding.

[0017] The formula for calculating the hidden state is:

[0018] ;

[0019] in:

[0020] ;

[0021] ;

[0022] ;

[0023] in, Indicates a hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the current state.

[0024] The weights of each sensor are computed using the hidden states in the output layer of the combined policy network;

[0025] Optimizing the performance of a combined policy network using reinforcement learning algorithms to update weights includes:

[0026] Calculate the reward function to evaluate the performance of the combined policy network;

[0027] The formula for calculating the reward function is as follows:

[0028] , ;

[0029] in, Represents the reward function, Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight;

[0030] By utilizing the alternating update strategy and value function approach in the PPO algorithm to optimize the performance of the combined policy network, a combined policy network that maximizes the reward function is obtained.

[0031] The alternating update strategy steps in the PPO algorithm are as follows:

[0032] Experience collection: Interact with the system using the current combined policy network to collect states, actions, rewards, and subsequent state sequences;

[0033] Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated through the generalized advantage estimation algorithm to evaluate the relative merits of the actions;

[0034] Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters:

[0035] ;

[0036] in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. Represents the clipping function;

[0037] Value function network update: With a fixed policy network, the following mean squared error loss is minimized through gradient descent to optimize the network parameters of the value function, making its predictions closer to the actual returns:

[0038] ;

[0039] in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. Indicates the state at time step t;

[0040] Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network.

[0041] Multiple image features are weighted and fused using multiple weights to obtain a fused feature vector. A target detection algorithm is then used to identify the fused feature vector to obtain the target result in the target region.

[0042] Preferably, multiple sensors deployed in the target area collect multiple sensor data in the target area, and perform spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor aligned data of the target area, including:

[0043] Multiple sensor data include visible light image data, infrared image data, and radar point cloud data; among them, visible light image data is three-dimensional data, infrared image data is two-dimensional data, and radar point cloud data is four-dimensional data.

[0044] Time synchronization and spatial alignment of data from multiple sensors are performed to obtain aligned data from multiple sensors in the target area.

[0045] Preferably, multiple sensor data are synchronized in time and aligned spatially to obtain multiple sensor aligned data for the target area, including:

[0046] The GPS_Sync function is used to synchronize visible light image data, infrared image data, and radar point cloud data in time, resulting in time-aligned visible light image data, infrared image data, and radar point cloud data, ensuring that the visible light image data, infrared image data, and radar point cloud data are collected within a unified time window.

[0047] Hardware synchronization of multiple sensors ensures a synchronization error of less than 1ms;

[0048] The intrinsic parameter matrix of the time-aligned visible light image data is obtained using the Zhang Zhengyou calibration method. The time-aligned visible light image data is then processed using the intrinsic parameter matrix to obtain the aligned visible light image data.

[0049] The SIFT feature matching algorithm is used to extract feature points from time-aligned visible light image data and time-aligned infrared image data. Affine transformation is used to register the feature points to obtain aligned infrared image data.

[0050] Perspective projection is used to project the time-aligned radar point cloud data onto a plane, and the resolution is unified with the time-aligned visible light image data to obtain aligned radar point cloud data.

[0051] Preferably, image features from multiple sensor aligned data are extracted, the multiple image features are concatenated into a feature vector, and the feature vector is converted into an encoded feature vector using an encoder, including:

[0052] Visible light features are obtained by extracting features from aligned visible light image data; infrared features are obtained by extracting features from aligned infrared image data; and radar features are obtained by extracting features from aligned radar point cloud data.

[0053] Visible light features, infrared features, and radar features are concatenated into a feature vector;

[0054] The Transformer encoder is used to convert feature vectors into encoded feature vectors.

[0055] Preferably, multiple environmental data points of the target area are collected, the multiple environmental data points are converted into environmental vectors, and the environmental vectors are encoded using a fully connected network to obtain environmental codes, including:

[0056] Environmental data includes rain and fog level data for the target area, target speed data, light intensity data, and cloud height data;

[0057] The environmental data is converted into an environmental vector; the environmental vector is then encoded using a fully connected network to obtain the environmental code.

[0058] Preferably, the weights of each sensor are calculated through a combined policy network based on the encoded feature vectors and the environment encoding, including:

[0059] In the fully connected layer of the combined policy network, the total encoding is calculated using the encoded feature vector and the environment encoding.

[0060] In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the context encoding, and the total encoding.

[0061] The weights of each sensor are calculated using the hidden states in the output layer of the combined policy network.

[0062] Preferably, optimizing the performance of the combined policy network using reinforcement learning algorithms includes:

[0063] Calculate the reward function to evaluate the performance of the combined policy network;

[0064] By utilizing the alternating update strategy and value function approach in the PPO algorithm to optimize the performance of the combined policy network, a combined policy network that maximizes the reward function is obtained.

[0065] Preferably, a fused feature vector is obtained by weighting and fusing multiple image features using multiple weights, including:

[0066] A fused feature vector is obtained by weighted fusion of visible light features, infrared features, radar features, and sensor weights.

[0067] Preferably, the target result in the target region is obtained by using a target detection algorithm to identify and fuse feature vectors, including:

[0068] The target detection algorithm is trained using a loss function;

[0069] Spatial pyramid pooling is used to extract multi-scale features from the fused feature vector. The first output dimension of the multi-scale features is... ;

[0070] Target detection is performed using three anchor points on multi-scale features to obtain target results within the target region. The second output dimension of the target results is... ;in, Indicates the number of categories.

[0071] Secondly, this application provides a multimodal and reinforcement learning-based search system for people who have fallen overboard at sea. The system is applied to the method described in the first aspect above, and includes:

[0072] The sensor data acquisition and processing unit is used to acquire multiple sensor data in the target area using multiple sensors deployed in the target area, and to perform spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor aligned data in the target area.

[0073] The sensor data feature extraction unit is electrically connected to the sensor data acquisition and processing unit. It is used to extract image features from the aligned data of multiple sensors, stitch the multiple image features into a feature vector, and use an encoder to convert the feature vector into an encoded feature vector.

[0074] The environmental data acquisition and processing unit is electrically connected to the sensor data feature extraction unit. It is used to acquire multiple environmental data of the target area, convert the multiple environmental data into environmental vectors, and encode the environmental vectors using a fully connected network to obtain environmental codes.

[0075] The sensor weight calculation and combined strategy network optimization unit is electrically connected to the environmental data acquisition and processing unit. It is used to calculate the weight of each sensor through the combined strategy network based on the encoded feature vector and the environmental code. This includes: in the fully connected layer of the combined strategy network, the total code is calculated using the encoded feature vector and the environmental code.

[0076] The formula for calculating the total code is as follows:

[0077] ;

[0078] in, Indicates the total code. Indicates environment encoding. Indicates the parameters for weight calculation. Offset calculation parameters, Represents the encoded feature vector;

[0079] In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the context encoding, and the total encoding.

[0080] The formula for calculating the hidden state is:

[0081] ;

[0082] in:

[0083] ;

[0084] ;

[0085] ;

[0086] in, Indicates a hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the current state.

[0087] The weights of each sensor are computed using the hidden states in the output layer of the combined policy network; the performance of the combined policy network is optimized using reinforcement learning algorithms to update the weights, including:

[0088] Calculate the reward function to evaluate the performance of the combined policy network;

[0089] The formula for calculating the reward function is as follows:

[0090] , ;

[0091] in, Represents the reward function, Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight;

[0092] By utilizing the alternating update strategy and value function approach in the PPO algorithm to optimize the performance of the combined policy network, a combined policy network that maximizes the reward function is obtained.

[0093] The alternating update strategy steps in the PPO algorithm are as follows:

[0094] Experience collection: Interact with the system using the current combined policy network to collect states, actions, rewards, and subsequent state sequences;

[0095] Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated through the generalized advantage estimation algorithm to evaluate the relative merits of the actions;

[0096] Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters:

[0097] ;

[0098] in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. Represents the clipping function;

[0099] Value function network update: With a fixed policy network, the following mean squared error loss is minimized through gradient descent to optimize the network parameters of the value function, making its predictions closer to the actual returns:

[0100] ;

[0101] in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. Indicates the state at time step t;

[0102] Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network.

[0103] The target recognition unit is electrically connected to the sensor weight calculation and combination strategy network optimization unit. It is used to perform weighted fusion of multiple image features using multiple weights to obtain a fused feature vector, and to use a target detection algorithm to identify the fused feature vector to obtain the target result in the target region.

[0104] This application provides a multimodal and reinforcement learning-based method and system for searching for people who have fallen into the sea. This method achieves efficient and accurate searching for people who have fallen into the sea by fusing visible light, infrared, and radar data and combining them with reinforcement learning intelligent decision-making. First, the multi-source sensor data is spatiotemporally aligned and features are extracted and fused into a unified feature code. At the same time, environmental data is collected and encoded, and input together with the perception features into a policy network based on LSTM and PPO to dynamically calculate the optimal weights of each sensor. Finally, the recognition result is output through weighted fusion of features and target detection algorithm. While ensuring accuracy, this method significantly optimizes energy consumption and response speed, effectively improving the success rate of search and rescue in complex sea conditions. Attached Figure Description

[0105] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0106] Figure 1 A flowchart illustrating a multimodal and reinforcement learning-based method for searching for people who have fallen overboard at sea, as provided in this application;

[0107] Figure 2 A flowchart of a multi-sensor adaptive fusion method for searching for people who have fallen into the water at sea, which combines multimodal and reinforcement learning, provided in this application;

[0108] Figure 3 A schematic diagram of the structure of a multimodal and reinforcement learning combined search system for people who have fallen into the water at sea, provided in this application;

[0109] Figure 4 A flowchart illustrating the target monitoring process of a multi-source sensor adaptive fusion method for searching for people who have fallen overboard at sea, which is a multimodal and reinforcement learning approach provided in this application.

[0110] Figure 5 A data processing flowchart of an unmanned system ground target monitoring system for a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea, provided in this application;

[0111] Figure 6 The flowchart of the unmanned system for ground target monitoring, which is a method for searching for people who have fallen into the water at sea that combines multimodal and reinforcement learning, is provided in this application. Detailed Implementation

[0112] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0113] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0114] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0115] This application provides a multimodal and reinforcement learning-based method and system for searching for people who have fallen overboard at sea. The method first performs precise spatiotemporal alignment and feature extraction on visible light image data, infrared image data, and radar point cloud data, fusing them into a unified encoded feature vector. Simultaneously, rain / fog level data, target velocity data, light intensity data, and cloud height data are encoded into environmental codes. Then, a combined policy network driven by LSTM (Long Short-Term Memory) and PPO (Proximal Policy Optimization) algorithms comprehensively analyzes perceived features (i.e., encoded feature vectors) and environmental states (i.e., environmental codes) to generate weights for each sensor. Finally, based on these weights, multimodal features (i.e., image features corresponding to sensor-aligned data) are adaptively weighted and fused, and a target detection algorithm is used to accurately identify and locate the person in the water (i.e., the target). This method overcomes the limitations of traditional fusion strategies, achieving joint optimization of search accuracy, system energy consumption, and response speed in complex and changing maritime environments.

[0116] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0117] Figure 1 The flowchart of a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea, provided in this application, includes: collecting multiple sensor data from multiple sensors deployed in the target area; performing spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor-aligned data for the target area; extracting image features from the multiple sensor-aligned data respectively; concatenating the multiple image features into a feature vector and converting the feature vector into an encoded feature vector using an encoder; collecting multiple environmental data from the target area; converting the multiple environmental data into an environmental vector; encoding the environmental vector using a fully connected network to obtain an environmental code; and calculating the weight of each sensor based on the encoded feature vector and the environmental code through a combined policy network, including: calculating the total code using the encoded feature vector and the environmental code in the fully connected layer of the combined policy network.

[0118] The formula for calculating the total code is as follows:

[0119] ;

[0120] in, Indicates the total code. Indicates environment encoding. Indicates the parameters for weight calculation. Offset calculation parameters, Represents the encoded feature vector;

[0121] In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the context encoding, and the total encoding.

[0122] The formula for calculating the hidden state is:

[0123] ;

[0124] in:

[0125] ;

[0126] ;

[0127] ;

[0128] in, Indicates a hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the current state.

[0129] The weights of each sensor are computed using the hidden states in the output layer of the combined policy network;

[0130] Optimizing the performance of a combined policy network using reinforcement learning algorithms to update weights includes:

[0131] Calculate the reward function to evaluate the performance of the combined policy network;

[0132] The formula for calculating the reward function is as follows:

[0133] , ;

[0134] in, Represents the reward function, Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight;

[0135] By utilizing the alternating update strategy and value function approach in the PPO algorithm to optimize the performance of the combined policy network, a combined policy network that maximizes the reward function is obtained.

[0136] The alternating update strategy steps in the PPO algorithm are as follows:

[0137] Experience collection: Interact with the system using the current combined policy network to collect states, actions, rewards, and subsequent state sequences;

[0138] Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated through the generalized advantage estimation algorithm to evaluate the relative merits of the actions;

[0139] Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters:

[0140] ;

[0141] in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. Represents the clipping function;

[0142] Value function network update: With a fixed policy network, the following mean squared error loss is minimized through gradient descent to optimize the network parameters of the value function, making its predictions closer to the actual returns:

[0143] ;

[0144] in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. Indicates the state at time step t;

[0145] Iterative optimization: Repeatedly execute policy network updates and value function network updates multiple times to complete alternating updates within one iteration. Then, use the updated policy network to collect new experience. This process is repeated until the reward function R converges to its maximum value, thus obtaining the optimal combined policy network. Multiple image features are weighted and fused using multiple weights to obtain a fused feature vector. The target detection algorithm is then used to identify the fused feature vector to obtain the target result in the target region.

[0146] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. The method first performs spatiotemporal alignment and feature extraction on visible light image data, infrared image data, and radar point cloud data, fusing them into a unified encoded feature vector. Simultaneously, rain / fog level data, target velocity data, light intensity data, and cloud height data are encoded into environmental codes. Then, a reinforcement learning-based combined policy network is used to comprehensively analyze these two types of information, dynamically calculating the weights of each sensor. Finally, the system adaptively weights and fuses the features of different modalities (i.e., image features corresponding to the sensor-aligned data) based on these weights, inputting the results into a target detection algorithm, ultimately outputting accurate target results.

[0147] This method overcomes the limitations of traditional fixed-weight fusion by using reinforcement learning for intelligent decision-making. It can dynamically adjust the sensor usage strategy according to real-time sea conditions, thereby maintaining high recognition accuracy in complex marine environments. At the same time, its innovative energy consumption-accuracy joint optimization mechanism can actively shut down redundant sensors, significantly reducing system power consumption and extending endurance while ensuring performance, achieving the best balance between search efficiency, accuracy, and energy consumption.

[0148] Specifically, multiple sensors deployed in the target area are used to collect multiple sensor data in the target area. The multiple sensor data are then spatiotemporally aligned to obtain multiple sensor aligned data for the target area. This data includes visible light image data, infrared image data, and radar point cloud data. The visible light image data is three-dimensional data, the infrared image data is two-dimensional data, and the radar point cloud data is four-dimensional data. The multiple sensor data are then time-synchronized and spatially aligned to obtain the multiple sensor aligned data for the target area.

[0149] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. This method uses multiple sensors, such as visible light cameras, infrared thermal imagers, and lidar, deployed in the target area to simultaneously acquire three-dimensional visible light image data, two-dimensional infrared image data, and four-dimensional radar point cloud data. Subsequently, a combination of GPS hardware synchronization and software timestamps is used to ensure that all sensor data achieves millisecond-level time synchronization. On this basis, through algorithms such as camera calibration, SIFT (scale-invariant feature transform) feature registration, and perspective projection, data from different viewpoints and resolutions are uniformly mapped to the same coordinate system, ultimately generating multiple sensor aligned data that are precisely aligned in both time and space.

[0150] This method fundamentally solves the basic problem of fusing data from multi-source heterogeneous sensors. Through high-precision time synchronization and spatial registration, it effectively eliminates information misalignment and fusion conflicts caused by differences in the physical characteristics of sensors and asynchronous data acquisition. This provides an accurate and consistent data foundation for subsequent feature-level deep fusion and intelligent decision-making, thereby significantly improving the reliability and accuracy of the entire system's target perception in complex marine environments.

[0151] Specifically, time synchronization and spatial alignment of multiple sensor data are performed to obtain aligned data for the target area. This includes: using the GPS_Sync function to synchronize visible light image data, infrared image data, and radar point cloud data in time, resulting in time-aligned visible light image data, infrared image data, and radar point cloud data, ensuring that these data are acquired within a unified time window; performing hardware synchronization of multiple sensors to ensure a synchronization error of less than 1ms; using the Zhang Zhengyou calibration method to obtain the intrinsic parameter matrix of the time-aligned visible light image data, and processing the time-aligned visible light image data using the intrinsic parameter matrix; extracting feature points from the time-aligned visible light image data and time-aligned infrared image data using the SIFT feature matching algorithm, and registering the feature points using affine transformation to obtain aligned infrared image data; and projecting the time-aligned radar point cloud data onto a plane using perspective projection, and unifying the resolution with the time-aligned visible light image data to obtain aligned radar point cloud data.

[0152] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. This method achieves hardware-level time synchronization of multi-sensor data in less than 1ms using the GPS_Sync function (a GPS synchronization function) and the U-Blox ZED-F9P GPS module (a commercially available high-precision GPS module). For spatial alignment, it uses the Zhang Zhengyou calibration method to obtain the intrinsic parameters of the visible light camera, employs SIFT feature matching and affine transformation to perform pixel-level registration between the infrared and visible light images, and uses perspective projection to accurately project the 3D lidar point cloud onto the 2D image plane, ultimately unifying the visible light image data, infrared image data, and radar point cloud data in the spatiotemporal dimensions.

[0153] The formula for calculating aligned visible light image data is:

[0154]

[0155] in, This represents the aligned visible light image data. This represents the pixels in the aligned visible light image data. This represents the intrinsic parameter matrix of the time-aligned visible light image data. , and These represent the coordinates of a point in three-dimensional space in the camera coordinate system.

[0156] The formula for registering feature points using affine transformation is as follows:

[0157]

[0158] in, , , and The parameters representing the affine matrix, and Denotes the translation parameter, and the matching error is minimized as follows: , This represents time-aligned visible light image data. This represents time-aligned infrared image data. This represents the coordinates of feature points in time-aligned visible light image data. This represents the coordinates of feature points in the time-aligned infrared image data. Represents a feature descriptor.

[0159] By using perspective projection to project the time-aligned radar point cloud data onto a plane, and then unifying the resolution with the time-aligned visible light image data, the calculation formula for the aligned radar point cloud data is obtained as follows:

[0160] ,

[0161] in, This represents the aligned radar point cloud data. Operators representing perspective projection, and Indicates focal length. Represents the coordinates of the optical center. Indicates 3 A rotation matrix of 3, Indicates 3 Translation vector of 1, Represents the camera intrinsic parameter matrix. This represents time-aligned radar point cloud data.

[0162] This method achieves precise pixel-level alignment of cross-modal data through high-precision hardware synchronization and multi-level spatial registration algorithms. This rigorous spatiotemporal unification effectively eliminates information misalignment caused by differences in acquisition timing and viewing angle among different sensors, laying a reliable data foundation for subsequent cross-modal feature fusion and fundamentally improving the collaborative sensing accuracy and reliability of multi-source sensing systems in complex environments.

[0163] Specifically, image features from aligned data from multiple sensors are extracted, and these multiple image features are concatenated into a feature vector. An encoder is then used to convert the feature vector into an encoded feature vector. This process includes: extracting features from aligned visible light image data to obtain visible light features; extracting features from aligned infrared image data to obtain infrared features; extracting features from aligned radar point cloud data to obtain radar features; concatenating the visible light features, infrared features, and radar features into a feature vector; and using a Transformer encoder to convert the feature vector into an encoded feature vector.

[0164] Specifically, the calculation formulas for extracting features from aligned visible light image data to obtain visible light features, extracting features from aligned infrared image data to obtain infrared features, and extracting features from aligned radar point cloud data to obtain radar features are as follows:

[0165] ;

[0166] ;

[0167] ;

[0168] in, Indicates the characteristics of visible light. This represents the aligned visible light image data. express Convolution operation, Indicates infrared characteristics, express Activation function express Convolution operation, This represents the aligned infrared image data. Indicates radar characteristics, This represents the aligned radar point cloud data, MLP represents the multilayer perceptron, and max represents the maximum value operation.

[0169] The formula for concatenating these features into a 256-dimensional feature vector is as follows:

[0170]

[0171] in, Represents the eigenvector. It represents the set of real numbers.

[0172] The formula for converting a feature vector into an encoded feature vector is as follows:

[0173]

[0174] in, Represents the encoded feature vector. This indicates a Transformer encoder.

[0175] This method achieves efficient and deep integration of multimodal features (i.e., multi-sensor data) through a "divide and conquer, then fuse" strategy. It ensures that the unique advantages of each modality's features (such as visible light texture, infrared heat sources, and radar geometry) are fully extracted, while the Transformer encoder intelligently captures the inherent relationships between different features, effectively resolving semantic conflicts and information misalignment issues between heterogeneous features. This results in fused features that are more robust and discriminative than simple splicing, significantly improving the accuracy of subsequent target recognition.

[0176] Specifically, multiple environmental data points of the target area are collected, converted into environmental vectors, and then encoded using a fully connected network to obtain environmental codes. The environmental data includes rain and fog level data, target speed data, light intensity data, and cloud height data of the target area; the environmental data is converted into environmental vectors; and the environmental vectors are encoded using a fully connected network to obtain environmental codes.

[0177] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. The method systematically collects four types of environmental data in the target area: rain and fog level data, target speed data, light intensity data, and cloud height data, and constructs them into a four-dimensional environmental vector. Subsequently, a two-layer fully connected neural network is used to perform nonlinear transformation and feature enhancement on the environmental vector. Finally, a 256-dimensional environmental code is output through the Sigmoid activation function, thereby transforming the abstract environmental data into a quantitative and structured representation that can be directly understood and processed by deep neural networks.

[0178] The formula for converting multiple environmental data points into an environmental vector is:

[0179]

[0180] in, Represents the environment vector. This indicates the rain / fog level data. This represents the target speed data. Represents light intensity data. This represents cloud height data.

[0181] The formula for calculating the environment code is as follows:

[0182] ,

[0183] in, Indicates the primary environment code. Indicates environment encoding. , Indicates the parameters for weight calculation. , Offset calculation parameters, Let represent the Sigmoid activation function, where , , , , , It represents the set of real numbers.

[0184] This method transforms discrete environmental data with different dimensions into a unified, high-dimensional environmental vector, enabling collaborative decision-making between environmental data and sensor data within the same semantic space. This encoding approach not only quantifies the complex impact of the environment on sensor performance but also provides crucial contextual information for the combined policy network, serving as a fundamental basis for achieving the system's adaptive capabilities.

[0185] Specifically, the weights of each sensor are calculated through a combined policy network based on the encoded feature vector and the environment encoding, including: calculating the total encoding using the encoded feature vector and the environment encoding in the fully connected layer of the combined policy network; calculating the hidden state using the encoded feature vector, the environment encoding, and the total encoding in the decision layer of the combined policy network; and calculating the weights of each sensor using the hidden state in the output layer of the combined policy network.

[0186] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. The method concatenates encoded feature vectors with environmental encodings and inputs them into a combined policy network consisting of a fully connected layer and an LSTM decision layer. The fully connected layer first performs feature integration, and then the LSTM unit processes temporal information through its gating mechanisms (including forget gates and input gates) to calculate the hidden state containing historical decision context. Finally, the output layer uses this hidden state to generate a set of normalized sensor weights through a Softmax function, thereby achieving dynamic weight allocation based on the current perception state and environmental context.

[0187] The formula for calculating the total code is as follows:

[0188]

[0189] in, Indicates the total code. Indicates environment encoding. Indicates the parameters for weight calculation. Offset calculation parameters, This represents the encoded feature vector.

[0190] The formula for calculating the hidden state is:

[0191] ;

[0192] in:

[0193] ;

[0194] ;

[0195] ;

[0196] in, Indicates a hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the state at the current moment.

[0197] The formula for calculating the weight of each sensor is as follows:

[0198]

[0199] in, Indicates the weight of the sensor, Indicates the parameters for weight calculation. Indicates the bias calculation parameters. express Activation function.

[0200] The decision-making process of this method combines environmental perception and historical memory capabilities. The introduction of the LSTM structure enables the system to make smooth and coherent decisions based on continuous environmental change trends, effectively avoiding drastic fluctuations in sensor weights. This significantly improves the system's adaptability and robustness in dynamic search and rescue environments, providing core support for achieving adaptive intelligent sensor management.

[0201] Specifically, the performance of the combined policy network is optimized using reinforcement learning algorithms, including: calculating the reward function to evaluate the performance of the combined policy network; and optimizing the performance of the combined policy network using the alternating update strategy and value function in the PPO algorithm to obtain a combined policy network that maximizes the reward function.

[0202] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. The method employs the Proximal Policy Optimization (PPO) algorithm to optimize the combined policy network. Its core is the design of a comprehensive reward function that integrates task completion (AP), system energy consumption (…), and other factors. ) and processing delay ( The PPO algorithm performs joint evaluation with a weight of 0.7:0.2:0.1. By calculating the probability ratio between the old and new strategies and using a pruning function, the algorithm carefully balances exploration and exploitation during the strategy update process, thereby guiding the strategy in a direction that maximizes the cumulative reward.

[0203] The formula for calculating the reward function is as follows:

[0204] ,

[0205] in, Represents the reward function, Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight.

[0206] The alternating update strategy steps in the PPO algorithm are as follows:

[0207] Experience collection: Using the current combined policy network to interact with the system, collect states (composed of encoded feature vectors and environment encodings), actions (sensor weights), rewards, and subsequent state sequences.

[0208] Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated using the generalized advantage estimation (GAE) algorithm to evaluate the relative merits of the actions.

[0209] Policy network update: A fixed-value function network is used to optimize the policy network parameters by maximizing the objective function of PPO-Clip (a policy gradient algorithm) through gradient ascent.

[0210]

[0211] in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. This represents the clipping function.

[0212] Value function network update: With a fixed policy network, the following mean squared error loss is minimized through gradient descent to optimize the network parameters of the value function, making its predictions closer to the actual returns:

[0213]

[0214] in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. This represents the state at time step t (i.e., the current observation information of the environment).

[0215] Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network.

[0216] The value function is a function with one parameter. A neural network whose input is a state The output is a scalar value representing the expected cumulative reward (state value) in that state.

[0217] This method achieves multi-objective joint optimization. Its reward function guides the combined policy network to no longer solely pursue recognition accuracy, but actively seek the optimal balance between accuracy, energy consumption, and speed. The introduction of the PPO algorithm ensures the stability of the optimization process and effectively avoids policy mutations. The final trained combined policy network can intelligently achieve "using the right sensor at the right time," thereby comprehensively improving the overall performance of the system in real deployment environments.

[0218] Specifically, a fused feature vector is obtained by weighting and fusing multiple image features using multiple weights, including weighting and fusing visible light features, infrared features, radar features, and sensor weights.

[0219] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. This method utilizes sensor weights dynamically calculated by a combined policy network to perform a weighted summation of visible light, infrared, and radar features, generating a final fused feature vector. Before fusion, the infrared and radar features are first subjected to layer normalization to eliminate dimensional differences between modes. Then, the features of each mode are multiplied by their corresponding weights and summed to obtain a fused feature vector that dynamically reflects the importance of each sensor in the current environment.

[0220] The formula for calculating the fused feature vector is:

[0221]

[0222] in, Represents the fused feature vector. Presentation layer normalization operation, This represents the visible light weight component in the weighting. This represents the infrared weight component in the weighting. This represents the radar weight component in the weighting.

[0223] This method achieves adaptive fusion at the feature level. Instead of treating all sensor data equally, it allows the combined policy network to intelligently decide "which data to trust more" based on the real-time environment and task requirements. This dynamic weighting mechanism effectively strengthens the features of sensors that perform better under specific conditions, while suppressing noise from unreliable sensors, directly improving the robustness and accuracy of subsequent target detection at the feature level.

[0224] Specifically, the target detection algorithm is used to identify the target result in the target region by fusing the feature vector, including: training the target detection algorithm using a loss function; and extracting multi-scale features from the fusing feature vector using spatial pyramid pooling, where the first output dimension of the multi-scale features is... ; Target detection is performed on multi-scale features using three anchor points to obtain target results in the target region. The second output dimension of the target results is ;in, Indicates the number of categories.

[0225] This application provides a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea. The method inputs an adaptively weighted fused feature vector into a target detection algorithm based on the YOLO architecture for final identification. The target detection algorithm first uses a Spatial Pyramid Pooling (SPPF) module to extract multi-scale features from the fused feature vector to enhance the perception of targets of different sizes. Then, it predicts on the multi-scale features using preset anchor points, with an output dimension of 3×(5+C), corresponding to bounding box coordinates, confidence score, and target class probability, respectively. End-to-end training and optimization are performed using a composite loss function that integrates bounding box regression (LIoU), target determination (Lobj), and classification (Lcls).

[0226] The formula for calculating the loss function used to train the object detection algorithm is as follows:

[0227]

[0228] in, , and These represent different loss weights. This represents the bounding box regression loss. Indicates the target detection loss. This represents the classification loss.

[0229] This method boasts powerful multi-scale target detection capabilities and end-to-end optimization efficiency. Spatial pyramid pooling ensures that the system can capture key features regardless of the distance and size of the person in the water (i.e., the target) in the image; while the loss function directly improves the reliability of the final target output by jointly optimizing the positioning accuracy and classification accuracy, thereby achieving rapid and accurate identification and positioning of maritime targets.

[0230] Figure 2 The multi-sensor adaptive fusion flowchart of a multimodal and reinforcement learning-based method for searching for people who have fallen into the water at sea provided in this application includes: firstly, spatiotemporally aligning the multi-source heterogeneous data collected by multi-source sensors (visible light camera, infrared sensor, and lidar), namely visible light image data, infrared image data, and radar point cloud data; then, fusing the features of the spatiotemporally aligned multi-source heterogeneous data (i.e., sensor aligned data) and feeding them into the combined policy network; finally, adding environmental coding to the combined policy network for target detection and recognition, and updating the combined policy network.

[0231] Figure 3 A schematic diagram of a multimodal and reinforcement learning-integrated search system for people missing at sea, provided in this application, includes:

[0232] The sensor data acquisition and processing unit is used to acquire multiple sensor data in the target area using multiple sensors deployed in the target area, and to perform spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor aligned data in the target area.

[0233] The sensor data feature extraction unit is electrically connected to the sensor data acquisition and processing unit. It is used to extract image features from multiple sensor aligned data, concatenate multiple image features into a feature vector, and use an encoder to convert the feature vector into an encoded feature vector.

[0234] The environmental data acquisition and processing unit is electrically connected to the sensor data feature extraction unit. It is used to acquire multiple environmental data of the target area, convert the multiple environmental data into environmental vectors, and encode the environmental vectors using a fully connected network to obtain environmental codes.

[0235] The sensor weight calculation and combined strategy network optimization unit is electrically connected to the environmental data acquisition and processing unit. It is used to calculate the weight of each sensor through the combined strategy network based on the encoded feature vector and the environmental code. This includes: in the fully connected layer of the combined strategy network, the total code is calculated using the encoded feature vector and the environmental code.

[0236] The formula for calculating the total code is as follows:

[0237] ;

[0238] in, Indicates the total code. Indicates environment encoding. Indicates the parameters for weight calculation. Offset calculation parameters, This represents the encoded feature vector.

[0239] In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the environment encoding, and the total encoding.

[0240] The formula for calculating the hidden state is:

[0241] ;

[0242] in:

[0243] ;

[0244] ;

[0245] ;

[0246] in, Indicates a hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the state at the current moment.

[0247] The weights of each sensor are computed using the hidden states in the output layer of the combined policy network; the performance of the combined policy network is optimized using reinforcement learning algorithms to update the weights, including:

[0248] Calculate the reward function to evaluate the performance of the combined policy network.

[0249] The formula for calculating the reward function is as follows:

[0250] , ;

[0251] in, Represents the reward function, Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight.

[0252] By utilizing the alternating update strategy and value function approach in the PPO algorithm to optimize the performance of the combined policy network, a combined policy network that maximizes the reward function is obtained.

[0253] The alternating update strategy steps in the PPO algorithm are as follows:

[0254] Experience collection: Interact with the system using the current combined policy network to collect states, actions, rewards, and subsequent state sequences.

[0255] Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated through the generalized advantage estimation algorithm to evaluate the relative merits of the actions.

[0256] Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters:

[0257] ;

[0258] in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. This represents the clipping function.

[0259] Value function network update: With a fixed policy network, the following mean squared error loss is minimized through gradient descent to optimize the network parameters of the value function, making its predictions closer to the actual returns:

[0260] ;

[0261] in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. This indicates the state at time step t.

[0262] Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network.

[0263] The target recognition unit is electrically connected to the sensor weight calculation and combination strategy network optimization unit. It is used to use multiple weights to perform weighted fusion of multiple image features to obtain a fused feature vector, and use a target detection algorithm to identify the fused feature vector to obtain the target result in the target region.

[0264] This application provides a multimodal and reinforcement learning-integrated search system for people who have fallen into the water at sea. It consists of five core units forming a collaborative whole: a sensor data acquisition and processing unit collects and spatiotemporally aligns multimodal data; a sensor data feature extraction unit extracts and fuses this multimodal data into a unified encoded feature vector; an environmental data acquisition and processing unit works in parallel, encoding environmental data into semantic vectors (i.e., environmental encoding); a sensor weight calculation and combination strategy network optimization unit integrates the encoded feature vector and environmental encoding, dynamically outputting the optimal sensor weights through reinforcement learning; finally, a target recognition unit performs adaptive fusion and accurate recognition of different image features based on the weights, outputting the target result.

[0265] This system achieves closed-loop optimization of perception, decision-making, and recognition through a modular architecture. Each unit has a clear division of labor and is closely connected, which not only ensures the efficiency and stability of the process from raw sensor data to the final target result processing, but more importantly, it introduces a reinforcement learning-based combinatorial policy network, making the system a self-optimizing organism. As a result, it exhibits excellent comprehensive advantages of adaptability, high precision, and high energy efficiency when facing complex and ever-changing marine environments.

[0266] Figure 4 The flowchart of the target monitoring process of the multi-source sensor adaptive fusion of the multimodal and reinforcement learning combined search method for people who have fallen into the water at sea provided in this application includes: processing multi-source heterogeneous data (i.e., multiple sensor data) obtained by different types of sensors and performing multi-sensor adaptive fusion with the positioning system and the high-precision map of the target area, and then performing target identification and monitoring. During the fusion process, the weights of each sensor are updated using reinforcement learning algorithms.

[0267] Figure 5 The data processing flowchart of the unmanned system ground target monitoring system for the search method for people who have fallen into the water at sea, which is a combination of multimodal and reinforcement learning provided in this application, includes: first, analyzing the task type of the UAV ground monitoring system based on the fusion of multi-source data; then, obtaining the initial weights of each sensor combination; collecting multi-source heterogeneous data (i.e., multiple sensor data) using different types of sensors; updating the weights of each sensor using a combination policy network; then, using a deep learning algorithm to fuse the multi-source heterogeneous data; and finally, performing target detection and recognition.

[0268] Figure 6The flowchart of the unmanned system for ground target monitoring in the search method for people who have fallen into the water at sea, which combines multimodal and reinforcement learning, provided in this application, includes: combining multiple airborne sensors such as visible light modules, lidar modules and infrared modules through a sensor combination strategy, and then putting them into a rotor micro-UAV; obtaining sensor data from multiple sensors through data interaction between the flight control platform and the rotor micro-UAV; and sending the multiple sensor data to a local server to achieve target recognition and monitoring.

[0269] The experimental results comparing this application with other methods are shown in Table 1 below:

[0270] Table 1

[0271]

[0272] This application, along with other methods, has been thoroughly validated on the UDT-3M dataset (a dataset naming convention). MetaSensor represents the meta-sensor (or meta-model), mAP@0.5 represents the average accuracy calculated when the Intersection over Union (IoU) threshold is 0.5, and MOTA represents the multi-target tracking accuracy. As shown in Table 1, the target recognition accuracy of the method in this application reaches 89.2% at mAP@0.5, with a feature similarity of 1.00, and power consumption as low as 18.3W. This represents a 9% reduction compared to traditional EKF (Extended Kalman Filter) fusion and a 41% reduction compared to CMT-Fusion (Context-Aware Multi-Scale Transform). The processing latency reaches 43.5ms, improving real-time performance by 5 times compared to traditional methods. The target loss rate is 4.2%, and the MOTA reaches 82.4%, effectively improving system stability and target recognition accuracy.

Claims

1. A multimodal search method combining reinforcement learning and searching for people who have fallen overboard at sea, characterized in that, include: Multiple sensor data in the target area are collected by multiple sensors deployed in the target area, and spatiotemporal alignment processing is performed on the multiple sensor data to obtain multiple sensor aligned data in the target area. Image features of the alignment data from multiple sensors are extracted respectively, the multiple image features are concatenated into a feature vector, and the feature vector is converted into an encoded feature vector using an encoder; Multiple environmental data points of the target area are collected, the multiple environmental data points are converted into environmental vectors, and the environmental vectors are encoded using a fully connected network to obtain environmental codes; The weight of each sensor is calculated through a combined policy network based on the encoded feature vector and the environment code, including: in the fully connected layer of the combined policy network, the total code is calculated using the encoded feature vector and the environment code; The formula for calculating the total code is as follows: ； in, This indicates the total code. This indicates the environment code. Indicates the parameters for weight calculation. Offset calculation parameters, This represents the encoded feature vector; In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the environment encoding, and the total encoding. The formula for calculating the hidden state is as follows: ； in: ； ； ； in, This indicates the hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the current state. The weights of each sensor are calculated using the hidden states in the output layer of the combined policy network; Optimizing the performance of the combined policy network using a reinforcement learning algorithm to update the weights, wherein optimizing the performance of the combined policy network using a reinforcement learning algorithm includes: Calculate the reward function to evaluate the performance of the combined policy network; The formula for calculating the reward function is as follows: ， ； in, This represents the reward function. Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight; The performance of the combined policy network is optimized by using the alternating update strategy and value function in the PPO algorithm to obtain the combined policy network that maximizes the reward function. The alternating update strategy steps in the PPO algorithm are as follows: Experience collection: Using the current combined policy network to interact with the system, collect states, actions, rewards and subsequent state sequences; Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated by the generalized advantage estimation algorithm to evaluate the relative merits of the actions; Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters: ； in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. Represents the clipping function; Value function network update: With the policy network fixed, optimize the network parameters of the value function by minimizing the following mean squared error loss through gradient descent, so that its predictions are closer to the actual returns: ； in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. Indicates the state at time step t; Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network. Multiple image features are weighted and fused using multiple weights to obtain a fused feature vector. A target detection algorithm is then used to identify the fused feature vector to obtain the target result in the target region.

2. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 1, is characterized in that... The process of collecting multiple sensor data from multiple sensors deployed in the target area, and performing spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor-aligned data for the target area includes: The sensor data includes visible light image data, infrared image data, and radar point cloud data; wherein the visible light image data is three-dimensional data, the infrared image data is two-dimensional data, and the radar point cloud data is four-dimensional data. The data from multiple sensors are synchronized in time and aligned in space to obtain the aligned data of multiple sensors for the target area.

3. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 2, is characterized in that... The step of synchronizing and spatially aligning the data from multiple sensors to obtain aligned sensor data for the target region includes: The visible light image data, the infrared image data, and the radar point cloud data are synchronized in time using the GPS_Sync function to obtain time-aligned visible light image data, infrared image data, and radar point cloud data, ensuring that the visible light image data, infrared image data, and radar point cloud data are collected within a unified time window. Hardware synchronization of multiple sensors ensures a synchronization error of less than 1ms; The intrinsic parameter matrix of the time-aligned visible light image data is obtained using the Zhang Zhengyou calibration method. The time-aligned visible light image data is then processed using the intrinsic parameter matrix to obtain aligned visible light image data. The SIFT feature matching algorithm is used to extract feature points from time-aligned visible light image data and time-aligned infrared image data. The feature points are then registered using affine transformation to obtain aligned infrared image data. Perspective projection is used to project the time-aligned radar point cloud data onto a plane, and the resolution is unified with the time-aligned visible light image data to obtain aligned radar point cloud data.

4. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 3, is characterized in that... The step of extracting image features from multiple sensor alignment data, concatenating the multiple image features into a feature vector, and converting the feature vector into an encoded feature vector using an encoder includes: Visible light features are obtained by extracting features from the aligned visible light image data; infrared features are obtained by extracting features from the aligned infrared image data; and radar features are obtained by extracting features from the aligned radar point cloud data. The visible light feature, the infrared feature, and the radar feature are concatenated into a feature vector; The feature vector is converted into the encoded feature vector using a Transformer encoder.

5. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 1, is characterized in that... The process involves collecting multiple environmental data points from the target area, converting these data points into environmental vectors, and encoding the environmental vectors using a fully connected network to obtain environmental codes. This includes: The environmental data includes rain and fog level data, target speed data, light intensity data, and cloud height data for the target area; The environmental data is converted into the environmental vector; the environmental vector is then encoded using a fully connected network to obtain the environmental code.

6. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 4, is characterized in that... The step of weighting and fusing multiple image features using multiple weights to obtain a fused feature vector includes: The fused feature vector is obtained by weighted fusion of the visible light features, the infrared features, the radar features, and the sensor weights.

7. The method for searching for people who have fallen into the water at sea, combining multimodal and reinforcement learning, as described in claim 1, is characterized in that... The step of using a target detection algorithm to identify the fused feature vector to obtain the target result in the target region includes: The target detection algorithm is trained using a loss function; Spatial pyramid pooling is used to extract multi-scale features from the fused feature vector, and the first output dimension of the multi-scale features is... ; Target detection is performed on the multi-scale features using three anchor points to obtain the target result in the target region. The second output dimension of the target result is... ;in, Indicates the number of categories.

8. A multimodal search system combining reinforcement learning and maritime personnel search, characterized in that, The system is applied to the method according to any one of claims 1 to 7, the system comprising: The sensor data acquisition and processing unit is used to acquire multiple sensor data in the target area using multiple sensors deployed in the target area, and to perform spatiotemporal alignment processing on the multiple sensor data to obtain multiple sensor aligned data in the target area. The sensor data feature extraction unit is electrically connected to the sensor data acquisition and processing unit. It is used to extract image features from multiple aligned data of the sensors, concatenate the multiple image features into a feature vector, and use an encoder to convert the feature vector into an encoded feature vector. An environmental data acquisition and processing unit is electrically connected to the sensor data feature extraction unit. It is used to acquire multiple environmental data of the target area, convert the multiple environmental data into environmental vectors, and encode the environmental vectors using a fully connected network to obtain environmental codes. The sensor weight calculation and combined strategy network optimization unit is electrically connected to the environmental data acquisition and processing unit. It is used to calculate the weight of each sensor through the combined strategy network based on the encoded feature vector and the environmental code. The calculation includes: in the fully connected layer of the combined strategy network, the total code is calculated using the encoded feature vector and the environmental code. The formula for calculating the total code is as follows: ； in, This indicates the total code. This indicates the environment code. Indicates the parameters for weight calculation. Offset calculation parameters, This represents the encoded feature vector; In the decision layer of the combined policy network, the hidden state is calculated using the encoded feature vector, the environment encoding, and the total encoding. The formula for calculating the hidden state is as follows: ； in: ； ； ； in, This indicates the hidden state. Indicates the current state. This represents the third output vector. This indicates element-wise multiplication. This represents the first output vector. This represents the Sigmoid activation function. This represents the second output vector. and These represent the weight matrices corresponding to the forget gate and the input gate, respectively. and This represents the bias vectors corresponding to the forget gate and the input gate. This represents the hyperbolic tangent activation function. This represents the weight matrix corresponding to the current state. This represents the bias vector corresponding to the current state. In the output layer of the combined policy network, the weights of each sensor are calculated using the hidden states; the performance of the combined policy network is optimized using a reinforcement learning algorithm to update the weights, wherein optimizing the performance of the combined policy network using a reinforcement learning algorithm includes: Calculate the reward function to evaluate the performance of the combined policy network; The formula for calculating the reward function is as follows: ， ； in, This represents the reward function. Indicates the task completion rate. Indicates energy consumption. This indicates a processing delay. Represents visible light weight, Indicates infrared weighting, Indicates radar weight; The performance of the combined policy network is optimized by using the alternating update strategy and value function in the PPO algorithm to obtain the combined policy network that maximizes the reward function. The alternating update strategy steps in the PPO algorithm are as follows: Experience collection: Using the current combined policy network to interact with the system, collect states, actions, rewards and subsequent state sequences; Advantage estimation: Using the current value function network and the collected rewards, the advantage value at each time step is calculated by the generalized advantage estimation algorithm to evaluate the relative merits of the actions; Policy network update: Fixed value function network, maximizing the objective function of PPO-Clip through gradient ascent to optimize policy network parameters: ； in, This represents the objective function of the PPO algorithm. Indicates the policy network parameters, This represents the mathematical expectation with respect to time step t. This represents the ratio of the probability of the new policy to the probability of the old policy in a combined policy network. This represents the advantage function calculated using generalized advantage estimation. Represents the clipping function; Value function network update: With the policy network fixed, optimize the network parameters of the value function by minimizing the following mean squared error loss through gradient descent, so that its predictions are closer to the actual returns: ； in, Indicates the mean square error loss. Network parameters representing the value function, Indicates the target return. The target reward can also be calculated using accumulated discount rewards. Indicates by The state value function of control. Indicates the state at time step t; Iterative optimization: Repeatedly perform policy network updates and value function network updates multiple times to complete the alternating updates within one iteration. Then, use the updated policy network to collect new experience. Repeat this iterative process until the reward function R converges to its maximum value, thereby obtaining the optimal combined policy network. The target recognition unit is electrically connected to the sensor weight calculation and combination strategy network optimization unit. It is used to perform weighted fusion of multiple image features using multiple weights to obtain a fused feature vector, and to use a target detection algorithm to identify the fused feature vector to obtain the target result in the target region.

Images