A multi-feature fusion-based falling object confirmation and multi-unmanned platform scheduling method

Abstract

The application discloses a kind of based on multi-feature fusion's falling water target confirmation and multi-unmanned platform scheduling method, it is related to intelligent search and rescue and unmanned system collaborative scheduling technical field, this method is patrolled by unmanned flight platform in water space and collects water surface data, obtains at least one suspected falling water target area marked with rectangular boundary box;Multi-dimensional feature extraction is carried out to suspected falling water target area;The confidence C that the suspected falling water target is real falling water personnel is obtained;If the confidence C is greater than pre-set confidence threshold, then confirm that the suspected falling water target is falling water personnel, carry out unmanned platform operating state parameter acquisition, calculate platform and task optimal allocation scheme, and generate executable scheduling instruction;The application can realize falling water target automatic, accurate confirmation, and can quickly generate multi-unmanned platform optimal scheduling scheme after confirmation, improve rescue efficiency, meet the timeliness and accuracy requirement of water emergency search and rescue.

Description

Technical Field

[0001] This invention belongs to the field of intelligent search and rescue and unmanned system collaborative scheduling technology, specifically involving a method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion. Background Technology

[0002] Search and rescue operations for people who have fallen into the water require extremely high response timeliness. The survival time of people who have fallen into the water in a cold water environment is only tens of minutes. Every minute that the rescue response time is shortened can significantly increase the survival probability of people who have fallen into the water. Therefore, the search and rescue process needs to complete a complete closed loop of finding the target, confirming the target, and dispatching in a very short time. This places stringent requirements on the rapid and accurate confirmation of the target and the efficient dispatch of rescue forces.

[0003] Existing search and rescue technologies for people falling into water mainly include three types of solutions: manual patrol or manned vessel search and rescue, single drone-assisted search and rescue, and rule-based unmanned platform dispatch. Among them, the manual patrol or manned vessel search and rescue solution involves manually patrolling the water area by human visual means, and manually dispatching manned vessels to carry out rescue after a suspected target is found. However, the manual patrol solution relies entirely on human experience, resulting in low search efficiency, long response time, and difficulty in adapting to the needs of search and rescue in large-area waters.

[0004] The single drone-assisted search and rescue solution uses drones equipped with optical or infrared cameras to collect images or video data of the water surface. The data transmitted back to suspected targets in the water is then handed over to ground operators to determine whether it is a real person who has fallen into the water. Subsequent rescue forces are then dispatched through manual command. However, although this solution automates water area patrols, the target confirmation process relies on manual judgment. Moreover, it uses only a single optical or infrared feature as the basis for judgment, which cannot effectively distinguish between people who have fallen into the water and objects that are interfering with the water surface. At the same time, the method of manually reviewing video data frame by frame results in low efficiency in target confirmation and makes it difficult to achieve real-time judgment.

[0005] After manual confirmation that no one has fallen into the water, the rule-based unmanned platform scheduling scheme selects platforms such as drones and unmanned boats to perform rescue missions based on simple fixed rules such as proximity and idle status, either manually or semi-automatically. However, since the scheduling logic does not comprehensively consider the matching degree between the real-time operating status of multiple unmanned platforms and the rescue mission, it is prone to suboptimal scheduling problems such as proximity but insufficient power, and flying platforms being assigned to water surface rescue missions. It is not easy to achieve the global optimal decision for multi-platform collaborative rescue.

[0006] The aforementioned solutions suffer from problems such as time-consuming rescue dispatch decisions, impacting rescue efficiency and failing to meet the timeliness and accuracy requirements of water emergency search and rescue. To address these issues, we propose a multi-feature fusion-based method for identifying targets in the water and scheduling multiple unmanned platforms. This method enables automatic and accurate identification of targets in the water and allows for the rapid generation of optimal scheduling schemes for multiple unmanned platforms after identification, thereby improving rescue efficiency and meeting the timeliness and accuracy requirements of water emergency search and rescue. Summary of the Invention

[0007] The purpose of this invention is to provide a method for identifying targets that have fallen into the water and scheduling multiple unmanned platforms based on multi-feature fusion. This method can automatically and accurately identify targets that have fallen into the water and quickly generate an optimal scheduling scheme for multiple unmanned platforms after identification, thereby improving rescue efficiency and meeting the timeliness and accuracy requirements of emergency search and rescue in water areas, thus solving the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion includes the following steps:

[0010] S1. An unmanned aerial vehicle (UAV) patrols the airspace above the water area and collects water surface data. A deep learning target detection algorithm is used to perform preliminary target detection on the water surface data to obtain at least one suspected water-falling target area marked with a rectangular bounding box. The water surface data includes water surface images and video data.

[0011] S2. Multi-dimensional feature extraction is performed on the suspected drowning target area. The multi-dimensional features include water surface floating morphology features, motion change features, thermal radiation continuity features, and background contrast features.

[0012] S3. Integrate the multi-dimensional features and comprehensively calculate the probability that the suspected drowning target is actually a drowning person, and obtain the confidence level C that the suspected drowning target is actually a drowning person.

[0013] S4. Compare the confidence level C with the preset confidence threshold. If the confidence level C is greater than the preset confidence threshold, then confirm that the suspected target is a person who has fallen into the water, acquire the unmanned platform operation status parameters, and proceed to step S5; otherwise, confirm that the suspected target is not a person who has fallen into the water, abandon the target or mark it as a target to be tracked.

[0014] S5. Real-time acquisition of operational status parameters of all available unmanned aerial platforms, and generation of platform status tables;

[0015] S6. Calculate the optimal allocation scheme between the platform and the task based on the location of the person who fell into the water and the operating status parameters of each unmanned aerial vehicle platform, and generate executable scheduling instructions;

[0016] S7. The dispatch instructions are sent to the corresponding unmanned aerial vehicle (UAV) platforms via wireless communication links, and each UAV platform parses and executes the rescue mission.

[0017] Preferably, in step S1, the water surface data is obtained by acquiring image data and continuous video frame data of the water surface in real time through an optical camera and infrared sensor mounted on an unmanned aerial vehicle platform. After being identified by a deep learning target detection algorithm, each suspected target that has fallen into the water is marked in the image or video frame in the form of a rectangular bounding box, so that the rectangular bounding box can accurately define the pixel area range of the suspected target in the image or video.

[0018] The preferred extraction process for water surface floating morphology features is as follows:

[0019] A1. The OTSU adaptive threshold segmentation algorithm is used to automatically determine the optimal segmentation threshold between the foreground and background in the suspected water-falling target area to obtain the binary mask image of the target.

[0020] A2. Extract the edge contours of the binary mask image to obtain the target outer contour point set, and use the least squares method to fit the target outer contour point set to an ellipse to obtain the length of the major axis and the length of the minor axis of the circumscribed ellipse.

[0021] A3. Calculate the ratio of major to minor axis, circularity, and water surface contact ratio based on the outline and the lengths of the major and minor axes of the circumscribed ellipse.

[0022] A4. Combine the calculated major-minor axis ratio, circularity, and water surface contact ratio to normalize the target area to the actual physical scale, and output the water surface floating morphology feature vector.

[0023] The preferred motion change feature extraction process is as follows:

[0024] B1. The Lucas-Kanade sparse optical flow method is used to track the suspected target area to obtain the pixel coordinate position sequence of the target centroid in each video frame.

[0025] B2. Calculate the ratio of the displacement distance of the target between adjacent video frames to the time, obtain the instantaneous drift velocity at each moment, and calculate the average drift velocity;

[0026] B3. Calculate the motion direction angle and average direction change rate at each moment based on the displacement between adjacent video frames;

[0027] B4. Calculate the coefficient of variation of the velocity sequence, and then combine the average instantaneous drift velocity and the average rate of change of direction to output the motion change feature vector.

[0028] The preferred procedure for extracting the continuity feature of thermal radiation is as follows:

[0029] C1. In each frame of infrared image, extract the thermal radiation intensity values ​​of all pixels in the suspected water-falling target area, and calculate the average thermal radiation intensity of the suspected water-falling target area.

[0030] C2. Collect the time series of average thermal radiation intensity within the observation window and calculate the thermal stability index within the observation window;

[0031] C3. Calculate the continuous temperature difference between the suspected water-falling target area and the surrounding water surface, take the average thermal contrast within the observation window, and output the thermal radiation continuity feature vector.

[0032] The preferred background contrast feature extraction process is as follows:

[0033] D1. Expand outward by 1.5 times the outer rectangle of the suspected drowning target area, and take the annular area between the expanded area and the original suspected drowning target area as the background area.

[0034] D2. Calculate the relative difference in the mean grayscale values ​​between the suspected drowning target area and the background area;

[0035] D3. Extract the frequency values ​​of LBP texture feature histograms for the suspected drowning target area and the background area respectively, and calculate the difference between the two LBP texture feature histograms using the chi-square distance;

[0036] D4. Calculate the relative difference between the suspected drowning target area and the background area in terms of thermal radiation.

[0037] D5. Output the background contrast feature vector based on the relative difference value of the grayscale mean, the difference value of the LBP texture feature histogram, and the relative difference value in the thermal radiation dimension.

[0038] The preferred multi-dimensional feature fusion process is as follows:

[0039] E1. Normalize the floating morphology features, motion change features, thermal radiation continuity features, and background contrast features on the water surface to obtain multiple sub-vectors corresponding to the features.

[0040] E2. Concatenate the normalized sub-vectors into a unified feature vector;

[0041] E3. Assign weights to each feature and calculate the comprehensive score through weighted fusion.

[0042] E4. Using the Sigmoid function, the comprehensive score value is mapped to the (0,1) interval to obtain the confidence level C of the suspected drowning target being a real drowning person.

[0043] Preferably, in step S4, when the confidence threshold is preset, a feature curve analysis is performed using a validation dataset containing labeled drowning personnel and non-drowning target samples to obtain the true positive rate and false positive rate under different thresholds. Then, based on the true positive rate and false positive rate, the threshold point that maximizes the F1 score is selected as the default threshold, and the default threshold is set to be a confidence threshold that can be adaptively adjusted according to the environment.

[0044] Preferably, the operating status parameters of the unmanned aerial platform include platform number, current location, remaining energy, maximum energy, current mission status, platform type, and cruise rate.

[0045] Preferably, in step S6, the executable scheduling instruction generation process is as follows:

[0046] F1. Select unmanned aerial platforms from the platform status table that meet the requirements of being idle in the current mission status and having remaining energy that meets the minimum reserve constraint, and form a candidate platform set.

[0047] F2. For each unmanned aerial vehicle in the candidate platform set, estimate the estimated arrival time by combining its distance from the water impact point and its cruising speed.

[0048] F3. Define a comprehensive cost function with arrival speed, remaining energy, and mission matching degree as factors, and calculate the comprehensive cost value;

[0049] F4. Construct a cost matrix from the costs of each rescue sub-task of each unmanned aerial vehicle platform, and use the Hungarian algorithm to solve the cost matrix to obtain the optimal platform and task allocation scheme that minimizes the total cost.

[0050] F5. Generate structured scheduling instructions for each selected unmanned aerial vehicle platform according to the optimal allocation scheme of platform and mission. The structured scheduling instructions include instruction number, target platform number, mission type, target location, waypoint sequence, priority, estimated arrival time and timestamp.

[0051] The present invention proposes a method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion, which has the following advantages compared with the prior art:

[0052] 1. This invention integrates and analyzes the floating morphology features, motion change features, thermal radiation continuity features, and background contrast features on the water surface. By utilizing the complementarity between these features, it effectively distinguishes people who have fallen into the water from floating objects, waves, reflections, and other interfering targets, thereby reducing the false alarm rate on the water surface. Furthermore, the algorithm automatically completes feature extraction, fusion, and confidence C calculation, enabling real-time and automatic confirmation of suspected falling targets without human intervention, thus improving the efficiency of target confirmation.

[0053] 2. This invention, through a confidence threshold determination mechanism, can ensure a high recognition rate of real people who have fallen into the water, while effectively filtering out non-falling targets. At the same time, the confidence threshold can be adaptively adjusted according to the actual environment, taking into account the confirmation needs in different scenarios, making the target confirmation results more in line with the complex actual scenarios of water search and rescue, and further improving the confirmation accuracy.

[0054] 3. After the confidence level C exceeds the confidence level threshold and the target falls into the water is automatically confirmed, the present invention directly triggers the unmanned aerial platform to acquire the status and perform scheduling calculations. No manual transmission of results or manual command and scheduling is required. The entire process of discovery, confirmation and scheduling is upgraded from a manual serial operation to an algorithm-driven automated closed loop, which reduces the rescue response time from minutes to seconds and improves the rescue success rate.

[0055] 4. This invention automates the entire process of suspected target detection, feature extraction, fusion confirmation, platform scheduling, and command issuance through algorithms, eliminating the need for intensive human intervention. It can adapt to the patrol and search and rescue needs of large water areas and supports the collaborative scheduling of multiple unmanned platforms. It can handle diverse water search and rescue scenarios such as multiple targets in the water and complex sea conditions, meeting the timeliness and accuracy requirements of water emergency search and rescue. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the process of the present invention;

[0057] Figure 2 This is a schematic diagram of the multi-dimensional feature fusion process of the present invention;

[0058] Figure 3 This is a schematic diagram of the scheduling process for multiple unmanned aerial platforms according to the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] This invention provides, for example Figure 1-3 The method for identifying targets that have fallen into the water and scheduling multiple unmanned platforms based on multi-feature fusion, as shown, includes the following steps:

[0061] S1. An unmanned aerial vehicle (UAV) patrols the airspace above the water area and collects water surface data. A deep learning target detection algorithm is used to perform preliminary target detection on the water surface data to obtain at least one suspected water-falling target area marked with a rectangular bounding box. The water surface data includes water surface images and video data.

[0062] Specifically, the water surface data is collected by using an unmanned aerial vehicle (UAV) equipped with an optical camera and infrared sensor. The UAV patrols the airspace above the water area to be searched and rescued along a preset route, collecting image data and continuous video frame data of the water surface in real time. After being identified by a deep learning target detection algorithm, each suspected target that has fallen into the water is marked with a rectangular bounding box in the image or video frame. This allows the rectangular bounding box to accurately define the pixel area range of the suspected target in the image or video, thus defining a precise analysis range for subsequent feature extraction.

[0063] S2. Multi-dimensional feature extraction is performed on the suspected drowning target area. The multi-dimensional features include water surface floating morphology features, motion change features, thermal radiation continuity features, and background contrast features.

[0064] Among them, the water surface floating morphology feature is used to determine whether the target outline conforms to the geometric features of a human body floating on water from a spatial geometric dimension, quickly eliminating floating objects with obvious morphological differences. The extraction process of this water surface floating morphology feature is as follows:

[0065] A1. The OTSU adaptive threshold segmentation algorithm is used to automatically determine the optimal segmentation threshold between the foreground and background in the suspected drowning target area to obtain the binary mask image of the target, where the target pixel value is 1 and the background pixel value is 0.

[0066] A2. Extract the edge contours of the binary mask image to obtain the target outer contour point set, and use the least squares method to fit the target outer contour point set to an ellipse to obtain the length of the major axis and the length of the minor axis of the circumscribed ellipse.

[0067] A3. Calculate the major-minor axis ratio, roundness, and water contact ratio based on the major and minor axis lengths of the outline and the circumscribed ellipse. The major-minor axis ratio measures the elongation of the overall outline of the target. Since a human body floating on the water in a side-lying or supine position exhibits a distinctly elliptical shape, while spherical floating objects are approximately of equal length and width, the formula for calculating the major-minor axis ratio is:

[0068] ,

[0069] In the formula, The ratio of the major axis to the minor axis of the circumscribed ellipse (typical range for human buoyancy). ; spherical floating objects (Close to 1.0) To fit the length of the major axis of the ellipse, Let be the length of the minor axis of the fitted ellipse;

[0070] Circularity is used to measure the degree of deviation of a target outline from a standard circle. A perfect circle has the largest area-to-circumference ratio, while the outline of a floating human body is irregular, such as with outstretched limbs or fluttering clothing, resulting in a smaller area relative to its circumference. The formula for calculating circularity is:

[0071] ,

[0072] In the formula, The circularity value (for a perfect circle) =1, the more irregular the shape, the smaller the value; the typical value for human buoyancy is 1. <0.5), A is the area of ​​the suspected drowning target region, and P is the perimeter of the suspected drowning target region;

[0073] The water-to-surface contact ratio measures the degree to which a target is submerged in water. Since most of a person's body is submerged, a significant portion of their outline is in contact with the water surface. Objects that float completely on the surface, such as plastic bottles, have a smaller proportion of water contact. The formula for calculating the water-to-surface contact ratio is:

[0074] ,

[0075] In the formula, Water-to-surface contact ratio (typical value for people who fall into the water) >0.3, objects floating on the water surface (smaller) denoted as the edge length of the target outline in contact with the water surface, specifically the arc length of the part where the bottom of the outline meets the water surface, and P is the perimeter of the suspected target area.

[0076] A4. Combine the calculated major-minor axis ratio, circularity, and water surface contact ratio to normalize the target area to the actual physical scale, and output the water surface floating morphology feature vector.

[0077] The formula for normalization is:

[0078] ,

[0079] In the formula, This is an approximate actual area of ​​the target's projection onto the ground, used to determine whether the target's scale corresponds to the size of a human body. The area represents the suspected crash site, and H represents the current flight schedule of the unmanned aerial vehicle (UAV), obtained from the onboard barometer or GPS altitude. The focal length of the camera;

[0080] The output form of the water surface floating morphology feature vector is: ;

[0081] The process for extracting motion change features is as follows:

[0082] B1. The Lucas-Kanade sparse optical flow method is used to track the suspected water-falling target area to obtain the pixel coordinate position sequence of the target centroid in each video frame. The expression for the pixel coordinate position sequence is:

[0083] In the formula, Let t be the pixel coordinates of the target centroid at time t, where T is the number of observation frames;

[0084] B2. Calculate the ratio of the displacement distance of the target between adjacent video frames to the time, obtain the instantaneous drift velocity at each moment, and calculate the average drift velocity;

[0085] The formula for calculating instantaneous drift velocity is:

[0086] ,

[0087] In the formula, Let be the instantaneous drift velocity at time t. Let be the pixel coordinates of the target centroid at time t. The pixel coordinates of the target centroid in the previous frame. The time interval between adjacent video frames;

[0088] The formula for calculating the average drift velocity is:

[0089] ,

[0090] In the formula, The average drift velocity is T, and the number of observed video frames is T. Let be the instantaneous drift velocity at time t;

[0091] B3. Calculate the motion direction angle and average direction change rate at each moment based on the displacement between adjacent video frames;

[0092] The formula for calculating the direction angle of motion is:

[0093] ,

[0094] In the formula, Let be the angle between the direction of motion and the horizontal direction at time t. , These are the pixel coordinates of the target centroid at time t. and They are respectively The target centroid pixel coordinates at any given time;

[0095] The formula for calculating the average directional rate of change is:

[0096] ,

[0097] ,

[0098] In the formula, The rate of change of the direction angle between adjacent time points. The average rate of change in direction. Let be the angle between the direction of motion and the horizontal direction at time t. for The angle between the direction of motion and the horizontal direction at any given moment.

[0099] B4. Calculate the coefficient of variation of the velocity sequence, and then combine the average instantaneous drift velocity and the average rate of change of direction to output the motion change feature vector;

[0100] The formula for calculating the coefficient of variation of a velocity sequence is:

[0101] ,

[0102] In the formula, The coefficient of variation of the velocity sequence ( The higher the value, the more irregular the movement. The movement of a person struggling in the water is usually highly irregular. A typical value is... >0.5, while floating objects that drift uniformly with the water flow have smaller velocity changes. (approaching 0) Velocity sequence within the observation window standard deviation Velocity sequence within the observation window The mean;

[0103] The output motion change feature vector is represented as follows: .

[0104] The process for extracting the continuity feature of thermal radiation is as follows:

[0105] C1. In each frame of the infrared image, extract the thermal radiation intensity values ​​of all pixels within the suspected water-falling target area, and calculate the average thermal radiation intensity of the suspected water-falling target area; the formula for calculating the average thermal radiation intensity is:

[0106] ,

[0107] In the formula, Let be the average thermal radiation intensity of the target region at time t, and N be the number of pixels contained in the target region. Infrared image coordinates ( The thermal radiation intensity value at time t;

[0108] C2. Collect the time series of average thermal radiation intensity within the observation window and calculate the thermal stability index within the observation window;

[0109] Let the observation window be [tW,t], then the time series of the collected average thermal radiation intensity is: The formula for calculating the thermal stability index is:

[0110] ,

[0111] In the formula, Stable value of thermal radiation intensity ( The closer the value is to 1, the more stable the target's thermal radiation. The human body, due to its own heat production, maintains relatively stable thermal radiation; a typical value is... >0.8, while non-living materials such as wooden boards and plastic gaps will fluctuate with ambient temperature. (The value is relatively low), W is the length of the observation time window. and These represent the maximum and minimum values ​​of the average thermal radiation intensity within the time window, respectively. This represents the average thermal radiation intensity within the time window.

[0112] C3. Calculate the continuous temperature difference between the suspected water-falling target area and the surrounding water surface, take the average thermal contrast within the observation window, and output the thermal radiation continuity feature vector.

[0113] The formula for calculating average thermal contrast ratio is:

[0114] ,

[0115] ,

[0116] In the formula, Let be the average thermal radiation intensity of the target area at time t. Let be the average thermal radiation intensity of the water surface area surrounding the target at time t. This indicates a continuous temperature difference between the target area and the surrounding water surface, where K is the number of sampling frames within the window. To measure the average thermal contrast within the observation window, the body temperature of the person who fell into the water was higher than the water temperature. >0;

[0117] The expression for the output thermal radiation continuity eigenvector is: .

[0118] The process of background contrast feature extraction is as follows:

[0119] D1. Expand outward by 1.5 times the outer rectangle of the suspected drowning target area, and take the annular area between the expanded area and the original suspected drowning target area as the background area.

[0120] D2. Calculate the relative difference in the mean grayscale values ​​between the suspected drowning target area and the background area;

[0121] The formula for calculating the relative difference of grayscale mean is:

[0122] ,

[0123] In the formula, This represents the relative difference in the mean grayscale values ​​between the target area and the background area. The higher the value, the more pronounced the difference in brightness between the target and the background. The grayscale average of all pixels in the target region. This represents the average grayscale value of all pixels in the background area.

[0124] D3. Extract the frequency values ​​of LBP (Local Binary Pattern) texture feature histograms from the suspected water-falling target area and the background area respectively, and calculate the difference between the two LBP texture feature histograms using the chi-square distance. The formula for calculating the difference between the two LBP texture feature histograms is as follows:

[0125] ,

[0126] In the formula, This represents the chi-square distance difference between the bin frequency values ​​of the histograms of the target region and the background region. The larger the value, the greater the difference in texture between the target and the background; the difference between the texture of the human body surface and the texture of water ripples is obvious. The frequency value of the i-th bin in the histogram of the target region. Let B be the frequency value of the i-th bin in the background region histogram, and let B be the number of bins in the LBP histogram.

[0127] D4. Calculate the relative difference between the suspected drowning target area and the background area in terms of thermal radiation.

[0128] The formula for calculating the relative difference in the thermal radiation dimension is:

[0129] ,

[0130] In the formula, This represents the relative difference between the target area and the background area in the dimension of thermal radiation. The average thermal radiation intensity of the target area. The average thermal radiation intensity of the background region;

[0131] The background contrast feature vector output expression is as follows: .

[0132] D5. Output the background contrast feature vector based on the relative difference value of the grayscale mean, the difference value of the LBP texture feature histogram, and the relative difference value in the thermal radiation dimension.

[0133] S3. Integrate the multi-dimensional features and comprehensively calculate the probability that the suspected drowning target is actually a drowning person, and obtain the confidence level C that the suspected drowning target is actually a drowning person.

[0134] Specifically, such as Figure 2 As shown, the process of multi-dimensional feature fusion is as follows:

[0135] E1. Normalize the floating morphology features, motion change features, thermal radiation continuity features, and background contrast features on the water surface to obtain multiple sub-vectors corresponding to the features.

[0136] The normalization formula is:

[0137] ,

[0138] In the formula, Let be the normalized eigenvalue of the i-th feature. Let be the original value of the i-th feature. and These are the minimum and maximum values ​​of the i-th feature in the training labeled dataset, respectively;

[0139] E2. Concatenate the normalized sub-vectors into a unified feature vector;

[0140] The unified feature vector expression is: ,

[0141] in, The dimension is 4;

[0142] The dimension is 3;

[0143] The dimension is 2;

[0144] The dimension is 3;

[0145] Therefore, the total dimension of the feature vector is d = 4 + 3 + 2 + 3 = 12.

[0146] E3. Assign weights to each feature and calculate the comprehensive score through weighted fusion.

[0147] The weighted fusion calculation formula is as follows:

[0148] ,

[0149] In the formula, The overall score is given by d, where d is the dimension of the total feature vector. Let b be the fusion weight of the i-th feature, reflecting the contribution of this feature to the identification of drowning victims, and b be the bias term. b is obtained by training on a labeled dataset, or it can be labeled by a domain expert based on experience. Let be the eigenvalue of the i-th feature after normalization;

[0150] E4. Using the Sigmoid function, the comprehensive score value is mapped to the (0,1) interval to obtain the confidence level C of the suspected drowning target being a real drowning person;

[0151] The formula for calculating the confidence level C is:

[0152] ,

[0153] In the formula, e is the natural constant, and C is the target confidence value, which ranges from (0, 1). The closer C is to 1, the higher the probability that the target is a real person who fell into the water. The closer C is to 0, the higher the probability that the target is not a person who fell into the water.

[0154] S4. Compare the confidence level C with the preset confidence threshold. If the confidence level C is greater than the preset confidence threshold, then confirm that the suspected target is a person who has fallen into the water, acquire the unmanned platform operation status parameters, and proceed to step S5; otherwise, confirm that the suspected target is not a person who has fallen into the water, abandon the target or mark it as a target to be tracked.

[0155] When setting the confidence threshold, feature curve analysis is performed using a validation dataset containing labeled individuals who have fallen into the water and non-submerged target samples to obtain the true positive rate and false positive rate under different thresholds. Then, based on the true positive rate and false positive rate, the threshold point that maximizes the F1 score is selected as the default threshold. The default threshold is set to be an adaptive confidence threshold that can be adjusted according to the environment. For example, the default threshold is 0.75. In practical applications, it can be adaptively adjusted within the range of [0.6, 0.9] according to environmental conditions. When visibility is low (such as at night or in foggy weather) or sea state is high, the confidence threshold is appropriately reduced, such as to 0.65, to improve recall and reduce false negatives. When environmental conditions are good, the confidence threshold is appropriately increased, such as to 0.85, to improve accuracy and reduce false alarms.

[0156] S5. Real-time acquisition of operational status parameters of all available unmanned aerial platforms, and generation of platform status tables;

[0157] The operational status parameters of the unmanned aerial platform include platform number, current location, remaining energy, maximum energy, current mission status, platform type, and cruise speed. Unmanned aerial platforms include unmanned surface platforms and unmanned aerial platforms, such as unmanned search and rescue vessels and drones.

[0158] S6. Calculate the optimal allocation scheme between the platform and the task based on the location of the person who fell into the water and the operating status parameters of each unmanned aerial vehicle platform, and generate executable scheduling instructions;

[0159] like Figure 3 As shown, the executable scheduling instruction generation process is as follows:

[0160] F1. Select unmanned aerial platforms from the platform status table that meet the requirements of being idle in the current mission status and having remaining energy that meets the minimum reserve constraint, and form a candidate platform set.

[0161] F2. For each unmanned aerial vehicle (UAV) in the candidate platform set, estimate the estimated arrival time by combining its distance from the landing point and its cruising speed. The formula for calculating the estimated arrival time is:

[0162] ,

[0163] In the formula, For the estimated arrival time, The cruising speed of platform j; This is the current location of platform j. to the location of the person who fell into the water Geographical distance, using Formula for calculating spherical distance; The formula is as follows:

[0164] ,

[0165] In the formula, =6371000 is the average radius of the Earth. and These are the latitudes and longitudes of the platform location and the target location, respectively. = - Due to latitude difference, Longitude difference;

[0166] F3. Define a comprehensive cost function with arrival speed, remaining energy, and mission matching degree as factors, and calculate the comprehensive cost value;

[0167] The formula for the comprehensive cost function is as follows:

[0168] ,

[0169] In the formula, For comprehensive value ( The smaller the value, the more suitable the platform is for performing the current rescue mission. The arrival time weighting coefficient reflects the importance of response speed. This is a weighting factor for energy consumption, reflecting the importance of range safety. Assigning weight coefficients to tasks reflects the importance of platform capability adaptation. + + =1, typically taking the value of =0.5, =0.3, =0.2, For the estimated arrival time, The maximum arrival time in the candidate set. This represents the remaining energy percentage of platform j; the higher the value, the lower the cost. For task matching, when the platform type matches the task requirements... =1, if the close-range water rescue mission is matched with an unmanned surface platform, otherwise =0.

[0170] F4. Construct a cost matrix from the costs of each rescue sub-task of each unmanned aerial vehicle platform, and use the Hungarian algorithm to solve the cost matrix to obtain the optimal platform and task allocation scheme that minimizes the total cost.

[0171] The formula for the Hungarian algorithm is:

[0172] ,

[0173] Constraints: ,

[0174] In the formula, For the optimal allocation scheme, To assign the j-th platform to the i-th subtask, The comprehensive cost of allocating the i-th subtask to the j-th platform, X is the allocation scheme of all subtasks and platforms, n is the number of subtasks, and m is the number of candidate platforms;

[0175] F5. Generate structured scheduling instructions for each selected unmanned aerial vehicle platform according to the optimal allocation scheme of platform and mission. The structured scheduling instructions include instruction number, target platform number, mission type, target location, waypoint sequence, priority, estimated arrival time and timestamp.

[0176] S7. The dispatch instructions are sent to the corresponding unmanned aerial platforms via wireless communication links, and each unmanned aerial platform parses and executes the rescue mission;

[0177] This process automatically completes the entire process, including suspected target detection, feature extraction, fusion confirmation, platform scheduling, and command issuance, without the need for intensive human intervention. It can adapt to the patrol and search and rescue needs of large water areas, and supports the collaborative scheduling of multiple unmanned platforms. It can cope with diverse water search and rescue scenarios such as multiple targets in the water and complex sea conditions, meeting the timeliness and accuracy requirements of water emergency search and rescue.

[0178] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion, characterized in that: Includes the following steps: S1. An unmanned aerial vehicle (UAV) patrols the airspace above the water area and collects water surface data. A deep learning target detection algorithm is used to perform preliminary target detection on the water surface data to obtain at least one suspected water-falling target area marked with a rectangular bounding box. The water surface data includes water surface images and video data. S2. Multi-dimensional feature extraction is performed on the suspected drowning target area. The multi-dimensional features include water surface floating morphology features, motion change features, thermal radiation continuity features, and background contrast features. S3. Integrate the multi-dimensional features and comprehensively calculate the probability that the suspected drowning target is actually a drowning person, and obtain the confidence level C that the suspected drowning target is actually a drowning person. S4. Compare the confidence level C with the preset confidence threshold. If the confidence level C is greater than the preset confidence threshold, then confirm that the suspected target is a person who has fallen into the water, acquire the unmanned platform operation status parameters, and proceed to step S5; otherwise, confirm that the suspected target is not a person who has fallen into the water, abandon the target or mark it as a target to be tracked. S5. Real-time acquisition of operational status parameters of all available unmanned aerial platforms, and generation of platform status tables; S6. Calculate the optimal allocation scheme between the platform and the task based on the location of the person who fell into the water and the operating status parameters of each unmanned aerial vehicle platform, and generate executable scheduling instructions; S7. The dispatch instructions are sent to the corresponding unmanned aerial vehicle (UAV) platforms via wireless communication links, and each UAV platform parses and executes the rescue mission.

2. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: In step S1, the water surface data is collected in real time by an optical camera and an infrared sensor mounted on an unmanned aerial vehicle platform, which collects image data and continuous video frame data of the water surface. After being identified by a deep learning target detection algorithm, each suspected target that has fallen into the water is marked in the image or video frame in the form of a rectangular bounding box, so that the rectangular bounding box can accurately define the pixel area range of the suspected target in the image or video.

3. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: The process for extracting the floating morphological features on the water surface is as follows: A1. The OTSU adaptive threshold segmentation algorithm is used to automatically determine the optimal segmentation threshold between the foreground and background in the suspected water-falling target area to obtain the binary mask image of the target. A2. Extract the edge contours of the binary mask image to obtain the target outer contour point set, and use the least squares method to fit the target outer contour point set to an ellipse to obtain the length of the major axis and the length of the minor axis of the circumscribed ellipse. A3. Calculate the ratio of major to minor axis, circularity, and water surface contact ratio based on the outline and the lengths of the major and minor axes of the circumscribed ellipse, respectively. A4. Combine the calculated aspect ratio, circularity, and water surface contact ratio to normalize the target area to the actual physical scale, and output the water surface floating morphology feature vector.

4. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 3, characterized in that: The process for extracting motion change features is as follows: B1. The Lucas-Kanade sparse optical flow method is used to track the suspected water-falling target area to obtain the pixel coordinate position sequence of the target centroid in each video frame. B2. Calculate the ratio of the displacement distance of the target between adjacent video frames to the time, obtain the instantaneous drift velocity at each moment, and calculate the average drift velocity; B3. Calculate the motion direction angle and average direction change rate at each moment based on the displacement between adjacent video frames; B4. Calculate the coefficient of variation of the velocity sequence, and then combine the average instantaneous drift velocity and the average rate of change of direction to output the motion change feature vector.

5. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 4, characterized in that: The process for extracting the continuity feature of thermal radiation is as follows: C1. In each frame of infrared image, extract the thermal radiation intensity values ​​of all pixels in the suspected water-falling target area, and calculate the average thermal radiation intensity of the suspected water-falling target area. C2. Collect the time series of average thermal radiation intensity within the observation window and calculate the thermal stability index within the observation window; C3. Calculate the continuous temperature difference between the suspected water-falling target area and the surrounding water surface, take the average thermal contrast within the observation window, and output the thermal radiation continuity feature vector.

6. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 5, characterized in that: The background contrast feature extraction process is as follows: D1. Expand outward by 1.5 times the outer rectangle of the suspected drowning target area, and take the annular area between the expanded area and the original suspected drowning target area as the background area. D2. Calculate the relative difference in the mean grayscale values ​​between the suspected drowning target area and the background area; D3. Extract the frequency values ​​of LBP texture feature histograms for the suspected drowning target area and the background area respectively, and calculate the difference between the two LBP texture feature histograms using the chi-square distance; D4. Calculate the relative difference between the suspected drowning target area and the background area in terms of thermal radiation. D5. Output the background contrast feature vector based on the relative difference value of the grayscale mean, the difference value of the LBP texture feature histogram, and the relative difference value in the thermal radiation dimension.

7. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: The process of multi-dimensional feature fusion is as follows: E1. Normalize the floating morphology features, motion change features, thermal radiation continuity features, and background contrast features on the water surface to obtain multiple sub-vectors corresponding to the features. E2. Concatenate the normalized sub-vectors into a unified feature vector; E3. Assign weights to each feature and calculate the comprehensive score through weighted fusion. E4. Using the Sigmoid function, the comprehensive score value is mapped to the (0,1) interval to obtain the confidence level C of the suspected drowning target being a real drowning person.

8. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: In step S4, when the confidence threshold is preset, a feature curve analysis is performed using a validation dataset containing labeled drowning personnel and non-drowning target samples to obtain the true positive rate and false positive rate under different thresholds. Then, based on the true positive rate and false positive rate, the threshold point that maximizes the F1 score is selected as the default threshold. The default threshold is set to be a confidence threshold that can be adaptively adjusted according to the environment.

9. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: The operational status parameters of the unmanned aerial platform include platform number, current location, remaining energy, maximum energy, current mission status, platform type, and cruise rate.

10. The method for identifying water-falling targets and scheduling multiple unmanned platforms based on multi-feature fusion according to claim 1, characterized in that: In step S6, the executable scheduling instruction generation process is as follows: F1. Select unmanned aerial platforms from the platform status table that meet the requirements of being idle in the current mission status and having remaining energy that meets the minimum reserve constraint, and form a candidate platform set. F2. For each unmanned aerial vehicle in the candidate platform set, estimate the estimated arrival time by combining its distance from the landing point and its cruising speed. F3. Define a comprehensive cost function with arrival speed, remaining energy, and mission matching degree as factors, and calculate the comprehensive cost value; F4. Construct a cost matrix from the costs of each rescue sub-task of each unmanned aerial vehicle platform, and use the Hungarian algorithm to solve the cost matrix to obtain the optimal allocation scheme of platform and task that minimizes the total cost. F5. Generate structured scheduling instructions for each selected unmanned aerial vehicle platform according to the optimal allocation scheme of platform and mission. The structured scheduling instructions include instruction number, target platform number, mission type, target location, waypoint sequence, priority, estimated arrival time and timestamp.

Images