Unmanned aerial vehicle black box control method and system based on deep learning

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle black box control method and system based on deep learning, and the method comprises the steps: obtaining a real-time flight region image of an unmanned aerial vehicle, and extracting a region image before the unmanned aerial vehicle falls according to the real-time flight region image; and based on a deep learning model, carrying out feature extraction and environment type determination on the regional image. Meanwhile, interaction data of the search and rescue end and the black box and positioning data of the search and rescue end are collected in real time. Afterwards, the environment types, the interaction data and the positioning data are input into a reinforcement learning model, state benefits of different prompt actions are calculated and selected through a strategy network, a black box control strategy including light, sound and smoke control is generated, and self-adaptive scheduling and energy consumption optimization of the prompt actions are achieved. The reachability and effectiveness of black box prompt signals are improved in a complex environment, and the search and rescue efficiency in a falling scene is improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more particularly to a UAV black box control method and system based on deep learning. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are widely used in aerial surveying, inspection, security, and logistics, and their operating environments are characterized by increasing complexity, long distances, and high risks. During long-term flights or high-load missions, UAVs may crash due to battery failure, environmental interference, attitude instability, or component damage. To locate crash sites and conduct timely search and rescue operations, existing UAVs are generally equipped with black boxes to record flight data and provide basic communication feedback. However, existing black boxes mainly rely on fixed-frequency wireless signal transmission or simple sound and light cues, resulting in a simplistic operating method that is difficult to actively adjust to environmental changes, thus limiting the effective search and rescue range and intensity.

[0003] In complex environments, traditional black boxes have significant shortcomings. On the one hand, drone crash sites may be located in forests, grasslands, water bodies, or gaps in buildings, where light obstruction, terrain refraction, and communication interference can make it difficult for search and rescue teams to quickly locate the target using conventional signals. On the other hand, black box alert methods lack intelligent adjustment capabilities, failing to execute differentiated commands based on changes in distance between the search and rescue team and the black box, signal strength, and environmental risk levels. Unrestricted use of sound and light alerts also easily wastes and depletes electrical resources, significantly reducing search and rescue efficiency. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a deep learning-based method and system for controlling the black box of a drone.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for controlling the black box of a drone includes the following steps: S1. Obtain real-time flight area images of the drone, and extract the area image of the drone before it crashes based on the real-time flight area images; S2. Determine the environmental type of the crash area based on the area image before the drone crashes; S3. Real-time acquisition of interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal; S4. Based on the reinforcement learning model, a black box control strategy is generated according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

[0006] Furthermore, the flight area images are acquired by installing a camera in the black box.

[0007] Furthermore, the step of extracting the area image of the UAV before it crashed based on the real-time flight area image includes the following steps: Temporal difference processing is performed on consecutive frames based on real-time flight area images, and a flight state change sequence is constructed based on the pixel change amplitude of adjacent frames; Based on the flight state change sequence, the abnormal moments of flight attitude are identified by the time-series feature detection model, and several frames of images before the abnormal moments are constructed as a candidate pre-crash image sequence. Feature aggregation processing is performed on the candidate pre-crash image sequence, and image frames are selected based on preset image sharpness evaluation index and scene stability index to obtain the regional image before the drone crashes.

[0008] Furthermore, the temporal feature detection model includes a bidirectional gated loop unit.

[0009] Further, S2 includes the following steps: Preprocessing operations are performed on the regional images of the drone before it crashes, including brightness equalization, noise suppression, and edge enhancement, to obtain a preprocessed image. The preprocessed image is input into the convolutional neural network model, and multi-scale convolution operations and feature extraction are performed on the image to obtain the environmental feature vector; Based on the environmental feature vector input to the environmental type determination model, the probability calculation of forest, grassland, water, built-up area and other environmental categories is performed through the classification layer to obtain the probability distribution of environmental type; The environment type corresponding to the maximum probability value is selected as the environment type of the drone crash area based on the probability distribution of environment types.

[0010] Furthermore, the interaction data between the search and rescue terminal and the black box includes: communication signal strength data, disconnection rate, recent connection time and current time difference.

[0011] Furthermore, the search and rescue terminal positioning data includes: real-time geographic coordinate data and displacement data of the search and rescue terminal, wherein the displacement data includes velocity parameters and motion direction parameters.

[0012] Further, S4 includes the following steps: A reinforcement learning state vector is constructed based on the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the location data of the search and rescue terminal. The state vector is input into the reinforcement learning policy network, and feature encoding and policy evaluation calculation are performed on the state vector. The immediate benefits of different control actions are evaluated based on the preset reward function to obtain the corresponding policy score. The optimal control action is selected based on the strategy score, and the control action is mapped to a light control command, a sound control command, or a smoke control command. The feedback data after the control action is executed is fed back based on the reward function, the policy network is updated, and a black box control policy is generated based on the updated policy network.

[0013] Furthermore, the formula for the reward function is as follows: ; in, This is the reward value; The increase in communication signal strength after the control action is executed, relative to before the control action; This represents the amount by which the distance between the search and rescue location and the black box location is shortened; The energy consumption value corresponding to the actions of light, sound and smoke control; This is the environmental risk assessment value; , , and These are positive weighting coefficients.

[0014] A deep learning-based drone black box control system, applied to any of the aforementioned deep learning-based drone black box control methods, includes: The image processing module is used to acquire real-time flight area images of the drone and extract images of the area before the drone crashes based on the real-time flight area images. The environment recognition module is used to determine the environmental type of the crash area based on the area image before the drone crashes; The search and rescue data acquisition module is used to acquire real-time interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal. The black box control module is used to generate a black box control strategy based on a reinforcement learning model, according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

[0015] The beneficial effects of this invention are as follows: By acquiring real-time flight area images of the UAV and performing differential analysis on consecutive frames to extract the area image before the crash, the black box can capture discriminative environmental visual information before the crash occurs. Furthermore, a deep learning model is used to perform feature extraction and environment type determination on the area image before the crash, enabling classification and recognition of various crash scenarios such as forests, grasslands, water areas, and built-up areas. By real-time collection of interaction data between the search and rescue terminal and the black box, as well as the search and rescue terminal's positioning data, a dynamic information input reflecting communication status, spatial relationship, and movement trend is constructed, enabling the black box to perform state modeling of the search and rescue process. The crash area environment type, interaction data, and positioning data are then input into a reinforcement learning model. A policy network calculates and decides on the state benefits of various prompting actions, generating a black box control strategy including light control, sound control, and smoke control. This achieves adaptive selection of control actions and optimized resource allocation, thereby improving the reachability and effectiveness of the black box's prompting signals under different environmental conditions and effectively saving energy resources. Attached Figure Description

[0016] Figure 1 This is a flowchart of the steps of a deep learning-based drone black box control method in this invention.

[0017] Figure 2 This is a flowchart of step S4 in this invention. Detailed Implementation

[0018] Please see Figures 1-2 As shown, this invention relates to a deep learning-based method for controlling the black box of a drone, comprising the following steps: S1. Obtain real-time flight area images of the drone, and extract the area image of the drone before it crashes based on the real-time flight area images; S2. Determine the environmental type of the crash area based on the area image before the drone crashes; S3. Real-time acquisition of interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal; S4. Based on the reinforcement learning model, a black box control strategy is generated according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

[0019] Furthermore, the flight area images are acquired by installing a camera in the black box.

[0020] In some embodiments, the black box integrates a camera module, a processor, and a wireless communication unit. When the UAV is conducting inspection or aerial survey missions, the black box uses a wide-angle camera fixed to the underside of the aircraft to continuously acquire images of the flight area at, for example, a resolution of 1920×1080 and a frame rate of 30 frames per second, and caches a certain length of image sequences locally. The processor performs inter-frame difference and optical flow estimation on consecutive frames to construct a temporal feature sequence reflecting scene changes and aircraft attitude changes. Once a sharp change in attitude parameters fed back by the underlying flight control or inertial navigation module is detected (such as pitch angle or roll angle exceeding a preset threshold in a very short time), the black box anomaly detection process is triggered. At this time, the system calls a pre-trained bidirectional temporal feature detection model to discriminate the image difference sequences of the most recent frames. After detecting a potentially critical moment before the crash, it backtracks from that moment to extract several clear and relatively stable images, forming a regional image dataset of the UAV before the crash. This approach, which first automatically locates the "pre-crash window" based on visual temporal features and then focuses on in-depth analysis of the image within that window, differs from traditional black box methods that rely solely on flight control logs or GPS trajectories. It introduces more refined environmental visual information, improving the specificity and reliability of subsequent environmental assessments. After brightness normalization, noise suppression, and edge enhancement of the pre-crash area image, it is input into a convolutional neural network pre-trained through transfer learning and fine-tuned on multiple outdoor scene datasets. Multi-scale convolution and pooling layers extract environmental feature vectors, including texture, color distribution, vegetation morphology, water surface reflection characteristics, and building outlines. Subsequently, a fully connected classification layer performs multi-class discrimination on the feature vectors, outputting probability distributions for environmental types such as forest, grassland, water, bare land, and built-up areas. The type with the highest probability is taken as the environmental type of the UAV crash area, and this type is encoded into a one-dimensional or multi-dimensional environmental semantic vector. Unlike existing technologies that rely solely on simple geographic information or coarse-grained map labels to determine the environment, this embodiment enables the black box to visually recognize specific fall environments by learning and classifying features from the real visual scene before the fall. Furthermore, through continuous interaction with the ground search and rescue terminal via a wireless link, real-time data is collected on the interaction between the search and rescue terminal and the black box, as well as the search and rescue terminal's positioning data. The interaction data includes communication signal strength (such as RSSI), the number of disconnections per unit time, and the difference between the time of the most recent successful communication and the current time, used to characterize link quality and communication reliability. The positioning data includes longitude, latitude, and altitude output by the search and rescue terminal equipment, as well as speed and direction of motion calculated based on continuous positioning, used to reflect the spatial relationship and movement trend of the search and rescue personnel relative to the estimated position of the black box. The processor encodes the above interaction data and positioning data into numerical features and constructs them, along with the fall area environment type, into a reinforcement learning state vector, thereby simultaneously integrating communication state, spatial topology, and environmental constraints in the state space.Finally, a reinforcement learning policy model based on a deep neural network optimizes the control behavior of the black box. Specifically, the state vector is input into the policy network to evaluate the value of various candidate control actions (e.g., combinations of lights with different flashing frequencies and brightness, sound cues with different intervals and volumes, and smoke signals with different release frequencies and durations) in the current search and rescue state. A preset reward function considers the increase in communication signal strength and the reduction in distance between the search and rescue location and the estimated location of the black box as positive rewards, and energy consumption and potential risks and interference in certain sensitive environments as penalties. A comprehensive reward value is calculated and fed back to the policy network to update parameters. As the search and rescue process progresses, the model continuously receives feedback and adjusts the control strategy under different environmental types and search and rescue paths, enabling the black box to adaptively select the combination and intensity of light, sound, and smoke control based on the real-time state, rather than emitting a single signal in a fixed period or pattern. Compared with existing technologies, this embodiment constructs a closed loop of "pre-fall visual environment recognition - search and rescue interaction state modeling - reinforcement learning control decision-making". It embeds environmental semantic information and search and rescue dynamic information into the generation process of control strategy, realizes joint optimization of prompting methods in spatial, temporal and energy consumption dimensions, significantly improves the location of black boxes and search and rescue efficiency in complex environments, and has the ability to continuously self-optimize through data-driven approaches.

[0021] Furthermore, the step of extracting the area image of the UAV before it crashed based on the real-time flight area image includes the following steps: Temporal difference processing is performed on consecutive frames based on real-time flight area images, and a flight state change sequence is constructed based on the pixel change amplitude of adjacent frames; Based on the flight state change sequence, the abnormal moments of flight attitude are identified by the time-series feature detection model, and several frames of images before the abnormal moments are constructed as a candidate pre-crash image sequence. Feature aggregation processing is performed on the candidate pre-crash image sequence, and image frames are selected based on preset image sharpness evaluation index and scene stability index to obtain the regional image before the drone crashes.

[0022] Furthermore, the temporal feature detection model includes a bidirectional gated loop unit.

[0023] In some embodiments, temporal difference processing is first performed on the image sequence within the time window, i.e., pixel-level subtraction is performed on adjacent frames, and combined with a gradient magnitude-based change detection algorithm to construct a flight state change sequence describing the degree of dynamic changes in the scene. This state change sequence not only includes traditional brightness difference information but also incorporates motion vector features obtained based on structural similarity (SSIM) or optical flow estimation, enabling the sequence to more sensitively reflect changes in the field of view caused by subtle attitude variations during flight, thus providing a more comprehensive temporal input for further identification of abnormal flight states. After constructing the flight state change sequence, a pre-trained bidirectional gated recurrent unit (Bi-GRU) model is used to perform temporal feature analysis on the sequence. This model uses forward and backward recursive units to bidirectionally encode the input sequence, enabling it to simultaneously capture the antecedent and subsequent features of state changes, thereby improving the temporal discriminative ability of anomaly detection. During training, the Bi-GRU model uses a labeled dataset containing samples of normal flight and abnormal attitudes to learn the temporal patterns of different types of attitude anomalies (including sharp pitch, roll instability, and spiral fall) on state change sequences, enabling rapid identification of abnormal moments in actual flight. When the model detects a moment in the sequence where the probability of an anomaly exceeds a threshold, the system automatically extracts several frames from the previous sequence to form a candidate pre-fall image sequence. These images typically correspond to the stable field of view before attitude instability and can be used as high-value visual information for subsequent environmental assessment. To ensure high discrimination quality of the final output pre-fall region images, feature aggregation and filtering operations are performed on the candidate image sequence. First, a convolutional feature extractor generates spatial texture features and global scene features for each frame in the sequence. Then, based on feature similarity, an inter-frame similarity matrix is ​​constructed to determine the scene consistency within the candidate sequence. Simultaneously, the system uses sharpness evaluation metrics (such as Laplacian transform gradient values) to assess the focus of each image and utilizes scene stability metrics (such as inter-frame optical flow stability and edge consistency scores) to measure whether there is significant blur or dynamic interference in the image. By combining the feature aggregation results with the above evaluation metrics, keyframes with high clarity, good scene consistency, and stable visual features are finally selected as the output images of the area before the fall.

[0024] Further, S2 includes the following steps: Preprocessing operations are performed on the regional images of the drone before it crashes, including brightness equalization, noise suppression, and edge enhancement, to obtain a preprocessed image. The preprocessed image is input into the convolutional neural network model, and multi-scale convolution operations and feature extraction are performed on the image to obtain the environmental feature vector; Based on the environmental feature vector input to the environmental type determination model, the probability calculation of forest, grassland, water, built-up area and other environmental categories is performed through the classification layer to obtain the probability distribution of environmental type; The environment type corresponding to the maximum probability value is selected as the environment type of the drone crash area based on the probability distribution of environment types.

[0025] In some embodiments, after receiving the image of the region before the fall, preprocessing operations are first performed on the image to ensure that the subsequent feature extraction stage can be performed under high-quality input conditions. The preprocessing process includes three types of operations: brightness equalization, noise suppression, and edge enhancement. Brightness equalization adjusts the image brightness distribution using histogram equalization or adaptive histogram equalization (CLAHE) to enhance detail visibility under insufficient or overexposed lighting conditions. Noise suppression uses Gaussian filters, bilateral filters, or adaptive mean filtering to suppress the interference of environmental and sensor noise on image texture. Edge enhancement utilizes the Laplacian operator or high-pass filtering to enhance the structural information of object boundaries in the scene, making the subsequent convolutional neural network more sensitive when extracting shape and contour features. Through these preprocessing operations, the system obtains a preprocessed image with a high signal-to-noise ratio and structural clarity, which significantly improves the stability of feature learning. Subsequently, the preprocessed image is input into the convolutional neural network model to perform multi-scale feature extraction. This embodiment employs a network structure containing shallow texture convolutional kernels and deep semantic convolutional kernels. Through convolutional operations at different scales, it captures multi-level features such as vegetation texture, water surface reflection, land surface structure, and building outlines. It then progressively extracts environmental feature vectors by combining batch normalization, ReLU activation function, and max pooling operations. The feature vectors output by the convolutional layers contain not only pixel-level texture information but also region-level semantic structure information, serving as important descriptors for environment classification. To enhance the model's robustness under different environmental conditions, the feature extraction network uses diverse scene datasets including forests, grasslands, water bodies, bare land, and built-up areas during training. Transfer learning and data augmentation techniques are used to improve the model's generalization ability to complex environments. After obtaining the environmental feature vectors, this embodiment inputs them into the environment type determination model. A classification structure constructed through fully connected layers calculates the probabilities of the features, outputting the probability distribution of each environmental category. The classification layer normalizes the category probabilities using the Softmax function, ensuring that it accurately reflects the matching degree of the input features across different environmental categories. The system then uses the category corresponding to the maximum probability as the final environmental type of the drone crash area, and uses this type as an important input parameter for the subsequent control strategy generation module.

[0026] Furthermore, the interaction data between the search and rescue terminal and the black box includes: communication signal strength data, disconnection rate, recent connection time and current time difference.

[0027] It should be noted that the interaction data between the search and rescue terminal and the black box is designed as a multi-dimensional feature vector including communication signal strength data, disconnection rate, and the time difference between the most recent connection time and the current time. The communication signal strength data uses Received Signal Strength Indicator (RSSI) or Channel Quality Indicator (CQI) as the basic metric. The processing module performs moving average and short-term fluctuation analysis on this indicator over multiple communication cycles to obtain signal strength characteristics that reflect link stability and distance change trends. The disconnection rate is calculated by statistically analyzing the number of communication link interruptions and their duration per unit time. This parameter can reveal occasional link failures caused by terrain obstruction, environmental absorption, or movement of the search and rescue terminal, and provides a basis for the robust design of the alert strategy. The time difference between the most recent connection time and the current time is obtained by recording the timestamp of the last successful connection between the search and rescue terminal and the current system time, and performing a difference operation to obtain a time difference feature that can be used as a characterization of link real-time performance. This feature can reflect whether the communication is in a state of prolonged instability or complete failure, and can serve as an important reference factor in the selection of reinforcement learning strategies to increase audio cues or enhance the frequency of light cues. To ensure that these time features are continuous enough to be used for model training and policy evaluation.

[0028] Furthermore, the search and rescue terminal positioning data includes: real-time geographic coordinate data and displacement data of the search and rescue terminal, wherein the displacement data includes velocity parameters and motion direction parameters.

[0029] It should be noted that once the search and rescue process is initiated, the search and rescue terminal continuously outputs real-time geographic coordinate data, including longitude, latitude, and altitude parameters, at a preset sampling frequency. The original coordinate sequence is then smoothed using a Kalman filter algorithm to reduce instantaneous jump errors caused by obstruction, multipath propagation, or poor satellite geometry, resulting in more stable spatial continuity of the coordinate information received by the black box. Based on this, differential operations are performed on continuous coordinate points to construct displacement data describing the movement behavior of the search and rescue terminal. Specifically, by performing distance and direction calculations on two adjacent coordinate points and combining the coordinate sampling time interval, the instantaneous velocity parameters of the search and rescue terminal can be derived; then, the motion direction parameters are calculated based on the spatial azimuth difference between the two points. To further improve the robustness of the displacement characteristics, a motion state estimation method based on Extended Kalman Filter (EKF) is adopted, fusing GNSS position data with IMU linear acceleration and angular velocity data to obtain more stable and noise-resistant velocity and direction estimation results. The real-time geographic coordinate data, velocity parameters, and motion direction parameters will be encoded into continuous numerical vectors and input into subsequent reinforcement learning state modeling to characterize the spatial relationship and motion trend of the search and rescue terminal relative to the possible location of the black box.

[0030] Further, S4 includes the following steps: A reinforcement learning state vector is constructed based on the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the location data of the search and rescue terminal. The state vector is input into the reinforcement learning policy network, and feature encoding and policy evaluation calculation are performed on the state vector. The immediate benefits of different control actions are evaluated based on the preset reward function to obtain the corresponding policy score. The optimal control action is selected based on the strategy score, and the control action is mapped to a light control command, a sound control command, or a smoke control command. The feedback data after the control action is executed is fed back based on the reward function, the policy network is updated, and a black box control policy is generated based on the updated policy network.

[0031] In some embodiments, the multi-source input data is first uniformly vectorized and encoded to construct a reinforcement learning state vector that can be used for policy learning. This state vector consists of three core parts: first, environmental semantic features, such as category vectors output by the environmental recognition model (e.g., "forest," "water area," "construction zone"), represented by one-hot encoding or environmental embedding vectors to reflect the impact of different environments on the propagation characteristics of the prompting method; second, interaction data between the search and rescue terminal and the black box, including signal strength, disconnection rate, and connection delay, processed through normalization and time decay functions to form continuous features that measure link reliability; and third, search and rescue terminal positioning data, including filtered and optimized geographic coordinates, speed, and direction of movement, constructed using coordinate difference and vector projection to describe the approach trend and spatial distance of the search and rescue terminal. These features are concatenated to form a high-dimensional state vector, used to comprehensively characterize the search and rescue environment state of the black box. After constructing the state vector, it is input into the reinforcement learning policy network for feature encoding and policy evaluation calculation. This embodiment employs a deep neural architecture comprising a state encoding layer and a policy inference layer. The state encoding layer utilizes a fully connected network or graph neural network to perform nonlinear mapping on input features, obtaining latent feature representations capable of characterizing complex environments and action dependencies. The policy inference layer, based on a policy gradient or actor-critic framework, calculates the policy score for each candidate action (e.g., light signals with different brightness values, sound signals with different interval frequencies, and smoke signals with different release intensities) in the current state. The model quantifies the immediate benefits of actions using a pre-defined reward function. For example, enhanced communication signals and reduced distance to search and rescue are considered positive benefits, while energy consumption costs and penalties for the impact of sound, light, and smoke in environmentally sensitive areas are considered negative benefits. This constructs an action value evaluation mechanism, providing a mathematical basis for policy selection. Based on the policy score, a greedy policy or an ε-greedy policy with an exploration factor is used to select the optimal control action in the current state, and this action is mapped to a specific black box execution command. For example, when the environment is dense forest and signal attenuation is significant, the policy network might output a combination of actions that enhance sound cues while reducing light signal power; when the search and rescue end is approaching in a clear direction, the network might choose a low-power flashing cues to increase endurance time; when the communication link is completely lost and the search and rescue end is far away, the policy network might output highly visible smoke signals as an auxiliary positioning method. This mapping process transforms abstract actions into executable command parameters, including brightness, frequency, sound intensity, and smoke release time. After the control action is executed, the system recalculates the reward value based on the latest feedback data from the search and rescue end, and inputs this reward, along with the state and action, as training samples into the critic network to update the policy network parameters.This embodiment employs a time-difference (TD)-based strategy optimization method, enabling the strategy network to learn the optimal action distribution pattern through continuous interactive feedback. This allows for the generation of highly adaptive black box control strategies under different environmental types, communication states, and search and rescue trajectory conditions. Compared to existing technologies that rely on fixed signal transmission methods, this embodiment utilizes a reinforcement learning-based dynamic strategy update mechanism, enabling the black box to self-adjust its prompting methods based on real-time status. This algorithmically optimizes prompting actions and minimizes energy consumption, significantly improving search and rescue efficiency and positioning accuracy.

[0032] Furthermore, the formula for the reward function is as follows: ; in, This is the reward value; The increase in communication signal strength after the control action is executed, relative to before the control action; This represents the amount by which the distance between the search and rescue location and the black box location is shortened; The energy consumption value corresponding to the actions of light, sound and smoke control; This is the environmental risk assessment value; , , and These are positive weighting coefficients.

[0033] It should be noted that the reward value Used to measure the contribution of current actions to improving search and rescue efficiency. It represents the increase in communication signal strength after the control action is executed relative to before the action was executed. It is usually calculated by comparing the difference between two RSSI measurements or by the change in channel quality (CQI), and is used to reflect the actual effect of the action in improving link stability. The reduction in distance between the search and rescue terminal location and the estimated location of the black box is calculated by comparing the Euclidean distance change obtained from the GPS and IMU fusion positioning results before and after the action is performed. The power management module inside the black box is used to measure the energy consumption differences of different prompt actions, including the power consumption corresponding to the light brightness level, the driving energy consumption corresponding to the sound output intensity, and the energy consumption required for the smoke release mechanism. It is calculated based on a preset model and can effectively prevent the strategy from wasting power due to the frequent use of high-energy-consuming actions. This is used to measure the potential negative impacts of control actions in specific environmental types. For example, using high-intensity smoke in forest and grassland environments may cause visual interference during searches, while using loud cues in built-up areas may create echo effects that affect judgment. The environmental risk value is calculated by the environmental recognition module based on a risk matrix constructed according to the environmental type and action type. Positive weight coefficients are pre-configured according to task requirements to balance the impact of signal improvement gains, distance reduction gains, energy consumption costs, and environmental risks in the reward function. Through the above reward function, the reinforcement learning model can continuously update the policy network parameters during training, ensuring that the selection of control actions is always optimized towards improving communication capabilities, shortening search and rescue time, and reducing energy consumption and environmental risks, forming a black box cue strategy that combines efficiency and robustness.

[0034] The present invention also includes a deep learning-based drone black box control system, applied to any of the foregoing deep learning-based drone black box control methods, comprising: The image processing module is used to acquire real-time flight area images of the drone and extract images of the area before the drone crashes based on the real-time flight area images. The environment recognition module is used to determine the environmental type of the crash area based on the area image before the drone crashes; The search and rescue data acquisition module is used to acquire real-time interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal. The black box control module is used to generate a black box control strategy based on a reinforcement learning model, according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

[0035] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A deep learning-based method for controlling the black box of a drone, characterized in that, Includes the following steps: S1. Obtain real-time flight area images of the drone, and extract the area image of the drone before it crashes based on the real-time flight area images; S2. Determine the environmental type of the crash area based on the area image before the drone crashes; S3. Real-time acquisition of interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal; S4. Based on the reinforcement learning model, a black box control strategy is generated according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

2. The deep learning-based UAV black box control method according to claim 1, characterized in that, The flight area images are acquired by installing a camera in the black box.

3. The deep learning-based UAV black box control method according to claim 1, characterized in that, The step of extracting the area image of the UAV before it crashed based on the real-time flight area image includes the following steps: Temporal difference processing is performed on consecutive frames based on real-time flight area images, and a flight state change sequence is constructed based on the pixel change amplitude of adjacent frames; Based on the flight state change sequence, the abnormal moments of flight attitude are identified by the time-series feature detection model, and several frames of images before the abnormal moments are constructed as a candidate pre-crash image sequence. Feature aggregation processing is performed on the candidate pre-crash image sequence, and image frames are selected based on preset image sharpness evaluation index and scene stability index to obtain the regional image before the drone crashes.

4. The deep learning-based UAV black box control method according to claim 3, characterized in that, The temporal feature detection model has a bidirectional gated loop unit.

5. The deep learning-based UAV black box control method according to claim 1, characterized in that, S2 includes the following steps: Preprocessing operations are performed on the regional images of the drone before it crashes, including brightness equalization, noise suppression, and edge enhancement, to obtain a preprocessed image. The preprocessed image is input into the convolutional neural network model, and multi-scale convolution operations and feature extraction are performed on the image to obtain the environmental feature vector; Based on the environmental feature vector input to the environmental type determination model, the probability calculation of forest, grassland, water, built-up area and other environmental categories is performed through the classification layer to obtain the probability distribution of environmental type; The environment type corresponding to the maximum probability value is selected as the environment type of the drone crash area based on the probability distribution of environment types.

6. The deep learning-based UAV black box control method according to claim 1, characterized in that, The interaction data between the search and rescue terminal and the black box includes: communication signal strength data, disconnection rate, recent connection time and current time difference.

7. The deep learning-based UAV black box control method according to claim 1, characterized in that, The search and rescue terminal positioning data includes: real-time geographic coordinate data and displacement data of the search and rescue terminal, wherein the displacement data includes velocity parameters and motion direction parameters.

8. The deep learning-based UAV black box control method according to claim 1, characterized in that, S4 includes the following steps: A reinforcement learning state vector is constructed based on the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the location data of the search and rescue terminal. The state vector is input into the reinforcement learning policy network, and feature encoding and policy evaluation calculation are performed on the state vector. The immediate benefits of different control actions are evaluated based on the preset reward function to obtain the corresponding policy score. The optimal control action is selected based on the strategy score, and the control action is mapped to a light control command, a sound control command, or a smoke control command. The feedback data after the control action is executed is fed back based on the reward function, the policy network is updated, and a black box control policy is generated based on the updated policy network.

9. A deep learning-based UAV black box control method according to claim 8, characterized in that, The formula for the reward function is as follows: ； in, This is the reward value; The increase in communication signal strength after the control action is executed, relative to before the control action; This represents the amount by which the distance between the search and rescue location and the black box location is shortened; The energy consumption value corresponding to the actions of light, sound and smoke control; This is the environmental risk assessment value; , , and These are positive weighting coefficients.

10. A deep learning-based UAV black box control system, applied to the deep learning-based UAV black box control method described in any one of claims 1-9, characterized in that, include: The image processing module is used to acquire real-time flight area images of the drone and extract images of the area before the drone crashes based on the real-time flight area images. The environment recognition module is used to determine the environmental type of the crash area based on the area image before the drone crashes; The search and rescue data acquisition module is used to acquire real-time interaction data between the search and rescue terminal and the black box, as well as the location data of the search and rescue terminal. The black box control module is used to generate a black box control strategy based on a reinforcement learning model, according to the environmental type of the crash area, the interaction data between the search and rescue terminal and the black box, and the positioning data of the search and rescue terminal. The types of black box control strategies include light control, sound control and smoke control.

Images