A positioning method of a drone remote controller

Abstract

The application relates to the technical field of unmanned aerial vehicle remote controller positioning, in particular to a positioning method of an unmanned aerial vehicle remote controller. The method comprises the following steps: obtaining the position of a target unmanned aerial vehicle remote controller according to the message information of the target unmanned aerial vehicle remote controller at a target moment, and defining a first region with the position as the center and a first preset length as the radius; obtaining the video of the target moment of the region, if the video contains at least one unmanned aerial vehicle remote controller, extracting the behavior characteristic data of each unmanned aerial vehicle remote controller in a target time period in the video; obtaining the matching probability of each unmanned aerial vehicle remote controller for the target unmanned aerial vehicle remote controller according to the behavior characteristic data of each unmanned aerial vehicle remote controller in the target time period and the state data of the target unmanned aerial vehicle in the target time period; if only one unmanned aerial vehicle remote controller corresponds to the matching probability greater than or equal to a threshold value, the unmanned aerial vehicle remote controller is determined as the target unmanned aerial vehicle remote controller. The application can improve the efficiency of determining the target unmanned aerial vehicle remote controller.

Description

Technical Field

[0001] This invention relates to the field of drone remote controller positioning technology, and in particular to a positioning method for drone remote controllers. Background Technology

[0002] In recent years, drone technology has developed rapidly and its applications have become increasingly widespread. However, the resulting low-altitude safety issues have also become increasingly prominent. Rapidly locating and identifying target drones using remote controllers has become a key technology in airspace management and public safety.

[0003] Currently, remote controller positioning mainly relies on the radio messages they emit (such as ADS-B-like signals) for coarse location. This method first estimates the remote controller's location information using techniques such as triangulation, and then dispatches management personnel to the area for manual investigation. However, this method has the following problems: First, the message positioning itself has a certain degree of error, which is particularly significant in complex urban environments; second, when there are multiple potential remote controllers in the positioning area (e.g., in a park where drone enthusiasts gather or at a large event), location information alone cannot accurately identify which remote controller belongs to the target drone. Management personnel need to question or observe each one individually, which is inefficient and cannot achieve the goal of accurate and rapid control. How to improve the efficiency of identifying the remote controller of a target drone is an urgent problem to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method for locating a drone remote controller, so as to improve the efficiency of remote controllers in determining the target drone.

[0005] According to the present invention, a positioning method for a drone remote controller is provided, the method comprising the following steps:

[0006] S100: Obtain the position of the remote controller of the target drone based on the message information of the remote controller of the target drone at the target time, and delineate a first area with the position as the center and a first preset length as the radius.

[0007] S200: Obtain the video of the target time in the first area. If the video of the target time in the first area contains at least one drone remote controller, proceed to S300.

[0008] S300, extract the behavioral feature data of each drone remote controller in the video of the target time in the first region during the target time period; the target time period is the time period including the target time.

[0009] S400, based on the behavioral feature data of each drone remote controller in the target time period and the state data of the target drone in the target time period, obtain the matching probability that each drone remote controller is the remote controller of the target drone.

[0010] S500: If there is only one drone remote controller with a matching probability greater than or equal to a preset probability threshold, then the drone remote controller is identified as the remote controller of the target drone.

[0011] Compared with the prior art, the present invention has at least the following beneficial effects:

[0012] This invention employs a dual verification mechanism of initial location screening and precise behavioral feature judgment. First, potential areas are delineated using message information. Then, the matching probability between the behavioral characteristics of the remote controllers within the area and the state data of the UAV is analyzed to make a judgment. This overcomes the misjudgment and missed judgment caused by relying solely on location information, achieving the effect of locating the person from the area. Moreover, by calculating the matching probability of each remote controller and setting a threshold, this invention can effectively handle complex situations where multiple remote controllers exist in an area. Even in areas with high remote controller density, it can accurately filter out a unique target remote controller through the strong correlation between behavior and state, improving the efficiency of identifying the remote controller of the target UAV. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a positioning method for a drone remote controller provided in Embodiment 1 of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1:

[0017] According to this embodiment, a positioning method for a drone remote controller is provided, the method including the following steps, such as... Figure 1 As shown:

[0018] S100: Obtain the position of the remote controller of the target drone based on the message information of the remote controller of the target drone at the target time, and delineate a first area with the position as the center and a first preset length as the radius.

[0019] In this embodiment, the location of the target drone's remote controller can be determined by monitoring the message information transmitted by the target drone's remote controller. It should be understood that the message information transmitted by the target drone's remote controller and the message information transmitted by the target drone itself contain the same identity ID. Based on the identity ID in the message information transmitted by the target drone, the message information transmitted by the drone's remote controller can be identified (that is, the drone remote controller whose transmitted message information contains the same identity ID is identified as the target drone's remote controller).

[0020] In this embodiment, the process of locating the remote controller based on the message information transmitted by the target drone remote controller (i.e., drone remote controller location) is existing technology and will not be described in detail here.

[0021] As an optional specific implementation, the first preset length is an empirical value, such as 20 meters or 50 meters.

[0022] In this embodiment, the target time is any time, for example, the target time is the current time.

[0023] S200: Obtain the video of the target time in the first area. If the video of the target time in the first area contains at least one drone remote controller, proceed to S300.

[0024] As an optional implementation method, existing security cameras or temporarily deployed monitoring equipment in the first area can be activated to obtain video at the target time.

[0025] As an optional implementation, an object detection model (such as YOLO, Faster R-CNN, etc.) can be used to automatically identify all drone remote controllers in the video frames. Those skilled in the art will understand that the process of object detection using an object detection model is prior art and will not be described further here.

[0026] In this embodiment, if the video of the target time in the first region does not contain the drone remote controller, the range is expanded to continue determining the target drone's remote controller. For example, a second region is defined with the location as the center and a second preset length as the radius, and it is determined whether the target drone's remote controller exists in the second region; the second preset length is greater than the first preset length. The process of determining whether the target drone's remote controller exists in the second region is similar to the process of determining whether the target drone's remote controller exists in the first region in this embodiment, and will not be described again here.

[0027] S300, extract the behavioral feature data of each drone remote controller in the video of the target time in the first region during the target time period; the target time period is the time period including the target time.

[0028] In this embodiment, the target time period is a time period centered on the target time (e.g., 10 seconds before and after the target time), or a time period starting from the target time (e.g., 20 seconds after the target time). The length of the target time period is an empirical value, such as 10 seconds, 20 seconds, etc.

[0029] In this embodiment, behavioral feature data is used to characterize the operation of the drone remote controller on the drone. As an optional implementation, the behavioral feature data includes the horizontal movement speed of hand key points. As an optional implementation, a human key point detection algorithm is used to extract hand key points, and the movement speed (pixels / second) is calculated by tracking the horizontal pixel coordinate changes of the hand key points in consecutive video frames. This directly corresponds to the left and right movements of the drone operator's hand on the remote controller. It should be understood that the behavioral feature data of each drone remote controller within the target time period is time-series data. For example, if each second includes n frames and the video includes 10 seconds, then the behavioral feature data of each drone remote controller within the target time period includes 10 × n data points.

[0030] S400, based on the behavioral feature data of each drone remote controller in the target time period and the state data of the target drone in the target time period, obtain the matching probability that each drone remote controller is the remote controller of the target drone.

[0031] In this embodiment, the target drone's state data is used to characterize the drone's response to the behavioral feature data of its remote controller. There is a correspondence between the target drone's state data and the aforementioned behavioral feature data. As a specific implementation, when the aforementioned behavioral feature data is the horizontal movement speed of a hand key point, the target drone's state data is the target drone's yaw rate. The drone's heading rate describes the rate at which the drone turns left or right.

[0032] As one specific implementation, S400 includes:

[0033] S410, perform time-series alignment of the behavioral feature data of each drone remote controller with the state data of the target drone.

[0034] In this embodiment, time series alignment refers to the alignment of the timelines of standard behavioral feature data and UAV status data, eliminating time differences between different devices and avoiding misjudgments caused by time misalignment. As a specific implementation, the time difference between the two sets of data is obtained through pre-calibration via synchronous testing, and time series alignment is performed based on this time difference. For example, if the result obtained through pre-calibration via synchronous testing is a 0.1-second delay in the behavioral feature data, then the time series alignment of the behavioral feature data and the status data is achieved by subtracting 0.1 seconds from the behavioral feature data.

[0035] S420, obtain the correlation between the behavioral feature data of each drone remote controller and the state data of the target drone, and use it as the matching score of the drone remote controller.

[0036] In one specific implementation, S420 includes:

[0037] S421, For any drone, within a preset time lag range, calculate the cross-correlation coefficient between the behavioral feature data of the drone remote controller and the state data of the target drone.

[0038] In this embodiment, lag cross-correlation analysis refers to analyzing the degree of correlation between two time series data at different time lags (e.g., the remote controller's action at time t, the drone's response at time t+t0, where t0 is the lag time); the cross-correlation coefficient is used to quantify the strength of the linear association between the two sets of time series data, where 1 indicates a perfect positive correlation and 0 indicates no correlation; the preset time lag range is the range of t0 values ​​set according to the drone's response characteristics, for example, 0-2 seconds, covering the maximum delay of signal transmission + drone response.

[0039] As a specific implementation method, for the two aligned sets of data, iterate through each t0 within the preset lag range (e.g., t0 = 0.1 seconds, 0.2 seconds, ..., 2 seconds); for each t0, extract the [t, t+T-t0] data of X and the [t+t0, t+T] data of Y (T is the target time period length, t is the start time of the target time period, and t+T is the end time of the target time period); calculate the correlation coefficient between the two sets of extracted data using the Pearson correlation coefficient formula, and determine the absolute value of the correlation coefficient as the cross-correlation coefficient; X is behavioral feature data, and Y is state data.

[0040] S422, the largest cross-correlation coefficient is determined as the correlation between the behavioral feature data of the drone remote controller and the state data of the target drone.

[0041] In this embodiment, the greater the correlation between the behavioral feature data of a certain drone remote controller and the state data of the target drone, the greater the probability that the drone remote controller is the remote controller of the target drone.

[0042] S430: Based on the matching scores of all drone remote controllers, the normalization process is used to calculate the matching probability of each drone remote controller being the remote controller of the target drone.

[0043] In this embodiment, normalization can be achieved by dividing the matching score of each drone remote controller by the sum of the matching scores of all drone remote controllers, thus obtaining the matching probability that each drone remote controller is the remote controller of the target drone.

[0044] S500: If there is only one drone remote controller with a matching probability greater than or equal to a preset probability threshold, then the drone remote controller is identified as the remote controller of the target drone.

[0045] As a specific implementation, S500 further includes: if the matching probability corresponding to the drone remote controller is not greater than or equal to a preset probability threshold, then a second region is defined with the location as the center and a second preset length as the radius, and it is determined whether the remote controller of the target drone exists in the second region; the second preset length is greater than the first preset length. Optionally, the second preset length is an empirical value, for example, the first preset length is 50 meters and the second preset length is 100 meters. The process of determining whether the remote controller of the target drone exists in the second region is similar to the process of determining whether the remote controller of the target drone exists in the first region in this embodiment, and will not be described again here. Optionally, if the remote controller of the target drone still does not exist in the second region, then the range is further expanded, or a failure message is output.

[0046] As a specific implementation, S500 further includes: if there are at least two drone remote controllers whose matching probabilities are greater than or equal to a preset probability threshold, then proceed to S510.

[0047] S510: Continuously acquire flight status data of the target UAV, form flight status time series data, and divide the flight status time series data into several continuous flight modes.

[0048] As one specific implementation, the flight modes include uniform linear flight, hovering, turning, climbing, and descending.

[0049] In this embodiment, the goal of flight mode segmentation is to label the flight state of the UAV. As a specific implementation, the flight state data of the target UAV includes synthetic velocity (reflecting the overall motion intensity), yaw rate (reflecting the turning intensity), vertical velocity (reflecting the rise and fall intensity), and acceleration modulus (reflecting the intensity of maneuverability).

[0050] As a specific implementation method, a series of thresholds are first defined based on physical knowledge and experimental data. For example: a velocity threshold (e.g., 0.5 m / s, below which is considered stationary), an angular velocity threshold (e.g., 0.3 rad / s, above which is considered turning), and a vertical velocity threshold (0.3 m / s, above which is considered climbing / descending). Then, rules are established. For example, when the combined velocity is less than the velocity threshold, the directional angular velocity is less than the angular velocity threshold, and the vertical velocity is less than the vertical velocity threshold, it is determined to be hovering; when the combined velocity is greater than or equal to the velocity threshold, the directional angular velocity is less than the angular velocity threshold, and the vertical velocity is less than the vertical velocity threshold, it is determined to be uniform straight-line flight; when the directional angular velocity is greater than or equal to the angular velocity threshold, it is determined to be turning; when the vertical velocity is greater than or equal to the vertical velocity threshold, it is determined to be climbing; and when the vertical velocity is less than or equal to the negative of the vertical velocity threshold, it is determined to be descending. Finally, by comparing the data of the target UAV at a certain moment with the above rules, the flight mode of the target UAV at that moment can be obtained.

[0051] S520: Continuously acquire video data of each designated drone remote controller and extract operation behavior data of each designated drone remote controller to form operation behavior time sequence data of each designated drone remote controller, and divide the operation behavior time sequence data of each designated drone remote controller into several continuous operation modes; the designated drone remote controller is a drone remote controller whose matching probability is greater than or equal to a preset probability threshold.

[0052] As one specific implementation, the control modes include static hold, horizontal lateral control, horizontal longitudinal control mode, vertical throttle control, and composite coordinated control.

[0053] In this embodiment, the goal of control mode segmentation is to label the operation behavior of the remote control.

[0054] In one specific implementation, the video data of each designated drone remote controller includes hand movement speed and hand movement direction, wherein the hand movement speed can be obtained based on the displacement of wrist key points between consecutive frames, and the hand movement direction can be obtained by decomposing the horizontal and vertical axes of the image.

[0055] As a specific implementation, if the average movement speed of both wrists of a specified drone remote controller remains below a threshold v0, the corresponding operation mode is determined to be static hold. If the absolute value of the horizontal movement speed of a single wrist of the specified drone remote controller exceeds v0, and the cumulative displacement exceeds a threshold D0, the corresponding operation mode is determined to be horizontal lateral control. If the absolute value of the vertical movement speed of a single wrist of the specified drone remote controller exceeds a threshold v0, and the cumulative displacement exceeds a threshold D0, the corresponding operation mode is determined to be horizontal longitudinal control. If the absolute value of the vertical movement speed of a single wrist of the specified drone remote controller exceeds a threshold v0, and the cumulative displacement exceeds a threshold D0, the corresponding operation mode is determined to be vertical throttle control. If the movement speed of both hands of the specified drone remote controller simultaneously exceeds v0, the corresponding operation mode is determined to be composite coordinated control. Based on this, the timing data of the control behavior of each specified drone remote controller can be divided into several continuous control modes.

[0056] S530: For each specified drone remote controller, obtain the matching degree between the control mode sequence of the specified drone remote controller and the flight mode sequence of the target drone.

[0057] As one specific implementation, S530 includes:

[0058] S531, identify several first time points in the flight mode sequence where flight mode switching occurs.

[0059] In this embodiment, the first time point is the time point of flight mode switching in the flight mode sequence, such as the 5th second from hovering to turning, the 8th second from turning to uniform straight flight, etc.

[0060] S532 identifies several second time points in the control mode sequence of each specified UAV remote controller where an operation mode switch occurs.

[0061] In this embodiment, the second time point is the time point of operation mode switching in the control mode sequence, for example, the 4.8th second from static hold to horizontal control, and the 7.9th second from horizontal control to compound control.

[0062] S533, calculate the lag time distribution between the first time point and the second time point corresponding to each designated drone remote controller.

[0063] As a specific implementation, based on preset pairing rules, the plurality of first time points and the plurality of second time points are paired to form valid event pairs, wherein each valid event pair includes a second time point and a first time point. As a specific implementation, the preset pairing rules include: the second time point is prior to the first time point; the time difference between the second time point and the first time point is within a preset delay range; and the control mode switch corresponding to the second time point and the flight mode switch corresponding to the first time point are physically semantically matched. Physically semantically matched means that the flight mode switch is the result of the operation mode switch. For example, when control switches from static hold to horizontal lateral control, flight should switch from hovering to turning, and the two are physically semantically matched; when control switches from horizontal lateral control to vertical throttle control, flight should switch from turning to climbing, and the two are physically semantically matched. Optionally, all physically semantically matched operation mode switches and flight mode switches can be pre-listed, and a comparison can be made to determine whether a certain operation mode switch and a certain flight mode switch are physically semantically matched (successful comparison indicates a match, unsuccessful comparison indicates a mismatch).

[0064] For each valid event pair, the time difference between the flight mode switch and the operation mode switch is obtained, i.e., the lag time. It should be understood that if p valid event pairs are obtained, then p lag times can be obtained.

[0065] S534, obtain the matching degree corresponding to each specified drone remote controller based on the standard deviation of the lag time distribution corresponding to each specified drone remote controller; the standard deviation of the lag time distribution corresponding to any specified drone remote controller is negatively correlated with the matching degree.

[0066] In this embodiment, for any designated drone remote controller, if the number of its corresponding valid event pairs is 0, its corresponding matching degree is determined to be 0; if the number of its corresponding valid event pairs is 1 and there are other designated players whose corresponding valid event pairs are greater than 1, its corresponding matching degree is determined to be 0; if the number of its corresponding valid event pairs is 1 and the maximum number of other designated players' corresponding valid event pairs is also 1, then the flight status data of the target drone and the video data of the designated drone remote controller corresponding to a longer time period are obtained, and valid event pairs are obtained again until there are other designated players whose corresponding valid event pairs are greater than or equal to 2; if the number of its corresponding valid event pairs is greater than or equal to 2, its corresponding matching degree is determined according to the standard deviation of its corresponding lag time distribution.

[0067] In this embodiment, the standard deviation of the lag time distribution corresponding to any specified drone remote controller is negatively correlated with the matching degree. Optionally, the standard deviation of the lag time distribution corresponding to any specified drone remote controller is normalized, and then the difference between 1 and the normalized result corresponding to the specified drone remote controller is taken as the matching degree corresponding to the specified drone remote controller. Those skilled in the art will know that the normalization process is prior art and will not be described in detail here.

[0068] S540 identifies the remote controller of the specified drone with the highest matching degree as the remote controller of the target drone.

[0069] In this embodiment, the operation mode switching of the designated drone remote controller corresponding to the highest matching degree is most matched with the flight state switching of the target drone, and the accuracy of identifying the designated drone remote controller corresponding to the highest matching degree as the remote controller of the target drone is relatively high.

[0070] This embodiment employs a dual verification mechanism of initial location screening and precise behavioral feature judgment. First, potential areas are delineated using message information. Then, the matching probability between the behavioral characteristics of the remote controllers within the area and the state data of the UAV is analyzed to make a judgment. This overcomes the misjudgment and missed judgment caused by relying solely on location information, achieving the effect of locating the person from the area. Moreover, by calculating the matching probability of each remote controller and setting a threshold, this embodiment can effectively handle the complex situation of multiple remote controllers in an area. Even in areas with high remote controller density, the strong correlation between behavior and state can accurately filter out a unique target remote controller, improving the efficiency of identifying the remote controller of the target UAV.

[0071] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. It should also be understood that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A positioning method for a drone remote controller, characterized in that, The method comprises the following steps: S100, obtaining the position of the remote controller of the target UAV according to the message information of the remote controller of the target UAV at the target time, and defining a first region with the position as the center and a first preset length as the radius; S200, obtaining the video of the target time of the first region, and if the video of the target time of the first region contains at least one UAV remote controller, entering S300; S300, extracting the behavior feature data of each UAV remote controller in the video of the target time of the first region in the target time period; the target time period is a time period including the target time; S400, obtaining the matching probability of each UAV remote controller for the remote controller of the target UAV according to the behavior feature data of each UAV remote controller in the target time period and the state data of the target UAV in the target time period; S500, if only one UAV remote controller has a matching probability greater than or equal to a preset probability threshold, determining the UAV remote controller as the remote controller of the target UAV. 2.The method of claim 1, wherein, S400 comprises: S410, time sequence alignment of the behavior feature data of each UAV remote controller and the state data of the target UAV; S420, obtaining the correlation degree between the behavior feature data of each UAV remote controller and the state data of the target UAV as the matching score of the UAV remote controller; S430, based on the matching scores of all UAV remote controllers, calculating the matching probability of each UAV remote controller for the remote controller of the target UAV through normalization processing. 3.The method of Claim 2, wherein, S420 comprises: S421, for any UAV, calculating the cross-correlation coefficient between the behavior feature data of the UAV remote controller and the state data of the target UAV within a preset time lag range; S422, determining the maximum cross-correlation coefficient as the correlation degree between the behavior feature data of the UAV remote controller and the state data of the target UAV. 4.The method of claim 1, wherein, The behavior feature data comprises the horizontal motion speed of the hand key point; and the state data of the target UAV comprises the yaw angular velocity of the target UAV. 5.The method of claim 1, wherein, S500 further comprises: if there are at least two UAV remote controllers with a matching probability greater than or equal to a preset probability threshold, entering S510; S510, continuously obtaining the flight state data of the target UAV, forming flight state time sequence data, and dividing the flight state time sequence data into a plurality of continuous flight modes; S520, continuously obtaining the video data of each specified UAV remote controller, and extracting the operation behavior data of each specified UAV remote controller, forming the operation behavior time sequence data of each specified UAV remote controller, and dividing the operation behavior time sequence data of each specified UAV remote controller into a plurality of continuous operation modes; the specified UAV remote controller is a UAV remote controller with a corresponding matching probability greater than or equal to a preset probability threshold; S530, for each specified UAV remote controller, obtaining the matching degree between the operation mode sequence of the specified UAV remote controller and the flight mode sequence of the target UAV; S540, determining the specified drone remote controller corresponding to the highest matching degree as the remote controller of the target drone. 6.The method of Claim 5, wherein, S530 includes: S531, identifying a plurality of first time points at which flight mode switching occurs in the flight mode sequence; S532, identifying a plurality of second time points at which operation mode switching occurs in the operation mode sequence of each specified drone remote controller; S533, calculating the lag time distribution between the first time points and the second time points corresponding to each specified drone remote controller; S534, obtaining the matching degree corresponding to each specified drone remote controller based on the standard deviation of the lag time distribution corresponding to each specified drone remote controller; the standard deviation of the lag time distribution corresponding to any specified drone remote controller is negatively correlated with the matching degree. 7.The method of Claim 5, wherein, The flight mode includes uniform straight flight, hovering, turning, climbing and descending; the operation mode includes static holding, horizontal lateral control, horizontal longitudinal control mode, vertical throttle control and composite coordinated control. 8.The method of claim 1, wherein, S500 further includes: if there is no drone remote controller corresponding to the matching probability greater than or equal to the preset probability threshold, a second region is circumscribed with the position as the center and the second preset length as the radius, and it is determined whether there is a remote controller of the target drone in the second region; the second preset length is greater than the first preset length.

Images