Method for matching cluster target consistency, unmanned aerial vehicle, server and storage medium

Abstract

The application provides a UAV cluster target consistency matching method, a UAV, a server and a storage medium. The method can include: when a target is detected within a preset range, acquiring collection information; selecting a best target detection model according to a current channel condition; processing the collection information through a target detection algorithm of the best target detection model to generate a target detection result, the target detection result including at least two of target classification information, target confidence information, target time information and target position information; and sending the target detection result to a server, the target detection result being used for matching by the server. The detection model can be reasonably selected from preset detection models according to the current channel condition, the UAV collection information is processed to obtain a target detection result, and the target detection result includes multi-dimensional information, so that the accuracy of judging whether the target is consistent can be improved.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicles (UAVs), and more particularly to a method for target consistency matching in UAV swarms, UAVs, servers, and storage media. Background Technology

[0002] Drones in a drone swarm are typically small and micro-sized, with limited battery capacity and payload. Traditional drone target consistency matching methods have certain drawbacks. Due to payload limitations, drones in a swarm cannot carry multiple payloads like larger drones. They generally only carry high-resolution sensor modules (such as high-definition cameras) to acquire target video images. Therefore, when fusion nodes perform consistency matching, the comparison dimension is singular, resulting in low accuracy in determining whether targets are consistent. Summary of the Invention

[0003] This application provides a method for target consistency matching in a drone swarm, a drone, a server, and a storage medium. Based on the current channel conditions, it rationally selects a detection model from preset detection models, processes the information collected by the drone, and obtains a target detection result. This target detection result includes multi-dimensional information, thus improving the accuracy of determining whether targets are consistent.

[0004] The first aspect of this application provides a method for target consistency matching in a drone swarm, the method being applied to drones, and the method may include:

[0005] When detecting a target within a preset range, acquire collected information;

[0006] Select the optimal target detection model based on the current channel conditions;

[0007] The collected information is processed by the target detection algorithm of the optimal target detection model to generate target detection results, which include at least two of the following: target classification information, target confidence information, target time information, and target location information.

[0008] The target detection result is sent to the server, and the target detection result is used by the server for matching.

[0009] Optionally, sending the target detection result to the server may include: structurally encapsulating the target detection result and sending it.

[0010] Optionally, selecting the optimal target detection model based on the current channel conditions may include:

[0011] Based on the current channel conditions, select a detection model from the preset models with a processing power not exceeding [a certain value]. Among the detection models, the detection model with the highest processing power is the optimal detection model;

[0012] Among them, P max For the maximum airborne power, σ 2 denoted as the noise power factor, h as the channel gain from the current UAV to the server, L as the amount of data generated by the current UAV, B as the bandwidth allocated to the current UAV, and t as the duration.

[0013] Optionally, the detection model includes FastRCNN, YOLO, YOLO-tiny, and LightGBM, with corresponding processing powers including P. FRCNN P YOLO P YOLOT P LGBM .

[0014] Optionally, the processing power is sorted by P. FRCNN >P YOLO >P YOLOT >P LGBM .

[0015] A second aspect of this application provides a method for target consistency matching in a drone swarm, the method being applied to a server, the method comprising:

[0016] The system receives a first target detection result sent by a first UAV and a second target detection result sent by a second UAV. The first target detection result includes at least two of the following: first target classification information, first target confidence information, first target time information, and first target location information. The second target detection result includes at least two of the following: second target classification information, second target confidence information, second target time information, and second target location information.

[0017] Based on the first target detection result and the second target detection result, target consistency matching is performed.

[0018] Optionally, the first target detection result and / or the second target detection result are structured encapsulated target detection results.

[0019] Optionally, the first target detection result includes first target classification information, first target confidence information, first target time information, and first target location information; the second target detection result includes second target classification information, second target confidence information, second target time information, and second target location information; the step of performing target consistency matching based on the first target detection result and the second target detection result may include:

[0020] A consistency matching is performed based on the first target classification information and the second target classification information;

[0021] If the first target classification information and the second target classification information match, then perform consistency matching on the first target confidence information and the second target confidence information;

[0022] If the first target confidence information and the second target confidence information match, then the first target time information and the second target time information are matched for consistency.

[0023] If the first target time information and the second target time information match, perform consistency matching on the first target location information and the second target location information;

[0024] If the first target location information and the second target location information match, the targets corresponding to the first target classification information and the second target classification information are determined to be consistent.

[0025] Optionally, when the first target classification information and the second target classification information match, performing consistency matching on the first target confidence information and the second target confidence information may include: when the first target classification information and the second target classification information are the same, determining that the first target classification information and the second target classification information match, and performing consistency matching on the first target confidence information and the second target confidence information.

[0026] Optionally, when the first target confidence information and the second target confidence information match, performing consistency matching on the first target time information and the second target time information may include: calculating a first difference between the first target confidence information and the second target confidence information; determining that the first target confidence information and the second target confidence information match if the absolute value of the first difference is greater than a confidence threshold; and performing consistency matching on the first target time information and the second target time information.

[0027] Optionally, when the first target time information and the second target time information match, performing consistency matching on the first target location information and the second target location information may include: calculating a second difference between the first target time information and the second target time information; if the absolute value of the second difference is greater than a time threshold, determining that the first target time information and the second target time information match; and performing consistency matching on the first target location information and the second target location information.

[0028] Optionally, determining that the targets corresponding to the first target classification information and the second target classification information are consistent when the first target location information and the second target location information match may include: calculating a third difference between the first target location information and the second target location information; and determining that the first target location information and the second target location information are consistent when the absolute value of the third difference is greater than a location threshold.

[0029] Optionally, the method may further include: when the targets corresponding to the first target detection result and the second target detection result are consistent, performing information fusion on the first target detection result and the second target detection result.

[0030] A third aspect of this application provides an unmanned aerial vehicle (UAV) that may include:

[0031] The target detection module is used to obtain the current channel conditions and select the best target detection model when detecting a target within a preset range; and to generate a target detection result based on the target detection algorithm of the best target detection model. The target detection result includes at least two of the following: target classification information, target confidence information, target time information, and target location information.

[0032] The transceiver module is used to send the target detection results to the server, and the target detection results are used by the server for matching.

[0033] A fourth aspect of this application provides a server that may include:

[0034] The transceiver module is used to receive a first target detection result sent by a first UAV and a second target detection result sent by a second UAV. The first target detection result includes at least two of the following: first target classification information, first target confidence information, first target time information, and first target location information. The second target detection result includes at least two of the following: second target classification information, second target confidence information, second target time information, and second target location information.

[0035] The processing module is used to perform target consistency matching based on the first target detection result and the second target detection result.

[0036] The fifth aspect of this application provides an unmanned aerial vehicle (UAV) that may include:

[0037] Memory containing executable program code;

[0038] A processor and a transceiver coupled to the memory;

[0039] The processor and the transceiver are used to perform the method as described in the first aspect of this application.

[0040] A sixth aspect of this application provides a server that may include:

[0041] Memory containing executable program code;

[0042] A processor and a transceiver coupled to the memory;

[0043] The processor and the transceiver are used to perform the method as described in the second aspect of this application.

[0044] Another aspect of this application provides a computer-readable storage medium including instructions that, when executed on a processor, cause the processor to perform the methods described in the first or second aspect of this application.

[0045] Another aspect of this invention discloses a computer program product that, when run on a computer, causes the computer to execute the method described in the first or second aspect of this application.

[0046] Another aspect of this invention discloses an application publishing platform for publishing computer program products, wherein when the computer program product is run on a computer, the computer executes the method described in the first or second aspect of this application.

[0047] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0048] In this embodiment, when detecting a target within a preset range, collected information is acquired; based on the current channel conditions, an optimal target detection model is selected; the collected information is processed using the target detection algorithm of the optimal target detection model to generate a target detection result, which includes at least two of the following: target classification information, target confidence information, target time information, and target location information; the target detection result is sent to a server, and the target detection result is used by the server for matching. By reasonably selecting a detection model from preset detection models based on the current channel conditions and processing the information collected by the UAV to obtain the target detection result, which includes multi-dimensional information, the accuracy of determining whether targets are consistent can be improved. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments and the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and other drawings can be obtained based on these drawings.

[0050] Figure 1 A structural diagram for target consistency matching in an existing drone swarm;

[0051] Figure 2 The structural block diagram of the UAV swarm target consistency matching applied in the embodiments of this application;

[0052] Figure 3 This is a schematic diagram of an embodiment of the method for target consistency matching in a drone swarm according to this application.

[0053] Figure 4 This is a schematic diagram of another embodiment of the method for target consistency matching in a drone swarm in this application.

[0054] Figure 5 This is a schematic diagram of another embodiment of the method for target consistency matching in a drone swarm in this application.

[0055] Figure 6 This is a schematic diagram of one embodiment of the UAV in this application;

[0056] Figure 7 This is a schematic diagram of one embodiment of the server in this application;

[0057] Figure 8 This is a schematic diagram of another embodiment of the drone in this application;

[0058] Figure 9 This is a schematic diagram of another embodiment of the server in this application. Detailed Implementation

[0059] This application provides a method for target consistency matching in a drone swarm, a drone, a server, and a storage medium. Based on the current channel conditions, it rationally selects a detection model from preset detection models, processes the information collected by the drone, and obtains a target detection result. This target detection result includes multi-dimensional information, thus improving the accuracy of determining whether targets are consistent.

[0060] To enable those skilled in the art to better understand the present application, the technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. All embodiments based on the present application should fall within the scope of protection of the present application.

[0061] As unmanned platform technology, with drones as a prime example, matures, swarming multiple drones together can achieve functions that are impossible for a single drone, such as collaborative target detection, identification, and tracking, thus improving work efficiency through multi-drone collaboration. In this process, due to the different positions and angles of the multiple drones, target consistency matching is necessary during collaborative work. This involves confirming whether different drones are monitoring the same target, thus resolving the issue of multiple drones monitoring different targets (i.e., drones monitoring the same target having different identification numbers within the swarm network). Furthermore, based on the target consistency matching results, the fusion node (a high-performance drone or ground information processing node within the swarm, hereinafter the same) can organically integrate the target monitoring information from multiple drones after acquiring it, thereby improving the effectiveness of detection, identification, and tracking.

[0062] Currently, using mature communication protocols for networking (such as the 802.11 standard wireless LAN protocol) to achieve collaborative information exchange is a common communication technology solution for UAV swarms. Each UAV within the swarm uses an onboard target detection module to transmit the collected target monitoring information via wireless channels. The fusion node then performs target consistency matching based on the target detection information from each UAV. Further, this enables collaborative detection, identification, and tracking of targets, such as... Figure 1 The diagram shown is a structural block diagram of target consistency matching in an existing drone swarm.

[0063] Collaborative detection, identification, and tracking of targets by a swarm of multiple drones requires addressing the target consistency problem. This means determining whether the targets detected, identified, and tracked by different drones are the same target. Only on this basis can effective collaborative information fusion be performed, improving work efficiency. Currently, target consistency matching typically employs similarity matching methods, such as appearance similarity, location similarity, and speed similarity. Target information collected by different drones is matched for similarity at the fusion node. Targets with similarity exceeding a certain threshold are identified as the same target. Furthermore, the fusion node integrates the information on the same target collected by different drones, improving detection, identification, and tracking effectiveness.

[0064] Drones in a drone swarm are typically small and micro-sized, with limited battery capacity and payload. Traditional drone target consistency matching methods have certain limitations. Due to payload constraints, individual drones in a swarm cannot carry multiple payloads like larger drones to acquire multi-dimensional information about the target, such as appearance, position, and speed. They generally only carry high-resolution sensor modules (such as high-definition cameras) to acquire target video images. The fusion nodes need to perform target consistency matching based on target detection models such as FastRCNN, YOLO, YOLO-tiny, and LightGBM (LightGradient Boosting Machine) (all image-based target detection models capable of visual analysis and target surveillance information).

[0065] Based on the above analysis, this application proposes a target consistency matching method for UAV swarms capable of operating under time-varying wireless network conditions. This method aims to solve the consistency matching problem required for multi-UAV collaborative target detection, identification, and tracking under limited wireless network channels. The UAV's target detection module selects the optimal detection model based on the wireless channel conditions and its total power to process the collected information and sends the target detection results to the fusion node. Optionally, the target detection results are encapsulated using structured messages and sent to the fusion node. The fusion node performs target consistency matching based on the target detection results from each UAV, achieving collaborative information fusion on the basis of target consistency. That is, this method can call the best detection model from different detection models according to the time-varying wireless network conditions under environmental influences, generating target detection results of varying accuracy, and sending the target detection results on the channel. The fusion node aggregates the target detection results from different UAVs, i.e., the detection and identification information, and completes target consistency matching at the fusion node. It unifies the detected target identifiers (IDs) (unique target identification numbers) from different UAVs, thus completing the target consistency matching. Figure 2 The diagram shown is a structural block diagram of the target consistency matching of the UAV swarm used in the embodiments of this application.

[0066] like Figure 3 The diagram shown is a schematic representation of an embodiment of the method for target consistency matching in a drone swarm according to this application, which may include:

[0067] 301. When the UAV detects a target within a preset range, it acquires and collects information.

[0068] Optionally, the information collected may include video data or image data.

[0069] Optionally, the drone stores the collected information in a buffer.

[0070] 302. The UAV selects the best target detection model based on the current channel conditions.

[0071] Optionally, the UAV selecting the optimal target detection model based on the current channel conditions may include: the UAV selecting a detection model from a preset pool with a processing power not exceeding [a certain threshold] based on the current channel conditions. Among the detection models, the detection model with the highest processing power is the optimal detection model; where P max For the maximum airborne power, σ 2 denoted as the noise power factor, h as the channel gain from the current UAV to the server, L as the amount of data generated by the current UAV, B as the bandwidth allocated to the current UAV, and t as the duration.

[0072] Optionally, the detection model includes FastRCNN, YOLO, YOLO-tiny, and LightGBM, with corresponding processing powers including P. FRCNN P YOLO P YOLOT P LGBM .

[0073] Optionally, the processing power is sorted by P. FRCNN >P YOLO >P YOLOT >P LGBM .

[0074] This application defines a detection-based target consistency model and its required parameters for time-varying wireless network scenarios. Furthermore, based on the parameter definitions, this application designs a corresponding method for target consistency matching in UAV swarms under time-varying wireless network conditions. The target consistency model description and parameter definitions for time-varying wireless network scenarios involved in this application are as follows:

[0075] Assume a drone swarm consists of n drones, where the i-th and j-th drones detect multiple targets in the same area. Each drone is equipped with a target detection module, capable of detecting targets locally and generating detection results. The target detection modules use several common models, including Fast R-CNN, YOLO, YOLO-tiny, and LightGBM, with each model requiring a processing power of Pi for target detection. FRCNN P YOLO P YOLOT P LGBM Detection models with higher detection power have higher detection accuracy, but they also require more detection power. Their detection power ranking is P. FRCNN >P YOLO >PYOLOT >P LGBM .

[0076] The amount of data generated by the drone is L, and the data needs to be sent out within a time period t. Therefore, the rate at which the model generates and sends data is... The drone swarm communicates via wireless networking, with a total available bandwidth of B. total Let B be the bandwidth allocated to each drone in the cluster, and P be the processing power of the detection model. c The drone's transmission power is P t The maximum onboard power is P max The channel gain from the UAV to the fusion node is h, and the noise power figure is σ. 2 .

[0077] According to the formula for the maximum information transmission rate under Gaussian white noise interference (Shannon's formula), the channel maximum... To ensure normal data transmission, the transmit power P must be less than the channel's maximum transmission rate r. t Must meet That is, the transmission power P t Not less than

[0078] The drone is using airborne transmission power P t While transmitting data via the wireless channel, a target detection module is used to detect and identify the target. The detection model used has a processing power of P. c The transmit power and processing power must meet the following constraints: P t +P c ≤P max Therefore, the processing power is P. c Cannot exceed P max -P t Transmit power P t Take the lower limit Then the processing power P c The upper limit is Model selection is based on the processing power required for different detection models, ensuring that the processing power of the detection model used does not exceed [the required processing power]. In this case, the detection model can complete the normal transmission of detection and identification results through the wireless network channel while ensuring detection accuracy.

[0079] In existing technologies, due to communication limitations, drone swarms are subject to environmental interference during communication, such as terrain obstruction and communication interference. In such cases, adjustments to communication parameters are necessary to reduce communication speed and improve communication stability. Furthermore, drones do not consider channel variations when processing collected information using their onboard detection models. In this application, the information collected by high-resolution sensors is processed locally using the target detection model in the onboard target detection module, reducing data volume before transmission. The rationality of the target detection model selection is addressed. Based on the wireless network channel model, and considering the current channel conditions and onboard power, multiple detection models are selected. A reasonable target detection model is chosen to process the drone's collected information, improving target detection efficiency while ensuring data transmission. Due to energy constraints, after acquiring target information, the onboard high-resolution sensor needs to select a suitable target detection model based on channel transmission power and total onboard power. When the detection model power is limited, the optimal detection model is selected for onboard target detection.

[0080] 303. The UAV processes the collected information using the target detection algorithm of the optimal target detection model to generate target detection results.

[0081] The target detection results include at least two of the following: target classification information, target confidence information, target time information, and target location information.

[0082] 304. The drone sends the target detection results to the server, and the target detection results are used by the server for matching.

[0083] Optionally, the drone sending the target detection result to the server may include: the drone performing structured encapsulation of the target detection result and sending it. In this embodiment, the target detection processing encapsulation is convenient and simple. Existing technical solutions do not decompose and encapsulate the target detection result in multiple dimensions when sending it. In this application, when processing the target detection result, the target detection result is characterized and encapsulated into a message in multiple dimensions, such as target classification, target confidence, target time, and target location. When the server, i.e., the fusion node, performs target consistency matching, it can simply parse the message and perform target consistency matching in sequence according to the four dimensions, which simplifies the information processing encapsulation process and is simple and reliable to use.

[0084] The server receives a first target detection result sent by a first UAV and a second target detection result sent by a second UAV. The first target detection result includes at least two of the following: first target classification information, first target confidence information, first target time information, and first target location information. The second target detection result includes at least two of the following: second target classification information, second target confidence information, second target time information, and second target location information.

[0085] For example, the UAV encapsulates the target detection results into a structured message and sends the resulting target detection structured message to the server, which can also be called a fusion node. Optionally, for each UAV in the UAV cluster, such as UAV i, the structure of the target detection structured message generated by the UAV itself is shown in Table 1. The message includes information about k targets detected by the target detection model, denoted as i1, i2, ... i1. s ,…,i k , with the s-th target i s For example, the detection information obtained by UAV i through the target detection module using the target detection model can be specifically defined as information in four dimensions, which can be represented as i s (τ s,1 ,τ s,2 ,τ s,3 ,τ s,4 ); where τ s,1 For classifying information for the target, τ s,2 For target confidence information, τ s,3 For target time information, τ s,4 The target location information is used to complete the information representation of the UAV surveillance target through the above four dimensions of information.

[0086]

[0087] Table 1

[0088] Similarly, for other drones in the cluster, such as drone j, the target detection result has the same message structure as drone i, including information about l targets detected by the target detection model. Let the l targets identified by the detection model be j1, j2, ... j t ,…,j l , with the t-th target j t For example, the detection information obtained by UAV j through the target detection module using the detection model is j. t (τ t,1 ,τ t,2 ,τ t,3 ,τ t,4 ); where τ t,1 For classifying information for the target, τt,2 For target confidence information, τ t,3 For target time information, τ t,4 This refers to the target location information.

[0089] 305. The server performs target consistency matching based on the first target detection result and the second target detection result.

[0090] Optionally, the first target detection result and / or the second target detection result are structured encapsulated target detection results.

[0091] Optionally, the first target detection result includes first target classification information, first target confidence information, first target time information, and first target location information; the second target detection result includes second target classification information, second target confidence information, second target time information, and second target location information; the server performs target consistency matching based on the first target detection result and the second target detection result, which may include:

[0092] The server performs consistency matching based on the first target classification information and the second target classification information; if the first target classification information and the second target classification information match, the server performs consistency matching on the first target confidence information and the second target confidence information; if the first target confidence information and the second target confidence information match, the server performs consistency matching on the first target time information and the second target time information; if the first target time information and the second target time information match, the server performs consistency matching on the first target location information and the second target location information; if the first target location information and the second target location information match, the server determines that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0093] Optionally, when the first target classification information and the second target classification information match, the server performs consistency matching on the first target confidence information and the second target confidence information, which may include: when the first target classification information and the second target classification information are the same, the server determines that the first target classification information and the second target classification information are consistent, and performs consistency matching on the first target confidence information and the second target confidence information.

[0094] Optionally, when the first target confidence information and the second target confidence information match, the server performs consistency matching on the first target time information and the second target time information, which may include: the server calculates a first difference between the first target confidence information and the second target confidence information; if the absolute value of the first difference is greater than a confidence threshold, the server determines that the first target confidence information and the second target confidence information match, and performs consistency matching on the first target time information and the second target time information.

[0095] Optionally, when the first target time information and the second target time information match, the server performs consistency matching on the first target location information and the second target location information, which may include: the server calculates a second difference between the first target time information and the second target time information; if the absolute value of the second difference is greater than a time threshold, the server determines that the first target time information and the second target time information match, and performs consistency matching on the first target location information and the second target location information.

[0096] Optionally, when the first target location information and the second target location information match, the server determines that the targets corresponding to the first target classification information and the second target classification information are consistent, which may include: the server calculating a third difference between the first target location information and the second target location information; if the absolute value of the third difference is greater than a location threshold, the server determines that the first target location information and the second target location information are consistent, and determines that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0097] For example, the server, i.e., the fusion node, receives target detection results from drone i and drone j. Based on the target detection results, it extracts k target information i1, i2, ... i from drone i. s ,…,i k And l target information j1, j2, ... j of UAV j t ,…,j l We perform a one-by-one matching and comparison to achieve target consistency matching.

[0098] Optionally, during the matching process, it is necessary to perform matching calculations on the four dimensions of UAV target detection information: target classification information, target confidence information, target time information, and target location information. The matching calculation can be based on the following criteria: the target classification information of the two targets to be matched is the same; and the difference in target confidence between the two targets cannot exceed a confidence threshold θ, and the difference in target time cannot exceed a time threshold. The target position difference cannot exceed the position threshold μ (where the confidence threshold θ and the time threshold are also present). The position threshold μ is preset, and its value can be obtained in advance through simulation verification.

[0099] This application provides a target consistency matching method based on multi-UAV target detection results. After obtaining the parsed target detection results from different UAVs, the fusion node performs consistency matching according to different information dimensions of the target detection results. This involves four stages: comparing the classification information of multiple targets, determining the threshold for confidence difference, determining the threshold for time difference, and determining the threshold for position difference. Finally, it determines whether the different targets detected by multiple UAVs are the same target.

[0100] This application improves the efficiency of target consistency matching. Existing technologies perform target consistency matching based on target video (images) collected by the UAV, matching based on appearance similarity, position similarity, or speed similarity, which places high demands on the processing of the fusion nodes. This application fully utilizes the detection models used by the target detection modules carried by each UAV within the UAV cluster, performing target consistency matching based on the target detection results. The processing is simple and efficient; target consistency matching can be completed by matching the target detection results detected by the UAV itself across multiple dimensions, demonstrating high efficiency in matching processing.

[0101] In this embodiment, when detecting a target within a preset range, collected information is obtained; based on the current channel conditions, an optimal target detection model is selected; the collected information is processed using the target detection algorithm of the optimal target detection model to generate a target detection result, the target detection result including at least two of target classification information, target confidence information, target time information, and target location information; the target detection result is sent to a server, and the target detection result is used by the server for matching; a first target detection result sent by a first UAV is received, and a second target detection result sent by a second UAV is received, the first target detection result including at least two of first target classification information, first target confidence information, first target time information, and first target location information, and the second target detection result including at least two of second target classification information, second target confidence information, second target time information, and second target location information; target consistency matching is performed based on the first target detection result and the second target detection result. The detection model can be reasonably selected from the preset detection models according to the current channel conditions, and the UAV-collected information can be processed to obtain the target detection result. This target detection result includes multi-dimensional information, thus improving the accuracy of determining whether targets are consistent.

[0102] like Figure 4The diagram shown is a schematic representation of another embodiment of the method for target consistency matching in a drone swarm according to this application, which may include:

[0103] 401. When the UAV detects a target within a preset range, it acquires and collects information.

[0104] 402. The UAV selects the best target detection model based on the current channel conditions.

[0105] 403. The UAV processes the collected information using the target detection algorithm of the optimal target detection model to generate target detection results.

[0106] 404. The drone sends the target detection result to the server, and the target detection result is used by the server for matching.

[0107] 405. The server performs target consistency matching based on the first target detection result and the second target detection result.

[0108] 406. If the targets corresponding to the first target detection result and the second target detection result are consistent, perform information fusion on the first target detection result and the second target detection result.

[0109] For example, such as Figure 5 The diagram shown is a schematic representation of another embodiment of the method for target consistency matching in a drone swarm according to this application, which may include:

[0110] If the fusion node receives target detection structured messages from more than two UAVs, it can perform target consistency matching on the target detection information from different UAVs according to the above model and the steps below, ensuring consistent results when different UAVs detect, identify, and track the same target. The steps of a UAV cluster target consistency matching method under time-varying wireless network conditions are as follows: Steps 1 to 5 involve the UAV selecting a detection model based on the time-varying wireless network conditions and using the optimal detection model to generate target detection results for message encapsulation. Steps 7 to 12 involve the fusion node performing target consistency matching using the target detection messages from UAV i and UAV j. The specific process is as follows... Figure 4 As shown.

[0111] Step 1: Data Acquisition and Input. The drone uses its onboard high-resolution sensor to acquire target images within a specific area, places the acquired target images into a buffer, and waits for the detection model to input the data.

[0112] Step 2: Loading Time-Varying Wireless Network Parameters. Assume the total available network bandwidth for the UAV swarm is B. totalThere are n drones in the drone swarm. Calculate and evenly distribute the bandwidth of each drone. Let h be the wireless channel gain from n drones to the fusion node (which can be a drone with higher computing power in the cluster or other ground computing nodes), and σ be the noise power figure from the drones to the fusion node. 2 The amount of data generated by the drone is L, and the shortest data transmission time is T. These parameters are loaded and set in each drone within the cluster.

[0113] Step 3: Detect and load model parameters. Obtain and load the local maximum power value P. max Let the power required by different detection models of the detection module be PF. RCNN P YOLO P YOLOT P LGBM (The corresponding target detection models are FastRCNN, YOLO, YOLO-tiny, and LightGBM, respectively.) Load and set the above parameters in each drone within the drone cluster.

[0114] Step 4: Calculate the transmit power P t The lower limit and processing power P c The upper limit of the processing power P c The upper limit value is selected by comparing it with the power value of the detection model. The lower limit value of the transmit power is selected using... Calculations are performed to process the upper limit of power. Calculations are performed to determine the upper limit of processing power and P. FRCNN P YOLO P YOLOT P LGBM For comparison, among the detection models FastRCNN, YOLO, YOLO-tiny, and LightGBM, the detection model with a power value less than the upper limit of processing power and the maximum power value is selected and loaded.

[0115] Step 5: Encapsulate and send the target detection results. Use one of the loaded target detection models, namely FastRCNN, YOLO, YOLO-tiny, and LightGBM, to detect the target region image acquired in the buffer. Encapsulate the detection results using the structured message in Table 1 above, and send the encapsulated message to the fusion node through a time-varying wireless channel.

[0116] Step 6: Parse the target detection messages from different UAVs at the fusion node. Following the structure shown in Table 1, remove message headers, message lengths, etc., to extract the multiple target detection information generated by the UAVs. For example, the k targets of UAV i are i1, i2, ... i s,…,i k The drone j has l targets, namely j1, j2, ..., j t ,…,j l Each target contains four types of target detection information (τ1, τ2, τ3, τ4), which correspond to target classification information, target confidence information, target time information, and target location information, respectively.

[0117] Step 7: Match all target detection information of UAV i with all target detection information of UAV j one by one according to steps 8 to 11. For example, match the s-th target i of UAV i. s (τ s,1 ,τ s,2 ,τ s,3 ,τ s,4 ) and the t-th target j of UAV j t (τ t,1 ,τ t,2 ,τ t,3 ,τ t,4 The matching process is as follows: First, the first target detection information of UAV i is matched sequentially with all target detection information of UAV j according to steps 8-11. If a target satisfying the target consistency condition (i.e., the target classification information is the same and the target confidence information, target time information, and target position information are all below the threshold) can be found in all target detection information of UAV j, then the matching of the current target of UAV i and all targets of UAV j is completed. Further, the next target of UAV i is matched sequentially with all target detection information of UAV j according to steps 8-11. If the current target of UAV i cannot be matched with all targets of UAV j, the next target of UAV i is matched again with all target detection information of UAV j according to steps 8-11, until the matching of all k targets of UAV i and all l targets of UAV j is completed.

[0118] Step 8: Perform target consistency matching based on multiple target detection information from different UAVs. First, extract the s-th target classification information of UAV i and the t-th target classification information of UAV j, i.e., τ s,1 and τ t,1 Compare, if τ s,1 =τ t,1 , that is, τ s,1 and τ t,1 If the target classification information is completely identical, proceed to step 9 for confidence matching; otherwise, repeat step 8 to select the next target of UAV i and all targets of UAV j for consistency matching.

[0119] Step 9: Target confidence information matching based on target consistency matching of multiple target detection information from different UAVs. First, extract the confidence information of the s-th target of UAV i and the confidence information of the t-th target of UAV j, i.e., τ s,2 and τ t,2 Compare and calculate ||τ s,2 -τ t,2 ||,if||τ s,2 -τ t,2 If the value of || is greater than the confidence threshold θ, proceed to step 10 for time matching; otherwise, jump to step 8 to select the next target of UAV i and all targets of UAV j for consistency matching.

[0120] Step 10: Time matching for target consistency matching based on multiple target detection information from different UAVs. First, extract the time information of the s-th target from UAV i and the time information of the t-th target from UAV j, i.e., τ s,3 and τ t,3 Compare and calculate ||τ s,3 -τ t,3 ||,if||τ s,3 -τ t,3 The value of || is higher than the time threshold. If the position is matched, proceed to step 11; otherwise, proceed to step 8 and select the next target of UAV i and all targets of UAV j for consistency matching.

[0121] Step 11: Position matching based on target consistency matching using multiple target detection information from different UAVs. First, extract the position information of the s-th target from UAV i and the position information of the t-th target from UAV j, i.e., τ s,4 and τ t,4 Compare and calculate ||τ s,4 -τ t,4 ||,if||τ s,4 -τ t,4 If the value of || is higher than the position threshold μ, then all target consistency matching is completed, confirming that the s-th target of UAV i and the t-th target of UAV j are the same target. Otherwise, jump to step 8 and select the next target of UAV i and all targets of UAV j for consistency matching.

[0122] Step 12: If all k targets of UAV i and l targets of UAV j have achieved target consistency matching, the entire process ends. Otherwise, continue to perform target consistency matching between the remaining targets of UAV i and l targets of UAV j in sequence according to the process of steps 7 to 12, until all targets of UAV i and all targets of UAV j have achieved consistency matching.

[0123] In this embodiment, when different UAVs detect targets within a certain range, each UAV needs to analyze the current wireless network channel conditions, select the optimal target detection model, use the target detection algorithm of that model to generate a target detection result, and encapsulate and send the target detection result using a structured message. Multiple UAVs share their respective target detection results within the network. The fusion node improves the accuracy of target detection, identification, and tracking through message interaction containing target detection results within the cluster. Based on collaborative fusion, the UAV cluster target consistency monitoring method is applied when multiple UAVs detect the same target. After parsing the target detection results from different UAVs, the fusion node uses a target consistency matching algorithm to determine whether the detected targets are the same target based on feature matching and spatiotemporal calibration, and then performs information fusion (to improve target detection accuracy, etc.).

[0124] This application provides a method for selecting a UAV local detection model based on time-varying wireless channel changes. After each UAV in a UAV swarm acquires target video (image) information using a high-resolution sensor, it can select the optimal detection model to detect the acquired information based on the current channel conditions (bandwidth, channel gain, noise figure, etc.) and onboard power, while ensuring normal data transmission on the time-varying wireless channel. This application also provides a method for formatting and encapsulating UAV target detection information: after multiple UAVs in the swarm detect targets using the selected target detection model, they encapsulate the target detection results (detection information of multiple targets) according to different information dimensions, generate UAV target detection messages, and send them. The fusion node parses the target detection messages from different UAVs and then processes them locally. Optionally, this application can be designed and used in a UAV swarm time-varying wireless network simulation demonstration and verification system.

[0125] like Figure 6 The diagram shown is a schematic representation of one embodiment of the UAV in this application, which may include:

[0126] The target detection module 601 is used to obtain the current channel conditions and select the best target detection model when detecting a target within a preset range; and to generate a target detection result according to the target detection algorithm of the best target detection model, wherein the target detection result includes at least two of the following: target classification information, target confidence information, target time information, and target location information.

[0127] The transceiver module 602 is used to send the target detection result to the server, and the target detection result is used by the server for matching.

[0128] Optionally, the transceiver module 602 is specifically used to perform structured encapsulation of the target detection results and send them.

[0129] Optionally, the target detection module 601 is specifically used to select, based on the current channel conditions, a target from a preset detection model whose processing power does not exceed a certain threshold. Among the detection models, the detection model with the highest processing power is the optimal detection model;

[0130] Among them, P max For the maximum airborne power, σ 2 denoted as the noise power factor, h as the channel gain from the current UAV to the server, L as the amount of data generated by the current UAV, B as the bandwidth allocated to the current UAV, and t as the duration.

[0131] Optionally, the detection model includes FastRCNN, YOLO, YOLO-tiny, and LightGBM, with corresponding processing powers including P. FRCNN P YOLO P YOLOT P LGBM .

[0132] Optionally, the processing power is sorted by P. FRCNN >P YOLO >P YOLOT >P LGBM .

[0133] like Figure 7 The diagram shown is a schematic representation of one embodiment of the server in this application, which may include:

[0134] Transceiver 701 is used to receive a first target detection result sent by a first UAV and a second target detection result sent by a second UAV. The first target detection result includes at least two of the following: first target classification information, first target confidence information, first target time information, and first target location information. The second target detection result includes at least two of the following: second target classification information, second target confidence information, second target time information, and second target location information.

[0135] The processor 702 is used to perform target consistency matching based on the first target detection result and the second target detection result.

[0136] Optionally, the first target detection result and / or the second target detection result are structured encapsulated target detection results.

[0137] Optionally, the first target detection result includes first target classification information, first target confidence information, first target time information, and first target location information, and the second target detection result includes second target classification information, second target confidence information, second target time information, and second target location information;

[0138] The processor 702 is specifically configured to: perform consistency matching based on the first target classification information and the second target classification information; when the first target classification information and the second target classification information match, perform consistency matching on the first target confidence information and the second target confidence information; when the first target confidence information and the second target confidence information match, perform consistency matching on the first target time information and the second target time information; when the first target time information and the second target time information match, perform consistency matching on the first target location information and the second target location information; and when the first target location information and the second target location information match, determine that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0139] Optionally, the processor 702 is specifically configured to determine that the first target classification information and the second target classification information are consistent when the first target classification information and the second target classification information are the same, and to perform consistency matching on the first target confidence information and the second target confidence information.

[0140] Optionally, the processor 702 is specifically configured to calculate a first difference between the first target confidence information and the second target confidence information, and if the absolute value of the first difference is greater than a confidence threshold, determine that the first target confidence information and the second target confidence information are consistent, and perform consistency matching on the first target time information and the second target time information.

[0141] Optionally, the processor 702 is specifically configured to calculate a second difference between the first target time information and the second target time information, and if the absolute value of the second difference is greater than a time threshold, determine that the first target time information and the second target time information are consistent, and perform consistency matching on the first target location information and the second target location information.

[0142] Optionally, the processor 702 is specifically configured to calculate a third difference between the first target location information and the second target location information, and if the absolute value of the third difference is greater than a location threshold, determine that the first target location information and the second target location information are consistent, and determine that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0143] Optionally, the processor 702 is further configured to perform information fusion on the first target detection result and the second target detection result when the targets corresponding to the first target detection result and the second target detection result are consistent.

[0144] like Figure 8 The diagram shown is a schematic representation of another embodiment of the drone in this application, which may include:

[0145] Memory 801 storing executable program code;

[0146] A processor 802 and a transceiver 803 are coupled to the memory 801;

[0147] The processor 802 is configured to, when detecting a target within a preset range, acquire the current channel conditions, select the optimal target detection model, and generate a target detection result based on the target detection algorithm of the optimal target detection model. The target detection result includes at least two of the following: target classification information, target confidence information, target time information, and target location information.

[0148] Transceiver 803 is used to send the target detection result to the server, and the target detection result is used by the server for matching.

[0149] Optionally, transceiver 803 is specifically used to perform structured encapsulation of the target detection results and send them.

[0150] Optionally, the processor 802 is specifically configured to select, based on the current channel conditions, a detection model from a preset set of models with a processing power not exceeding [a certain threshold]. Among the detection models, the detection model with the highest processing power is the optimal detection model;

[0151] Among them, P max For the maximum airborne power, σ 2 denoted as the noise power factor, h as the channel gain from the current UAV to the server, L as the amount of data generated by the current UAV, B as the bandwidth allocated to the current UAV, and t as the duration.

[0152] Optionally, the detection model includes FastRCNN, YOLO, YOLO-tiny, and LightGBM, with corresponding processing powers including P. FRCNN P YOLO P YOLOT P LGBM .

[0153] Optionally, the processing power is sorted by P. FRCNN >P YOLO >P YOLOT >P LGBM .

[0154] like Figure 9 The diagram shown is a schematic representation of another embodiment of the server in this application, which may include:

[0155] Memory 901 storing executable program code;

[0156] A processor 902 and a transceiver 903 are coupled to a memory 901;

[0157] Transceiver 903 is used to receive a first target detection result sent by a first UAV and a second target detection result sent by a second UAV. The first target detection result includes at least two of the following: first target classification information, first target confidence information, first target time information, and first target location information. The second target detection result includes at least two of the following: second target classification information, second target confidence information, second target time information, and second target location information.

[0158] The processor 902 is used to perform target consistency matching based on the first target detection result and the second target detection result.

[0159] Optionally, the first target detection result and / or the second target detection result are structured encapsulated target detection results.

[0160] Optionally, the first target detection result includes first target classification information, first target confidence information, first target time information, and first target location information, and the second target detection result includes second target classification information, second target confidence information, second target time information, and second target location information;

[0161] The processor 902 is specifically configured to: perform consistency matching based on the first target classification information and the second target classification information; when the first target classification information and the second target classification information match, perform consistency matching on the first target confidence information and the second target confidence information; when the first target confidence information and the second target confidence information match, perform consistency matching on the first target time information and the second target time information; when the first target time information and the second target time information match, perform consistency matching on the first target location information and the second target location information; and when the first target location information and the second target location information match, determine that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0162] Optionally, the processor 902 is specifically configured to determine that the first target classification information and the second target classification information are consistent when the first target classification information and the second target classification information are the same, and to perform consistency matching on the first target confidence information and the second target confidence information.

[0163] Optionally, the processor 902 is specifically configured to calculate a first difference between the first target confidence information and the second target confidence information, and if the absolute value of the first difference is greater than a confidence threshold, determine that the first target confidence information and the second target confidence information are consistent, and perform consistency matching on the first target time information and the second target time information.

[0164] Optionally, the processor 902 is specifically configured to calculate a second difference between the first target time information and the second target time information, and if the absolute value of the second difference is greater than a time threshold, determine that the first target time information and the second target time information are consistent, and perform consistency matching on the first target location information and the second target location information.

[0165] Optionally, the processor 902 is specifically configured to calculate a third difference between the first target location information and the second target location information, and if the absolute value of the third difference is greater than a location threshold, determine that the first target location information and the second target location information are consistent, and determine that the targets corresponding to the first target classification information and the second target classification information are consistent.

[0166] Optionally, the processor 902 is further configured to perform information fusion on the first target detection result and the second target detection result when the targets corresponding to the first target detection result and the second target detection result are consistent.

[0167] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for target consistency matching in a drone swarm, characterized in that, The method is applied to a drone, and the method includes: When detecting a target within a preset range, acquire collected information; Select the optimal target detection model based on the current channel conditions; The step of selecting the optimal target detection model based on the current channel conditions includes: Based on the current channel conditions, select a detection model from the preset models with a processing power not exceeding [a certain value]. Among the detection models, the detection model with the highest processing power is the optimal target detection model; in, This is the maximum onboard power. The noise power factor is... This represents the current channel gain from the drone to the server. The amount of data generated by the current drone. The bandwidth allocated to the current drone, Duration; The collected information is processed by the target detection algorithm of the optimal target detection model to generate target detection results, which include at least two of the following: target classification information, target confidence information, target time information, and target location information. The target detection result is sent to the server, and the target detection result is used by the server to perform target consistency matching.

2. The method according to claim 1, characterized in that, Sending the target detection result to the server includes: The target detection results are structured, encapsulated, and sent.

3. The method according to claim 1 or 2, characterized in that, The detection models include FastR-CNN, YOLO, YOLO-tiny, and LightGBM, with corresponding processing powers of [list of powers to be filled in]. , , , .

4. The method according to claim 3, characterized in that, The processing power is sorted as follows 。 5. A drone, characterized in that, include: The target detection module is used to acquire collected information and select the best target detection model based on the current channel conditions when detecting targets within a preset range. The collected information is processed by the target detection algorithm of the optimal target detection model to generate target detection results, which include at least two of the following: target classification information, target confidence information, target time information, and target location information. The target detection module selects the optimal target detection model based on the current channel conditions in the following ways: The target detection module is used to select, from a preset detection model, a target with a processing power not exceeding a certain threshold, based on the current channel conditions. Among the detection models, the detection model with the highest processing power is the optimal target detection model; whereby... This is the maximum onboard power. The noise power factor is... This represents the current channel gain from the drone to the server. The amount of data generated by the current drone. The bandwidth allocated to the current drone, Duration; The transceiver module is used to send the target detection result to the server, and the target detection result is used by the server to perform target consistency matching.

6. A drone, characterized in that, include: Memory containing executable program code; A processor and a transceiver coupled to the memory; The processor and the transceiver are used to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium comprising instructions that, when executed on a processor, cause the processor to perform the method as described in any one of claims 1-4.

Images