Visible light unmanned aerial vehicle air situation positioning method, device and equipment and storage medium

Abstract

The invention discloses a visible light unmanned aerial vehicle air situation positioning method, device and equipment and a storage medium, and relates to the technical field of target detection, and the method comprises the steps: training a target detection model, and deploying the target detection model to ground monitoring equipment and at least two optical cameras; shooting the target airspace through the ground monitoring equipment to obtain a first image, and recognizing the unmanned aerial vehicle according to the first image by using the target detection model; shooting the target airspace by using each optical camera to obtain a second image, and judging and identifying whether the unmanned aerial vehicle exists in the target airspace according to the second image by using the target detection model; and if yes, identifying the position of the unmanned aerial vehicle according to the positions of the ground monitoring equipment and the optical cameras. By training and deploying the target detection model, the position of the unmanned aerial vehicle can be detected only through at least two optical cameras except for the ground monitoring equipment, the cost of positioning the unmanned aerial vehicle can be greatly reduced, and the position of the unmanned aerial vehicle can be accurately recognized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a visible light unmanned aerial vehicle air situation positioning method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of low-altitude economy, how to regulate airspace has become a big problem for maintaining public safety. Most of the existing unmanned aerial vehicle identification and positioning technologies use the reflection characteristics of radio waves to determine the position of the unmanned aerial vehicle by receiving the radio signals emitted by the unmanned aerial vehicle. This technology is called radio frequency scanning technology, but the hardware cost of such equipment is high. Another method is to use a visible light identification detection algorithm to identify the unmanned aerial vehicle in the picture by using deep learning, but this method can only identify whether there is an unmanned aerial vehicle, and cannot locate the position of the unmanned aerial vehicle. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a visible light unmanned aerial vehicle air situation positioning method, device, equipment and storage medium to reduce the cost of identifying the position of the unmanned aerial vehicle and accurately identify the position of the unmanned aerial vehicle.

[0004] To achieve the above purpose, one aspect of the embodiments of the present application provides a visible light unmanned aerial vehicle air situation positioning method, which comprises the following steps: training a target detection model and then deploying the target detection model to a ground monitoring device and at least two optical cameras respectively; wherein the positions of the ground monitoring device and each optical camera are different; capturing a first image of a target airspace by the ground monitoring device and identifying an unmanned aerial vehicle from the first image by using the target detection model; capturing a second image of the target airspace by each optical camera and judging whether there is an unmanned aerial vehicle in the target airspace according to the second image by using the target detection model; if yes, identifying the position of the unmanned aerial vehicle according to the positions of the ground monitoring device and each optical camera.

[0005] In some embodiments, the training of the target detection model comprises the following steps: obtaining an unmanned aerial vehicle image data set; improving an original yolov11 target detection model to obtain an improved yolov11 target detection model as the target detection model; constructing a first loss function for optimizing the classification accuracy of the target class and a second loss function for optimizing the position of the detection frame, and then constructing a final loss function; training the improved yolov11 target detection model according to the unmanned aerial vehicle image dataset, the first loss function and the second loss function.

[0006] In some embodiments, the unmanned aerial vehicle image dataset is obtained, including the following steps: obtaining a third image including a bird and an unmanned aerial vehicle; obtaining a fourth image containing an unmanned aerial vehicle by a non-special camera device; obtaining a fifth image including an interference term and a random background; obtaining a sixth image of an unmanned aerial vehicle under different weather, different light conditions, different detection distances and different models; obtaining an initial image dataset by blurring and adding noise points to the third image, the fourth image, the fifth image and the sixth image; obtaining the unmanned aerial vehicle image dataset by labeling the detection frame of the unmanned aerial vehicle in the initial image dataset.

[0007] In some embodiments, the original yolov11 target detection model is improved to obtain an improved yolov11 target detection model as the target detection model, including the following steps: replacing the C2PSA module of the backbone network in the original yolov11 target detection model with a C2LSKA module; replacing the PSA module of the backbone network in the original yolov11 target detection model with a LSKA module; wherein the backbone network of the improved yolov11 target detection model further comprises a C3K2 structure, an SPPF structure and a detection head structure.

[0008] In some embodiments, the first loss function for optimizing the classification accuracy of the target class and the second loss function for optimizing the position of the detection frame are constructed, and then the final loss function is constructed, including the following steps: the first loss function is constructed as follows: ; wherein, the first loss function is represented by L, a coefficient for balancing the positive and negative sample weights; represents the predicted probability of the target detection model for the correct class; represents the focusing parameter for adjusting the difficulty sample weight; the second loss function is constructed as follows: ; ; ; ; ; ; ; wherein, represents the second loss function; represents a detection frame predicted by the target detection model, represents a real detection frame, and respectively represent the width and height of the real detection frame, and scale is a scaling factor; and respectively represent the x-axis coordinates of the center points of the predicted detection frame and the real detection frame, and respectively represent the y-axis coordinates of the center points of the predicted detection frame and the real detection frame; and the final loss function of the target detection model is constructed as: ; wherein, represents the final loss function, , are balance factors.

[0009] In some embodiments, the step of identifying a UAV from the image by using the target detection model comprises the following steps: annotating a detection frame for a UAV on the image by using the target detection model; determining a confidence degree that the detection frame contains a UAV; if the confidence degree reaches a preset threshold, determining that the image contains a UAV.

[0010] In some embodiments, the step of identifying the position of the UAV according to the positions of the ground monitoring device and each optical camera comprises the following steps: drawing corresponding rays according to the positions, azimuth angles and elevation angles of the ground monitoring device and each optical camera; solving the points closest to each ray in the space as the position of the UAV.

[0011] To achieve the above object, another aspect of the embodiment of the present application proposes a visible light UAV air situation positioning device, which comprises: a model training and deployment unit, configured to train a target detection model and then deploy the target detection model to a ground monitoring device and at least two optical cameras, respectively; wherein the ground monitoring device and each of the optical cameras are located at different locations; A preliminary identification unit, configured to obtain a first image by photographing the target airspace with the ground monitoring equipment and identify a UAV based on the first image using the target detection model; a confirmation and identification unit, configured to use each of the optical cameras to respectively photograph the target airspace to obtain a second image and use the target detection model to determine whether a drone exists in the target airspace based on the second image; A drone positioning unit is used to obtain the position of the drone based on the position of the ground monitoring equipment and each of the optical cameras if the target detection model is used to determine the presence of a drone in the target airspace based on the second image.

[0012] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0013] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned method when executed by a processor.

[0014] The embodiments of the present application include at least the following beneficial effects: The present application can train a target detection model and then deploy the target detection model to a ground monitoring device and at least two optical cameras respectively; wherein the ground monitoring device and each optical camera are located at different locations; the ground monitoring device shoots the target airspace to obtain a first image and uses the target detection model to identify a drone based on the first image; each optical camera shoots the target airspace to obtain a second image and uses the target detection model to determine whether a drone exists in the target airspace based on the second image; if so, the position of the drone is obtained based on the position identification of the ground monitoring device and each optical camera. By training and deploying the target detection model, the present application can detect the position of the drone using at least two optical cameras in addition to the ground monitoring device, which can greatly reduce the cost of locating the drone, and can accurately identify the position of the drone based on the ground monitoring device and each optical camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

[0016] Figure 1 A flowchart of a visible light unmanned aerial vehicle air situation positioning method provided by an embodiment of the present application is shown in the figure. Fig. 2 (a)~Fig. 2 (e) are respectively unmanned aerial vehicle example diagrams of different data sets provided by an embodiment of the present application. Figure 3 An example diagram of an image after blur processing provided by an embodiment of the present application is shown in the figure. Figure 4 An example diagram of an image after adding noise provided by an embodiment of the present application is shown in the figure. Figure 5 A schematic diagram of an original yolov11 target detection model structure provided by an embodiment of the present application is shown in the figure. Figure 6 A schematic diagram of an improved yolov11 target detection model structure provided by an embodiment of the present application is shown in the figure. Figure 7 A schematic diagram of a C3K2 structure provided by an embodiment of the present application is shown in the figure. Figure 8 A schematic diagram of an SPPF structure provided by an embodiment of the present application is shown in the figure. Figure 9 A schematic diagram of a C2LSKA structure provided by an embodiment of the present application is shown in the figure. Figure 10 A schematic diagram of a detection head structure provided by an embodiment of the present application is shown in the figure. Figure 11 A schematic diagram of marking a ray in an image provided by an embodiment of the present application is shown in the figure. Fig. 12 (a), Fig. 12 (b) are respectively front view and top view diagrams for solving a target point position by using two rays provided by an embodiment of the present application. Fig. 13 (a), Fig. 13 (b) are respectively front view and top view diagrams for solving a target point position by using three rays provided by an embodiment of the present application. Figure 14 A structural schematic diagram of a visible light unmanned aerial vehicle air situation positioning device provided by an embodiment of the present application is shown in the figure. Figure 15 A hardware structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, further described are the present application in combination with the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not intended to limit the present application. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with embodiments of the present application. They are merely examples of apparatuses and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0018] It can be understood that the terms “first”, “second” and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, without departing from the scope of the embodiments of the present application. Depending on the context, the word “if” as used herein can be interpreted as “when” or “upon determination” or “in response to determining”.

[0019] The terms “at least one”, “multiple”, “each”, “any” and the like used in the present application include one, two or more than two, multiple includes two or more than two, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0021] The embodiment of the present application provides a visible light unmanned aerial vehicle air and space positioning method, device and equipment and storage medium, relates to the target detection technical field. The embodiment of the present application provides a visible light unmanned aerial vehicle air and space positioning method, device and equipment and storage medium, which can be applied to a terminal, can also be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch and a vehicle-mounted terminal, but is not limited thereto; the server end can be configured as an independent physical server, can also be configured as a server cluster or a distributed system formed by multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN and big data and artificial intelligence platform, and the server can also be a node server in a blockchain network; the software can be an application for implementing a visible light unmanned aerial vehicle air and space positioning method, but is not limited to the above forms.

[0022] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0023] Reference Figure 1 The embodiment of the present application provides a visible light unmanned aerial vehicle air and space positioning method, which can include but is not limited to S100 to S130, and specifically as follows: S100: training a target detection model and then deploying the target detection model to a ground monitoring device and at least two optical cameras respectively; wherein the ground monitoring device and each of the optical cameras are located at different positions; S110: obtaining a first image by photographing a target airspace through the ground monitoring device, and identifying an unmanned aerial vehicle according to the first image by using the target detection model; S120: capturing a second image of the target space by each optical camera and determining whether a UAV exists in the target space according to the second image by using the target detection model; S130: if yes, determining the position of the UAV according to the positions of the ground monitoring device and each optical camera.

[0024] Optionally, the training of the target detection model comprises the following steps: obtaining a UAV image dataset; improving an original yolov11 target detection model to obtain an improved yolov11 target detection model as the target detection model; constructing a first loss function for optimizing the classification accuracy of target categories and a second loss function for optimizing the position of a detection frame, and then constructing a final loss function; training the improved yolov11 target detection model according to the UAV image dataset, the first loss function and the second loss function.

[0025] Optionally, the obtaining of the UAV image dataset comprises the following steps: obtaining a third image including birds and UAVs; capturing a fourth image containing a UAV by a non-special camera device; obtaining a fifth image including interference items and random backgrounds; obtaining a sixth image of a UAV under different weather conditions, different light conditions, different detection distances and different models; blurring and adding noise points to the third image, the fourth image, the fifth image and the sixth image to obtain an initial image dataset; annotating a detection frame of a UAV in the initial image dataset to obtain the UAV image dataset.

[0026] Optionally, the improving of the original yolov11 target detection model to obtain the improved yolov11 target detection model as the target detection model comprises the following steps: replacing a C2PSA module of a backbone network in the original yolov11 target detection model with a C2LSKA module; replacing a PSA module of the backbone network in the original yolov11 target detection model with a LSKA module; wherein the backbone network of the improved yolov11 target detection model further comprises a C3K2 structure, an SPPF structure and a detection head structure.

[0027] Optionally, the first loss function for optimizing the target category classification accuracy and the second loss function for optimizing the detection frame position are constructed, and then the final loss function is constructed, comprising the following steps: The first loss function is constructed as follows: ; wherein, represents the first loss function, represents a coefficient for balancing the positive and negative sample weights; represents the prediction probability of the target detection model for the correct category; represents a focusing parameter for adjusting the difficult sample weight; The second loss function is constructed as follows: ; ; ; ; ; ; ; wherein, represents the second loss function; represents the detection frame predicted by the target detection model, represents the real detection frame, and respectively represent the width and height of the real detection frame, and scale is a scaling factor; and respectively represent the x-axis coordinates of the center points of the predicted detection frame and the real detection frame, and respectively represent the y-axis coordinates of the center points of the predicted detection frame and the real detection frame; The final loss function of the target detection model is constructed as follows: ; wherein, represents the final loss function, , are both balancing factors.

[0028] Optionally, the step of identifying a UAV from the image by using the target detection model comprises the following steps: annotating a detection frame for the UAV on the image by using the target detection model; determining the confidence that the UAV is contained in the detection frame; If the confidence reaches a preset threshold, it is determined that the image contains a UAV.

[0029] Optionally, the position of the UAV is identified according to the positions of the ground monitoring device and each optical camera, and includes the following steps: According to the positions, azimuth angles and elevation angles of the ground monitoring device and each optical camera, corresponding rays are drawn respectively. The distances from each ray to the nearest point in the space are solved as the position of the UAV.

[0030] Next, the scheme of the embodiments of the present application will be described and explained in detail in conjunction with specific application examples.

[0031] The present embodiment aims to provide a low-cost UAV positioning scheme, which can complete the positioning of the UAV by relying on only two optical cameras at least; combining deep learning algorithm with traditional algorithm can not only identify the model of the UAV, but also position the position of the UAV.

[0032] Specifically, the present embodiment can include the following technical solutions: 1. Data acquisition.

[0033] Firstly, the relevant data of the unmanned aerial vehicle is collected based on the online open source dataset, which includes Drone vs Birds, drone tracking datasets, Cranfield-Synthetic-Drone-Detection, DroneDataset, etc. In addition to rich unmanned aerial vehicle and environmental data, the Drone-vs-Bird dataset also contains some bird data. When the unmanned aerial vehicle is far away, it is similar to the bird in appearance. Therefore, the emergence of this dataset can help researchers to study the problem of distinguishing unmanned aerial vehicles and birds at a distance; drone tracking datasets are mainly used to study the tracking and positioning technology of unmanned aerial vehicles, but they also contain multiple consumer camera (smartphone, gopro, etc.) images of unmanned aerial vehicles, which also meet the ground perspective requirements of this project; Cranfield-Synthetic-Drone-Detection uses structured domain randomization to generate a completely synthetic dataset of unmanned aerial vehicles. It contains multiple datasets generated using different styles: unmanned aerial vehicles only, unmanned aerial vehicles and birds, general distractors, realistic distractors, random backgrounds, etc. At the same time, this dataset also simulates the size of the unmanned aerial vehicle at different distances (20 meters, 40 meters, 80 meters, 160 meters, 320 meters); Drone Dataset is mainly prepared for academic and competition purposes, but it also contains about 1400 unmanned aerial vehicle images and label files. Most of the images come from Google and Yandex image search tools, which are obtained through image (network) crawlers, and other images are cropped from unmanned aerial vehicle videos downloaded from YouTube.

[0034] However, this method can only cover part of the unmanned aerial vehicle models, so this embodiment also manually collects unmanned aerial vehicle related materials, including unmanned aerial vehicle visible light images under different weather conditions, different light conditions, different detection distances, different models, etc. The dataset examples are shown in FIG. 2. Among them, FIG. 2 (a)~FIG. 2 (e) are respectively Drone-vs-Bird dataset examples, drone tracking datasets dataset examples, Cranfield-Synthetic-Drone-Detection dataset examples, Drone Dataset dataset examples, and manually collected unmanned aerial vehicle image examples.

[0035] 2, data augmentation and data labeling.

[0036] In actual work, due to the movement of the drone and the influence of light on the acquisition equipment, the real-time images captured will be motion blurred and may contain noise. In addition, the data in the dataset are relatively clear. Therefore, this embodiment uses the method of randomly adding blur and noise to perform data enhancement. The image in Figure 2 (e) is blurred. Figure 3 As shown in Figure 2(e), after adding noise, Figure 4 shown.

[0037] Using X-anylabeling software, we annotated the dataset images using the YOLO object detection format. We then divided the annotated data into a training set, a validation set, and a test set in an 8:1:1 ratio. We then trained and tested the model based on this dataset.

[0038] 3. Model improvement.

[0039] The model of this embodiment is improved based on the yolov11 algorithm. The original yolov11 target detection model structure is as follows Figure 5 shown.

[0040] This embodiment improves the C2PSA module and adopts the C2LSKA module with smaller computational complexity but higher performance. The PSA module is modified into the LSKA module. The improved yolov11 target detection model structure is as follows: Figure 6 shown.

[0041] First, the input is processed by the first and second 3×3 convolution blocks, where the 3×3 convolution block consists of a regular convolution with a kernel of 3×3, a stride of 2, and a padding of 1, a batch normalization layer, and a SiLU activation function. Then it is processed by the first C3K2 module, where the network structure of C3K2 is as follows: Figure 7 shown.

[0042] The input of the C3K2 module is first processed by a 3×3 convolution block (same as above), and then undergoes a channel split operation. Part of it is directly input to the splicing layer, and the other part is directly input to the splicing layer and also processed by two layers of C3K modules. The result of the first layer of C3K module also participates in the splicing operation. Finally, the spliced ​​result is processed by another 3×3 convolution block to obtain the final output. The C2K module structure Figure 7 It is also shown that the input is processed by two 3×3 convolution blocks on the left and right respectively. The output of the left 3×3 convolution block is processed by two bottleneck modules and then input to the splicing layer. The output of the right 3×3 convolution block is directly input to the splicing layer. The spliced ​​output is processed by the second 3×3 convolution block to obtain the final output. The structure of the bottleneck layer is as follows Figure 7 As shown in the left part of , the input is first processed by two layers of 3×3 convolution blocks, and then added through jump connections to obtain the final output.

[0043] The output of the first C3K2 module is input to the second C3K2 module after being processed by the third 3x3 convolution block, and then input to the fourth 3x3 convolution block, and then processed by the third C3K2 module, and then input to the SPPF layer. The structure of the SPPF layer is as shown in Figure 8 .

[0044] The input of the SPPF is first processed by a 1x1 convolution block, which includes a 1x1 regular convolution, a batch normalization layer, and a SiLU activation function layer. The output of the 1x1 convolution block is directly input to the concatenation layer and also processed by three max pooling layers with a pool kernel size of 5x5. The output of each pooling layer is input to the concatenation layer. The output of the concatenation layer is processed by another 1x1 convolution block to obtain the final output.

[0045] The output of the SPPF layer is processed by the C2LSKA layer, and then by the first upsampling layer. The structure of the C2LSKA module is as shown in Figure 9 .

[0046] The input of the C2LSKA is processed by a 1x1 regular convolution and a batch normalization layer. The result is input to the concatenation layer through a skip connection and also processed by N LSKA layers, where N is 2. The output of the second LSKA layer is also input to the concatenation layer. The output of the concatenation layer is processed by a 1x1 regular convolution and a batch normalization layer to obtain the final output. The structure of the LSKA is taken from the paper "Large Separable Kernel Attention: Rethinking the Large Kernel Attention Design in CNN", and the structure is as shown in the right part of Figure 9 . First, it is processed by a first depth separable convolution with a kernel size of 1x3 to extract horizontal direction features. Then it is processed by a second depth separable convolution with a kernel size of 3x1 to extract vertical direction features. Then it is processed by a first depth separable dilated convolution with a dilation rate of 2 and a kernel size of 1x3. Then it is processed by a second depth separable dilated convolution with a dilation rate of 2 and a kernel size of 3x1. Then it is processed by a 1x1 regular convolution layer. The processing result is Hadamard multiplied with the output to obtain the final output.

[0047] The output of the first upsampling layer is spliced ​​with the output of the second C3K2 module, and then processed by the fourth C3K2 module. After processing, it is input to the second upsampling layer, and then spliced ​​with the output of the first C3K2. After splicing, it is input to the fifth C3K2 layer. The output of the fifth C3K2 layer is directly input to the first detection head and input to the fifth 3×3 convolution block for processing. After processing, it is spliced ​​with the output of the fourth C3K2 and input to the sixth C3K2 module. The output of the sixth C3K2 module is input to the second detection head and also input to the sixth 3×3 convolution block. The output of the sixth 3×3 convolution is spliced ​​with the output of the C2LSKA layer, and then input to the seventh C3K2 module for processing. After processing, it is input to the third detection head. The structure of the detection head is as follows: Figure 10 shown.

[0048] The input of the detection head is processed on the left and right sides respectively. The left side first undergoes two 3×3 convolutions with a stride and padding of 1, and then undergoes a 1×1 convolution with a stride of 1 and padding of 0 to obtain the final output; the right side first undergoes a depthwise separable convolution with a stride and padding of 1 and a convolution kernel of 3×3, and then undergoes a conventional convolution with a stride and padding of 1 and a convolution kernel of 1×1. After processing, it is input to the second depthwise separable convolution with a stride and padding of 1 and a convolution kernel of 3×3, and then undergoes a second conventional convolution with a stride and padding of 1 and a convolution kernel of 1×1, and finally undergoes a 1×1 convolution with a stride of 1 and padding of 0 to obtain the final output.

[0049] 4. Model training and deployment.

[0050] Model training mainly includes two loss functions. One is Focal Loss, which is responsible for the accuracy of the target category. It is a loss function used to solve the problem of class imbalance, especially in target detection. It adjusts the weights of difficult and easy samples so that the model pays more attention to difficult-to-classify samples and suppresses simple samples. Its loss function is: ; Among them is The coefficient for balancing the weights of positive and negative samples is 0.2 here. Represents the model's predicted probability for the correct category, defined as: ; To adjust the focus parameter of the weight of difficult and easy samples, it is taken as 0.5 here.

[0051] Another loss function is responsible for optimizing the position of the detection box. The Shape-IoU loss function is used in this embodiment. Existing detection box regression methods usually consider the geometric relationship between the true GT (Ground Truth) box and the predicted box, calculate the loss by the relative position and shape of the detection box, but ignore the inherent properties of the detection box itself such as shape and scale. In order to make up for the shortcomings of existing research, Shape-IoU proposes a detection box regression method that focuses on the shape and scale of the detection box itself. The Shape-IoU loss function expression includes: ; ; ; ; ; ; wherein represents the detection box predicted by the model, represents the true detection box, and respectively represent the width and height of the true detection box, scale is a scaling factor related to the target size in the data set, and respectively represent the x-axis coordinates of the center points of the predicted detection box and the true detection box, and respectively represent the y-axis coordinates of the center points of the predicted detection box and the true detection box, wherein is defined as follows: ; Therefore, the final loss function is: ; wherein , are balance factors, and their values are 1.0 and 7.5. During model training, the model weight with the minimum loss error is saved, converted from pytorch format to onnx format, and then converted to tensorrt format for deployment on a camera chip.

[0052] Next, the application will be described in more detail in a more specific embodiment.

[0053] First, the trained target detection model is deployed to the chip of the ground monitoring device. In operation, the real-time picture of the ground monitoring device is input to the trained target detection model. When a suspected unidentified flying object is detected using the trained target detection model, the zoom function is started. When the target confidence (i.e., the probability that the algorithm considers it to be a drone) is greater than 0.5, the terminal platform is used to mobilize nearby optical cameras for joint search. After the nearby optical cameras receive the joint signal, the gimbal is pointed to the target airspace. When some of the optical cameras (at least two) also detect a drone, the air defense positioning algorithm is started.

[0054] The air defense positioning principle is as follows: first, based on the GPS coordinates of the ground monitoring device, several rays are drawn in combination with the azimuth and elevation angles of the gimbal, as shown in FIG. 11. Figure 11

[0055] Then, the rays are combined in pairs without repetition, for example, A, B, and C are combined in AB, AC, and BC. Then, the point closest to the two rays in space is solved. At this time, the principle of the common perpendicular is introduced, for example, the principle of solving the point closest to the A and B rays is as follows: first, the point E closest to the A ray is solved, and then the point F closest to the B ray is solved. The coordinate expression of point E is as follows: ; where represents the vector AB, A represents point A, B represents point B, v is the direction vector of AE, and w is the direction vector of BF. represents the dot product, represents the cross product. The calculation formula of point F is as follows: ; The coordinates of the point P closest to the two rays are as follows: ; where x, y, and z represent the sizes of the spatial coordinates x, y, and z of the point, respectively. When there are only two rays, the coordinates of point P are the position of the drone, as shown in FIGS. 12(a) and 12(b). FIGS. 12(a) and 12(b) are a front view and a top view of solving the target point position using two rays, respectively.

[0056] When there are three available rays, the points P AB , P AC , and P BC are solved in pairs, and the coordinates of the point P closest to the three rays are as follows: ; ​The x, y, and z of P AB , P AC , P BC are averaged, that is, the final target is obtained, and the effect is shown in FIG. 13. Among them, FIG. 13(a) and FIG. 13(b) are respectively an elevation view of solving the target point position by using three rays and a top view of solving the target point position by using three rays.

[0057] Similarly, four or more rays can be solved according to this logic. The method of the embodiment greatly saves the equipment cost, and only relies on at least two ground cameras to complete the positioning of the unmanned aerial vehicle.

[0058] In summary, the beneficial effects of the embodiment include: at least only two optical cameras can be used to complete the air situation positioning of the unidentified flying object, greatly reducing the equipment cost; the target detection model is improved, the recognition accuracy is improved, and the calculation amount is reduced; the training loss function is improved, the detection accuracy is improved; the air situation positioning algorithm is used to reduce the difficulty of solving, and the positioning speed of the unidentified flying object is improved.

[0059] With reference to Figure 14 , the application also provides a visible light unmanned aerial vehicle air situation positioning device, which can implement the above-mentioned visible light unmanned aerial vehicle air situation positioning method. The device comprises: a model training and deployment unit configured to train a target detection model and then deploy the target detection model to a ground monitoring device and at least two optical cameras; wherein the ground monitoring device and each of the optical cameras are located at different positions; a preliminary identification unit configured to obtain a first image by photographing a target airspace by the ground monitoring device and identify an unmanned aerial vehicle from the first image by using the target detection model; a confirmation identification unit configured to obtain a second image by photographing the target airspace by each of the optical cameras and determine whether the target airspace contains an unmanned aerial vehicle by using the target detection model to judge the second image; an unmanned aerial vehicle positioning unit configured to obtain the position of the unmanned aerial vehicle according to the positions of the ground monitoring device and each of the optical cameras if the target airspace contains an unmanned aerial vehicle is determined by using the target detection model to judge the second image.

[0060] It can be understood that the contents in the above method embodiments are applicable to the device embodiments, the device embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0061] The embodiment of the present application further provides an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor executes the computer program to realize the method of the embodiment of the present application. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0062] It can be understood that the contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically realize the functions of the method of the present application, and achieve the same beneficial effects as the method of the present application.

[0063] Please refer to Figure 15 , Figure 15 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises: The processor 1501 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of the present application. The memory 1502 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 1502 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1502 and are called and executed by the processor 1501 to realize the method of the present application. The input / output interface 1503 is used to realize information input and output. The communication interface 1504 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.). The bus 1505 is used to transmit information between various components (such as the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504) of the device. The processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504 are connected to each other through the bus 1505 to realize the communication connection between them in the device.

[0064] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the present application.

[0065] It can be understood that the contents in the above method embodiments are all applicable to the present storage medium embodiment, the present storage medium embodiment specifically implements the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments.

[0066] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0067] The embodiments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0068] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0069] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment.

[0070] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0071] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological mundane operation, reverse order operation, based on circuitry availability, based on stated preference or the like, and that "default" or other orderings are thus permissible. Further, the terms "comprise", "comprising", "include", "including", and the like, are specifically intended to be open-ended. That is, references to individual steps and the like do not suhstantially exclude the presence of two or more of a recited step or its integral sub-steps or additional steps whether or not readily ascertainable from the description or the like. Further, the words "a" or "an", as used herein in the disclosure and elsewhere, are used indiscriminately and are to be interpreted in the same way, i.e. as meaning "one or more".

[0072] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or", used to describe the relationship between associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or the like means any combination of these items, including single or multiple combinations. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0073] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0074] The units described above as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0075] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0076] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0077] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A visible light UAV air situation positioning method, characterized in that: The method comprises the following steps: Training a target detection model and then deploying the target detection model on a ground monitoring device and at least two optical cameras, respectively; wherein the ground monitoring device and each of the optical cameras are located at different locations; Using the ground monitoring equipment to capture the target airspace to obtain a first image and using the target detection model to identify the drone based on the first image; Using each of the optical cameras to capture the target airspace to obtain a second image, and using the target detection model to determine whether a drone exists in the target airspace based on the second image; If so, the position of the UAV is obtained according to the position identification of the ground monitoring equipment and each of the optical cameras.

2. The visible light UAV air situation positioning method according to claim 1 is characterized in that: The training target detection model includes the following steps: Obtain a drone image dataset; The original yolov11 target detection model is improved to obtain an improved yolov11 target detection model as the target detection model; Constructing a first loss function for optimizing the accuracy of target category classification and a second loss function for optimizing the position of the detection box, and then constructing a final loss function; The improved yolov11 target detection model is trained according to the drone image dataset, the first loss function and the second loss function.

3. The visible light UAV air situation positioning method according to claim 2 is characterized in that: The method of obtaining a drone image dataset includes the following steps: Acquire a third image that includes both the bird and the drone; capturing a fourth image containing the drone through a non-dedicated camera device; acquiring a fifth image including distractors and a random background; Acquire a sixth image taken by the drone under different weather conditions, different lighting conditions, different detection distances, and different models; blurring and adding noise to the third image, the fourth image, the fifth image, and the sixth image to obtain an initial image dataset; The detection frames of the drones in the initial image dataset are marked to obtain the drone image dataset.

4. The visible light UAV air situation positioning method according to claim 2, characterized in that: The original yolov11 target detection model is improved to obtain the improved yolov11 target detection model as the target detection model, comprising the following steps: The C2PSA module of the backbone network in the original yolov11 target detection model is replaced by the C2LSKA module; The PSA module of the backbone network in the original yolov11 target detection model is replaced by the LSKA module; Among them, the backbone network of the improved yolov11 target detection model also includes a C3K2 structure, an SPPF structure and a detection head structure.

5. The visible light UAV air situation positioning method according to claim 2, characterized in that: The first loss function for optimizing the accuracy of target category classification and the second loss function for optimizing the position of the detection frame are constructed, and then the final loss function is constructed, including the following steps: The first loss function is constructed as follows: ； in, represents the first loss function, The coefficient representing the balance of positive and negative sample weights; Represents the predicted probability of the target detection model for the correct category; Represents the focus parameter that adjusts the weight of difficult and easy samples; The second loss function is constructed as follows: ； ； ； ； ； ； ； in, represents the second loss function; represents the detection box predicted by the target detection model, represents the real detection box, and Represent the width and height of the real detection frame respectively, and scale is the scaling factor; and Represents the x-axis coordinates of the center points of the predicted detection frame and the real detection frame, respectively. and Represents the y-axis coordinates of the center points of the predicted detection box and the real detection box respectively; Then the final loss function of the target detection model is constructed as follows: ； in, represents the final loss function, 、 Both are balancing factors.

6. The visible light UAV air situation positioning method according to claim 1, characterized in that: The step of obtaining a drone based on the image recognition using the target detection model includes the following steps: Using the target detection model to mark a detection frame for the drone on the image; Determining the confidence level that the detection frame contains the drone; If the confidence reaches a preset threshold, it is determined that a drone exists in the image.

7. The visible light UAV air situation positioning method according to any one of claims 1 to 6, characterized in that: Obtaining the position of the drone based on the position identification of the ground monitoring equipment and each of the optical cameras includes the following steps: Draw corresponding rays according to the position, azimuth and pitch angle of the ground monitoring equipment and each of the optical cameras; The distance to each ray and the nearest point in space are solved as the position of the drone.

8. A visible light UAV air situation positioning device, characterized in that: The device comprises: a model training and deployment unit, configured to train a target detection model and then deploy the target detection model to a ground monitoring device and at least two optical cameras, respectively; wherein the ground monitoring device and each of the optical cameras are located at different locations; A preliminary identification unit, configured to obtain a first image by photographing the target airspace with the ground monitoring equipment and identify a UAV based on the first image using the target detection model; a confirmation and identification unit, configured to use each of the optical cameras to respectively photograph the target airspace to obtain a second image and use the target detection model to determine whether a drone exists in the target airspace based on the second image; A drone positioning unit is used to obtain the position of the drone based on the position of the ground monitoring equipment and each of the optical cameras if the target detection model is used to determine the presence of a drone in the target airspace based on the second image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images