UAV monitoring method, system and equipment

Abstract

The present invention discloses a drone monitoring method, system, and computer device, comprising: obtaining drone images captured by a first camera and multiple second cameras; performing drone detection on each image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each second camera to obtain multiple second detection result sets; performing fusion matching based on the first detection result set, parameters of the first camera, the second detection result set, and parameters of the second camera, and monitoring the drone based on the fusion matching results. The solution proposed by the present invention utilizes multiple cameras to capture drone images, performs target detection based on the captured images to obtain multiple detection results, then performs fusion matching using the multiple detection results to obtain a fusion matching result, and then, based on the fusion matching results, supplements the drone's flight trajectory within the blind spot, thereby obtaining a complete drone flight trajectory.

Description

Technical Field

[0001] The present invention relates to the field of drones, and in particular to a drone monitoring method, system and equipment. Background Art

[0002] During a drone's flight, its status and location need to be monitored and evaluated in real time or regularly. This is to ensure flight safety and avoid accidents caused by drone malfunction or improper operation.

[0003] Unlike vehicles, which are restricted to limited roads, drones have a vast range. To achieve full coverage without blind spots, multiple surveillance cameras must be densely deployed. However, cameras have limited field of view; each camera can only cover a certain area. Furthermore, due to cost constraints, it's difficult to deploy enough cameras to fully cover the entire drone's flight area.

[0004] In actual monitoring, drones often look similar, making it difficult to correlate them based solely on image texture information. Therefore, it's more important to leverage camera position information for correlation. Consequently, it's currently impossible to fully utilize the information captured by cameras to accurately determine whether multiple cameras are capturing the same drone. Furthermore, when a drone briefly appears in a blind spot within several cameras, it's impossible to correlate it with monitoring data from other time periods, making it impossible to accurately determine the drone's complete flight path. Summary of the Invention

[0005] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention provides a drone monitoring method, comprising the following steps:

[0006] Acquire drone images captured by a first camera and a plurality of second cameras;

[0007] Performing drone detection on each image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each second camera to obtain multiple second detection result sets;

[0008] Fusion matching is performed based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and the drone is monitored according to the fusion matching result.

[0009] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0010] Constructing a training set using historical images captured by the first camera and each historical image captured by the second camera;

[0011] Use the training set to train the target detection model;

[0012] The trained object detection model is used to detect the image captured by the first camera and each image captured by each second camera.

[0013] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0014] Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera;

[0015] Training a first object detection model using the first training set, and training at least one second object detection model using at least one second training set;

[0016] Detecting the image captured by the first camera using the trained first object detection model;

[0017] The image captured by the corresponding second camera is detected using at least one trained second object detection model.

[0018] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result further includes:

[0019] constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters;

[0020] Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes world coordinates and a timestamp;

[0021] The position trajectory of the UAV is obtained according to the fusion result.

[0022] In some embodiments, constructing an input matrix based on the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters further includes:

[0023] obtaining a plurality of first parameters based on each first detection result and a first camera parameter in the first detection result set, and using a timestamp of a detection image corresponding to the first detection result as a first timestamp of the first parameters;

[0024] Obtaining multiple second parameters based on each second detection result in the second detection result set currently undergoing fusion matching and the corresponding second camera parameters, and using the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter;

[0025] An input matrix is ​​constructed based on each of the first parameters, each of the second parameters, and the difference between the corresponding first timestamp and the second timestamp.

[0026] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes:

[0027] In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same UAV, calculating the input matrix using a fusion network to obtain a fusion result;

[0028] In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0029] In some embodiments, further comprising:

[0030] Determine whether there are multiple fusion results with the same timestamp;

[0031] In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0032] In some embodiments, obtaining the position trajectory of the UAV according to the fusion result further includes:

[0033] The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0034] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a drone monitoring system, comprising:

[0035] an acquisition module configured to acquire drone images captured by the first camera and the plurality of second cameras;

[0036] a detection module configured to perform drone detection on each image captured by the first camera to obtain a first detection result set, and to perform drone detection on each image captured by each second camera to obtain multiple second detection result sets;

[0037] The fusion module is configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result.

[0038] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including:

[0039] at least one processor; and

[0040] A memory storing a computer program that can be run on the processor, wherein the processor executes the steps of any one of the above-mentioned drone monitoring methods when executing the program.

[0041] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the drone monitoring methods described above are performed.

[0042] The present invention has one of the following beneficial technical effects: the solution proposed in the present invention utilizes multiple cameras to collect drone images, performs target detection based on the collected images to obtain multiple detection results, and then uses the multiple detection results to perform fusion matching to obtain a fusion matching result. In this way, even if the drone stays briefly in the blind spot of the camera, the flight trajectory of the drone in the blind spot can be supplemented based on the fusion matching result, and then the complete flight trajectory of the drone is obtained, which effectively solves the problem that the complete trajectory of the drone cannot be obtained due to the blind spot of the camera monitoring, and improves the reliability and effectiveness of drone monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a drone monitoring method provided by an embodiment of the present invention;

[0045] Figure 2A schematic diagram of a converged network provided by an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of the structure of a drone monitoring system provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention;

[0048] Figure 5 A schematic structural diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0050] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. Subsequent embodiments will not explain this one by one.

[0051] According to one aspect of the present invention, an embodiment of the present invention provides a drone monitoring method, such as Figure 1 As shown, it may include the steps of:

[0052] S1, obtaining drone images captured by a first camera and multiple second cameras;

[0053] S2, performing drone detection on each image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each second camera to obtain multiple second detection result sets;

[0054] S3, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result.

[0055] The solution proposed by the present invention utilizes multiple cameras to capture drone images, performs target detection based on the captured images to obtain multiple detection results, and then fuses and matches the multiple detection results to obtain a fused matching result. In this way, even if the drone briefly stops in the camera's blind spot, the flight trajectory of the drone in the blind spot can be supplemented based on the fused matching result, thereby obtaining the drone's complete flight trajectory. This effectively solves the problem of being unable to obtain the drone's complete trajectory due to the camera's monitoring blind spot, and improves the reliability and effectiveness of drone monitoring. The first camera and multiple second cameras can be set to face different directions and / or areas.

[0056] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0057] Constructing a training set using historical images captured by the first camera and each historical image captured by the second camera;

[0058] Use the training set to train the target detection model;

[0059] The trained object detection model is used to detect the image captured by the first camera and each image captured by each second camera.

[0060] Specifically, an object detection model Det can be constructed, such as a common object detection model like YOLO or DETR. Training the model requires a large amount of annotated image data (including the drone's category and coordinates in the image). This data can be sourced from historical images collected by each camera. The model is then trained based on these annotated images. During training, the model can be optimized using a backpropagation algorithm, such as a classification loss and a position regression loss. Once the model is optimized, it can be used to detect the object's type C and detection bounding box Bbox in real-time from the first and second cameras. The Bbox contains the object's coordinates. By combining historical images from all cameras to create a training set, the model can integrate multi-source data, providing a degree of adaptability to variations in camera angle, lighting, and other factors. Furthermore, it eliminates the need to maintain a separate model for each camera, simplifying management and deployment and reducing costs.

[0061] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0062] Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera;

[0063] Training a first object detection model using the first training set, and training at least one second object detection model using at least one second training set;

[0064] Detecting the image captured by the first camera using the trained first object detection model;

[0065] The image captured by the corresponding second camera is detected using at least one trained second object detection model.

[0066] Specifically, a target detection model can be constructed for each camera, such as a common target detection model like YOLO or DETR. The detection model types for each camera can be the same or different. A first training set is then constructed using historical images captured by the first camera, and multiple second training sets are constructed using historical images captured by each second camera. The first target detection model is trained using the first training set, and the corresponding second target detection models are trained using the multiple second training sets. During training, the models can be optimized using classification loss, position regression loss, and backpropagation algorithms. Once model optimization is complete, the first target detection model can be used to detect the images captured in real time by the first camera, and the multiple second target detection models can be used to detect the images captured in real time by the corresponding second cameras. The target type C and detection box Bbox are obtained, respectively. The detection box Bbox includes the target's coordinates and, in some embodiments, may also include a timestamp. Thus, each camera has a corresponding training set and detection model. The models can be trained and optimized specifically for the specific camera's shooting characteristics, viewing angle, and environment, better capturing the features and patterns in the images captured by that camera, improving the accuracy and effectiveness of target detection in images captured by that camera. In addition, if the performance of a camera needs to be improved, or the characteristics of the data it collects change, only the corresponding first target detection model or second target detection model needs to be updated or retrained separately, which will not affect the detection models of other cameras and is conducive to the optimization of the overall system.

[0067] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result further includes:

[0068] constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters;

[0069] Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes world coordinates and a timestamp;

[0070] The position trajectory of the UAV is obtained according to the fusion result.

[0071] Specifically, during fusion matching, the first detection result set obtained by the first camera is fused and matched with each of the second detection result sets. That is, if there are three second cameras, A, B, and C, the first detection result set of the first camera is fused and matched with the second detection result set of second camera A, the first detection result set of the first camera is fused and matched with the second detection result set of second camera B, and the first detection result set of the first camera is fused and matched with the second detection result set of second camera C. An input matrix is ​​then constructed using the first camera parameters, the first detection result set, the second camera parameters, and the second detection result set, and the fusion network is used to calculate the input matrix to obtain a fusion result. This utilizes camera parameter information during the fusion calculation, i.e., utilizes richer information during the data fusion calculation, thereby improving the accuracy of fusion detection.

[0072] In some embodiments, constructing an input matrix based on the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters further includes:

[0073] obtaining a plurality of first parameters based on each first detection result and a first camera parameter in the first detection result set, and using a timestamp of a detection image corresponding to the first detection result as a first timestamp of the first parameters;

[0074] Obtaining multiple second parameters based on each second detection result in the second detection result set currently undergoing fusion matching and the corresponding second camera parameters, and using the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter;

[0075] An input matrix is ​​constructed based on each of the first parameters, each of the second parameters, and the difference between the corresponding first timestamp and the second timestamp.

[0076] Specifically, each camera can be pre-calibrated to obtain the intrinsic parameter matrix K, distortion matrix R, and extrinsic parameter matrix M, and then the above matrices and detection results can be used to construct input parameters. For example, using the first detection result D (C i , Bbox i ), internal parameter matrix K i , distortion matrix R i and the external parameter matrix M i Construct the first parameter, namely [D (C i , Bbox i ), K i , R i , M i ], the timestamp corresponding to the first parameter is the first timestamp of the image corresponding to the second detection result D. Using the second detection result D (C j , Bbox j ), internal parameter matrix K j , distortion matrix R j and the external parameter matrix M j Construct the second parameter, namely [D (C j , Bbox j ), K j , R j , M j ], the timestamp corresponding to the second parameter is the second timestamp of the image corresponding to the second detection result D.

[0077] Since the first detection result set obtained based on the first camera includes multiple first detection results, multiple first parameters can be obtained based on the first detection result set. Similarly, the second detection result set obtained based on the second camera currently undergoing fusion matching includes multiple second detection results, so multiple second parameters can also be obtained based on the second detection result set. Multiple input matrices are constructed using the difference between each first parameter and each second parameter, and the timestamps corresponding to the two parameters. Each input matrix is ​​input into the fusion network to obtain a fusion result. In this way, by fusing the drone's detection results with the camera's parameters and timestamps, spatial information is comprehensively considered, conforming to the physical laws of drone flight, thereby improving the accuracy of fusion detection.

[0078] For example, if two first parameters A and B can be obtained based on the first detection result set, the first parameter A includes the first detection result D of the first camera. A , internal parameter matrix K i , distortion matrix R i and the external parameter matrix M i , the first parameter B includes the first detection result D of the first camera B , internal parameter matrix K i , distortion matrix Ri and the external parameter matrix M i , the corresponding first timestamps are TA and TB respectively. Based on the second detection result set, two second parameters a and b can be obtained. The second parameter a includes the second detection result D a , internal parameter matrix K j , distortion matrix R j and the external parameter matrix M j , the second parameter b includes the second detection result D b , internal parameter matrix K j , distortion matrix R j and the external parameter matrix M j , the corresponding first timestamps are Ta and Tb respectively, then the constructed input matrix can be (A, a, TA-Ta), (A, b, TA-Tb), (B, a, TB-Ta), (B, b, TB-Tb).

[0079] In some embodiments, as Figure 2 As shown, the fusion network can be an Encoder-Decoder structure similar to the Transformer. Similarly, the first and second parameters can be constructed using historical images and historical detection results to train the fusion network.

[0080] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes:

[0081] In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same UAV, calculating the input matrix using a fusion network to obtain a fusion result;

[0082] In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0083] Specifically, after the input matrix is ​​fed into the fusion network, calculations are performed within the fusion network based on the input matrix to determine whether the drones observed by different cameras are the same or different drones. Specifically, this determines whether the first detection result from the first camera and the second detection result from the second camera represent the same drone. Because drone flight conforms to certain physical laws, the fusion network can calculate the two detection results to determine whether they represent the same drone. If so, the world coordinates of the drone are calculated based on the difference between the first and second parameters in the input matrix and the timestamp, and the first timestamp is used as the timestamp for the world coordinates. If not, the calculation proceeds directly to the next input matrix.

[0084] In some embodiments, further comprising:

[0085] Determine whether there are multiple fusion results with the same timestamp;

[0086] In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0087] Specifically, when multiple input matrices are input into the fusion network, the fusion result may produce multiple world coordinates at the same time. For example, if the timestamps corresponding to the input matrices (A, a, TA-Ta) and (A, b, TA-Tb) are both TA, then the timestamps of the two world coordinates output based on the above input matrices are the same. In this case, the two world coordinates are fused, for example, the average value can be calculated.

[0088] In some embodiments, obtaining the position trajectory of the UAV according to the fusion result further includes:

[0089] The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0090] Specifically, since each fusion result is timestamped and contains the coordinates of the target drone in the world coordinate system, the world coordinates of each fusion result can be connected in sequence according to the order of the timestamps in the fusion results, ultimately forming a continuous path. This path is the position trajectory of the target drone, thereby obtaining the position changes of the target drone at different time points.

[0091] The solution proposed in the present invention utilizes multiple cameras to collect drone images, performs target detection based on the collected images to obtain multiple detection results, and then uses the multiple detection results to perform fusion matching to obtain a fusion matching result. In this way, even if the drone briefly stays in the blind spot of the camera, the flight trajectory of the drone in the blind spot can be supplemented based on the fusion matching result, thereby obtaining the complete flight trajectory of the drone. This effectively solves the problem of being unable to obtain the complete trajectory of the drone due to the blind spot of camera monitoring, and improves the reliability and effectiveness of drone monitoring.

[0092] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a drone monitoring system 400, such as Figure 3 Shown, including:

[0093] An acquisition module 401 is configured to acquire drone images captured by a first camera and a plurality of second cameras;

[0094] A detection module 402 is configured to perform drone detection on each image captured by the first camera to obtain a first detection result set, and to perform drone detection on each image captured by each second camera to obtain multiple second detection result sets;

[0095] The fusion module 403 is configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result.

[0096] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0097] Constructing a training set using historical images captured by the first camera and each historical image captured by the second camera;

[0098] Use the training set to train the target detection model;

[0099] The trained object detection model is used to detect the image captured by the first camera and each image captured by each second camera.

[0100] In some embodiments, performing drone detection on an image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further includes:

[0101] Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera;

[0102] Training a first object detection model using the first training set, and training at least one second object detection model using at least one second training set;

[0103] Detecting the image captured by the first camera using the trained first object detection model;

[0104] The image captured by the corresponding second camera is detected using at least one trained second object detection model.

[0105] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result further includes:

[0106] constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters;

[0107] Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes world coordinates and a timestamp;

[0108] The position trajectory of the UAV is obtained according to the fusion result.

[0109] In some embodiments, constructing an input matrix based on the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters further includes:

[0110] obtaining a plurality of first parameters based on each first detection result and a first camera parameter in the first detection result set, and using a timestamp of a detection image corresponding to the first detection result as a first timestamp of the first parameters;

[0111] Obtaining multiple second parameters based on each second detection result in the second detection result set currently undergoing fusion matching and the corresponding second camera parameters, and using the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter;

[0112] An input matrix is ​​constructed based on each of the first parameters, each of the second parameters, and the difference between the corresponding first timestamp and the second timestamp.

[0113] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes:

[0114] In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same UAV, calculating the input matrix using a fusion network to obtain a fusion result;

[0115] In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0116] In some embodiments, further comprising:

[0117] Determine whether there are multiple fusion results with the same timestamp;

[0118] In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0119] In some embodiments, obtaining the position trajectory of the UAV according to the fusion result further includes:

[0120] The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0121] Based on the same inventive concept, according to another aspect of the present invention, Figure 4 As shown, an embodiment of the present invention further provides a computer device 501, including:

[0122] at least one processor 520; and

[0123] The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any of the above drone monitoring methods are performed.

[0124] Based on the same inventive concept, according to another aspect of the present invention, Figure 5 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above drone monitoring methods are performed.

[0125] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0126] Furthermore, it should be appreciated that the computer-readable storage media (eg, memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory.

[0127] It will also be appreciated by those skilled in the art that the various exemplary logic blocks, modules, circuits and algorithmic steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given of the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0128] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope of the embodiments disclosed in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless expressly limited to the singular.

[0129] It should be understood that, as used herein, the singular forms "a" and "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" is intended to include any and all possible combinations of one or more of the associated listed items.

[0130] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0131] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program instructing the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

[0132] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to limit the scope of the disclosure of the present invention (including the claims) to these examples. Within the spirit of the present invention, the technical features of the above embodiments or different embodiments may be combined, and many other variations exist in different aspects of the above embodiments, which are not provided in detail for the sake of clarity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A drone monitoring method, characterized in that: The following steps are involved: Acquire drone images captured by a first camera and a plurality of second cameras; Performing drone detection on each image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each second camera to obtain multiple second detection result sets; Performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result, wherein monitoring the drone further includes: constructing an input matrix based on a plurality of first detection results in the first detection result set, first camera parameters, a plurality of second detection results in the second detection result set, second camera parameters, and differences between timestamps corresponding to the first detection results and the second detection results; Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes world coordinates and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

2. The method according to claim 1, wherein Performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a training set using historical images captured by the first camera and each historical image captured by the second camera; Use the training set to train the target detection model; The trained object detection model is used to detect the image captured by the first camera and each image captured by each second camera.

3. The method according to claim 1, wherein Performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera; Training a first object detection model using the first training set, and training at least one second object detection model using at least one second training set; Detecting the image captured by the first camera using the trained first object detection model; The image captured by the corresponding second camera is detected using at least one trained second object detection model.

4. The method according to claim 1, wherein Constructing multiple input matrices based on multiple first detection results in the first detection result set, first camera parameters, multiple second detection results in the second detection result set, second camera parameters, and differences between timestamps corresponding to the first detection results and the second detection results, further comprising: obtaining a plurality of first parameters based on each first detection result and a first camera parameter in the first detection result set, and using a timestamp of a detection image corresponding to the first detection result as a first timestamp of the first parameters; Obtaining multiple second parameters based on each second detection result in the second detection result set currently undergoing fusion matching and the corresponding second camera parameters, and using the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter; An input matrix is ​​constructed based on each of the first parameters, each of the second parameters, and the difference between the corresponding first timestamp and the second timestamp.

5. The method according to claim 4, wherein Calculating the input matrix using a fusion network to obtain a fusion result further includes: In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same UAV, calculating the input matrix using a fusion network to obtain a fusion result; In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

6. The method according to claim 1, wherein Also includes: Determine whether there are multiple fusion results with the same timestamp; In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

7. The method according to claim 6, wherein Obtaining the position trajectory of the UAV according to the fusion result further includes: The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

8. A drone monitoring system, characterized in that: include: an acquisition module configured to acquire drone images captured by the first camera and the plurality of second cameras; a detection module configured to perform drone detection on each image captured by the first camera to obtain a first detection result set, and to perform drone detection on each image captured by each second camera to obtain multiple second detection result sets; a fusion module configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result; The fusion module configuration is: constructing an input matrix based on a plurality of first detection results in the first detection result set, first camera parameters, a plurality of second detection results in the second detection result set, second camera parameters, and differences between timestamps corresponding to the first detection results and the second detection results; Utilizing a fusion network to calculate the plurality of input matrices to obtain a fusion result, wherein the fusion result includes a world coordinate and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

9. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when executing the program.

Images