Unmanned aerial vehicle monitoring method, system and equipment

Abstract

The invention discloses an unmanned aerial vehicle monitoring method and system and computer equipment. The method comprises the following steps: acquiring unmanned aerial vehicle images acquired by a first camera and a plurality of second cameras; performing unmanned aerial vehicle detection on each image acquired by the first camera to obtain a first detection result set, and performing unmanned aerial vehicle detection on each image acquired by each second camera to obtain a plurality of second detection result sets; and performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set and the parameters of the second camera, and monitoring the unmanned aerial vehicle according to a fusion matching result. According to the scheme provided by the invention, unmanned aerial vehicle image acquisition is carried out by using a plurality of cameras, target detection is carried out based on the acquired images to obtain a plurality of detection results, then fusion matching is carried out by using the plurality of detection results to obtain the fusion matching result, and then the flight path of the unmanned aerial vehicle in the blind area is supplemented based on the fusion matching result. And a complete flight path of the unmanned aerial vehicle is obtained.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicles (UAVs), and in particular to a method, system and equipment for monitoring UAVs. Background Art

[0002] During the flight of a drone, its status and location need to be monitored and evaluated in real time or regularly. The main purpose of this is to ensure flight safety and avoid accidents caused by drone failure or improper operation.

[0003] Unlike vehicles that can only travel on limited roads, drones have a very wide range of flight. In order to achieve full coverage without blind spots, multiple monitoring cameras need to be densely deployed. However, cameras have limited field of view, that is, each camera can only cover a certain spatial range, and due to cost constraints, it is difficult to deploy enough cameras to fully cover the entire drone's flight area.

[0004] In actual monitoring, since drones usually look similar, it is difficult to associate drones simply by relying on the texture information of the image. Therefore, it is more necessary to use the location information of the camera to make association judgments. Therefore, it is currently impossible to fully utilize the information collected by the camera to accurately determine whether the drones captured by multiple cameras are the same. In addition, when a drone briefly appears in the blind spot of several cameras, it is impossible to associate it with monitoring information from other time periods, and thus it is impossible to accurately grasp the complete flight path of the drone. Summary of the invention

[0005] In view of this, in order to overcome at least one aspect of the above problems, an embodiment of the present invention proposes a drone monitoring method, comprising the following steps: Acquire drone images captured by a first camera and a plurality of second cameras; Perform drone detection on each image captured by the first camera to obtain a first detection result set, and perform drone detection on each image captured by each second camera to obtain multiple second detection result sets; Fusion matching is performed based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and the drone is monitored according to the fusion matching result.

[0006] In some embodiments, performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a training set with the historical images acquired by the first camera and each of the historical images acquired by the second camera; Use the training set to train the target detection model; The trained target detection model is used to detect the image captured by the first camera and each image captured by the second camera.

[0007] In some embodiments, performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera; Using the first training set to train a first object detection model, and using at least one second training set to train at least one second object detection model; Using the trained first target detection model to detect the image captured by the first camera; The image captured by the corresponding second camera is detected using at least one trained second target detection model.

[0008] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera and monitoring the drone according to the fusion matching result further includes: Constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters; Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes a world coordinate and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

[0009] In some embodiments, constructing an input matrix according to the first detection result set, the first camera parameter, the second detection result set, and the second camera parameter further includes: Obtain multiple first parameters based on each first detection result in the first detection result set and the first camera parameter, and use the timestamp of the detection image corresponding to the first detection result as the first timestamp of the first parameter; Obtain multiple second parameters based on each second detection result in the second detection result set currently being fused and matched and the corresponding second camera parameter, and use the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter; An input matrix is ​​constructed based on each of the first parameters, each of the second parameters and the difference between the corresponding first timestamp and the second timestamp.

[0010] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes: In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same drone, the input matrix is ​​calculated using a fusion network to obtain a fusion result; In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0011] In some embodiments, it also includes: Determine whether there are multiple fusion results with the same timestamp; In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0012] In some embodiments, obtaining the position trajectory of the drone according to the fusion result further includes: The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0013] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a drone monitoring system, including: An acquisition module, configured to acquire drone images captured by the first camera and multiple second cameras; a detection module configured to perform drone detection on each image acquired by the first camera to obtain a first detection result set, and to perform drone detection on each image acquired by the second camera to obtain multiple second detection result sets; The fusion module is configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result.

[0014] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer device, including: at least one processor; and A memory, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps of any one of the drone monitoring methods described above are performed.

[0015] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the drone monitoring methods described above are performed.

[0016] The present invention has one of the following beneficial technical effects: the solution proposed in the present invention utilizes multiple cameras to collect UAV images, performs target detection based on the collected images to obtain multiple detection results, and then uses the multiple detection results to perform fusion matching to obtain a fusion matching result. In this way, even if the UAV stays briefly in the blind spot of the camera, the flight trajectory of the UAV in the blind spot can be supplemented based on the fusion matching result, and then the complete flight trajectory of the UAV is obtained, which effectively solves the problem of being unable to obtain the complete trajectory of the UAV due to the camera monitoring blind spot, and improves the reliability and effectiveness of UAV monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a drone monitoring method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a fusion network provided for an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a drone monitoring system provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0020] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0021] According to one aspect of the present invention, an embodiment of the present invention provides a drone monitoring method, such as Figure 1 As shown, it may include the steps of: S1, obtaining drone images captured by a first camera and multiple second cameras; S2, performing drone detection on each image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by the second camera to obtain multiple second detection result sets; S3, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitoring the drone according to the fusion matching result.

[0022] The solution proposed by the present invention uses multiple cameras to collect drone images, and performs target detection based on the collected images to obtain multiple detection results, and then uses the multiple detection results to perform fusion matching to obtain fusion matching results. In this way, even if the drone stays briefly in the blind spot of the camera, the flight trajectory of the drone in the blind spot can be supplemented based on the fusion matching results, and then the complete flight trajectory of the drone is obtained, which effectively solves the problem of not being able to obtain the complete trajectory of the drone due to the camera monitoring blind spot, and improves the reliability and effectiveness of drone monitoring. The first camera and multiple second cameras mentioned above can be set to face different directions and / or areas.

[0023] In some embodiments, performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a training set with the historical images acquired by the first camera and each of the historical images acquired by the second camera; Use the training set to train the target detection model; The trained target detection model is used to detect the image captured by the first camera and each image captured by the second camera.

[0024] Specifically, a target detection model Det can be constructed, for example, it can be a common target detection model such as YOLO, DETR, etc. In order to train the target detection model, a large amount of image data with annotation information (the category and coordinates of the drone on the image) is required. These images can come from the historical images collected by each camera in the past. Then, the target detection model Det is trained based on the historical images with annotation information. During training, the classification loss and position regression loss and the back propagation algorithm can be used to optimize the model. When the model optimization is completed, the model can be used to detect the images collected by the first camera and the second camera in real time to obtain the type C and the detection box Bbox of the target, where the detection box Bbox includes the coordinate value of the target. In this way, by integrating the historical images of all cameras together to build a training set, it is beneficial for the model to integrate multi-source data information. In this way, when facing differences in shooting angles, light, etc. of different cameras, it has a certain degree of adaptability, and there is no need to maintain a model for each camera separately. The management and deployment are relatively simple, and the cost is also saved.

[0025] In some embodiments, performing drone detection on an image acquired by the first camera to obtain a first detection result set, and performing drone detection on each image acquired by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera; Using the first training set to train a first object detection model, and using at least one second training set to train at least one second object detection model; Using the trained first target detection model to detect the image captured by the first camera; The image captured by the corresponding second camera is detected using at least one trained second target detection model.

[0026] Specifically, a target detection model can be constructed for each camera, for example, a common target detection model such as YOLO, DETR, etc. The type of detection model corresponding to each camera can be the same or different. Then, the historical images collected by the first camera are constructed as the first training set, and multiple second training sets are constructed with the historical images collected by each second camera. In this way, the first target detection model is trained using the first training set, and the corresponding second target detection models are trained using multiple second training sets. During training, the classification loss and position regression loss, etc., and the back propagation algorithm can be used to optimize the model. After the model optimization is completed, the first target detection model can be used to detect the image collected by the first camera in real time, and the multiple second target detection models can be used to detect the image collected by the corresponding second camera in real time, and the type C and detection box Bbox of the target are obtained respectively, wherein the detection box Bbox includes the coordinate value of the target, and in some embodiments, the detection box can also include a timestamp. In this way, each camera has a corresponding training set and detection model, and the model can be specially trained and optimized for the shooting characteristics, viewing angle, environment, etc. of a specific camera, so as to better capture the features and laws in the camera image and improve the detection accuracy and effect of the target in the image collected by the camera. In addition, if the performance of a camera needs to be improved, or the characteristics of the data it collects change, only the corresponding first target detection model or second target detection model needs to be updated or retrained separately, which will not affect the detection models of other cameras and is beneficial to the overall system optimization.

[0027] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera and monitoring the drone according to the fusion matching result further includes: Constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters; Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes a world coordinate and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

[0028] Specifically, when performing fusion matching, the first detection result set obtained by the first camera is fused and matched with each of the second detection result sets, that is, if there are three second cameras A, B, and C, the first detection result set of the first camera is fused and matched with the second detection result set of the second camera A, the first detection result set of the first camera is fused and matched with the second detection result set of the second camera B, and the first detection result set of the first camera is fused and matched with the second detection result set of the second camera C. Then, the input matrix is ​​constructed using the first camera parameters, the first detection result set, the second camera parameters, and the second detection result set, and the fusion network is used to calculate the input matrix to obtain the fusion result. In this way, since the camera parameter information is used in the fusion calculation, that is, more abundant information is used in the data fusion calculation, the accuracy of the fusion detection is improved.

[0029] In some embodiments, constructing an input matrix according to the first detection result set, the first camera parameter, the second detection result set, and the second camera parameter further includes: Obtain multiple first parameters based on each first detection result in the first detection result set and the first camera parameter, and use the timestamp of the detection image corresponding to the first detection result as the first timestamp of the first parameter; Obtain multiple second parameters based on each second detection result in the second detection result set currently being fused and matched and the corresponding second camera parameter, and use the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter; An input matrix is ​​constructed based on each of the first parameters, each of the second parameters and the difference between the corresponding first timestamp and the second timestamp.

[0030] Specifically, each camera can be pre-calibrated to obtain an internal parameter matrix K, a distortion matrix R, and an external parameter matrix M, and then the above matrices and the detection results are used to construct input parameters. For example, using the first detection result D (C i , Bbox i )、internal parameter matrix K i , distortion matrix R i and the external parameter matrix M i Construct the first parameter, namely [D (C i , Bbox i ), K i , R i , M i ], the timestamp corresponding to the first parameter is the first timestamp of the image corresponding to the second detection result D. Using the second detection result D (C j , Bbox j )、internal parameter matrix Kj , distortion matrix R j and the external parameter matrix M j Construct the second parameter, namely [D (C j , Bbox j ), K j , R j , M j ], the timestamp corresponding to the second parameter is the second timestamp of the image corresponding to the second detection result D.

[0031] Since the first detection result set obtained based on the first camera includes multiple first detection results, multiple first parameters can be obtained based on the first detection result set. Similarly, the second detection result set obtained based on the second camera currently undergoing fusion matching includes multiple second detection results, so multiple second parameters can also be obtained based on the second detection result set. Multiple input matrices are constructed using the difference between each first parameter and each second parameter and the timestamps corresponding to the two parameters. Each input matrix is ​​input into the fusion network to obtain the fusion result. In this way, by fusing the detection results of the drone with the parameters and timestamps of the camera, the spatial information is comprehensively considered, which conforms to the physical laws of drone flight, thereby improving the accuracy of fusion detection.

[0032] For example, if two first parameters A and B can be obtained based on the first detection result set, the first parameter A includes the first detection result D of the first camera. A , internal parameter matrix K i , distortion matrix R i and the external parameter matrix M i , the first parameter B includes the first detection result D of the first camera B , internal parameter matrix K i , distortion matrix R i and the external parameter matrix M i , the corresponding first timestamps are TA and TB respectively. Based on the second detection result set, two second parameters a and b can be obtained. The second parameter a includes the second detection result D a , internal parameter matrix K j , distortion matrix R j and the external parameter matrix M j , the second parameter b includes the second detection result D b , internal parameter matrix K j , distortion matrix R j and the external parameter matrix M j , the corresponding first timestamps are Ta and Tb respectively, then the constructed input matrix can be (A, a, TA-Ta), (A, b, TA-Tb), (B, a, TB-Ta), (B, b, TB -Tb).

[0033] In some embodiments, Figure 2 As shown, the fusion network can be an Encoder-Decoder structure similar to Transformer. Similarly, the first parameter and the second parameter can be constructed by using historical images and historical detection results to train the fusion network.

[0034] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes: In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same drone, the input matrix is ​​calculated using a fusion network to obtain a fusion result; In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0035] Specifically, after the input matrix is ​​input into the fusion network, the fusion network performs calculations based on the input matrix to determine whether the drones observed by different cameras are the same or different drones, that is, to determine whether the first detection result detected by the first camera and the second detection result detected by the second camera are the same drone. Since the flight of drones conforms to certain physical laws, the two detection results can be calculated through the fusion network to determine whether they are the same drone. If they are the same drone, the world coordinates of the drone are obtained according to the difference between the first parameter, the second parameter and the timestamp in the input matrix, and the first timestamp is used as the timestamp of the world coordinates. If they are not the same drone, the next input matrix calculation is performed directly.

[0036] In some embodiments, it also includes: Determine whether there are multiple fusion results with the same timestamp; In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0037] Specifically, when multiple input matrices are input into the fusion network, the fusion result may produce multiple world coordinates at the same time. For example, if the timestamps corresponding to the input matrices (A, a, TA-Ta) and (A, b, TA-Tb) are both TA, then the timestamps of the two world coordinates output based on the above input matrices are the same. In this case, the two world coordinates are fused, for example, the average value can be calculated.

[0038] In some embodiments, obtaining the position trajectory of the drone according to the fusion result further includes: The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0039] Specifically, since each fusion result has a timestamp and contains the coordinates of the target UAV in the world coordinate system, the world coordinates in each fusion result can be connected in sequence according to the order of the timestamps in the fusion result, and finally a continuous path is formed. This path is the position trajectory of the target UAV, thereby obtaining the position change of the target UAV at different time points.

[0040] The solution proposed in the present invention utilizes multiple cameras to collect UAV images, performs target detection based on the collected images to obtain multiple detection results, and then uses the multiple detection results to perform fusion matching to obtain a fusion matching result. In this way, even if the UAV stays briefly in the blind spot of the camera, the flight trajectory of the UAV in the blind spot can be supplemented based on the fusion matching result, and then the complete flight trajectory of the UAV is obtained, which effectively solves the problem of being unable to obtain the complete trajectory of the UAV due to the camera monitoring blind spot, and improves the reliability and effectiveness of UAV monitoring.

[0041] Based on the same inventive concept, according to another aspect of the present invention, an embodiment of the present invention further provides a drone monitoring system 400, such as Figure 3 As shown, including: An acquisition module 401 is configured to acquire drone images captured by a first camera and a plurality of second cameras; The detection module 402 is configured to perform drone detection on each image acquired by the first camera to obtain a first detection result set, and perform drone detection on each image acquired by the second camera to obtain multiple second detection result sets; The fusion module 403 is configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result.

[0042] In some embodiments, performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a training set with the historical images acquired by the first camera and each of the historical images acquired by the second camera; Use the training set to train the target detection model; The trained target detection model is used to detect the image captured by the first camera and each image captured by the second camera.

[0043] In some embodiments, performing drone detection on the image captured by the first camera to obtain a first detection result set, and performing drone detection on each image captured by each of the second cameras to obtain multiple second detection result sets, further comprising: Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera; Using the first training set to train a first object detection model, and using at least one second training set to train at least one second object detection model; Using the trained first target detection model to detect the image captured by the first camera; The image captured by the corresponding second camera is detected using at least one trained second target detection model.

[0044] In some embodiments, performing fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera and monitoring the drone according to the fusion matching result further includes: Constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters; Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes a world coordinate and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

[0045] In some embodiments, constructing an input matrix according to the first detection result set, the first camera parameter, the second detection result set, and the second camera parameter further includes: Obtain multiple first parameters based on each first detection result in the first detection result set and the first camera parameter, and use the timestamp of the detection image corresponding to the first detection result as the first timestamp of the first parameter; Obtain multiple second parameters based on each second detection result in the second detection result set currently being fused and matched and the corresponding second camera parameter, and use the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter; An input matrix is ​​constructed based on each of the first parameters, each of the second parameters and the difference between the corresponding first timestamp and the second timestamp.

[0046] In some embodiments, calculating the input matrix using a fusion network to obtain a fusion result further includes: In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same drone, the input matrix is ​​calculated using a fusion network to obtain a fusion result; In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

[0047] In some embodiments, it also includes: Determine whether there are multiple fusion results with the same timestamp; In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

[0048] In some embodiments, obtaining the position trajectory of the drone according to the fusion result further includes: The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

[0049] Based on the same inventive concept, according to another aspect of the present invention, Figure 4 As shown, an embodiment of the present invention further provides a computer device 501, including: at least one processor 520; and The memory 510 stores a computer program 511 that can be run on the processor. When the processor 520 executes the program, the steps of any of the above-mentioned drone monitoring methods are performed.

[0050] Based on the same inventive concept, according to another aspect of the present invention, Figure 5 As shown, an embodiment of the present invention further provides a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, the steps of any of the above drone monitoring methods are performed.

[0051] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0052] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.

[0053] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0054] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope disclosed in the embodiments of the present invention as defined in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. In addition, although the elements disclosed in the embodiments of the present invention may be described or required in individual form, they may also be understood as multiple unless explicitly limited to the singular.

[0055] It should be understood that, as used herein, the singular forms "a", "an" are intended to include the plural forms as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations including one or more of the associated listed items.

[0056] The serial numbers of the embodiments disclosed in the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0057] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0058] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. A drone monitoring method, characterized in that: The following steps are involved: Acquire drone images captured by a first camera and a plurality of second cameras; Perform drone detection on each image captured by the first camera to obtain a first detection result set, and perform drone detection on each image captured by each second camera to obtain multiple second detection result sets; Fusion matching is performed based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and the drone is monitored according to the fusion matching result.

2. The method according to claim 1, characterized in that The method further includes: performing drone detection on the image acquired by the first camera to obtain a first detection result set, and performing drone detection on each image acquired by each of the second cameras to obtain multiple second detection result sets. Constructing a training set with the historical images acquired by the first camera and each of the historical images acquired by the second camera; Use the training set to train the target detection model; The trained target detection model is used to detect the image captured by the first camera and each image captured by the second camera.

3. The method according to claim 1, characterized in that The method further includes: performing drone detection on the image acquired by the first camera to obtain a first detection result set, and performing drone detection on each image acquired by each of the second cameras to obtain multiple second detection result sets. Constructing a first training set using historical images acquired by the first camera, and constructing at least one second training set using historical images acquired by the second camera; Using the first training set to train a first object detection model, and using at least one second training set to train at least one second object detection model; Using the trained first target detection model to detect the image captured by the first camera; The image captured by the corresponding second camera is detected using at least one trained second target detection model.

4. The method according to claim 1, characterized in that Based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, fusion matching is performed and the UAV is monitored according to the fusion matching result, further comprising: Constructing an input matrix according to the first detection result set, the first camera parameters, the second detection result set, and the second camera parameters; Calculating the input matrix using a fusion network to obtain a fusion result, wherein the fusion result includes a world coordinate and a timestamp; The position trajectory of the UAV is obtained according to the fusion result.

5. The method according to claim 4, characterized in that Constructing an input matrix according to the first detection result set, the first camera parameter, the second detection result set, and the second camera parameter further includes: Obtain multiple first parameters based on each first detection result in the first detection result set and the first camera parameter, and use the timestamp of the detection image corresponding to the first detection result as the first timestamp of the first parameter; Obtain multiple second parameters based on each second detection result in the second detection result set currently being fused and matched and the corresponding second camera parameter, and use the timestamp of the detection image corresponding to the second detection result as the second timestamp of the second parameter; An input matrix is ​​constructed based on each of the first parameters, each of the second parameters and the difference between the corresponding first timestamp and the second timestamp.

6. The method according to claim 5, characterized in that Calculating the input matrix using a fusion network to obtain a fusion result further includes: In response to a first detection result of a first parameter in the input matrix and a second detection result of a second parameter corresponding to the same drone, the input matrix is ​​calculated using a fusion network to obtain a fusion result; In response to the first detection result of the first parameter and the second detection result of the second parameter in the input matrix not corresponding to the same drone, calculation of the next input matrix is ​​performed.

7. The method according to claim 4, characterized in that Also includes: Determine whether there are multiple fusion results with the same timestamp; In response to the existence of multiple fusion results with the same timestamp, the world coordinates of the multiple fusion results are fused.

8. The method according to claim 7, characterized in that Obtaining the position trajectory of the UAV according to the fusion result further includes: The world coordinates of each fusion result are connected according to the timestamp of the fusion result to obtain the position trajectory of the UAV.

9. A drone monitoring system, characterized in that: include: An acquisition module, configured to acquire drone images captured by the first camera and multiple second cameras; a detection module configured to perform drone detection on each image acquired by the first camera to obtain a first detection result set, and to perform drone detection on each image acquired by the second camera to obtain multiple second detection result sets; The fusion module is configured to perform fusion matching based on the first detection result set, the parameters of the first camera, the second detection result set, and the parameters of the second camera, and monitor the drone according to the fusion matching result.

10. A computer device comprising: at least one processor; as well as A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 8 when executing the program.

Images