Position determination method of drone cluster, edge device cluster system, storage medium and electronic device

Abstract

The present application discloses a method for determining the position of a drone cluster and an edge device cluster system, a storage medium and an electronic device, the method comprising: performing image recognition processing on collected video frames in real time; when a target drone is identified, performing a first matching operation in a first storage table based on feature data of the target drone; when the first matching operation is successful, updating the first storage table using the current position information of the target drone; when the first matching operation fails, storing the feature data of the target drone in association with the current position information in the first storage table, and performing a second matching operation in sequence in the second storage table; when the second matching operation is successful, generating an entry deletion instruction, and sending the entry deletion instruction to a second video acquisition device corresponding to the second storage table that has successfully matched, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method for determining the position of a drone cluster, an edge device cluster system, a storage medium, and an electronic device. Background Art

[0002] As living standards improve, drones are increasingly being used in our daily lives and work. In particular, drones, due to their ability to self-locate and fly autonomously, are widely used in high-altitude surveillance and even unmanned transportation. Furthermore, with the increasing use of drones, swarms of drones have emerged to cover a specific area. In this scenario, a large number of drones fly along a predetermined trajectory. However, due to the influence of environmental conditions such as airflow and the movement of surrounding drones, the actual flight position of drones at high altitudes can deviate from the predetermined position. Since a drone's flight trajectory depends on its real-time position in the air, a solution is needed to determine the position of a drone swarm in real time. Summary of the Invention

[0003] The embodiments of the present application provide a method for determining the position of a drone cluster, an edge device cluster system, a storage medium, and an electronic device to address the defect of poor accuracy in position monitoring of drone clusters in the prior art.

[0004] To achieve the above objectives, an embodiment of the present application provides a method for determining the position of a drone cluster based on an edge device cluster system, wherein the edge device cluster system includes: multiple video acquisition devices, each of the video acquisition devices stores multiple storage tables for associating and storing feature data and location information of drones collected by the video acquisition devices, and the multiple storage tables include a first storage table and multiple second storage tables, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device. The method includes:

[0005] Perform image recognition processing on the collected video frames in real time;

[0006] When a target UAV is identified, performing a first matching operation in the first storage table according to the feature data of the target UAV;

[0007] When the first matching operation is successful, updating the first storage table using the current location information of the target drone;

[0008] When the first matching operation fails, the characteristic data of the target drone is associated with the current location information and stored in the first storage table, and a second matching operation is sequentially performed in the second storage table;

[0009] When the second matching operation is successful, an entry deletion instruction is generated and sent to the second video acquisition device corresponding to the second storage table that has successfully matched, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

[0010] An embodiment of the present application further provides an edge device cluster system, comprising: multiple video capture devices, each of which stores multiple storage tables for associating and storing feature data and location information of drones captured by the video capture device, and the multiple storage tables include a first storage table and multiple second storage tables, wherein the first storage table is generated by the current first video capture device, and the multiple second storage tables are respectively generated by multiple second video capture devices other than the first video capture device, wherein each of the video capture devices performs the following operations as the first video capture device:

[0011] Image recognition processing is performed on the collected video frames in real time. When a target drone is identified, a feature matching operation is performed in the first storage table based on the feature data of the target drone. If the match is successful, the first storage table is updated with the current position information of the target drone. If the match fails, the feature data of the target drone is associated with the current position information and stored in the first storage table, and feature matching operations are performed in sequence in the second storage table. If the match is successful, an entry deletion instruction is generated and sent to the second video acquisition device corresponding to the second storage table that successfully matches, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

[0012] An embodiment of the present application further provides an electronic device, including:

[0013] Memory, used to store programs;

[0014] The processor is used to run the program stored in the memory, and when the program is run, the position determination method provided in the embodiment of the present application is executed.

[0015] An embodiment of the present application further provides a computer-readable storage medium on which a computer program executable by a processor is stored, wherein the program, when executed by the processor, implements the position determination method provided in the embodiment of the present application.

[0016] The embodiment of the present application provides a method for determining the position of a drone cluster, an edge device cluster system, a storage medium, and an electronic device. By storing a first storage table and multiple second storage tables in each video acquisition device, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device, each video acquisition device can perform recognition and processing on the captured video frames in real time. When a target drone is identified, a first matching operation can be performed in the first storage table based on the feature data of the target drone. When the match is successful, the first storage table can be updated with the location information of the target drone. When the first matching operation fails, the first storage table can be updated. The characteristic data and location information of the target UAV are associated and stored in a first storage table, and a second matching operation is performed on multiple second storage tables in sequence. When the second matching operation is successful, an entry deletion instruction is generated to instruct the corresponding second video acquisition device to delete the entry of the target UAV from its corresponding storage table. Therefore, multiple video acquisition devices can share the characteristics and locations of the UAVs they have discovered and tracked through multiple storage tables, and can also achieve mutual response and cooperation among the video acquisition devices by matching in both shared tables when the target UAV is discovered, thereby closely linking multiple video acquisition devices together and greatly improving the efficiency and accuracy of determining the location of the UAV cluster through collaboration.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0019] Figure 1 Schematic diagram of an application scenario of a solution for determining the location of a drone cluster based on an edge device cluster system provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of an embodiment of a method for determining the location of a drone cluster based on an edge device cluster system provided by this application;

[0021] Figure 3 A system block diagram of an embodiment of the edge device cluster system provided by this application;

[0022] Figure 4 This is a schematic structural diagram of an electronic device embodiment provided in this application. DETAILED DESCRIPTION

[0023] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0024] Example 1

[0025] The solution provided in the embodiments of the present application can be applied to any system with image recognition capabilities, such as an image recognition terminal, etc. Figure 1 Schematic diagram of an application scenario of a location determination solution for a drone cluster based on an edge device cluster system provided in an embodiment of the present application. Figure 1 The principle shown is only one example of the principle of the technical solution of this application.

[0026] Drones are now widely used in our daily lives. Drones, in particular, are widely used for high-altitude surveillance and even unmanned transportation due to their self-addressing and autonomous flight capabilities. Furthermore, with the increasing popularity of drones, swarms of drones have emerged to cover a specific area for operations. In this scenario, a large number of drones fly along a predetermined trajectory. However, due to the influence of environmental conditions such as airflow and the movement of surrounding drones, their actual flight positions at high altitudes can deviate from the predetermined position. A drone's flight trajectory relies on its real-time position in the air. This positioning typically utilizes GPS location information, such as the drone's internal positioning chip. However, due to the drone's high flight speed and the complex aerial environment it navigates, relying solely on GPS location information to locate the drone may not accurately reflect its true location. As drones become increasingly widespread, drone regulation is increasingly facing the threat of intrusive drones. Consequently, regulators are unable to communicate with these drones to obtain their location information, making it difficult to accurately monitor and manage them. In addition, as the cost of drones decreases, drones are increasingly being used in large groups to achieve better regional coverage. When faced with a drone swarm consisting of such a large number of drones, even when communication is possible, it is difficult to monitor and manage each one in the swarm based on the location information sent back by the drones, not to mention that in a scenario where such a drone swarm breaks in, it is even more difficult to effectively monitor the drone swarm.

[0027] For this reason, Figure 1 As shown in Figure 1 This is a schematic diagram showing an application scenario of a solution for determining the location of a drone cluster based on an edge device cluster system according to an embodiment of the present application. Figure 1 In the scenario shown in , multiple video acquisition devices can communicate with each other to form an edge device cluster system, in which each video acquisition device can capture video frames within its respective field of view and store them in its respective storage space. In addition, each video acquisition device can store multiple storage tables to associatively store the feature data and location information of the drone collected by the current video acquisition device and other video acquisition devices. For example, in an embodiment of the present application, each video acquisition device can store a first storage table and multiple second storage tables. The first storage table can be generated by the current video acquisition device, and the multiple second storage tables can be generated by other video acquisition devices other than the current video acquisition device.

[0028] In an embodiment of the present application, these video capture devices serve as nodes in an edge device cluster system, collectively responsible for monitoring drones entering the monitored area. For example, one of the multiple video capture devices serves as the current first video capture device, and the other video capture devices serve as second video capture devices for generating a second storage table. Therefore, after capturing a video image, the first video capture device can perform target recognition processing on the captured video frame. For example, it can extract features of each object in the video frame and, based on the extracted features, determine whether each object is a drone. Furthermore, if one or more objects are determined to be drones, they can be identified as target drones. Specifically, if multiple drone-type objects are identified in a video frame, these objects can be identified as different drones based on the differences in their respective features. For example, if three target drones are identified in the current first video frame, they can be numbered as target drone 1, target drone 2, and target drone 3 according to their respective features. The first video capture device can then store the features and detected location information of these three identified target drones in the first storage table it generates. At the same time, the second video acquisition device located near the first video acquisition device can also perform similar identification from the first video frame it captures simultaneously with the first video frame of the first video acquisition device, and for example, can identify two target drones, and therefore can number them as target drone 4 and target drone 5 respectively according to their respective characteristics, and the second video acquisition device can store the characteristics of target drones 4 and 5 and the detected location information in a first storage table created by the second video acquisition device in an associated manner, and send the storage table to other video acquisition devices including the first video acquisition device, so that the first video acquisition device can store the characteristics and location information of the target drones 1-3 detected by the first video acquisition device from the first video frame it captures, and can also store the characteristics and location information of the target drones 4 and 5 detected by the second video acquisition device from the first video frame it captures by storing the second storage table sent by the second video acquisition device.

[0029] Afterwards, in the following second and third video frames, the first video capture device can continuously identify the target objects in the video frames, and for example, two target drones can be identified in the second video frame captured by the first video capture device, and at this time, the first video capture device can match the features of the two identified target drones with the features stored in the first storage table that has been created. For example, through the matching operation, the features of the two target drones are matched with the features of target drones 2 and 3 stored in the first storage table, then it can be considered that the feature matching operation for the two target drones identified in the second video frame is successful, that is, the first video capture device continues to monitor target drones 2 and 3 in its field of view. Therefore, at this time, the features and position information of the target drones 2 and 3 identified by the first video capture device in the second video frame can be used to update the relevant information of target drones 2 and 3 originally stored in the first storage table based on the recognition result of the first video frame. For example, in an embodiment of the present application, the latest position of the target drone identified by the current first video capture device can be always updated and stored in the first storage table, or all position information detected before the current video frame can be stored in the first storage table to form the movement trajectory of each monitored target drone. Next, the first video capture device can continue to perform feature extraction and object recognition on the third video frame it has captured, and can, for example, identify two target drones. At this time, the first video capture device can match the features of the two identified target drones with the features stored in the first storage table that has been previously updated based on the recognition results of the second video frame. For example, through the matching operation, the features of only one of the two target drones identified by the first video capture device in the third video frame it has captured are matched with the features of target drone 2 stored in the first storage table that has been updated based on the recognition results of, for example, the second video frame. In other words, in the field of view of the first video capture device, at the moment identified by the third video frame, although two drones are monitored, only one of them is drone 2, which has been previously identified and monitored, and the other is the target drone that has newly entered the field of view of the first video capture device at the moment identified by, for example, the third video frame. In this case, on the one hand, similar to the above-mentioned processing method for the second video frame, the characteristics and position information of the target drone 2 that have been successfully matched with the characteristics in the first storage table can be updated to the first storage table; on the other hand, the characteristics of the target drone that failed to be matched in the first storage table can be further matched with the drone characteristics stored in the second storage table received from other second video acquisition devices other than the current first video acquisition device.For example, through a matching operation, the first video acquisition device matches the characteristics of the target drone 5 in the second storage table sent by, for example, a second video acquisition device located nearby, that is, the drone identified by the first video acquisition device in the third video frame it captured is moved from the field of view of the second video acquisition device to the field of view of the current first video acquisition device. In other words, for example, the target drone 5 identified by the second video acquisition device in, for example, the second video frame it captures moves into the acquisition range of the first video acquisition device at the moment corresponding to the third video frame, and therefore, the target drone 5 is identified in the third video frame captured by the first video acquisition device. For example, by successfully matching the identified drone features in the second storage table stored in the first video acquisition device, it is confirmed that the drone identified in the third video frame is the target drone 5 identified by the second video acquisition device before that moment, and therefore, an entry deletion instruction can be generated for the entry corresponding to the target drone 5 stored in the second storage table, and the entry deletion instruction can be sent to the corresponding second video acquisition device, so that the second video acquisition device can delete the entry related to the target drone 5 recorded in its own generated local first storage table according to the received entry deletion instruction, and can send the storage table with the entry deleted to other video acquisition devices in the system as the second storage table. In addition, in an embodiment of the present application, the second video acquisition device that receives the entry deletion instruction can further send all the information of the target drone 5 that has been recorded locally before the deletion moment, such as the characteristics and historical location information of the target drone 5 recorded in the local first storage table, to the sender of the entry deletion instruction before deleting the entry. For example, in the above case, it is the first video acquisition device, so that the first video acquisition device can transfer the historical information of the target drone 5 originally identified and recorded by the second video acquisition device, such as the location and feature information monitored at each moment, to the first video acquisition device that currently identifies the target drone 5 for storage, so that the first video acquisition device can achieve continuous trajectory monitoring and control of the target drone 5 by continuing to record the movement trajectory of the target drone 5.

[0030] In addition, in an embodiment of the present application, each video acquisition device can send its locally stored first storage table to other video acquisition devices at a predetermined time interval to be stored as a second storage table. The predetermined time interval can be a predetermined number of video frames or a predetermined time period. The present application has no restrictions on this.

[0031] In addition, in an embodiment of the present application, when, for example, a matching operation fails in the second storage table stored in the first video acquisition device for the target drone identified in the third video frame captured by the first video acquisition device, that is, when the first video acquisition device does not find a corresponding record in the second storage table generated by the other video acquisition device for the drone identified in the current video frame, the first video acquisition device can request the other second video acquisition device to resend the second storage table in response to the matching failure in the second storage table to update the second storage table stored in the first video acquisition device. In other words, when the first video acquisition device does not find a corresponding entry in the currently stored second storage table for the drone identified in the current video frame, it may be because the second storage table stored by the first video acquisition device is not the latest table, and therefore the first video acquisition device can request the other video acquisition device to send the latest locally stored storage table to the first video acquisition device, and then perform another matching operation to confirm whether the newly found drone is a target drone that has been previously monitored and recorded by the other video acquisition device.

[0032] In addition, in an embodiment of the present application, the first video acquisition device can also count the number of unmatched entries in the first storage table that have not been successfully matched when performing object recognition on each video frame, and when the number reaches a certain threshold, delete the entry from the first storage table, or the first video acquisition device can also set a time threshold for each entry in the first storage table. If it is not matched during the predetermined time threshold, or its position information is not updated, the entry can be set to be deleted from the first storage table when the time threshold is exceeded. In particular, in an embodiment of the present application, in response to the deletion of these entries from the first storage table, the first video acquisition device can also put them into a third storage table for further storage for a period of time, and when the drone identified in each subsequent video frame fails to match in both the first storage table and the second storage table, further feature matching can be performed in the third storage table, and if a corresponding record is matched in the third storage table, the entry can be restored from the third storage table to the first storage table.

[0033] In addition, when the above-mentioned entry is not updated or matched for more than a predetermined time interval, the characteristic data and location information of the drone in the entry can also be sent to multiple other second video acquisition devices for further matching and storage in the first storage table locally stored in these second video acquisition devices.

[0034] An embodiment of the present application provides a location determination solution for a drone cluster based on an edge device cluster system, which stores a first storage table and multiple second storage tables in each video acquisition device, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device. Therefore, for each video acquisition device, when the captured video frame is recognized in real time, a first matching operation can be performed in the first storage table based on the feature data of the target drone, and when the match is successful, the first storage table can be updated with the location information of the target drone, and when the first matching operation fails, the target drone will be updated. The characteristic data of the drone and its location information are associated and stored in the first storage table, and the second matching operation is performed on multiple second storage tables in sequence. When the second matching operation is successful, an entry deletion instruction is generated to instruct the corresponding second video acquisition device to delete the entry of the target drone from its corresponding storage table. Therefore, multiple video acquisition devices can share the characteristics and locations of drones that they have discovered and tracked through multiple storage tables, and can also achieve mutual response and cooperation among video acquisition devices by matching in both shared tables when the target drone is discovered, so as to closely link multiple video acquisition devices together, and greatly improve the efficiency and accuracy of determining the location of the drone cluster through collaboration.

[0035] The above embodiments are illustrations of the technical principles and exemplary application frameworks of the embodiments of the present application. The specific technical solutions of the embodiments of the present application are further described in detail below through multiple embodiments.

[0036] Example 2

[0037] Figure 2 This is a flowchart of an embodiment of the method for determining the location of a drone cluster based on an edge device cluster system provided by this application. The execution subject of this method can be various terminals or server devices with image recognition capabilities, or devices or chips integrated on these devices. Figure 2 As shown, the method for determining the location of a drone cluster based on an edge device cluster system includes the following steps:

[0038] S201, performing image recognition processing on the collected video frames in real time.

[0039] In step S201, an edge device cluster system can be formed by using multiple video acquisition devices that can communicate with each other. In this system, each video acquisition device can capture video frames within its own field of view and store them in its own storage space. In step S201, each video acquisition device can perform real-time image recognition processing on the video frames it has captured. In particular, in an embodiment of the present application, the video acquisition device currently performing image recognition processing can serve as a first video acquisition device, and other video acquisition devices can serve as second video acquisition devices relative to the first video acquisition device. Multiple storage tables can be stored in each video acquisition device to associatively store the feature data and location information of the drone captured by the first video acquisition device and the second video acquisition device.

[0040] For example, in an embodiment of the present application, the first video acquisition device may store a first storage table and multiple second storage tables. The first storage table may be generated by the first video acquisition device, and the multiple second storage tables may be generated by the second video acquisition device.

[0041] Therefore, in step S201, after capturing the video image, the first video capture device can perform target recognition processing on the captured video frame. For example, in step S201, features of each object in the video frame can be extracted, and based on the extracted features, whether each object is a drone type can be determined. Furthermore, if a particular object or objects are determined to be a drone type, then that object or objects can be identified as the target drone. In particular, if multiple drone-type objects are identified in a video frame, these multiple objects can be identified as different drones based on the differences in their respective features.

[0042] S202: When the first matching operation is successful, the first storage table is updated using the current location information of the target UAV.

[0043] In step S202, the first video acquisition device may perform a matching operation in the first storage table based on the target drone information identified from the video frame it has already captured in step S201, to find out whether there is a matching relationship between the target drone currently identified in the video frame and the features of the drone identified by the first video acquisition device in the video frame before the current video frame, which have been recorded in the first storage table.

[0044] For example, in step S202, the first video capture device can match the features of the identified target drone with the features stored in the first storage table that has been created. For example, if, through the matching operation, the features of the target drone identified in step S201 match the features of the target drone stored in the first storage table, then the feature matching operation for the target drone identified in step S201 for the video frame can be considered successful, that is, the first video capture device has detected the previously detected target drone in its field of view. Therefore, in step S202, the features and location information of the target drone identified by the first video capture device in the current video frame can be used to update the relevant information of the target drone originally stored in the first storage table based on the recognition result of the first video frame. For example, in step S202, the latest location of the target drone identified by the current first video capture device can be constantly updated and stored in the first storage table, or all location information detected before the current video frame can be stored in the first storage table to form the movement trajectory of each monitored target drone.

[0045] S203: When the first matching operation fails, the characteristic data of the target UAV is associated with the current location information and stored in the first storage table, and a second matching operation is sequentially performed in the second storage table.

[0046] In step S203, when the target drone identified in the current video frame is not matched in the first storage table of the first video acquisition device, the feature data and the current position information can be associated with each other and stored in the first storage table, and then a second matching operation can be performed one by one in the second storage table, so as to confirm whether there is a matching record in the record of the target drone previously identified by the second video acquisition device.

[0047] S204, when the second matching operation is successful, generates an entry deletion instruction, and sends the entry deletion instruction to the second video acquisition device corresponding to the second storage table that successfully matches, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

[0048] In step S204, when a drone identified in the current video frame captured by the second video capture device is matched in the second storage table created by the second video capture device and stored in the first video capture device, an entry deletion instruction can be generated for the matched entry and sent to the corresponding second video capture device, instructing the video capture device that the drone it previously monitored has moved into the current field of view of the first video capture device and has been identified by the first video capture device, thereby instructing the second video capture device to delete the entry from its local storage table.

[0049] For example, in step S204, when the first video capture device matches the characteristics of the target drone in the second storage table sent by, for example, a second video capture device located nearby, that is, the drone identified by the first video capture device in the current video frame captured by the first video capture device has moved from the field of view of a second video capture device to the field of view of the current first video capture device. In other words, for example, the target drone identified by the second video capture device in a video frame captured by the second video capture device has moved into the capture range of the first video capture device at the time corresponding to the current video frame, and therefore, the target drone is identified in the current video frame captured by the first video capture device. Therefore, an entry deletion instruction can be generated for the entry corresponding to the target drone stored in the second storage table and sent to the corresponding second video capture device. The second video capture device can then delete the entry related to the target drone recorded in its own local first storage table based on the received entry deletion instruction and send the storage table with the deleted entry to other video capture devices in the system as a second storage table.

[0050] In addition, in an embodiment of the present application, the second video acquisition device that receives the entry deletion instruction can further send all the information of the target drone that has been recorded locally before the deletion moment, such as the characteristics and historical location information of the target drone recorded in the local first storage table, to the sender of the entry deletion instruction before deleting the entry. For example, in the above case, it is the first video acquisition device, so that the first video acquisition device can transfer the historical information of the target drone originally identified and recorded by the second video acquisition device, such as the location and feature information monitored at each moment, to the first video acquisition device that currently identifies the target drone for storage, so that the first video acquisition device can achieve continuous trajectory monitoring and control of the target drone by continuing to record the moving trajectory of the target drone.

[0051] In addition, in an embodiment of the present application, each video acquisition device can send its locally stored first storage table to other video acquisition devices at a predetermined time interval to be stored as a second storage table. The predetermined time interval can be a predetermined number of video frames or a predetermined time period. The present application has no restrictions on this.

[0052] In addition, in an embodiment of the present application, when, for example, a matching operation fails in the second storage table stored in the first video acquisition device for the target drone identified in the third video frame captured by the first video acquisition device, that is, when the first video acquisition device does not find a corresponding record in the second storage table generated by the other video acquisition device for the drone identified in the current video frame, the first video acquisition device can request the other second video acquisition device to resend the second storage table in response to the matching failure in the second storage table to update the second storage table stored in the first video acquisition device. In other words, when the first video acquisition device does not find a corresponding entry in the currently stored second storage table for the drone identified in the current video frame, it may be because the second storage table stored by the first video acquisition device is not the latest table, and therefore the first video acquisition device can request the other video acquisition device to send the latest locally stored storage table to the first video acquisition device, and then perform another matching operation to confirm whether the newly found drone is a target drone that has been previously monitored and recorded by the other video acquisition device.

[0053] In addition, in an embodiment of the present application, the first video acquisition device can also count the number of unmatched entries in the first storage table that have not been successfully matched when performing object recognition on each video frame, and when the number reaches a certain threshold, delete the entry from the first storage table, or the first video acquisition device can also set a time threshold for each entry in the first storage table. If it is not matched during the predetermined time threshold, or its position information is not updated, the entry can be set to be deleted from the first storage table when the time threshold is exceeded. In particular, in an embodiment of the present application, in response to the deletion of these entries from the first storage table, the first video acquisition device can also put them into a third storage table for further storage for a period of time, and when the drone identified in each subsequent video frame fails to match in both the first storage table and the second storage table, further feature matching can be performed in the third storage table, and if a corresponding record is matched in the third storage table, the entry can be restored from the third storage table to the first storage table.

[0054] In addition, when the above-mentioned entry is not updated or matched for more than a predetermined time interval, the characteristic data and location information of the drone in the entry can also be sent to multiple other second video acquisition devices for further matching and storage in the first storage table locally stored in these second video acquisition devices.

[0055] An embodiment of the present application provides a location determination solution for a drone cluster based on an edge device cluster system, which stores a first storage table and multiple second storage tables in each video acquisition device, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device. Therefore, for each video acquisition device, when the captured video frame is recognized in real time, a first matching operation can be performed in the first storage table based on the feature data of the target drone, and when the match is successful, the first storage table can be updated with the location information of the target drone, and when the first matching operation fails, the target drone will be updated. The characteristic data of the drone and its location information are associated and stored in the first storage table, and the second matching operation is performed on multiple second storage tables in sequence. When the second matching operation is successful, an entry deletion instruction is generated to instruct the corresponding second video acquisition device to delete the entry of the target drone from its corresponding storage table. Therefore, multiple video acquisition devices can share the characteristics and locations of drones that they have discovered and tracked through multiple storage tables, and can also achieve mutual response and cooperation among video acquisition devices by matching in both shared tables when the target drone is discovered, so as to closely link multiple video acquisition devices together, and greatly improve the efficiency and accuracy of determining the location of the drone cluster through collaboration.

[0056] Example 3

[0057] Figure 3 This is a schematic diagram of the structure of an embodiment of the edge device cluster system provided by this application, which can be used to perform the following Figure 2 The method steps shown are as follows. Figure 3 As shown, the edge device cluster system may include: a first video acquisition device 31 and multiple second video acquisition devices 32.

[0058] The first video capture device 31 and multiple second video capture devices can communicate with each other to form a multi-node collaborative positioning system. In this system, the first video capture device 31 and the second video capture device 32 can each capture video frames within their respective fields of view and store them in their respective storage spaces. The first video capture device 31 can perform real-time image recognition processing on the captured video frames. The first video capture device 31 can store multiple storage tables to associate and store the characteristic data and location information of the drone collected by the first video capture device 31 and the second video capture device 32.

[0059] For example, in the embodiment of the present application, the first video capture device 31 may store a first storage table and multiple second storage tables. The first storage table may be generated by the first video capture device 31, and the multiple second storage tables may be generated by the multiple second video capture devices 32 respectively.

[0060] Therefore, after capturing video images, the first video capture device 31 can perform target recognition processing on the captured video frames. For example, it can extract features of each object in the video frame and determine whether each object is a drone based on the extracted features. Furthermore, if a particular object or objects are determined to be drones, then that object or objects can be identified as target drones. In particular, if multiple drone-type objects are identified in a video frame, these objects can be identified as different drones based on the differences in their respective features.

[0061] The first video acquisition device 31 can perform a matching operation in the first storage table based on the target drone information identified from the video frame it has already captured, to find out whether there is a matching relationship between the target drone currently identified in the video frame and the features of the drone identified by the first video acquisition device in the video frame before the current video frame, which have been recorded in the first storage table.

[0062] For example, the first video acquisition device 31 can match the features of the identified target drone with the features stored in the first storage table that has been created. For example, through the matching operation, the features of the identified target drone are matched with the features of the target drone stored in the first storage table, then it can be considered that the feature matching operation for the target drone identified for the video frame is successful, that is, the first video acquisition device 31 monitors the target drone that has been monitored before in its field of view. Therefore, the features and position information of the target drone identified by the first video acquisition device 31 in the current video frame can be used to update the relevant information of the target drone originally stored in the first storage table based on the recognition result of the first video frame. For example, the latest position of the target drone identified by the current first video acquisition device 31 can be always updated and stored in the first storage table, or all position information detected before the current video frame can be stored in the first storage table to form the movement trajectory of each monitored target drone.

[0063] When the target drone identified in the current video frame is not matched in the first storage table of the first video acquisition device 31, the feature data and the current position information can be stored in the first storage table in association with each other, and then a second matching operation can be performed one by one in the second storage table, so as to confirm whether there is a matching record in the record of the target drone identified before by the second video acquisition device 32.

[0064] When a drone identified in the current video frame captured by the second video capture device 32 is matched in the second storage table stored in the first video capture device 31 and created by the second video capture device 32, an entry deletion instruction can be generated for the matched entry and sent to the corresponding second video capture device 32, indicating to the video capture device 32 that the drone it previously monitored has moved to the current field of view of the first video capture device 31 and has been identified by the first video capture device 31, thereby instructing the second video capture device 32 to delete the entry from its local storage table.

[0065] For example, when the first video capture device 31 matches the characteristics of a target drone in the second storage table sent by, for example, a second video capture device 32 located nearby, that is, the drone identified by the first video capture device 31 in the current video frame it captures has moved from the field of view of a second video capture device 32 to the field of view of the current first video capture device 31. In other words, for example, the target drone identified by the second video capture device 32 in a video frame it captures has moved into the capture range of the first video capture device 31 at the time corresponding to the current video frame, and therefore, the target drone is identified in the current video frame captured by the first video capture device 31. Therefore, an entry deletion instruction can be generated for the entry corresponding to the target drone stored in the second storage table and sent to the corresponding second video capture device 32. In response to the received entry deletion instruction, the second video capture device 32 can delete the entry related to the target drone from its own local first storage table and send the storage table with the deleted entry to other video capture devices in the system as a second storage table.

[0066] In addition, in an embodiment of the present application, the second video acquisition device 32 that receives the entry deletion instruction can further send all the information of the target drone that has been recorded locally before the deletion moment, such as the characteristics and historical location information of the target drone recorded in the local first storage table, to the sender of the entry deletion instruction, that is, the first video acquisition device 31, before deleting the entry. In this way, the first video acquisition device 31 can transfer the historical information of the target drone originally identified and recorded by the second video acquisition device 32, such as the location and feature information monitored at each moment, to the first video acquisition device 31 that currently identifies the target drone for storage, so that the first video acquisition device 31 can achieve continuous trajectory monitoring and control of the target drone by continuing to record the movement trajectory of the target drone.

[0067] In addition, in an embodiment of the present application, each video acquisition device can send its locally stored first storage table to other video acquisition devices at a predetermined time interval to be stored as a second storage table. The predetermined time interval can be a predetermined number of video frames or a predetermined time period. The present application has no restrictions on this.

[0068] Furthermore, in an embodiment of the present application, when, for example, a matching operation fails in the second storage table stored in the first video capture device 31 for the target drone identified in the current video frame captured by the first video capture device 31, that is, when the first video capture device 31 does not find a corresponding record in the second storage table generated by another video capture device for the drone identified in the current video frame, the first video capture device 31 can, in response to the matching failure in the second storage table, request the other second video capture device 32 to resend the second storage table to update the second storage table stored in the first video capture device 31. In other words, when the first video capture device 31 does not find a corresponding entry in the currently stored second storage table for the drone identified in the current video frame, it may be because the second storage table stored by the first video capture device 31 is not the most recent table. Therefore, the first video capture device 31 can request the other video capture device 32 to send the latest locally stored storage table to the first video capture device, and then perform another matching operation to confirm whether the newly found drone is a target drone that has been previously monitored and recorded by the other video capture device.

[0069] In addition, in an embodiment of the present application, the first video acquisition device 31 can also count the number of unmatched entries in the first storage table that have not been successfully matched when performing object recognition on each video frame, and when the number reaches a certain threshold, delete the entry from the first storage table, or the first video acquisition device 31 can also set a time threshold for each entry in the first storage table. If it is not matched during the predetermined time threshold, or its position information is not updated, the entry can be set to be deleted from the first storage table when the time threshold is exceeded. In particular, in an embodiment of the present application, in response to the deletion of these entries from the first storage table, the first video acquisition device can also put them into a third storage table for further storage for a period of time, and when the drone identified in each subsequent video frame fails to match in both the first storage table and the second storage table, further feature matching can be performed in the third storage table, and if a corresponding record is matched in the third storage table, the entry can be restored from the third storage table to the first storage table.

[0070] In addition, when the above-mentioned entry is not updated or matched for more than a predetermined time interval, the characteristic data and location information of the drone in the entry can also be sent to multiple other second video acquisition devices for further matching and storage in the first storage table locally stored in these second video acquisition devices.

[0071] The edge device cluster system provided by the embodiment of the present application stores a first storage table and multiple second storage tables in each video acquisition device, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device. Therefore, for each video acquisition device, when the captured video frame is recognized in real time, a first matching operation can be performed in the first storage table according to the feature data of the target drone, and when the match is successful, the first storage table can be updated with the location information of the target drone. When the first matching operation fails, the feature data of the target drone is matched with the first storage table. The position information association pairs are stored in the first storage table, and the second matching operation is performed on multiple second storage tables in sequence. When the second matching operation is successful, an entry deletion instruction is generated to instruct the corresponding second video acquisition device to delete the entry of the target drone from its corresponding storage table. Therefore, multiple video acquisition devices can share the characteristics and positions of drones that they have discovered and tracked through multiple storage tables, and can also achieve mutual response and cooperation among video acquisition devices by matching in both shared tables when the target drone is discovered, thereby closely linking multiple video acquisition devices together and greatly improving the efficiency and accuracy of position determination of drone clusters through collaboration.

[0072] Example 4

[0073] The above describes the internal functions and structure of the edge device cluster system. The device can be implemented as an electronic device. Figure 4 This is a schematic diagram of the structure of an electronic device embodiment provided by this application. Figure 4 As shown, the electronic device includes a memory 41 and a processor 42 .

[0074] Memory 41 is used to store programs. In addition to the aforementioned programs, memory 41 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, images, videos, etc.

[0075] The memory 41 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0076] Processor 42 is not limited to a central processing unit (CPU) but may also be a processing chip such as a graphics processing unit (GPU), a field programmable gate array (FPGA), an embedded neural network processor (NPU), or an artificial intelligence (AI) chip. Processor 42 is coupled to memory 41 and executes a program stored in memory 41. When the program is executed, the method for determining the location of a drone cluster based on an edge device cluster system of the second embodiment described above is executed.

[0077] Further, if Figure 4 As shown, the electronic device may further include: a communication component 43, a power component 44, an audio component 45, a display 46 and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 Components shown.

[0078] The communication component 43 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 43 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 43 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0079] The power supply assembly 44 provides power to various components of the electronic device. The power supply assembly 44 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0080] The audio component 45 is configured to output and / or input audio signals. For example, the audio component 45 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 41 or transmitted via the communication component 43. In some embodiments, the audio component 45 also includes a speaker for outputting audio signals.

[0081] The display 46 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0082] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining the location of a drone cluster based on an edge device cluster system, characterized in that: The edge device cluster system includes: multiple video acquisition devices, each of the video acquisition devices stores multiple storage tables for associating and storing feature data and location information of drones acquired by the video acquisition devices, and the multiple storage tables include a first storage table and multiple second storage tables, wherein the first storage table is generated by the current first video acquisition device, and the multiple second storage tables are respectively generated by multiple second video acquisition devices other than the first video acquisition device, wherein the method includes: Perform image recognition processing on the collected video frames in real time; When a target UAV is identified, performing a first matching operation in the first storage table according to the feature data of the target UAV; When the first matching operation is successful, updating the first storage table using the current location information of the target drone; When the first matching operation fails, the characteristic data of the target drone is associated with the current location information and stored in the first storage table, and a second matching operation is sequentially performed in the second storage table; When the second matching operation is successful, an entry deletion instruction is generated and sent to the second video acquisition device corresponding to the second storage table that has successfully matched, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

2. The position determination method according to claim 1, wherein: The method further comprises: Every first preset time period, the current first storage table is sent to multiple second video acquisition devices for storage.

3. The method for determining a position according to claim 1, wherein: The method further comprises: When the second matching operation fails, a storage table update request is generated, and the storage table update request is sent to the plurality of second video acquisition devices respectively, so that the second video acquisition devices update the second storage table.

4. The method for determining a position according to claim 1, wherein: The method further comprises: The first storage table is monitored, and when a target entry has not been updated for a period exceeding a second preset time period, the target entry is deleted.

5. The position determination method according to claim 1, wherein: The method further comprises: The first storage table is monitored, and when the target entry has not been updated for more than a second preset time period, the characteristic data and location information of the drone in the target entry are pushed to multiple second video acquisition devices respectively, and the target entry is deleted.

6. The method for determining a position according to claim 4, wherein: The method further includes: in response to the deletion of the target entry, placing the target entry into a third storage table, where the third storage table is created by the first video acquisition device.

7. The position determination method according to claim 6, characterized in that: The method further comprises: When the feature data of the drone received and pushed by the second video acquisition device does not match the feature data of the target drone identified in the current video frame of the first video acquisition device, the feature data and location information of the drone pushed by the second video acquisition device are associated and stored in the third storage table.

8. An edge device cluster system, characterized in that: include: Multiple video capture devices, each of which stores multiple storage tables for associating and storing characteristic data and location information of drones captured by the video capture device, and the multiple storage tables include a first storage table and multiple second storage tables, wherein the first storage table is generated by the current first video capture device, and the multiple second storage tables are respectively generated by multiple second video capture devices other than the first video capture device, wherein each of the video capture devices performs the following operations as the first video capture device: Image recognition processing is performed on the collected video frames in real time. When a target drone is identified, a feature matching operation is performed in the first storage table based on the feature data of the target drone. If the match is successful, the first storage table is updated with the current position information of the target drone. If the match fails, the feature data of the target drone is associated with the current position information and stored in the first storage table, and feature matching operations are performed in sequence in the second storage table. If the match is successful, an entry deletion instruction is generated and sent to the second video acquisition device corresponding to the second storage table that successfully matches, so that the second video acquisition device deletes the entry about the target drone in its corresponding storage table according to the entry deletion instruction.

9. A computer-readable storage medium having stored thereon a computer program executable by a processor, wherein: When the program is executed by a processor, the position determination method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: Memory, used to store programs; A processor, configured to run the program stored in the memory to execute the position determination method according to any one of claims 1 to 7.

Images