Unmanned aerial vehicle supervision method and related device

Abstract

The unmanned aerial vehicle supervision method and the related device are applied to ground communication equipment, the ground communication equipment is in wireless communication with a target unmanned aerial vehicle, the ground communication equipment comprises a directional antenna unit and an image acquisition unit, the visual distance of the image acquisition unit is larger than that of human eyes, and a current image is acquired through the image acquisition unit; determining unmanned aerial vehicle position information in the current image; and generating a rotation angle adjustment instruction of the ground communication equipment according to the unmanned aerial vehicle position information, so that the target unmanned aerial vehicle is located in a directional signal coverage range of the directional antenna unit. The target unmanned aerial vehicle is monitored through the image information acquired by the image acquisition unit, and the angle range between the ground communication equipment and the target unmanned aerial vehicle is dynamically regulated and controlled to form a closed-loop tracking prompt system, so that the ground communication equipment and the high-gain antenna of the target unmanned aerial vehicle can be aligned with each other, and the communication quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of drones, and in particular to a drone monitoring method and related devices. Background Art

[0002] Unmanned drone flight is a growing trend, currently evolving from semi-automated to automated. Beyond the current semi-automated state, active safety measures need to be added. During operations, drones must maintain wireless communication with ground equipment to transmit data and control commands. This wireless communication method, for example, uses Wi-Fi.

[0003] When the distance between the drone and the ground equipment is far, the communication between the two is easily interfered, which affects the communication quality, reduces the communication rate, and affects the image transmission effect. Summary of the Invention

[0004] The purpose of the present invention is to provide a drone monitoring method and related devices to improve the above-mentioned problems.

[0005] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for monitoring a drone, which is applied to a ground communication device that wirelessly communicates with a target drone. The ground communication device includes a directional antenna unit and an image acquisition unit. The image acquisition unit has a visual range greater than that of the human eye. The method includes:

[0007] Acquire the current image through the image acquisition unit;

[0008] Determining the position information of the UAV in the current image, wherein the UAV position information is the position information of the target UAV in wireless communication with the ground communication device in the current image;

[0009] A rotation angle adjustment instruction for the ground communication device is generated according to the UAV position information so that the target UAV is within the directional signal coverage range of the directional antenna unit.

[0010] In a second aspect, an embodiment of the present invention provides a drone monitoring device, which is applied to a ground communication device. The ground communication device wirelessly communicates with a target drone. The ground communication device includes a directional antenna unit and an image acquisition unit. The image acquisition unit has a visual range greater than that of the human eye. The device includes:

[0011] a first processing unit, configured to acquire a current image through the image acquisition unit;

[0012] The first processing unit is further configured to determine the position information of the UAV in the current image, wherein the UAV position information is the position information of the target UAV in wireless communication with the ground communication device in the current image;

[0013] The second processing unit is used to generate a rotation angle adjustment instruction for the ground communication equipment according to the UAV position information, so that the target UAV is within the directional signal coverage range of the directional antenna unit.

[0014] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0015] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above method is implemented.

[0016] In a fifth aspect, an embodiment of the present invention provides a monitoring system, comprising a ground communication device and a target drone, wherein the target drone wirelessly communicates with the ground communication device, the ground communication device comprising a directional antenna unit and an image acquisition unit, wherein the image acquisition unit has a visual range greater than that of the human eye;

[0017] The supervision system is used to execute the above method.

[0018] Compared to the prior art, the drone monitoring method and related device provided in an embodiment of the present invention are applied to ground communication equipment. The ground communication equipment wirelessly communicates with the target drone. The ground communication equipment includes a directional antenna unit and an image acquisition unit. The visual range of the image acquisition unit is greater than the visual range of the human eye. The method includes: obtaining the current image through the image acquisition unit; determining the drone position information in the current image, wherein the drone position information is the position information of the target drone in wireless communication with the ground communication equipment in the current image; and generating a rotation angle adjustment instruction for the ground communication equipment based on the drone position information to place the target drone within the directional signal coverage range of the directional antenna unit. The target drone is monitored using the image information collected by the image acquisition unit, and the angle range between the ground communication equipment and the target drone is dynamically adjusted to form a closed-loop tracking and prompting system. This allows the high-gain antennas of the ground communication equipment and the target drone to be aligned with each other, thereby improving communication quality.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of the architecture of a monitoring system provided by an embodiment of the present invention.

[0022] Figure 2 This is a flow chart of a method for monitoring a drone according to an embodiment of the present invention.

[0023] Figure 3 A schematic diagram of a blind spot provided by an embodiment of the present invention.

[0024] Figure 4 A schematic diagram of the camera offset angle provided for the implementation of the present invention.

[0025] Figure 5 The second flowchart of the drone monitoring method provided by the embodiment of the present invention.

[0026] Figure 6 A schematic diagram of zooming provided by an embodiment of the present invention.

[0027] Figure 7 The third flowchart of the drone monitoring method provided by the embodiment of the present invention.

[0028] Figure 8 This is a fourth flow chart of the drone monitoring method provided in an embodiment of the present invention.

[0029] Figure 9 A schematic diagram of the units of a drone monitoring device provided in an embodiment of the present invention.

[0030] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0031] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 501 - first processing unit; 502 - second processing unit. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0034] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance.

[0035] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0036] In the description of the present invention, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is usually placed when in use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limiting the present invention.

[0037] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "disposed" and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections, or electrical connections; direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0038] The following embodiments of the present invention are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0039] The embodiment of the present invention provides a monitoring system for drones, please refer to Figure 1 , Figure 1 A schematic diagram of the architecture of a monitoring system provided by an embodiment of the present invention.

[0040] The surveillance system includes ground communication equipment and a target drone. The target drone wirelessly communicates with the ground communication equipment. The ground communication equipment includes a directional antenna unit and an image acquisition unit. The image acquisition unit has a visual range greater than that of the human eye. Optionally, the directional antenna unit and the image acquisition unit are located on the same side.

[0041] Among them, the ground communication equipment can be but is not limited to a handheld remote control. During the supervision of the drone, the staff can adjust the horizontal angle and / or vertical angle of the remote control according to the prompts to ensure that the target drone is within the directional signal coverage range of the directional antenna unit.

[0042] To improve communication distance and quality at long distances, directional antennas are used to increase range and anti-interference capabilities. Directional antennas have high gain within their effective horizontal and vertical angles. Therefore, at extreme distances, communication is only possible within the horizontal angle range. Outside this range, communication quality is affected. The vertical angle is fixed using other methods (such as the gimbal described below).

[0043] The human eye's field of view is approximately 95° outward, 60° inward, 60° upward, and 75° downward. The maximum visual distance is approximately 200 times the object's height. A drone's outer dimensions are approximately 1 meter, so visual control and aiming at distances greater than 200 meters is difficult. When a drone flies beyond 200 meters, visual observation is essentially impossible. Therefore, it is more suitable to integrate an image acquisition unit for automatic control. This adjusts the angular range between ground communication equipment and the target drone, improving communication distance and quality, while also enhancing the safety of automated flight and meeting basic monitoring and control requirements.

[0044] Optionally, the gain direction of the directional antenna unit is substantially consistent with the orientation of the image acquisition unit, and a deviation therebetween is less than a preset deviation amplitude.

[0045] When the viewing angle corresponding to the image acquisition unit is greater than or equal to the horizontal angle of the directional signal, and when the viewing angle of the camera used by the image acquisition unit is wide enough (greater than or equal to the horizontal angle of the directional signal), the image acquisition unit can use only one camera. When the viewing angle corresponding to the camera is less than the horizontal angle of the directional signal, the image acquisition unit needs to use multiple cameras (such as Figure 1 two cameras shown).

[0046] In order to achieve accurate tracking and control during the UAV monitoring process, ensure the communication quality between the ground communication equipment and the target UAV, and avoid errors caused by human operation, the embodiment of the present invention also provides an optional implementation method, please continue to refer to Figure 1 ,The supervision system can also include a pan-tilt platform where ground ,communication equipment is deployed.

[0047] The gimbal is used to execute a rotation angle adjustment command to bring the target drone within the directional signal coverage of the directional antenna unit. This rotation angle adjustment command can be a horizontal rotation angle adjustment command, which is a prompt command for adjusting the horizontal rotation angle of ground communication equipment. It can be a prompt command indicating counterclockwise or clockwise rotation in the horizontal direction, and can also include a specific horizontal rotation angle, which is not limited here.

[0048] It should be noted that the vertical angle of the gimbal is adjustable and the height of the gimbal is adjustable.

[0049] Please continue to refer to Figure 1 The monitoring system may also include a control center that is wirelessly connected to the ground communication equipment. The control center may be, but is not limited to, a mobile phone, a computer, or other smart terminal devices. The wireless communication connection between the two may be, but is not limited to, WiFi connection or Bluetooth connection.

[0050] Ground communication equipment can serve as a relay between the control center and the target drone, transmitting commands from the control center to the target drone and returning data collected by the target drone to the control center. During this process, only the angular range between the ground communication equipment and the target drone needs to be adjusted, allowing the control center (such as a mobile phone) to move flexibly, avoiding any restrictions on the staff holding the control center.

[0051] In an embodiment of the present invention, the following UAV monitoring method may be executed by a control center or ground communication equipment to obtain corresponding rotation angle adjustment instructions.

[0052] When the control center obtains the rotation angle adjustment instruction, the control center can be connected to the pan-tilt head via wireless communication, thereby sending the rotation angle adjustment instruction to the pan-tilt head.

[0053] When the ground communication device obtains the rotation angle adjustment instruction, the ground communication device can communicate with the pan-tilt platform (which can be a wired or wireless communication connection) to send the rotation angle adjustment instruction to the pan-tilt platform.

[0054] The drone monitoring method provided by the embodiment of the present invention can be applied to, but not limited to, Figure 1 For detailed procedures, please refer to the control center or ground communication equipment shown. Figure 2 ,UAV supervision methods include: S11, S12 and S13, which are described in detail as follows.

[0055] S11, acquiring a current image through an image acquisition unit.

[0056] S12, determining the position information of the UAV in the current image.

[0057] The UAV location information is the location information of the target UAV in the current image that is in wireless communication with the ground communication equipment.

[0058] S13, generating a rotation angle adjustment instruction for the ground communication equipment according to the UAV position information, so that the target UAV is within the directional signal coverage range of the directional antenna unit.

[0059] Among them, the rotation angle adjustment instruction can be a horizontal rotation angle adjustment instruction, which is a prompt instruction for adjusting the horizontal rotation angle of the ground communication equipment. It can be a prompt instruction, prompting counterclockwise or clockwise rotation in the horizontal direction, and can also include a specific horizontal rotation angle.

[0060] When the ground communication device is a handheld remote controller, the operator can adjust the horizontal and / or vertical angles of the remote controller in response to the rotation angle adjustment command to bring the target UAV within the directional signal coverage of the directional antenna unit. When the ground communication device is deployed on a gimbal, the gimbal can execute the rotation angle adjustment command to bring the target UAV within the directional signal coverage of the directional antenna unit.

[0061] The target UAV is within the directional signal coverage of the directional antenna unit, which may mean that the horizontal angle between the target UAV and the directional antenna unit is less than or equal to 1 / 2 of the horizontal antenna angle.

[0062] In the drone monitoring method provided in an embodiment of the present invention, the target drone is monitored through image information collected by the image acquisition unit, and the angle range between the ground communication equipment and the target drone is dynamically adjusted to form a closed-loop tracking prompt system, so that the high-gain antennas of the ground communication equipment and the target drone can be aligned with each other, thereby improving communication quality.

[0063] Regarding the structure of the directional antenna unit and the image acquisition unit, the embodiment of the present invention also provides an optional implementation method, please continue to refer to Figure 1 The directional antenna unit includes a first directional antenna and a second directional antenna (not shown in the figure), and the image acquisition unit includes a first camera and a second camera. The first camera is located at the left end of the directional antenna unit, and the second camera is located at the right end of the directional antenna unit. The maximum gain direction of the first directional antenna is consistent with the direction of the first camera, and the maximum gain direction of the second directional antenna is consistent with the direction of the second camera. The first directional antenna and the second directional antenna can be set on the same side.

[0064] The current image includes a first current image captured by the first camera and a second current image captured by the second camera.

[0065] When the first directional antenna and the second directional antenna inside the ground communication device are simultaneously aimed at the target UAV, the strength of the wireless communication signal (Wi-Fi RSSI) between the ground communication device and the target UAV is the highest and the communication quality is the best.

[0066] The viewing distance and viewing angle of a camera are related to its focal length. The larger the focal length, the longer the corresponding viewing distance and the smaller the (horizontal) viewing angle. A 120mm focal length increases the viewing distance by 2.4 times compared to a 50mm focal length (50mm is close to the focal length of the human eye and can observe drones 200 meters away). 120mm is equivalent to being able to observe drones 480 meters away. At 50mm, the horizontal viewing angle is 39.6°; but at 120mm, the viewing angle b is only 18.45°. At this time, the horizontal viewing angle of the camera is smaller than the horizontal angle of the directional signal, so there is a blind spot such as Figure 3 As shown, Figure 3 A schematic diagram of a blind spot provided by an embodiment of the present invention. Figure 3 In the figure, the dotted line shows the horizontal viewing angle of 12° corresponding to the 200mm focal length lens; the solid line shows the horizontal angle of the directional signal of 30°. Obviously, the horizontal viewing angle of the camera does not cover the horizontal angle of the directional signal, and there is a blind spot.

[0067] When the horizontal viewing angle of the camera using the standard focal length is smaller than the horizontal angle of the directional signal, in order to overcome the blind spot problem, two cameras can be used. Both the first camera and the second camera need to be offset from the directional antenna unit by a preset angle. The preset angle corresponds to the shortest horizontal distance of the operation and the standard focal length. When the UAV is directly in front of the ground terminal device and the distance is the shortest horizontal distance, the camera can take pictures according to the standard focal length and can clearly capture the UAV.

[0068] Please refer to Figure 4 , Figure 4 Schematic diagram of the camera offset angle provided for the implementation of the present invention. Figure 4 In the example, the first camera is initially offset 11.55° to the right for supplementation; the second camera is initially offset 11.55° to the left for supplementation, so that both cameras can simultaneously observe the target drone at a distance of 800 meters.

[0069] Take the Sony IMX586 phone parameters, 48 ​​million pixel periscope telephoto lens, F4.1 aperture, 1 / 2 inch bottom (6.4mm X 4.8mm), 120mm equivalent focal length.

[0070] The 120mm equivalent focal length (derived by an equivalent factor of 5.4; the actual physical focal length is only 22.2mm) and the aperture is F4.1. The physical aperture is 22.2 / 4.1 = 5.41mm, and the equivalent aperture is F4.1 x 5.4 = F22.14. Therefore, the imaging effect of this camera is darker, requiring a longer exposure, and there is a tailing phenomenon when shooting dynamic objects.

[0071] At different focal lengths, the distance at which the image can be clearly focused is at least greater than the hyperfocal distance:

[0072]

[0073] Where f is the focal length, N is the aperture, c is the diameter of the discrete circle (the degree of blur that humans can accept varies with different sensor sizes, and can be set to 0.03mm), and H is the hyperfocal distance.

[0074] Different focal lengths have different minimum clear object distances:

[0075]

[0076] Where d represents the current focus distance (the distance between the camera and the drone).

[0077] A 120mm focal length corresponds to a hyperfocal distance of 117,193mm, allowing focus to be achieved at a distance of 117,193 meters. Assuming the drone is 800 meters (800,000mm) from the ground, with a 120mm focal length and focusing at 800,000mm, the corresponding minimum clear object distance is 511,161mm.

[0078] Building on the above, regarding the content in S13, how to ensure that the angle between the ground communication equipment and the target drone is within the optimal communication angle range to ensure high-quality communication between the two, this embodiment of the present invention also provides an optional implementation method, which is referenced below. S13, generating a rotation angle adjustment instruction for the ground communication equipment based on the drone's position information, includes: S134, S135, and S136, which are described in detail below.

[0079] S134 , when the target UAV is located to the right of the center line of the first current image and to the left of the center line of the second current image, a counterclockwise rotation instruction is generated for the ground communication equipment.

[0080] This counterclockwise rotation command instructs ground communication equipment to rotate counterclockwise horizontally to achieve angle correction. When a device such as a gimbal executes this command, the amplitude of a single rotation can be either the preset minimum rotation amplitude or the amplitude specified in the counterclockwise rotation command.

[0081] S135 , when the target UAV is located on the left side of the center line of the first current image and on the right side of the center line of the second current image, a clockwise rotation instruction is generated for the ground communication device.

[0082] The clockwise rotation command instructs ground communication equipment to rotate horizontally clockwise to achieve angle correction. When the clockwise rotation command is executed by a device such as a gimbal, the amplitude of a single rotation can be the preset minimum rotation amplitude or the amplitude specified in the counterclockwise rotation command.

[0083] S136, when the target UAV is located on the center line of the first current image and the center line of the second current image, the horizontal rotation angle of the ground communication equipment remains unchanged.

[0084] When the target drone is located on the center line of the first current image and the center line of the second current image, it means that the first directional antenna and the second directional antenna inside the ground communication device are simultaneously aimed at the target drone. The wireless communication signal (Wi-Fi RSSI) strength between the ground communication device and the target drone is the highest, the communication quality is the best, and the horizontal rotation angle of the ground communication device remains unchanged.

[0085] Optionally, when the distance between the target drone and the center line in the image is less than a preset distance, it can be determined that the target drone is located on the center line.

[0086] During the operation, the distance between the target drone and the ground communication equipment may change. If the current focal length of the image acquisition unit on the ground communication equipment remains unchanged, when the target drone gradually moves away from the ground communication equipment, the number of actual display pixels of the target drone will gradually decrease at the current focal length, resulting in a decrease in the recognition features of the target drone, affecting the recognition accuracy of the target drone. To overcome this problem, the embodiment of the present invention also provides an optional implementation method, please refer to Figure 5 When the target drone is located on the central line of the first current image and on the central line of the second current image, the drone supervision method further includes: S21, S22, S23, S24 and S25, which are specifically described as follows.

[0087] S21, obtaining the pixel ratio of the target drone in the current image.

[0088] Optionally, the pixel percentage of the target drone in the first current image and / or the second current image is obtained. It should be noted that, in equivalent solutions of the embodiments of the present invention, the number of pixels of the target drone in the current image (instead of the pixel percentage) may also be obtained, and the corresponding threshold may be adaptively adjusted, which is not further described here.

[0089] S22: When the pixel ratio is less than the first category current threshold, determine whether the currently set focal length is the maximum focal length of the image acquisition unit. If not, execute S23; if so, execute S24.

[0090] The first category current threshold is the pixel ratio threshold corresponding to the currently set focal length for clear identification. When the pixel ratio is greater than or equal to the first category current threshold, it indicates that the target drone can be clearly identified and the focal length can be adjusted.

[0091] S23, controlling the first camera and the second camera to improve the current set focal length based on the current set focal length to execute the new current set focal length and perform image acquisition.

[0092] After executing the new current set focal length and performing image acquisition, S11 and its subsequent steps can be repeated, repeatedly acquiring the current image through the image acquisition unit, determining the drone position information in the current image, and generating a rotation angle adjustment instruction for the ground communication equipment based on the drone position information, so that the target drone is within the directional signal coverage range of the directional antenna unit.

[0093] Optical zoom can be used to achieve camera zoom. If optical zoom is too expensive, the camera pixel count can be increased (the sensor size also needs to be increased, and the tolerance for discrete circle diameters can be improved). Digital cropping (digital zoom, cropping to the same viewing angle as optical zoom) can essentially achieve the same logical effect as optical zoom, but the maximum distance is limited by the maximum pixel count, light aperture size, and sensor area of ​​currently available cameras.

[0094] Building on the above, the present invention also provides an optional implementation for increasing the focal length of the first and second cameras in S23 to utilize digital zoom, reduce frequent optical focus adjustments, and increase response speed. Please refer to the following. After each optical zoom, if the focal length of the first and second cameras needs to be increased, digital zoom is performed according to a preset digital zoom range. The number of digital zooms is counted. When the number of digital zooms reaches a set threshold, optical zoom is performed and the digital zoom count is reset.

[0095] Please refer to Figure 6 , Figure 6 This is a zoom diagram provided by an embodiment of the present invention. When focusing on a target drone at a distance of 800 meters, the target drone is clearly visible within the field of view of the ground communication equipment at a distance of 511-800 meters at this focal length. Therefore, digital zoom can be used within this range, reducing the need for frequent optical focusing and increasing response speed.

[0096] Regarding the detectable space of ground communication equipment, the maximum distance boundary (not shown) is determined by the upper limits of the sensor pixels, camera frame, and light aperture size of the camera in the image acquisition unit. The minimum distance boundary (not shown) is determined by factors such as the focus distance and focal length.

[0097] If the pixel ratio is less than the first category current threshold, and the current focal length is the maximum focal length of the image acquisition unit, the focal length cannot be further increased. If clear recognition is still performed, the target may be lost. Therefore, it is necessary to switch to fuzzy (pixel) recognition, and then execute S24.

[0098] S24, controlling the first camera and the second camera to lower the focal length based on the currently set focal length to execute the new currently set focal length and perform image acquisition.

[0099] By lowering the current set focal length to achieve a fallback focal length, introducing more light, and using fuzzy pixel recognition, the target drone is identified and tracked, reducing the possibility of target loss.

[0100] After executing the new current focus setting and performing image acquisition, S11 and subsequent steps can be repeated. At this time, the target drone is identified using the second category current threshold corresponding to fuzzy identification as a condition. However, as the target drone continues to move away, the pixel ratio value will still be less than the second category current threshold. At this time, the focus needs to be adjusted again, and S25 is executed.

[0101] S25, when the pixel ratio value is less than the second category current threshold, controlling the first camera and the second camera to reduce the current set focal length based on the current set focal length to execute the new current set focal length for image acquisition.

[0102] Among them, the second type of current threshold is the pixel ratio threshold corresponding to the currently set focal length during blur recognition.

[0103] Building on the above, regarding the content in S13, how to ensure that the angle between the ground communication equipment and the target drone is within the optimal communication angle range to ensure high-quality communication between the two, this embodiment of the present invention also provides an optional implementation method, which is referenced below. S13, generating a rotation angle adjustment instruction for the ground communication equipment based on the drone's position information, includes: S131, S132, and S133, which are described in detail below.

[0104] S131 , when the target UAV appears in the first current image and does not appear in the second current image, generating a counterclockwise rotation instruction for the ground communication equipment.

[0105] S132 , when the target UAV does not appear in the second current image, and appears in the second current image, generating a clockwise rotation instruction for the ground communication equipment.

[0106] S133, when the target UAV appears in the first current image and the second current image at the same time, the horizontal rotation angle of the ground communication device remains unchanged.

[0107] When the target UAV appears in both the first current image and the second current image, the above steps S134 to S136 may be continued.

[0108] On the basis of the above, regarding how to track and lock the target drone when the target drone does not appear in the field of view of the image acquisition unit, that is, when the current image does not include the target drone, the embodiment of the present invention also provides an optional implementation method, please refer to Figure 7 When the target drone does not appear in the first current image and the second current image, the drone monitoring method further includes: S31, S32, S33, S34 and S35, which are described in detail as follows.

[0109] S31, determining whether the currently set focal length is the minimum focal length of the image acquisition unit. If not, executing S32; if so, executing S35.

[0110] S32, controlling the first camera and the second camera to lower the focal length based on the currently set focal length to execute the new currently set focal length to perform image acquisition.

[0111] S33: Determine whether the drone appears in the first current image or the second current image based on the first category current threshold. If the drone appears in the first current image or the second current image, then execute S34. If the drone does not appear in either the first current image or the second current image, then repeat S31 to determine whether the currently set focal length is the minimum focal length of the image acquisition unit.

[0112] Among them, the first type of current threshold is the pixel ratio threshold corresponding to the currently set focal length during clear recognition.

[0113] S34, determining the position information of the UAV in the current image, and generating a rotation angle adjustment instruction for the ground communication equipment according to the UAV position information.

[0114] The rotation angle adjustment instruction of the ground communication equipment is generated according to the position information of the UAV. The execution method is the same as S13 in the previous text, which will not be repeated here.

[0115] S35 , determining whether the drone appears in the first current image or the second current image based on a second current threshold, wherein the second current threshold is a pixel ratio threshold corresponding to the currently set focal length during blur recognition.

[0116] Optionally, when the ground communication device is started, if the target drone cannot be identified and locked, the first camera and the second camera can be directly adjusted to the minimum focal length, and fuzzy recognition can be performed to determine whether the drone appears in the first current image or the second current image based on the second type of current threshold.

[0117] Based on the second current threshold, a determination is made as to whether the drone appears in the first current image or the second current image. If the drone appears in the first current image or the second current image, the process returns to step S34. If the target drone is still not identified, an error message is issued, or the ground communication device is prompted to perform a horizontal angle correction. The communication signal strength between the ground communication device and the target drone is obtained during the rotation of the ground communication device. An initial azimuth adjustment is performed based on the changing trend of the communication signal strength. After the adjustment is completed, the target drone is repositioned and identified based on the image information captured by the image acquisition unit, and the aforementioned steps are executed.

[0118] Taking the current focal length as 120mm and the target drone not being in the field of view of the first and second cameras as an example, the drone identification and positioning process is as follows.

[0119] The search requires a reduced focal length (<120mm). For example, a 60mm focal length results in a 33.4° horizontal viewing angle. Whether the camera can actually identify the drone at a distance of 800 meters depends on its size and camera resolution. Under sufficient light, at a distance of 800 meters, the drone is 1 meter long, 240 meters to the left, 240 meters to the right, and 1 / 240 of the center. Converted to camera pixels, the camera has 48 megapixels (typically a standard aspect ratio, but simplified here by assuming a square). Taking the square root, each side of the square has 692,000 pixels, with half the plane occupying 346,000 pixels. If the half plane is visible, the drone appears as a 1441 x 1441 pixel image, sufficient to identify the drone on the left or right side of the field of view based on its outline.

[0120] If the drone still can't be found at a 60mm focal length, further reduce the focal length to expand the field of view. For example, a 30mm focal length results in a 61.9° angle of view. When the drone is 800 meters away, it's 1 meter long, 480 meters to the left, 480 meters to the right, and 1 / 480th of the distance in the middle. Converted to camera pixels, assuming a 48-megapixel square sensor, the square root yields 692,000 pixels per side, with half the plane occupying 346,000 pixels. If half the plane is visible, the drone appears as a 720x720 pixel image, sufficient to identify the drone's position on the left or right side of the field of view based on its outline.

[0121] If the drone still can't be found at a 30mm focal length, the focal length needs to be further reduced to expand the field of view, such as to 22.2mm (due to physical focal length limitations, the minimum focal length is 22.2mm), resulting in a 78° angle of view. When the drone is 800 meters away, it is 1 meter long, 648 meters to the left, 648 meters to the right, and 1 / 648 of the center. Converted to camera pixels, assuming a 48-megapixel square sensor, the square root yields 692,000 pixels per side, with half the plane occupying 346,000 pixels. If half the plane is visible, the drone appears as a 534x 534 pixel image, sufficient to identify the drone's position on the left or right side of the field of view by its outline.

[0122] Based on the above, regarding the contents of S33, this embodiment of the present invention further provides an optional implementation, which is described below. S33, based on the first current threshold, determines whether the drone appears in the first current image or the second current image, including: S331, S332, S333, S334, S335, and S336.

[0123] S331: Identify whether there is a suspected target in the current image whose pixel ratio is greater than or equal to the first category current threshold. If there is a suspected target, execute S332; if not, execute S336.

[0124] S332: Obtain suspected target location information in the current image.

[0125] S333: Adjust the horizontal rotation angle of the ground communication device according to the suspected target location information, and obtain the communication signal strength between the ground communication device and the target UAV.

[0126] S334: Determine whether the suspected target is the target drone based on the changing trend of the communication signal strength. If the suspected target is the target drone, execute S335; if all suspected targets are not the target drone, execute S336.

[0127] S335 , determining whether the drone appears in the first current image or the second current image.

[0128] S336 , determining that the drone does not appear in the first current image and the second current image.

[0129] Based on the above, regarding the contents of S35, this embodiment of the present invention further provides an optional implementation, which is described below. S35, based on the second current threshold, determines whether the drone appears in the first current image or the second current image, including: S351, S352, S353, S354, S355, and S356.

[0130] S351: Identify whether there is a suspected target in the current image whose pixel ratio is greater than or equal to the second category current threshold. If there is a suspected target, execute S352; if not, execute S356.

[0131] S352: Obtain suspected target location information in the current image.

[0132] S353: Adjust the horizontal rotation angle of the ground communication equipment according to the suspected target location information, and obtain the communication signal strength between the ground communication equipment and the target UAV.

[0133] S354: Determine whether the suspected target is the target drone based on the changing trend of the communication signal strength. If the suspected target is the target drone, execute S355; if all suspected targets are not the target drone, execute S356.

[0134] S355 , determining whether the drone appears in the first current image or the second current image.

[0135] S356 , determining that the drone does not appear in the first current image and the second current image.

[0136] In an optional embodiment, the monitoring system further includes a gimbal, and the ground communication equipment is deployed on the gimbal; the gimbal is used to execute rotation angle adjustment instructions so that the target UAV is within the directional signal coverage range of the directional antenna unit.

[0137] In an optional embodiment, the ground communication equipment is provided with a counterclockwise indicator light and a clockwise indicator light.

[0138] When the rotation angle adjustment instruction is a counterclockwise rotation instruction, the counterclockwise indicator light is turned on; when the rotation angle adjustment instruction is a clockwise rotation instruction, the clockwise indicator light is turned on.

[0139] The counterclockwise and clockwise indicators serve as prompts, and staff can rotate the ground communication equipment (remote control) according to the prompts to ensure that the target UAV is within the directional signal coverage of the directional antenna unit.

[0140] When a drone is flying in the air, it has a weaker perception of the surrounding environment, especially in the non-forward direction, which is usually not equipped with radar or other sensing equipment. Therefore, whether there are other drones with higher flying speeds (usually agricultural drones are limited to 13 meters per second) or other surveying aircraft or other flying objects, there may be a risk of collision with the drone from the non-forward direction, and timely warning and decision-making are required. To this end, the embodiment of the present invention also provides an optional implementation method, please refer to Figure 8 ,The drone supervision method also includes : S41, S42 and S43, which are described in detail as follows.

[0141] S41 , when the target UAV is locked with the currently set focal length, the image acquisition unit is controlled to dynamically adjust the focal length during the corresponding search time period to obtain an environmental image of the target UAV.

[0142] S42, controlling the target UAV to avoid danger based on the environmental image.

[0143] The visual dynamic focus system identifies target drones, distinguishing between controlled drones and uncontrolled flying objects, and thus identifying potential risks. The visual system offers a high cost-effectiveness during daytime operations. This system maintains a certain viewing angle between the ground communication equipment and the target drone (the ground communication equipment is aimed at the target drone). This allows for switching focus back and forth. While maintaining focus on the target drone (providing a longer communication range), by switching to a shorter focal length, other flying objects can be identified within a wider range. This allows analysis of potential flying objects near the target drone to determine if they pose a threat (collision or other impact), allowing the target drone to be controlled and avoided.

[0144] S43: After the search period ends, the image acquisition unit is restored to the currently set focal length to continue tracking the target drone.

[0145] The target drone tracking process includes: Figure 2 and Figure 5 Steps shown.

[0146] See also Figure 9 , Figure 9 An embodiment of the present invention provides a drone monitoring device, which can be optionally applied to the control center or ground communication equipment mentioned above. The drone monitoring device includes a first processing unit 501 and a second processing unit 502.

[0147] The first processing unit 501 is configured to obtain a current image through an image acquisition unit.

[0148] The first processing unit 501 is further configured to determine the position information of the UAV in the current image, wherein the UAV position information is the position information of the target UAV in the current image that is in wireless communication with the ground communication device.

[0149] The second processing unit 502 is configured to generate a rotation angle adjustment instruction for the ground communication device according to the UAV position information, so as to place the target UAV within the directional signal coverage range of the directional antenna unit.

[0150] Optionally, the second processing unit 502 may execute the above S13, and the first processing unit 501 may execute other steps in the above method embodiment.

[0151] It should be noted that the drone monitoring device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any part not mentioned in this embodiment can be referred to the corresponding content in the above embodiment.

[0152] Embodiments of the present invention further provide a storage medium storing computer instructions or programs that, when read and executed, execute the drone monitoring method of the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0153] The following provides an electronic device, which can be a control center or ground communication equipment in a monitoring system. Figure 10As shown, the above-mentioned drone monitoring method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the drone monitoring method of the above-mentioned embodiment is executed.

[0154] The processor 10 can be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the drone supervision method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0155] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0156] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 10 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus 12 or one type of bus 12.

[0157] Memory 11 is used to store programs, such as a program for a drone monitoring device. The drone monitoring device includes at least one software module, which can be stored in memory 11 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, processor 10 executes the program to implement the drone monitoring method.

[0158] Optionally, the electronic device provided by the embodiment of the present invention further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0159] It should be understood that Figure 10 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 10 More or fewer components than shown, or with Figure 10 Different configurations shown. Figure 10 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0160] An embodiment of the present invention also provides a monitoring system, which includes ground communication equipment and a target drone. The target drone wirelessly communicates with the ground communication equipment. The ground communication equipment includes a directional antenna unit and an image acquisition unit. The visible distance of the image acquisition unit is greater than the visible distance of the human eye. The monitoring system is used to execute the above-mentioned drone monitoring method.

[0161] In summary, the embodiment of the present invention provides a drone monitoring method and related device, which are applied to ground communication equipment. The ground communication equipment wirelessly communicates with the target drone. The ground communication equipment includes a directional antenna unit and an image acquisition unit. The visual range of the image acquisition unit is greater than the visual range of the human eye. The method includes: obtaining the current image through the image acquisition unit; determining the drone position information in the current image, wherein the drone position information is the position information of the target drone in wireless communication with the ground communication equipment in the current image; generating a rotation angle adjustment instruction for the ground communication equipment based on the drone position information, so that the target drone is within the directional signal coverage range of the directional antenna unit. The target drone is monitored through the image information collected by the image acquisition unit, and the angle range between the ground communication equipment and the target drone is dynamically adjusted to form a closed-loop tracking prompt system, so that the high-gain antennas of the ground communication equipment and the target drone can be aligned with each other, thereby improving communication quality.

[0162] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0163] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A drone monitoring method, characterized in that: Applied to a ground communication device, the ground communication device wirelessly communicates with a target UAV, the ground communication device includes a directional antenna unit and an image acquisition unit, the visual range of the image acquisition unit is greater than the visual range of the human eye, the method includes: Acquire the current image through the image acquisition unit; Determining the position information of the UAV in the current image, wherein the UAV position information is the position information of the target UAV in wireless communication with the ground communication device in the current image; A rotation angle adjustment instruction for the ground communication device is generated according to the UAV position information so that the target UAV is within the directional signal coverage range of the directional antenna unit.

2. The drone monitoring method according to claim 1, wherein: The directional antenna unit includes a first directional antenna and a second directional antenna, and the image acquisition unit includes a first camera and a second camera, the first camera is located at the left end of the directional antenna unit, and the second camera is located at the right end of the directional antenna unit, the maximum gain direction of the first directional antenna is consistent with the direction of the first camera, and the maximum gain direction of the second directional antenna is consistent with the direction of the second camera; The current image includes a first current image captured by the first camera and a second current image captured by the second camera.

3. The drone monitoring method according to claim 2, wherein: Generating a rotation angle adjustment instruction for the ground communication device according to the drone position information includes: When the target UAV is located to the right of the center line of the first current image and to the left of the center line of the second current image, generating a counterclockwise rotation instruction for the ground communication device; When the target UAV is located to the left of the center line of the first current image and to the right of the center line of the second current image, generating a clockwise rotation instruction for the ground communication device; When the target UAV is located on the central line of the first current image and the central line of the second current image, the horizontal rotation angle of the ground communication device remains unchanged.

4. The drone monitoring method according to claim 2, wherein: When the target drone is located on the center line of the first current image and the center line of the second current image, the method further includes: Obtaining a pixel ratio of the target drone in the current image; When the pixel ratio value is less than the first category current threshold, controlling the first camera and the second camera to improve based on the currently set focal length to execute the new currently set focal length for image acquisition; Among them, the first type of current threshold is the pixel ratio threshold corresponding to the currently set focal length during clear recognition.

5. The drone monitoring method according to claim 4, wherein: When the pixel ratio is less than the first category current threshold, if the currently set focal length is the maximum focal length of the image acquisition unit, the method further includes: Controlling the first camera and the second camera to lower the focal length based on the currently set focal length to execute the new currently set focal length and perform image acquisition; When the pixel ratio value is less than the second category current threshold, controlling the first camera and the second camera to reduce the focal length based on the currently set focal length to execute the new currently set focal length for image acquisition; Among them, the second type of current threshold is the pixel ratio threshold corresponding to the currently set focal length during blur recognition.

6. The drone monitoring method according to claim 2, wherein: Generating a rotation angle adjustment instruction for the ground communication device according to the drone position information includes: When the target UAV appears in the first current image and does not appear in the second current image, generating a counterclockwise rotation instruction for the ground communication device; generating a clockwise rotation instruction for the ground communication device when the target UAV does not appear in the second current image and appears in the second current image; When the target UAV appears in both the first current image and the second current image, the horizontal rotation angle of the ground communication device remains unchanged.

7. The drone monitoring method according to claim 6, wherein: The method further comprises: When the target UAV does not appear in the first current image and the second current image, determining whether the currently set focal length is the minimum focal length of the image acquisition unit; If not, controlling the first camera and the second camera to lower the focal length based on the currently set focal length to execute the new currently set focal length to perform image acquisition; Determining whether the drone appears in the first current image or the second current image according to a first current threshold, wherein the first current threshold is a pixel ratio threshold corresponding to a currently set focal length for clear identification; When the drone appears in the first current image or the second current image, determining the drone position information in the current image; generating a rotation angle adjustment instruction for the ground communication device according to the drone position information; When the drone does not appear in the first current image and the second current image, repeatedly determining whether the currently set focal length is the minimum focal length of the image acquisition unit; If the currently set focal length is the minimum focal length of the image acquisition unit, then determine whether the drone appears in the first current image or the second current image based on the second current threshold, wherein the second current threshold is a pixel ratio threshold corresponding to the currently set focal length during fuzzy recognition.

8. The drone monitoring method according to claim 7, wherein: The determining, based on the first type of current threshold, whether the drone appears in the first current image or the second current image includes: Identify whether there is a suspected target in the current image whose pixel ratio is greater than or equal to the current threshold of the first category; If there is a suspected target, obtain the suspected target location information in the current image; adjusting the horizontal rotation angle of the ground communication device according to the suspected target location information, and obtaining the communication signal strength between the ground communication device and the target UAV; Determining whether the suspected target is a target drone based on a change trend of the communication signal strength; When the suspected target is a target drone, determining that the drone appears in the first current image or the second current image; When all suspected targets are not the target drone, it is determined that the drone does not appear in the first current image and the second current image.

9. The drone monitoring method according to claim 1, wherein: The ground communication equipment is deployed on the cloud platform; The gimbal is used to execute the rotation angle adjustment instruction so that the target UAV is within the directional signal coverage range of the directional antenna unit.

10. The drone monitoring method according to claim 1, wherein: The ground communication equipment is provided with a counterclockwise indicator light and a clockwise indicator light; When the rotation angle adjustment instruction is a counterclockwise rotation instruction, controlling the counterclockwise indicator light to turn on; When the rotation angle adjustment instruction is a clockwise rotation instruction, the clockwise indicator light is controlled to turn on.

11. The drone monitoring method according to claim 1, wherein: The method further comprises: When the target UAV is locked with the currently set focal length, the image acquisition unit is controlled to dynamically adjust the focal length during the corresponding search time period to obtain an environmental image of the target UAV; Controlling the target UAV to avoid danger based on the environmental image; After the search time period ends, the image acquisition unit is restored to the currently set focal length to continue tracking the target drone.

12. A drone monitoring device, characterized in that: Applied to ground communication equipment, the ground communication equipment wirelessly communicates with a target UAV, the ground communication equipment includes a directional antenna unit and an image acquisition unit, the visual range of the image acquisition unit is greater than the visual range of the human eye, and the device includes: a first processing unit, configured to acquire a current image through the image acquisition unit; The first processing unit is further configured to determine the position information of the UAV in the current image, wherein the UAV position information is the position information of the target UAV in wireless communication with the ground communication device in the current image; The second processing unit is used to generate a rotation angle adjustment instruction for the ground communication equipment according to the UAV position information, so that the target UAV is within the directional signal coverage range of the directional antenna unit.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. An electronic device, characterized in that: include: a processor and a memory, the memory being configured to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 11 is implemented.

15. A supervision system, characterized in that: The monitoring system includes a ground communication device and a target UAV, wherein the target UAV communicates wirelessly with the ground communication device, and the ground communication device includes a directional antenna unit and an image acquisition unit, wherein the visual range of the image acquisition unit is greater than the visual range of the human eye; The supervision system is used to execute the method according to any one of claims 1 to 11.

Images