Method, device and system for measuring density of aerial ecological target objects

Abstract

The present disclosure relates to a method, device and system for measuring the density of aerial ecological target objects. The method comprises: for any one side area, determining the volume of the photographic cone area of ​​each camera and the number of aerial ecological target objects according to the principle of photogrammetry; and determining the target object density of the side area according to the photographic cone volume and the number of target objects. The present application adopts a drone carrying a camera, and determines the volume of the photographic cone of each camera and the number of target objects therein according to the principle of photogrammetry, and then determines the target object density. The auxiliary radar completes the task of large-scale quantitative monitoring of aerial ecological targets, which helps to realize real-time reporting of insect pests and insect disasters, as well as early warning of areas where the disaster is about to spread. The relevant data of the above-mentioned target objects also helps to study the relationship between meteorology and insect disasters, thereby improving the quality of meteorological services.

Description

Technical Field

[0001] The present disclosure relates to the technical field of insect pest detection, and in particular to a method, device and system for measuring the density of aerial ecological target objects. Background Art

[0002] Aerial ecological targets, such as insect and bird activity, are crucial for ecological monitoring. Traditional insect monitoring techniques primarily focus on monitoring ground insect distribution, such as using radar remote sensing to detect insect distribution on the ground. However, because insects move quickly and spend most of their time in the air, it's difficult to understand their development and changes in mid-air. Summary of the Invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, device and system for measuring the density of aerial ecological target objects to solve the above problems.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for measuring target object density is provided, the method comprising:

[0005] Multiple groups of cameras are set on the drone; wherein each group of cameras is responsible for capturing images of target objects on each side of the drone;

[0006] For any area on one side, determine the photographic cone volume of each camera and the number of target objects according to the principles of photogrammetry;

[0007] The density of the target objects in the side area is determined according to the imaging cone volume and the number of target objects.

[0008] In one embodiment, determining the volume of the photographic cone of each camera based on the quantitative measurement area of ​​the side region comprises:

[0009] Determine the quantitative measurement area on each side according to the measurement range of each group of cameras on each side;

[0010] determining the coordinates of each target object within the quantitative measurement area in a real space coordinate system;

[0011] Determine, according to the coordinates of each target object, a target object with the largest distance in a first direction and a distance in the first direction, wherein the distance in the first direction is the distance between the target object and the plane of the camera lens group;

[0012] determining a base radius of the photographic cone according to the distance and a predetermined camera shooting angle;

[0013] The volume of the photographic cone and the quantitative measurement area are determined according to the base radius and the distance.

[0014] In one embodiment, determining the coordinates of each target object in the quantitative measurement area in the real space coordinate system includes:

[0015] Determine first image space auxiliary coordinates and second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras;

[0016] determining the length of a baseline between the two cameras; and a projection vector of the baseline;

[0017] determining a first projection coefficient or a second projection coefficient according to the projection vector of the baseline, the first image space auxiliary coordinates, and the second image space auxiliary coordinates;

[0018] Determine the first direction distance according to the first projection coefficient and the focal length of a camera; or,

[0019] The first direction distance is determined according to the second projection coefficient and a focal length of another camera.

[0020] In one embodiment, determining first image space auxiliary coordinates of the target object in the first camera coordinate system of the group of cameras includes:

[0021] determining first image space coordinates of a camera in the set of cameras;

[0022] The first image space auxiliary coordinates are determined according to a pre-determined first rotation matrix and the first image space coordinates.

[0023] In one embodiment, determining the second image space auxiliary coordinates of the target object in the second camera coordinate system of the group of cameras includes:

[0024] determining a second image space coordinate in another camera in the set of cameras;

[0025] The second image space auxiliary coordinates are determined according to a pre-determined second rotation matrix and the second image space coordinates.

[0026] In one embodiment, determining the number of target objects within the photographic cone based on the quantitative measurement area of ​​the side region includes:

[0027] determining a first quantity of target objects within the quantitative measurement region;

[0028] determining a second number of target objects in a non-quantitative measurement area based on the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area;

[0029] Specifically comprising: for each target object in the non-quantitative measurement area, in response to the image of the target object being larger than the image of the target object with the largest first direction distance in the quantitative measurement area, determining that the target object is a valid target object in the non-quantitative measurement area;

[0030] Counting a second number of the valid target objects;

[0031] The number of target objects within the photography cone is determined according to the first number and the second number.

[0032] In one embodiment, after determining the density of the target objects in the side area according to the imaging cone volume and the number of target objects, the method further includes:

[0033] In response to the density value of the target object being greater than a predetermined density threshold, sending an insect plague warning message to a server of a meteorological department, wherein the insect plague warning message includes an identifier of the area where the drone is located, so that the meteorological department server obtains historical meteorological data of the area based on the area identifier;

[0034] Determine areas where insect infestation may occur and need early warning based on the historical meteorological data, and send early warning information to the areas where insect infestation may occur.

[0035] In one embodiment, a first crossbar is provided at the front end of the drone, and a first camera and a second camera are provided at the first end of the first crossbar, respectively. The camera direction of the first camera is toward the first side of the drone; the camera direction of the second camera is toward the second side of the drone;

[0036] A third camera and a fourth camera are respectively provided at the second end of the first crossbar, wherein the camera direction of the third camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone;

[0037] A second crossbar is provided at the rear end of the drone, and a fifth camera and a sixth camera are provided at the first end of the second crossbar, wherein the camera direction of the fifth camera is toward the fourth side of the drone; and the camera direction of the sixth camera is toward the second side of the drone;

[0038] A seventh camera and an eighth camera are respectively provided at the second end of the second crossbar, wherein the camera direction of the seventh camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone;

[0039] The first camera and the third camera form a first group, used to photograph an aerial target object located on a first side of the drone; the second camera and the sixth camera form a second group of cameras, used to photograph an aerial target object located on a second side of the drone; the fourth camera and the eighth camera form a third group, used to photograph an aerial target object located on a third side of the drone; the fifth camera and the seventh camera form a fourth group, used to photograph an aerial target object located on a fourth side of the drone.

[0040] In one embodiment, when in operation, the first camera and the third camera are on the same horizontal line, symmetrically arranged relative to the central axis of the drone, and perpendicular to the central axis;

[0041] The fifth camera and the seventh camera are located on the same horizontal line, symmetrically arranged relative to the central axis of the drone, and perpendicular to the central axis;

[0042] The second camera and the sixth camera are on the same horizontal line and are arranged parallel to the central axis of the drone;

[0043] The fourth camera and the eighth camera are located on the same horizontal line and are arranged parallel to the central axis of the drone.

[0044] According to a second aspect of an embodiment of the present disclosure, there is provided an aerial ecological target object density measurement device, comprising:

[0045] The first determination module determines the photographic cone volume of each camera and the number of target objects based on the principles of photogrammetry for any one side area;

[0046] The second determining module is configured to determine the density of the target objects in the side area according to the imaging cone volume and the number of the target objects.

[0047] Wherein, multiple groups of cameras are set on the drone; wherein, each group of cameras is responsible for capturing images of target objects on each side of the drone.

[0048] In one embodiment, the first determination module is further configured to determine a quantitative measurement area on each side according to the measurement range of each camera group on each side; wherein the quantitative measurement area is an intersection area of ​​the photographic cone volumes of the two cameras in each group;

[0049] determining the coordinates of each target object within the quantitative measurement area in a real space coordinate system;

[0050] Determine, according to the coordinates of each target object, a target object with the largest distance in a first direction and a distance in the first direction, wherein the distance in the first direction is the distance between the target object and the plane of the camera lens group;

[0051] determining a base radius of the photographic cone according to the distance and a predetermined shooting angle range of the camera;

[0052] The volume of the photographic cone and the quantitative measurement area are determined according to the base radius and the distance.

[0053] In one embodiment, the first determining module is further configured to determine first image space auxiliary coordinates and second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras;

[0054] determining the length of a baseline between the two cameras; and a projection vector of the baseline;

[0055] determining a first projection coefficient or a second projection coefficient according to the projection vector of the baseline, the first image space auxiliary coordinates, and the second image space auxiliary coordinates;

[0056] Determine the first direction distance according to the first projection coefficient and the focal length of a camera; or,

[0057] The first direction distance is determined according to the second projection coefficient and a focal length of another camera.

[0058] In one embodiment, the second determining module is further configured to determine a first image space coordinate of a camera in the group of cameras;

[0059] The first image space auxiliary coordinates are determined according to a pre-determined first rotation matrix and the first image space coordinates.

[0060] In one embodiment, the first determination module is further configured to determine a second image space coordinate in another camera in the group of cameras;

[0061] The second image space auxiliary coordinates are determined according to a pre-determined second rotation matrix and the second image space coordinates.

[0062] In one embodiment, the first determination module is further configured to determine a first number of target objects within the quantitative measurement area;

[0063] Determining a second number of target objects in a non-quantitative measurement area according to the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area; specifically comprising:

[0064] For each target object in the non-quantitative measurement area, in response to an image of the target object being larger than an image of a target object with the largest first direction distance in the quantitative measurement area, determining the target object as a valid target object in the non-quantitative measurement area;

[0065] Counting a second number of the valid target objects;

[0066] The number of target objects within the photography cone is determined according to the first number and the second number.

[0067] In one embodiment, the invention further includes an early warning module configured to, after the second determination module determines the density of target objects in the side area based on the photographic cone volume and the number of target objects, send an insect plague early warning message to a meteorological department server in response to the density value of the target objects being greater than a predetermined density threshold, wherein the insect plague early warning message includes an identifier of the area where the drone is located, so that the meteorological department server obtains historical meteorological data for the area based on the area identifier;

[0068] Determine areas where insect infestation may occur and need early warning based on the historical meteorological data, and send early warning information to the areas where insect infestation may occur.

[0069] According to a third aspect of an embodiment of the present disclosure, there is provided an aerial ecological target object density measurement device, comprising:

[0070] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to run the executable instructions to implement the steps of the aerial ecological target object density measurement method provided in the first aspect of the present disclosure.

[0071] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the aerial ecological target object density measurement method provided by the first aspect of the present disclosure are implemented.

[0072] According to a fifth aspect of the embodiments of the present disclosure, the present application proposes an aerial ecological target object density measurement system, including a controller, a first camera, a second camera, a third camera, a fourth camera, a fifth camera, a sixth camera, a seventh camera, and an eighth camera respectively connected to the controller;

[0073] A first crossbar is provided at the front end of the drone, and a first camera and a second camera are provided at the first end of the first crossbar, wherein the camera direction of the first camera faces the direction of the first side of the drone; and the camera direction of the second camera faces the direction of the second side of the drone;

[0074] A third camera and a fourth camera are respectively provided at the second end of the first crossbar, wherein the camera direction of the third camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone;

[0075] A second crossbar is provided at the rear end of the drone, and a fifth camera and a sixth camera are provided at the first end of the second crossbar, wherein the camera direction of the fifth camera is toward the fourth side of the drone; and the camera direction of the sixth camera is toward the second side of the drone;

[0076] A seventh camera and an eighth camera are respectively provided at the second end of the second crossbar, wherein the camera direction of the seventh camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone;

[0077] The first camera and the third camera form a first group, and are used to photograph an aerial target object located on a first side of the drone; the second camera and the sixth camera form a second group, and are used to photograph an aerial target object located on a second side of the drone; the fourth camera and the eighth camera form a third group, and are used to photograph an aerial target object located on a third side of the drone; the fifth camera and the seventh camera form a fourth group, and are used to photograph an aerial target object located on a fourth side of the drone;

[0078] The controller is used to control the four groups of cameras to take images simultaneously at regular intervals;

[0079] For any one side area, the photographic cone volume of each camera and the number of target objects are determined according to the quantitative measurement area of ​​the side area;

[0080] The density of the target objects in the side area is determined according to the imaging cone volume and the number of target objects.

[0081] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by arranging multiple groups of cameras on the drone, the density of target objects in the side area is determined based on the volume of the photographic cone in each group of cameras and the number of target objects. The technology in photogrammetry is fully utilized, and multiple cameras in each group of cameras are used to determine the farthest target object in the quantitative measurement area, as well as the vertical distance between the farthest target object and the camera. However, in the related art, a single camera is used to determine the distance between the target object in the image and the camera, so the volume of the photographic cone cannot be calculated. The present application can use two cameras to determine the three-dimensional coordinates of the target object, determine the distance between the target object farthest from the camera and the camera, and use it as the height value of the photographic cone, so as to calculate the volume of the photographic cone. The technical solution has low cost and can achieve the density measurement of target objects in the air without high cost. This will help to further provide meteorological services and insect plague warning work.

[0082] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0084] Figure 1 is a flow chart showing a method for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0085] Figure 2 is a schematic diagram of a drone measurement system according to an exemplary embodiment;

[0086] Figure 3 is a flow chart showing another method for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0087] Figure 4 is a flow chart showing another method for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0088] Figure 5 is a flow chart showing another method for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0089] Figure 6 is a schematic diagram showing a principle for measuring the position coordinates of a target object according to an exemplary embodiment;

[0090] Figure 7 is a block diagram of a device for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0091] Figure 8 is a block diagram of a device for measuring density of aerial ecological target objects according to an exemplary embodiment;

[0092] Figure 9 The figure is a block diagram of a system for measuring the density of aerial ecological target objects according to an exemplary embodiment. DETAILED DESCRIPTION

[0093] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0094] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0095] This application proposes a method for measuring the density of a target object. Figure 1 A flow chart of a method for measuring the density of an aerial ecological target object is shown; the method may include the following steps:

[0096] In step S102 , multiple groups of cameras are set on the drone; wherein each group of cameras is responsible for capturing an image of a target object on each side of the drone.

[0097] Target objects can include insects, birds, and insects including locusts. Specifically, an identifier can be pre-set for each drone, and a group identifier can be set for each camera group. Each drone sends captured images to a controller on the drone. The controller can then group and categorize the received images based on the drone's group identifier and the drone's identifier. For example, the groups can be divided into four groups: Group 1, group identifier 01; Group 2, group identifier 02; Group 3, group identifier 03; and Group 4, group identifier 04. In the first group, one camera identifier can be set to 0101, and another camera identifier can be set to 0102. In the second group, one camera identifier can be set to 0201, and another camera identifier can be set to 0202. In the third group, one camera identifier can be set to 0301, and another camera identifier can be set to 0302. In the fourth group, one camera identifier can be set to 0401, and another camera identifier can be set to 0402.

[0098] In step S104 , for any one side area, the photographic cone volume of each camera and the number of target objects are determined according to the principles of photogrammetry.

[0099] In this embodiment, photogrammetry principles are used to determine the spatial coordinates of target objects in an image. These coordinates include two-dimensional coordinates on a plane or three-dimensional coordinates in space. The number of target objects in an image can also be counted. A camera is used to capture a conical three-dimensional space. By determining the geometric parameters of the cone, including its height and base radius, the cone's volume can be determined.

[0100] In step S106 , the target object density in the side area is determined according to the imaging cone volume and the number of target objects.

[0101] Specifically, the density value of the target objects on this side is obtained by dividing the value of the number of the target objects by the volume of the photographic cone.

[0102] Each of these cameras can send captured images to the drone's central controller. The central controller executes each step of the target object density measurement method. The central controller also controls the drone's flight and the timing of each camera's photos.

[0103] In the above-mentioned technical solution, multiple camera groups are installed on a drone, and the density of target objects is determined based on the volume of the photographic cone of each camera group and the number of target objects. This technical solution is low-cost and easy to use, enabling aerial density measurement of target objects without high-cost methods. This will further enhance meteorological services and insect pest warnings.

[0104] In some embodiments, see Appendix Figure 2 A first cross bar 01 is set at the front end of the drone, and a first camera 21 and a second camera 22 are respectively set at the first end of the first cross bar 01. The camera direction of the first camera 21 is toward the first side of the drone; the camera direction of the second camera 22 is toward the second side of the drone.

[0105] A third camera 23 and a fourth camera 24 are respectively provided at the second end of the first crossbar 01. The camera direction of the third camera 23 is toward the first side of the drone; the camera direction of the fourth camera 24 is toward the third side of the drone.

[0106] A second cross bar 02 is provided at the rear end of the drone, and a fifth camera 25 and a sixth camera 26 are provided at the first end of the second cross bar 02, respectively. The camera direction of the fifth camera 25 is toward the fourth side of the drone; the camera direction of the sixth camera 26 is toward the second side of the drone.

[0107] A seventh camera 27 and an eighth camera 28 are respectively provided at the second end of the second crossbar 02. The camera direction of the seventh camera 27 is toward the first side of the drone; the camera direction of the eighth camera 28 is toward the third side of the drone.

[0108] The first camera 21 and the third camera 23 form a first group, and are used to photograph an aerial target object located on a first side of the drone. The second camera 22 and the sixth camera 26 form a second group, and are used to photograph an aerial target object located on a second side of the drone. The fourth camera 24 and the eighth camera 28 form a third group, and are used to photograph an aerial target object located on a third side of the drone. The fifth camera 25 and the seventh camera 27 form a fourth group, and are used to photograph an aerial target object located on a fourth side of the drone.

[0109] In some embodiments, when the drone is operating in the air, it takes off, reaches a set altitude, and then flies at a constant altitude. While in the air, photos can be taken at regular intervals, using eight cameras to capture photos simultaneously. While the cameras are capturing photos, the drone should be level in the air, ensuring that the eight cameras are in a horizontal position.

[0110] When in operation, the first camera 21 and the third camera 23 are on the same horizontal line, symmetrically arranged relative to the central axis of the drone, and perpendicular to the central axis; the fifth camera 25 and the seventh camera 27 are on the same horizontal line, symmetrically arranged relative to the central axis of the drone, and perpendicular to the central axis; the second camera 22 and the sixth camera 26 are on the same horizontal line, parallel to the central axis of the drone; the fourth camera 24 and the eighth camera 28 are on the same horizontal line, parallel to the central axis of the drone.

[0111] Preferably, the distance between the two cameras in each group can be set to be equal.

[0112] The distance between the first camera 21 and the third camera 23 is equal to the distance between the fifth camera 25 and the seventh camera 27. The distance between the second camera 22 and the sixth camera 26 is equal to the distance between the fourth camera 24 and the eighth camera 28.

[0113] Furthermore, the distance between the first camera 21 and the third camera 23 may also be equal to the distance between the second camera 22 and the sixth camera 26 .

[0114] In some embodiments, after measuring the densities of the target objects in the four areas, the average of the four density values ​​may be calculated to obtain an approximate density value of the target objects at the location of the drone.

[0115] In some embodiments, the drone can fly in the air, constantly changing its position to measure density values ​​at different locations. For example, it can fly continuously in the air, controlling eight cameras to take a photo every 10 minutes to obtain a density value. The drone can also hover in the air for a long time at a certain location, measuring the density trend at that location over time, and obtaining a two-dimensional curve graph of the density change at that location over time.

[0116] In one embodiment, see Figure 3 In step S106, determining the volume of the photographic cone may further include the following steps:

[0117] In step S1061 , the quantitative measurement area on each side is determined according to the measurement range of each group of cameras on each side.

[0118] In this embodiment, the quantitative measurement area is the intersection of the photographic cone volumes of each set of two cameras. According to the knowledge of photogrammetry, while a single camera can determine the two-dimensional coordinate position of the target object in the image, it cannot determine the distance of the target object from the lens, and therefore, the three-dimensional coordinate position of the target object cannot be determined. However, using two cameras can determine the three-dimensional coordinates of each target object in the overlapping area. Therefore, the number of target objects in the overlapping area and the coordinates of each target object can be determined. Specifically, the performance parameters of the camera determine the vertex angle range of the photographic cone, and the spacing between the two cameras is a set value, so the overlapping area of ​​the two photographic cones can be determined. The height value of the photographic cone can be further determined, and the volume of the photographic cone can be calculated.

[0119] See also Figure 2 The photographic cone includes a quantitative measurement area and a non-quantitative measurement area; that is, overlapping areas and non-overlapping areas. Photogrammetry techniques can be used to determine the number of target objects within the quantitative measurement area and the 3D coordinates of each target object.

[0120] In the non-quantitative measurement area, a single camera cannot determine the distance of the target object in the image from the camera lens. Therefore, the target object's 3D coordinates cannot be determined, and it is impossible to determine whether the target object is valid. If the target object is far away from the camera lens, exceeding the height value of the predetermined photographic cone volume, it can be considered invalid and ignored. Therefore, in the non-quantitative measurement area, invalid target objects need to be excluded to determine the number of target objects in the non-quantitative measurement area.

[0121] The height of the cone can be determined based on the coordinates of the target object within the quantitative measurement area. Furthermore, the coordinates of the target object include the distance from the target object to the camera's lens plane. Based on the height and vertex angle, the base radius of the photographic cone, and thus its volume, can be determined.

[0122] In step S1062 , the coordinates of each target object in the quantitative measurement area in the real space coordinate system are determined.

[0123] In step S1063, the target object with the largest first direction distance and the first direction distance are determined according to the coordinates of each target object, wherein the first direction distance is the distance between the target object and the lens plane of the group of cameras.

[0124] In some embodiments, the lenses of two cameras in the camera group are arranged on the same plane. The first direction is a direction perpendicular to the lens plane. In this spatial coordinate system, the first direction can be determined to be the Z-axis direction, and the lens plane is provided with an X-axis and a Y-axis.

[0125] In step S1064, the base radius of the photographic cone is determined according to the distance and a predetermined shooting angle range of the camera.

[0126] The predetermined shooting angle range of the camera is determined by the camera lens.

[0127] See attached Figure 4 In , the shooting angle range is represented by θ. The models of the two cameras in each group are set to be the same to ensure that the two cameras can have the same shooting angle range.

[0128] In step S1065 , the volume and quantitative measurement area of ​​the photographic cone are determined according to the base radius and the distance.

[0129] For details, see the attached Figure 4 , the formula for calculating the volume of the photographic cone is:

[0130] V=(1 / 3)π(r 2 )Rmax;

[0131] r=Rmax╳tg(θ / 2);

[0132] Among them, V is the camera's photographic cone volume, r is the radius of the cone base, and θ is the camera's photographic angle.

[0133] Once the volumes of the two photographic cones are determined, the three-dimensional spatial range of the quantitative measurement area can be determined.

[0134] In one embodiment, see the attached Figure 5 In step S1062, determining the coordinates of each target object in the quantitative measurement area in the real space coordinate system may further include the following steps:

[0135] In step S502 , first image space auxiliary coordinates and second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras are determined.

[0136] In this embodiment, the first image space auxiliary coordinates of the target object in the first camera coordinate system of the group of cameras are determined by the following steps:

[0137] determining first image space coordinates of a camera in the set of cameras;

[0138] The first image space auxiliary coordinates are determined according to a pre-determined first rotation matrix and the first image space coordinates.

[0139] For example, see the attached Figure 6, the first image space coordinates of point A in a camera image point a1 are (x1, y1, -f), where f is the focal length of the camera, and the first image space auxiliary coordinates are (u1, v1, w1).

[0140]

[0141] Among them, R1 is the rotation matrix of camera 1, which can be obtained by measuring the rotation matrix R1 through a dual-axis high-precision inclinometer.

[0142] The following steps may be used to determine the second image space auxiliary coordinates of the target object in the second camera coordinate system of the group of cameras:

[0143] determining a second image space coordinate in another camera in the set of cameras;

[0144] The second image space auxiliary coordinates are determined according to a pre-determined second rotation matrix and the second image space coordinates.

[0145] For example, the second image space coordinates of the image point a2 of point A in another camera are (x2, y2, -f), and the second image space auxiliary coordinates are (u2, v2, w2), then,

[0146]

[0147] Among them, R2 is the rotation matrix of camera 2, which can be obtained by measuring the dual-axis high-precision inclinometer provided by the drone.

[0148] In step S504 , the length of the baseline between the two cameras and the projection vector of the baseline are determined.

[0149] In step S506, a first projection coefficient or a second projection coefficient is determined according to the projection vector of the baseline, the first image space auxiliary coordinates, and the second image space auxiliary coordinates.

[0150] According to the collinearity of the camera, the image point, and the target point, the first projection coefficient N1 and the second projection coefficient N2 are determined to have the following equation relationship:

[0151]

[0152] Wherein, N1 is the first projection coefficient, and (U1, V1, W1) is the coordinate of insect A in the coordinate system S1-U1V1W1 of camera 1.

[0153]

[0154] Where N2 is the second projection coefficient, and (U2, V2, W2) is the coordinate of insect A in the coordinate system S2-U2V2W2 of camera 2. The following formula can be obtained:

[0155]

[0156] Determine the projection vector B of the baseline u 、B v and B w With the following equality relationship:

[0157]

[0158] According to the above formula, the joint solution is:

[0159]

[0160] In step S508, the first direction distance is determined according to the first projection coefficient and the focal length of a camera; or, the first direction distance is determined according to the second projection coefficient and the focal length of another camera.

[0161] In one embodiment, determining the number of target objects within the photographic cone based on the quantitative measurement area of ​​the side region may further include the following steps:

[0162] determining a first quantity of target objects within the quantitative measurement region;

[0163] determining a second number of target objects in a non-quantitative measurement area based on the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area;

[0164] Specifically, it includes: for each target object in the non-quantitative measurement area, in response to the image of the target object being larger than the image of the target object with the largest first direction distance in the quantitative measurement area, determining the target object as a valid target object in the non-quantitative measurement area.

[0165] In some embodiments, the image size of a target object can be determined based on the pixel area occupied by the target object. For example, if target object A in the non-quantitative measurement area occupies a pixel matrix of 100 x 50 in the image, while target object B, which is the largest distance in the first direction in the quantitative measurement area, occupies a pixel matrix of 40 x 30 in the image, the image of target object A can be determined to be larger than the image of target object B. Target object A is considered valid. Otherwise, target object A is considered invalid.

[0166] Counting a second number of the valid target objects; and determining the number of target objects within the photographic cone according to the first number and the second number.

[0167] In response to the image of the target object being smaller than or equal to the image of the target object with the largest first direction distance in the quantitative measurement area, the target object is determined to be an invalid target object in the non-quantitative measurement area.

[0168] The above method compares the images of the first target object in the non-quantitative measurement area with the second target object, located at the farthest end of the quantitative measurement area, through image size comparison. If the image of the first target object is larger than the image of the second target object, it indicates that the distance of the first target object from the camera lens is less than the distance Rmax of the second target object, indicating that the first target object is within the range of the photographic cone and is valid and should be counted. Otherwise, the first target object is outside the range of the photographic cone and is not considered and counted. In this way, the validity of each target object in the non-quantitative measurement area can be determined, thereby completing the counting and statistics task.

[0169] In some embodiments, a target detection algorithm can also be used to detect target objects in an image and determine the type of the target object. Types include locusts, birds, moths, etc. A large number of image samples of the target objects can be collected in advance to train a target detection model. The target detection model can be implemented using a convolutional neural network, which includes an input layer, a convolution layer, a pooling layer, and an output layer. After extensive training, the target detection model can accurately identify the type of target object. The number of target objects of the same type in an image can be counted, and target objects of different types can be removed. For example, if there is a single bird among a large number of locusts, the bird can be removed to improve statistical accuracy.

[0170] In one embodiment, after determining the density of the target objects in the side area according to the imaging cone volume and the number of target objects, the following steps may be further included:

[0171] In response to the density value of the target object being greater than a predetermined density threshold, sending an insect plague warning message to a server of a meteorological department, wherein the insect plague warning message includes an identifier of the area where the drone is located, so that the meteorological department server obtains historical meteorological data of the area based on the area identifier;

[0172] Determine areas where insect infestation may occur and need early warning based on the historical meteorological data, and send early warning information to the areas where insect infestation may occur.

[0173] In this embodiment, after determining the density of target objects, the target object density data is stored to provide guidance for meteorological research. If the density exceeds a predetermined threshold, an insect plague is determined to have occurred. The threshold can be set flexibly and is not limited in this application. After determining that an insect plague has occurred, an alarm is promptly issued. An alarm message can be sent to the relevant area or to a meteorological unit, allowing the unit to review historical meteorological data based on the density data. After receiving the alarm message, the meteorological unit's server can retrieve historical climate data for the area from a historical meteorological database based on the area identifier carried in the alarm message. This climate data includes at least historical precipitation and temperature data. For example, locust plagues are primarily caused by meteorological factors. Temperature and precipitation are the main meteorological factors affecting the occurrence and growth of locusts, and abnormal climate can lead to locust plagues. Based on this historical climate data, areas at risk of locust plagues are identified, and locust plague warning information is then sent to the relevant areas. For example, if a locust plague occurs in a county in a certain province, historical climate data for that county is obtained, and counties B, C, and D are identified as having similar historical climate data to County A, an alarm message is then sent to counties B, C, and D.

[0174] To determine the target object's trajectory in the air, in some embodiments, after determining the target object's density, the drone can continue to fly forward, change positions, and continue measuring the target object's density at the changed position. In this way, the drone can measure multiple target object density values ​​in any direction of interest. Based on the magnitudes of these multiple target object density values, the density distribution trend in that direction can be determined. Based on this density distribution trend, an alarm message can be sent to the relevant area.

[0175] For example, if the locust density increases from north to south, it means that a large number of locusts are migrating from north to south. In this case, an insect plague alarm message can be sent to one or more neighboring areas in the south to prepare for emergency response.

[0176] Secondly, this application also proposes an early warning method, which is applied to a server of a meteorological department, and the method includes the following steps:

[0177] Receiving disaster warning information sent by a drone, wherein the disaster warning information carries an identifier of the area where the drone is located;

[0178] Obtaining historical climate data for the region;

[0179] Determining areas where disasters may occur based on the historical climate data;

[0180] Specifically, determine the historical climate data of the above-mentioned area, which includes factors such as temperature and precipitation.

[0181] Based on the above climate data, areas with similar climate data during the same historical period are identified as areas where disasters may occur.

[0182] Sending early warning information to the area where disaster may occur.

[0183] In the third aspect, this application proposes an aerial ecological target object density measurement device, see the attached Figure 7 As shown, the apparatus 700 includes:

[0184] A first determination module 71 determines, for any one side area, the photographic cone volume of each camera and the number of target objects according to photogrammetry principles;

[0185] The second determination module 72 is configured to determine the density of the target objects in the side area based on the volume of the photographic cone and the number of target objects. A plurality of camera groups are provided on the drone, each camera group being responsible for capturing images of the target objects on each side of the drone.

[0186] In one embodiment, the first determining module 71 is further configured to determine a quantitative measurement area on each side according to the measurement range of each camera group on each side; wherein the quantitative measurement area is the intersection area of ​​the photographic cone volumes of the two cameras in each group;

[0187] determining the coordinates of each target object within the quantitative measurement area in a real space coordinate system;

[0188] Determine, according to the coordinates of each target object, a target object with the largest distance in a first direction and a distance in the first direction, wherein the distance in the first direction is the distance between the target object and the plane of the camera lens group;

[0189] determining a base radius of the photographic cone according to the distance and a predetermined shooting angle range of the camera;

[0190] The volume of the photographic cone is determined according to the base radius and the distance.

[0191] In one embodiment, the first determining module 71 is further configured to determine first image space auxiliary coordinates and second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras;

[0192] determining the length of a baseline between the two cameras; and a projection vector of the baseline;

[0193] determining a first projection coefficient or a second projection coefficient according to the projection vector of the baseline, the first image space auxiliary coordinates, and the second image space auxiliary coordinates;

[0194] Determine the first direction distance according to the first projection coefficient and the focal length of a camera; or,

[0195] The first direction distance is determined according to the second projection coefficient and a focal length of another camera.

[0196] In one embodiment, the first determining module 71 is further configured to determine the first image space coordinates of a camera in the group of cameras; and determine the first image space auxiliary coordinates according to a pre-determined first rotation matrix and the first image space coordinates.

[0197] In one embodiment, the first determining module 71 is further configured to determine a second image space coordinate in another camera in the group of cameras; and determine the second image space auxiliary coordinate according to a pre-determined second rotation matrix and the second image space coordinate.

[0198] In one embodiment, an early warning module is further included, which is used for the second determination module 72 to determine the density of the target objects in the side area according to the volume of the photographic cone and the number of target objects. In response to the density value of the target objects being greater than a predetermined density threshold, an early warning information on insect plague is sent to the server of the meteorological department. The early warning information on insect plague includes the area identifier where the drone is located, so that the server of the meteorological department obtains the historical meteorological data of the area according to the area identifier, determines the area where insect plague may occur and needs early warning according to the historical meteorological data, and sends early warning information to the area where insect plague may occur.

[0199] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0200] In a fourth aspect, the present application also proposes an electronic device, see the attached Figure 8 The electronic device includes: a processor 81; a memory 82 for storing processor-executable instructions; wherein the processor 81 is configured to run the executable instructions to implement any of the above methods.

[0201] In the fifth aspect, this application also proposes an aerial ecological target object density measurement system, see the attached Figure 9 The system includes a controller 91, which can be installed on a drone. The system also includes a first camera 21, a second camera 22, a third camera 23, a fourth camera 24, a fifth camera 25, a sixth camera 26, a seventh camera 27, and an eighth camera 28, respectively connected to the controller.

[0202] A first cross bar is set at the front end of the drone, and a first camera 21 and a second camera 22 are respectively set at the first end of the first cross bar. The camera direction of the first camera 21 is toward the first side of the drone; the camera direction of the second camera 22 is toward the second side of the drone.

[0203] A third camera 23 and a fourth camera 24 are respectively provided at the second end of the first crossbar. The camera direction of the third camera 23 is toward the first side of the drone; the camera direction of the fourth camera 24 is toward the third side of the drone.

[0204] A second cross bar is provided at the rear end of the drone, and a fifth camera 25 and a sixth camera 26 are provided at the first end of the second cross bar, respectively. The camera direction of the fifth camera 25 is toward the fourth side of the drone; the camera direction of the sixth camera 26 is toward the second side of the drone.

[0205] A seventh camera 27 and an eighth camera 28 are respectively disposed at the second end of the second crossbar. The seventh camera 27 is oriented toward the first side of the drone, while the eighth camera 28 is oriented toward the third side of the drone. The first camera 21 and the third camera 23 form a first group, used to photograph aerial targets located on the first side of the drone. The second camera 22 and the sixth camera 26 form a second group, used to photograph aerial targets located on the second side of the drone. The fourth camera 24 and the eighth camera 28 form a third group, used to photograph aerial targets located on the third side of the drone. The fifth camera 25 and the seventh camera 27 form a fourth group, used to photograph aerial targets located on the fourth side of the drone.

[0206] The controller 91 is used to control the four groups of cameras to capture images simultaneously at regular intervals; for any side area, the photographic cone volume and the number of target objects of each camera are determined according to the principles of photogrammetry; and the density of the target objects in the side area is determined according to the photographic cone volume and the number of target objects.

[0207] The controller 91 can also be used to control the flight of the drone, including various operations such as take-off, landing, and hovering.

[0208] In some embodiments, the controller 91 is also used to determine the quantitative measurement area on each side according to the measurement range of each group of cameras on each side; wherein the quantitative measurement area is the intersection area of ​​the photographic cone volume of each group of two cameras; determine the coordinates of each target object in the quantitative measurement area in the real space coordinate system; determine the target object with the largest distance in the first direction and the distance in the first direction according to the coordinates of each target object, wherein the distance in the first direction is the distance between the target object and the lens plane of the group of cameras; determine the base radius of the photographic cone according to the distance and the predetermined camera shooting angle range; determine the volume of the photographic cone and the quantitative measurement area according to the base radius and the distance.

[0209] In some embodiments, the controller 91 is also used to determine the first image space auxiliary coordinates and the second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras; determine the length of the baseline between the two cameras; and the projection vector of the baseline; determine the first projection coefficient, or the second projection coefficient based on the projection vector of the baseline, the first image space auxiliary coordinates and the second image space auxiliary coordinates; determine the first direction distance based on the first projection coefficient and the focal length of a camera; or determine the first direction distance based on the second projection coefficient and the focal length of another camera.

[0210] In some embodiments, the controller 91 is further configured to determine a first image space coordinate of a camera in the group of cameras; and determine the first image space auxiliary coordinate according to a pre-determined first rotation matrix and the first image space coordinate.

[0211] In some embodiments, the controller 91 is further configured to determine a second image space coordinate in another camera in the group of cameras; and determine the second image space auxiliary coordinate according to a pre-determined second rotation matrix and the second image space coordinate.

[0212] In some embodiments, the controller 91 is further configured to determine a first number of target objects in the quantitative measurement area; and determine a second number of target objects in the non-quantitative measurement area based on the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area.

[0213] Specifically, the method includes: for each target object in the non-quantitative measurement area, in response to an image of the target object being larger than an image of the target object with the largest first direction distance in the quantitative measurement area, determining that the target object is a valid target object in the non-quantitative measurement area; counting a second number of the valid target objects; and determining the number of target objects in the photographic cone based on the first number and the second number.

[0214] In some embodiments, the controller 91 is also used to, after determining the density of the target objects in the side area based on the photographic cone volume and the number of target objects, in response to the density value of the target objects being greater than a predetermined density threshold, send insect plague warning information to the server of the meteorological department, wherein the insect plague warning information includes the area identifier where the drone is located, so that the meteorological department server obtains the historical meteorological data of the area based on the area identifier; determines the area where insect plague may occur that needs warning based on the historical meteorological data, and sends warning information to the area where insect plague may occur.

[0215] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0216] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for measuring the density of aerial ecological target objects, characterized in that: include: Multiple groups of cameras are set on the drone, wherein each group of two cameras is responsible for capturing images of the target object on each side of the drone; For any area on one side, determine the photographic cone volume of each camera and the number of target objects according to the principles of photogrammetry; Determining the density of the target objects in the side area according to the volume of the photographic cone and the number of the target objects; and determining the volume of the photographic cone of each camera according to photogrammetry principles, including: Determine a quantitative measurement area on each side according to the measurement range of each camera group on each side; wherein the quantitative measurement area is the intersection area of ​​the photographic cone volumes of the two cameras in each group; determining the coordinates of each target object within the quantitative measurement area in a real space coordinate system; Determine, according to the coordinates of each target object, a target object with the largest distance in a first direction and a distance in the first direction, wherein the distance in the first direction is the distance between the target object and the plane of the camera lens group; Determining the base radius of the photographic cone according to the distance and a predetermined camera shooting angle range; determining the volume and quantitative measurement area of ​​the photographic cone according to the base radius and the distance; Determining the number of target objects within the photographic cone according to the quantitative measurement area of ​​the side area includes: determining a first quantity of target objects within the quantitative measurement region; determining a second number of target objects in a non-quantitative measurement area based on the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area; Specifically include: For each target object in the non-quantitative measurement area, in response to an image of the target object being larger than an image of a target object with the largest first direction distance in the quantitative measurement area, determining the target object as a valid target object in the non-quantitative measurement area; Counting a second number of valid target objects; The number of target objects within the photography cone is determined according to the first number and the second number.

2. The method for measuring the density of aerial ecological target objects according to claim 1, characterized in that: Determining the coordinates of each target object in the quantitative measurement area in a real space coordinate system includes: Determine first image space auxiliary coordinates and second image space auxiliary coordinates of the target object in each camera coordinate system of a corresponding set of cameras; determining the length of a baseline between the two cameras; and a projection vector of the baseline; determining a first projection coefficient or a second projection coefficient according to the projection vector of the baseline, the first image space auxiliary coordinates, and the second image space auxiliary coordinates; Determine the first direction distance according to the first projection coefficient and the focal length of a camera; or, The first direction distance is determined according to the second projection coefficient and a focal length of another camera.

3. The method for measuring the density of aerial ecological target objects according to claim 2, characterized in that: Determining first image space auxiliary coordinates of the target object in the first camera coordinate system of the group of cameras respectively includes: determining first image space coordinates of a camera in the set of cameras; The first image space auxiliary coordinates are determined according to a pre-determined first rotation matrix and the first image space coordinates.

4. The method for measuring the density of aerial ecological target objects according to claim 2, characterized in that: Determining the second image space auxiliary coordinates of the target object in the second camera coordinate system of the group of cameras respectively includes: determining a second image space coordinate in another camera in the set of cameras; The second image space auxiliary coordinates are determined according to a pre-determined second rotation matrix and the second image space coordinates.

5. The method for measuring the density of aerial ecological target objects according to claim 1, characterized in that: After determining the density of the target objects in the side area according to the photographic cone volume and the number of the target objects, the method further includes: In response to the density value of the target object being greater than a predetermined density threshold, sending an insect plague warning message to a server of a meteorological department, wherein the insect plague warning message includes an identifier of the area where the drone is located, so that the meteorological department server obtains historical meteorological data of the area based on the area identifier; Determine areas where insect infestation may occur and need early warning based on the historical meteorological data, and send early warning information to the areas where insect infestation may occur.

6. The method for measuring the density of aerial ecological target objects according to claim 1, characterized in that: A first crossbar is provided at the front end of the drone, and a first camera and a second camera are provided at the first end of the first crossbar, wherein the camera direction of the first camera faces the direction of the first side of the drone; and the camera direction of the second camera faces the direction of the second side of the drone; A third camera and a fourth camera are respectively provided at the second end of the first crossbar, wherein the camera direction of the third camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone; A second crossbar is provided at the rear end of the drone, and a fifth camera and a sixth camera are provided at the first end of the second crossbar, wherein the camera direction of the fifth camera is toward the fourth side of the drone; and the camera direction of the sixth camera is toward the second side of the drone; A seventh camera and an eighth camera are respectively provided at the second end of the second crossbar, wherein the camera direction of the seventh camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone; The first camera and the third camera form a first group for photographing an aerial target object located on a first side of the drone; The second camera and the sixth camera form a second group of cameras, configured to photograph an aerial target object located on a second side of the drone; The fourth camera and the eighth camera form a third group, and are used to photograph an aerial target object located on a third side of the drone; The fifth camera and the seventh camera form a fourth group, and are used to photograph an aerial target object located on a fourth side of the drone.

7. An electronic device, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the method according to any one of claims 1 to 6.

8. An aerial ecological target object density measurement system, characterized in that: comprising a controller, and a first camera, a second camera, a third camera, a fourth camera, a fifth camera, a sixth camera, a seventh camera, and an eighth camera respectively connected to the controller; A first crossbar is provided at the front end of the drone, and a first camera and a second camera are provided at the first end of the first crossbar, wherein the camera direction of the first camera faces the direction of the first side of the drone; and the camera direction of the second camera faces the direction of the second side of the drone; A third camera and a fourth camera are respectively provided at the second end of the first crossbar, wherein the camera direction of the third camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone; A second crossbar is provided at the rear end of the drone, and a fifth camera and a sixth camera are provided at the first end of the second crossbar, wherein the camera direction of the fifth camera is toward the fourth side of the drone; and the camera direction of the sixth camera is toward the second side of the drone; A seventh camera and an eighth camera are respectively provided at the second end of the second crossbar, wherein the camera direction of the seventh camera is toward the first side of the drone; and the camera direction of the fourth camera is toward the third side of the drone; The first camera and the third camera form a first group, and are used to photograph an aerial target object located on a first side of the drone; The second camera and the sixth camera form a second group of cameras, configured to photograph an aerial target object located on a second side of the drone; The fourth camera and the eighth camera form a third group, and are used to photograph an aerial target object located on a third side of the drone; The fifth camera and the seventh camera form a fourth group, and are used to photograph an aerial target object located on a fourth side of the drone; The controller is used to control the four groups of cameras to take images simultaneously at regular intervals; For any side area, determine the photographic cone volume of each camera group and the number of target objects according to the principles of photogrammetry; determine the density of the target objects in the side area according to the photographic cone volume and the number of target objects; The volume of the photographic cone of each camera is determined according to the principles of photogrammetry, including: Determine a quantitative measurement area on each side according to the measurement range of each camera group on each side; wherein the quantitative measurement area is the intersection area of ​​the photographic cone volumes of the two cameras in each group; determining the coordinates of each target object within the quantitative measurement area in a real space coordinate system; Determine, according to the coordinates of each target object, a target object with the largest distance in a first direction and a distance in the first direction, wherein the distance in the first direction is the distance between the target object and the plane of the camera lens group; Determining the base radius of the photographic cone according to the distance and a predetermined camera shooting angle range; determining the volume and quantitative measurement area of ​​the photographic cone according to the base radius and the distance; Determining the number of target objects within the photographic cone according to the quantitative measurement area of ​​the side area includes: determining a first quantity of target objects within the quantitative measurement region; determining a second number of target objects in a non-quantitative measurement area based on the quantitative measurement area, wherein the photographic cone includes the quantitative measurement area and the non-quantitative measurement area; Specifically include: For each target object in the non-quantitative measurement area, in response to an image of the target object being larger than an image of a target object with the largest first direction distance in the quantitative measurement area, determining the target object as a valid target object in the non-quantitative measurement area; Counting a second number of valid target objects; The number of target objects within the photography cone is determined according to the first number and the second number.

Images