Unmanned aerial vehicle positioning method, device and equipment and storage medium

Abstract

The invention discloses an unmanned aerial vehicle positioning method and device, equipment and a storage medium. Wherein the unmanned aerial vehicle comprises at least two layers of cameras; the visual area of each layer of camera is gradually reduced along with the progressive increase of the layers; the method comprises the following steps: acquiring first information of a visual area acquired by a first-layer camera; the first layer of camera is any layer of camera in the at least one layer of camera; determining first positioning information corresponding to a to-be-positioned object based on the first information and a first preset model; determining a second-layer camera connected with the first-layer camera according to the first positioning information; acquiring second information of a visual area acquired by a second-layer camera; and performing image processing on the second information to obtain second positioning information corresponding to the to-be-positioned object.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a method, device, equipment and storage medium for positioning an unmanned aerial vehicle. Background Art

[0002] In the drone scenario, since the drone flies at a high altitude and the camera is usually far away, the pixels of a monocular camera are usually not enough to meet the needs of high-precision positioning. There is currently no effective solution to this problem. Summary of the invention

[0003] In order to solve the existing technical problems, the main purpose of the present invention is to provide a drone positioning method, device, equipment and storage medium.

[0004] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for positioning a drone, wherein the drone includes at least two layers of cameras; the visible area of ​​each layer of the cameras decreases as the layers increase; the method includes:

[0006] Acquire first information of a visible area collected by a first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras;

[0007] Determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model;

[0008] Determine a second-layer camera connected to the first-layer camera according to the first positioning information;

[0009] Acquire the second information of the visible area collected by the second layer camera;

[0010] Perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

[0011] In the above solution, before determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model, the method further includes:

[0012] Obtain image information and annotation information corresponding to the preset object;

[0013] The image information and the annotation information are input into an initial model for training to obtain the first preset model.

[0014] In the above solution, the method further includes:

[0015] Obtain the first parameter and the second parameter corresponding to the camera of each layer; the first parameter represents the parameter corresponding to the different focal lengths of the camera of each layer; the second parameter represents the parameter corresponding to the visible area controlled by the camera of each layer; the first parameter and the second parameter are used for the camera of each layer to collect information of the visible area.

[0016] In the above solution, the determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model includes:

[0017] Using the first preset model to determine whether the object to be located exists in the first information;

[0018] When the object to be located exists in the first information, obtaining pixel coordinate information corresponding to the object to be located;

[0019] The first positioning information is determined based on the pixel coordinate information, a first parameter corresponding to the first layer camera, and a second parameter.

[0020] In the above solution, performing image processing on the second information to obtain second positioning information corresponding to the object to be positioned includes:

[0021] Processing the second information using a second preset model to obtain contour area information corresponding to the object to be located;

[0022] Determine contour information corresponding to the object to be located according to the contour area information;

[0023] The second positioning information corresponding to the object to be positioned is determined based on the contour information, the first parameter and the second parameter corresponding to the second layer camera.

[0024] In the above solution, the second information is processed by using the second preset model to obtain the contour area information corresponding to the object to be located, including:

[0025] Processing the second information using the second preset model to obtain pixel position information of the object to be located in the second information;

[0026] Determine contour area information corresponding to the object to be located according to the pixel position information.

[0027] In the above solution, the determining the second positioning information corresponding to the object to be positioned based on the contour information, the first parameter and the second parameter corresponding to the second layer camera includes:

[0028] Acquire first coordinate information according to the contour information; the first coordinate information represents coordinate information of edge points of the contour corresponding to the object to be located;

[0029] Determine second coordinate information based on the first coordinate information, the first parameter and the second parameter corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information;

[0030] The second positioning information corresponding to the object to be positioned is determined according to the second coordinate information, the focal length of the camera and the position information of the drone.

[0031] In a second aspect, the present invention further provides a drone positioning device, the drone comprising at least two layers of cameras; the visible area of ​​the cameras in each layer decreases as the layers increase; the device comprises a first acquisition unit, a first determination unit, a second determination unit, a second acquisition unit and a processing unit, wherein:

[0032] The first acquisition unit is used to acquire first information of a visible area captured by a first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras;

[0033] The first determining unit is used to determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model;

[0034] The second determining unit is used to determine a second-layer camera connected to the first-layer camera according to the first positioning information;

[0035] The second acquisition unit is used to acquire second information of the visible area captured by the second layer camera;

[0036] The processing unit is used to perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

[0037] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon; when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0038] In a fourth aspect, an embodiment of the present invention provides a drone positioning device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor executes the steps of any one of the above methods when running the computer program.

[0039] The embodiment of the present invention provides a method, device, equipment and storage medium for positioning a drone. The drone includes at least two layers of cameras; the visible area of ​​each layer of the cameras decreases as the layers increase; the method includes: obtaining first information of the visible area collected by the first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras; determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model; determining the second layer of cameras connected to the first layer of cameras according to the first positioning information; obtaining the second information of the visible area collected by the second layer of cameras; performing image processing on the second information to obtain the second positioning information corresponding to the object to be positioned. The technical solution of the embodiment of the present invention is adopted, at least two layers of cameras are set on the drone, and the visible area of ​​each layer of the cameras decreases as the layers increase, and the first positioning information of the object to be positioned is determined according to the first information of the visible area collected by the first layer of cameras and the first preset model, so as to achieve rough positioning; performing image processing on the second information of the visible area collected by the second layer of cameras connected to the first layer of cameras to obtain the second positioning information corresponding to the object to be positioned, so as to achieve precise positioning, meet the requirements of high-precision positioning of the pixels of the cameras in the drone, and improve the positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of a method for positioning a drone provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of camera distribution provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of another camera distribution provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the distance relationship between an object to be located and a drone provided in an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of the relationship between the coordinates of an object to be located and the coordinates of the GPS of a drone provided in an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of a flow chart of another drone positioning method provided by an embodiment of the present invention;

[0046] Figure 7 A schematic structural diagram of a UAV positioning device provided by an embodiment of the present invention;

[0047] Figure 8 The figure is a schematic diagram of the hardware structure of the UAV positioning device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the specific technical solution of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Figure 1 The following is a flow chart of a method for positioning a drone provided by an embodiment of the present invention. Figure 1 As shown, the drone includes at least two layers of cameras; the visible area of ​​the cameras in each layer decreases as the layers increase; the method includes:

[0051] S101: Acquire first information of a visible area collected by a first-layer camera; the first-layer camera is any camera of the at least one layer of cameras;

[0052] S102: Determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model;

[0053] S103: Determine a second-layer camera connected to the first-layer camera according to the first positioning information;

[0054] S104: Acquire second information of a visible area captured by the second layer camera;

[0055] S105: Perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

[0056] It should be noted that the drone includes at least two layers of cameras, and the number of layers of cameras can be determined according to actual conditions and is not limited here. The visible area of ​​the cameras in each layer decreases as the layers increase, which can be understood as the focal length of the cameras in each layer increases as the layers increase, and the visible area decreases as the layers increase.

[0057] In S101, the first information may be image information collected by a first-layer camera.

[0058] In S102, the first preset model is a trained target detection model; the first positioning information corresponding to the object to be positioned may be rough coordinate information corresponding to the object to be positioned.

[0059] Before S103, the method also includes: determining the calibration of the cameras between the layers according to the focal length of the camera, and determining the range division of the area of ​​the cameras of the connected layers, which can be understood as a fine division of the coverage range of the cameras of the previous layer by the cameras of the next layer. It can be illustrated by an example that the focal length of the first-level camera is 15 millimeters (mm), and the collection radius is within the range of 100 meters (m). The focal length of the second-level camera is 50mm, covering a range of 20m in radius. About 25 photos of the second level can cover the range taken by the first level. The focal length of the third level is 100mm, covering an area with a radius of about 8m. The number of cameras in each layer is proportional to the focal length. For example, if the first layer has 1 camera and the focal length of the second-level camera is about 4 times that of the first layer, 4 cameras can be set. Similarly, 8 cameras are set for the third layer. Figure 2 A schematic diagram of camera distribution provided by an embodiment of the present invention, such as Figure 2 As shown, Figure 2 1 represents the layout position of the first-level camera, 2 represents the layout position of the second-level camera, and 3 represents the layout position of the third-level camera frame; Figure 3 A schematic diagram of another camera distribution provided by an embodiment of the present invention is shown as follows: Figure 3 As shown, Figure 3 1 represents the layout position of the first-level camera, 2 represents the layout position of the second-level camera, and 3 represents the layout position of the third-level camera frame. It should be noted that Figure 2 It can be understood as a top view of the camera distribution. Figure 3 It can be understood as the main view of the camera distribution.

[0060] In S103, determining the second layer camera connected to the first layer camera according to the first positioning information can be understood as determining the corresponding next layer camera according to the layer list determined above and the current location of the object to be positioned.

[0061] In S104, the second information can be understood as the image information collected by the second layer camera. It should be noted that, through the determined layer list, the camera of the next layer corresponding to the current object position is found, and after accessing this camera and obtaining the camera image data, this step needs to be repeated until the camera of the last layer is found.

[0062] In S105, the second positioning information can be understood as the precise coordinate information of the object to be positioned, that is, the accuracy of the coordinates meets the target requirements (decimeter level, centimeter level).

[0063] It should be noted that after determining the second positioning information, the second information is transmitted to the terminal; the terminal may be a drone or a server.

[0064] By adopting the technical solution of the embodiment of the present invention, the imaging range of the multi-level camera is gradually reduced, and the imaging accuracy is gradually improved, thereby improving the positioning accuracy of the moving object to the target accuracy.

[0065] In an optional embodiment of the present invention, before determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model, the method also includes: obtaining image information and annotation information corresponding to the preset object; inputting the image information and the annotation information into an initial model for training to obtain the first preset model.

[0066] In this embodiment, the preset object can be understood as a moving object sample, the image information can be understood as an image of a moving object under the camera perspective of each level, and the annotation information can be understood as the world coordinate point information of the region of interest (RIO) under the camera perspective of each level. The initial model can be a target detection algorithm model, and the type of the target detection algorithm model is not limited here. As an example, the initial model can be a target detection model (You Only Look Once version 5, yolov5) model. The first preset model can be understood as a target detection algorithm model after training.

[0067] For ease of understanding, an example can be given to collect ROI environmental data and moving object samples, including images of moving objects under the camera perspectives of each level and world coordinate point information of ROI, and train deep learning neural networks based on object sample data (images and corresponding annotations) under a wide perspective. The trained target detection algorithm model can recognize objects and output the bounding box of objects: The specific training details are as follows (taking yolov5 as an example): To facilitate the positioning of other cameras of subsequent drones, the input image is square, the input training size is 640 wide and 640 high, the batch size is 16, and 300 cycles of training are based on the pre-trained model. The training ends when the loss function no longer decreases or reaches 300 times; the model is accelerated and optimized using Open Neural Network Exchange (onnx) and tensorRT to facilitate better accelerated reasoning on NVIDIA Corporation (NVIDIA) hardware platforms.

[0068] By adopting the technical solution of the embodiment of the present invention, the trained target detection algorithm model can be used to improve positioning accuracy.

[0069] In an optional embodiment of the present invention, the method also includes: obtaining a first parameter and a second parameter corresponding to the camera of each layer; the first parameter represents the parameters corresponding to different focal lengths of the camera of each layer; the second parameter represents the parameters corresponding to the visible area controlled by the camera of each layer; the first parameter and the second parameter are used for the camera of each layer to collect information of the visible area.

[0070] In this embodiment, the first parameter can be understood as an intrinsic parameter of the camera, and the second parameter can be understood as an extrinsic parameter of the camera.

[0071] For ease of understanding, an example can be given as follows: calibrating the required N (N≥2) internal parameters of the cameras at different focal lengths to obtain the internal parameter matrix K1…K N After the hierarchical list of the camera is determined, it is determined that the camera's viewing angle covers the entire RIO area, and the corresponding camera extrinsic parameters are calculated based on the world coordinate point information of the ROI to obtain the extrinsic parameter matrix M1...M N .

[0072] By adopting the technical solution of the embodiment of the present invention, the intrinsic parameters and extrinsic parameters of the camera can be used to calculate the coordinates of the object to be located.

[0073] In an optional embodiment of the present invention, determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model includes: using the first preset model to determine whether the object to be positioned exists in the first information; when the object to be positioned exists in the first information, obtaining pixel coordinate information corresponding to the object to be positioned; determining the first positioning information based on the pixel coordinate information, the first parameters and the second parameters corresponding to the first layer camera.

[0074] In this embodiment, the use of the first preset model to determine whether the positioning object exists in the first information can be understood as passing the image taken by the first layer camera into the trained target detection algorithm model to detect whether there is an object to be detected in the image.

[0075] The determining of the first positioning information based on the pixel coordinate information, the first parameter and the second parameter corresponding to the first layer camera can be understood as converting the pixel coordinate information into image coordinate information, i.e., the first positioning information, according to the intrinsic parameters and extrinsic parameters corresponding to the first layer camera.

[0076] For ease of understanding, an example can be given: the first layer of cameras, i.e., the wide-angle camera, collects relevant data in real time. First, the image captured by the wide-angle camera is passed to the first target detection algorithm model to detect whether there is an object to be detected in the image; if so, the center point of the bounding box is obtained based on the detected object bounding box, and the rough position of the current moving object is determined through coordinate transformation operation. The coordinate transformation formula is shown in formula (1). Formula (1) is as follows:

[0077]

[0078] In formula (1), M1 represents the internal parameter corresponding to the first-level camera, and K1 represents the external parameter corresponding to the first-level camera.

[0079] By adopting the technical solution of the embodiment of the present invention, the target detection algorithm model is used to obtain the center point of the object to be detected, and the rough center point is used to provide positioning for subsequent detection.

[0080] In an optional embodiment of the present invention, the image processing of the second information to obtain second positioning information corresponding to the object to be located includes: processing the second information using a second preset model to obtain contour area information corresponding to the object to be located; determining contour information corresponding to the object to be located based on the contour area information; and determining the second positioning information corresponding to the object to be located based on the contour information, the first parameters and the second parameters corresponding to the second layer camera.

[0081] In this embodiment, the second preset model may be a Gaussian mixture model. The contour area information may be understood as the overall contour area of ​​the object to be located; the contour information may be understood as the contour line of the object to be located.

[0082] The method of processing the second information using the second preset model to obtain the contour area information corresponding to the object to be located can be understood as processing the second information using the second preset model to obtain the pixel position information of the object to be located in the second information; and determining the contour area information corresponding to the object to be located based on the pixel position information.

[0083] Determining the contour information corresponding to the object to be located according to the contour area information may be understood as obtaining the contour line of the object to be located (moving object) according to an edge detection (canny) operator.

[0084] The determining of the second positioning information corresponding to the object to be located based on the contour information and the first and second parameters corresponding to the second layer camera can be understood as: obtaining the first coordinate information according to the contour information; the first coordinate information represents the coordinate information of the edge points of the contour corresponding to the object to be located; determining the second coordinate information based on the first coordinate information and the first and second parameters corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information; and determining the second positioning information corresponding to the object to be located according to the second coordinate information, the focal length of the camera and the position information of the drone.

[0085] By adopting the technical solution of the embodiment of the present invention, the coordinate conversion operation is performed in combination with the contour line and the internal and external parameters of the corresponding level camera, the position of the moving object is output, and a more precise target is obtained.

[0086] In an optional embodiment of the present invention, the second information is processed using a second preset model to obtain contour area information corresponding to the object to be located, including: processing the second information using the second preset model to obtain pixel position information of the object to be located in the second information; and determining the contour area information corresponding to the object to be located based on the pixel position information.

[0087] In this embodiment, for ease of understanding, it can be illustrated as follows: since the image to be detected is not necessarily a regular shape, the outline of the object needs to be marked. In order to improve the segmentation efficiency, the segmentation model in deep learning is not used in this step, and the Gaussian mixture model filtering plus expansion erosion method is used to obtain the area of ​​the target to be detected. The specific method is: use a Gaussian mixture model to obtain the pixel position of the image that may be a moving object, establish a black mask of the same size, assign the pixels that may be moving objects to white, and use the closing operation-erosion-opening operation in the computer vision library (Open Source Computer Vision Library, opencv) to obtain the overall contour area of ​​the moving object.

[0088] In an optional embodiment of the present invention, determining the second positioning information corresponding to the object to be positioned based on the contour information, the first parameter and the second parameter corresponding to the second layer camera includes: acquiring first coordinate information according to the contour information; the first coordinate information represents the coordinate information of the edge point of the contour corresponding to the object to be positioned; determining the second coordinate information based on the first coordinate information, the first parameter and the second parameter corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information; determining the second positioning information corresponding to the object to be positioned according to the second coordinate information, the focal length of the camera and the position information of the drone.

[0089] In this embodiment, the first coordinate information is obtained according to the contour information; the first coordinate information represents the coordinate information of the edge of the contour corresponding to the object to be located, which can be understood as obtaining the edge information of the contour line, that is, drawing a box. Suppose the point set constituting the contour line is D = {(x1, y1), (x2, y2)…(x n ,y n )}, get the edge points of the contour line, that is, use the x-coordinate and y-coordinate of the maximum (max) and minimum (min) operating points to get the top, bottom, leftmost and rightmost points, recorded as d top , d bottom , d left , d right .

[0090] The second coordinate information is determined based on the first coordinate information, the first parameter and the second parameter corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information, which can be illustrated as mapping the four coordinates to the corresponding four points of the fixed focal length on the image with a fixed focal length by using the internal and external parameters of the camera, The coordinates of the center point of the box formed by the above four points are (x, y), where x and y are calculated by formula (2) and formula (3). Formula (2) and formula (3) are as follows:

[0091] x=(x(dd left )-x(dd right )) / twenty two)

[0092] y=(y(dd bottom )-y(dd top )) / twenty three)

[0093] Determining the second positioning information corresponding to the object to be positioned according to the second coordinate information, the focal length of the camera and the position information of the drone can be illustrated as follows: using the global positioning system (GPS) position of the drone and the focal length of the camera to obtain the parameters of the target position, where the GPS coordinate position of the drone is (XX, YY, ZZ) and the focal length is N; Figure 4 A schematic diagram of the distance relationship between an object to be located and a drone provided in an embodiment of the present invention, Figure 5 A schematic diagram of the relationship between the coordinates of an object to be located and the coordinates of the GPS of a drone provided in an embodiment of the present invention, wherein the object distance, i.e., the straight-line distance between the object to be measured and the camera, is calculated by formula (4), and formula (4) is as follows:

[0094] 1 / a=1 / N-1 / ZZ (4)

[0095] In formula (4), a represents the object distance. The horizontal distance between the object to be measured and the camera is solved using the Pythagorean theorem, as shown in formula (5). Formula (5) is as follows:

[0096] a 2 =b 2 +ZZ 2 (5)

[0097] In formula (5), b represents the horizontal distance, and the angle α in the horizontal direction is calculated by formula (6), which is as follows:

[0098] tanα=|y-YY| / |x-XX| (6)

[0099] The inverse trigonometric function is used to obtain α. Then the coordinates (X, Y) of the object to be measured on the GPS can be calculated by formula (7) and formula (8), which are as follows:

[0100] X=XX±b*cosα (7)

[0101] Y=YY±b*sinα (8)

[0102] For ease of understanding, a specific example is given here. Three cameras are arranged on the drone. The first camera is a 15mm focal length wide-angle camera, the second is a 50mm focal length standard camera, and the third is a 100mm focal length lens. First, use a 15mm focal length wide-angle lens to shoot. The wide-angle camera collects image data in real time and transmits it to the graphics processing unit (GPU) in real time. The deep learning algorithm module pre-processes the image of the wide-angle camera and inputs it into the network model for reasoning. The deep learning network completes the recognition of the container truck and obtains the detection frame of the object. According to the coordinates of the detection frame, use camera 2 to shoot the area where the target is located to obtain more accurate object coordinates. Camera 3 repeats this action to obtain its more accurate outline. Based on the fine outline and the internal and external parameters of the camera at the current focal length, the target container truck is accurately positioned, and the position information is transmitted back to the server, thereby completing the positioning of the object to be detected.

[0103] Figure 6 A flowchart of another method for positioning a drone provided by an embodiment of the present invention is shown in FIG. Figure 6As shown, the deployed multi-level cameras can be deployed at different distances or through zooming to achieve different levels, with levels from 1 to k (k ≥ 1). The first level (Level 1) generally obtains ROI image data under a wide range of vision (for example, the camera has a focal length of 15mm), and the second level (Level 2) refines the imaging area of ​​the first level (for example, the camera has a focal length of 50mm). The cameras are deployed layer by layer according to the level, and the visible area of ​​the cameras at each level decreases as the level increases. The larger the focal length, the smaller the ROI. For example, the first level controls an area with a radius of 100 meters, and the second level camera controls the range of the first level and divides the area into a radius of 20 meters. The smaller the range controlled by the cameras at each level, the smaller the actual distance represented by the pixels on the image at the same resolution, and the higher the level, the higher the positioning accuracy of the output. The number of levels can be selected according to the accuracy requirements.

[0104] Based on the same inventive concept as above, Figure 7 A schematic diagram of the structure of a drone positioning device provided by an embodiment of the present invention, wherein the drone includes at least two layers of cameras; the visible area of ​​each layer of the cameras decreases as the layers increase; the device 700 includes a first acquisition unit 701, a first determination unit 702, a second determination unit 703, a second acquisition unit 704 and a processing unit 705, wherein:

[0105] The first acquisition unit 701 is used to acquire first information of a visible area captured by a first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras;

[0106] The first determining unit 702 is used to determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model;

[0107] The second determining unit 703 is used to determine the second layer camera connected to the first layer camera according to the first positioning information;

[0108] The second acquisition unit 704 is used to acquire second information of the visible area captured by the second layer camera;

[0109] The processing unit 705 is used to perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

[0110] In some embodiments, the device 700 further includes a training unit for acquiring image information and annotation information corresponding to a preset object; inputting the image information and the annotation information into an initial model for training to obtain the first preset model.

[0111] In some embodiments, the device 700 also includes a third acquisition unit, which is used to obtain the first parameter and the second parameter corresponding to the camera of each layer; the first parameter represents the parameters corresponding to the different focal lengths of the camera of each layer; the second parameter represents the parameters corresponding to the visible area controlled by the camera of each layer; the first parameter and the second parameter are used for the camera of each layer to collect information of the visible area.

[0112] In some embodiments, the first determination unit 702 is further used to determine whether the object to be located exists in the first information by using the first preset model; if the object to be located exists in the first information, obtain pixel coordinate information corresponding to the object to be located; and determine the first positioning information based on the pixel coordinate information, the first parameters and the second parameters corresponding to the first layer camera.

[0113] In some embodiments, the processing unit 705 is further used to process the second information using a second preset model to obtain contour area information corresponding to the object to be located; determine the contour information corresponding to the object to be located based on the contour area information; and determine the second positioning information corresponding to the object to be located based on the contour information, the first parameters and the second parameters corresponding to the second layer camera.

[0114] In some embodiments, the processing unit 705 is further used to process the second information using the second preset model to obtain pixel position information of the object to be located in the second information; and determine contour area information corresponding to the object to be located according to the pixel position information.

[0115] In some embodiments, the processing unit 705 is also used to obtain first coordinate information based on the contour information; the first coordinate information represents the coordinate information of the edge point of the contour corresponding to the object to be located; the second coordinate information is determined based on the first coordinate information, the first parameter and the second parameter corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information; the second positioning information corresponding to the object to be located is determined according to the second coordinate information, the focal length of the camera and the position information of the drone.

[0116] It should be noted that the drone positioning device provided in the embodiment of the present invention and the drone positioning method provided in the aforementioned embodiment of the present invention belong to the same inventive concept, and the meanings of the terms appearing here have been described in detail above and will not be repeated here.

[0117] An embodiment of the present invention further provides a storage medium having a computer program stored thereon. When the computer program processor is executed by a processor, the steps of the above-mentioned method embodiment are implemented. The aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0118] An embodiment of the present invention also provides a drone positioning device, which includes: a processor and a memory for storing a computer program that can be run on the processor, wherein when the processor is used to run the computer program, it executes the steps of the above-mentioned method embodiment stored in the memory.

[0119] Figure 8 This is a hardware structure diagram of the drone positioning device of an embodiment of the present invention. The drone positioning device 800 includes: at least one processor 801, a memory 802, and optionally, the drone positioning device 800 may further include at least one communication interface 803. The various components in the drone positioning device 800 are coupled together via a bus system 804. It can be understood that the bus system 804 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Figure 8 Various buses are labeled as bus system 804 .

[0120] It can be understood that the memory 804 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 802 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memories.

[0121] The memory 802 in the embodiment of the present invention is used to store various types of data to support the operation of the drone positioning device 800. Examples of such data include: any computer program for operating on the drone positioning device 800, and the program for implementing the method of the embodiment of the present invention may be included in the memory 802.

[0122] The method disclosed in the above embodiment of the present invention can be applied to the processor 801, or implemented by the processor 801. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0123] In an exemplary embodiment, the drone positioning device 800 can be implemented by one or more application-specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to perform the above method.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0125] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0126] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0127] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0128] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0129] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for positioning a drone, characterized in that: The drone includes at least two layers of cameras; the visible area of ​​the cameras in each layer decreases as the layers increase; and the method includes: Acquire first information of a visible area collected by a first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras; Determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model; Determine a second-layer camera connected to the first-layer camera according to the first positioning information; Acquire the second information of the visible area collected by the second layer camera; Perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

2. The method according to claim 1, characterized in that Before determining the first positioning information corresponding to the object to be positioned based on the first information and the first preset model, the method further includes: Obtain image information and annotation information corresponding to the preset object; The image information and the annotation information are input into an initial model for training to obtain the first preset model.

3. The method according to claim 1, characterized in that The method further comprises: Obtain the first parameter and the second parameter corresponding to the camera of each layer; the first parameter represents the parameter corresponding to the different focal lengths of the camera of each layer; the second parameter represents the parameter corresponding to the visible area controlled by the camera of each layer; the first parameter and the second parameter are used for the camera of each layer to collect information of the visible area.

4. The method according to claim 3, characterized in that The determining, based on the first information and the first preset model, first positioning information corresponding to the object to be positioned includes: Using the first preset model to determine whether the object to be located exists in the first information; When the object to be located exists in the first information, obtaining pixel coordinate information corresponding to the object to be located; The first positioning information is determined based on the pixel coordinate information, a first parameter corresponding to the first layer camera, and a second parameter.

5. The method according to claim 3, characterized in that: The performing image processing on the second information to obtain second positioning information corresponding to the object to be positioned includes: Processing the second information using a second preset model to obtain contour area information corresponding to the object to be located; Determine contour information corresponding to the object to be located according to the contour area information; The second positioning information corresponding to the object to be positioned is determined based on the contour information, the first parameter and the second parameter corresponding to the second layer camera.

6. The method according to claim 5, characterized in that The using the second preset model to process the second information to obtain the contour area information corresponding to the object to be located includes: Processing the second information using the second preset model to obtain pixel position information of the object to be located in the second information; Determine contour area information corresponding to the object to be located according to the pixel position information.

7. The method according to claim 5, characterized in that The determining the second positioning information corresponding to the object to be positioned based on the contour information, the first parameter and the second parameter corresponding to the second layer camera includes: Acquire first coordinate information according to the contour information; the first coordinate information represents coordinate information of edge points of the contour corresponding to the object to be located; Determine second coordinate information based on the first coordinate information, the first parameter and the second parameter corresponding to the second layer camera; the second coordinate information represents the image coordinate information corresponding to the first coordinate information; The second positioning information corresponding to the object to be positioned is determined according to the second coordinate information, the focal length of the camera and the position information of the drone.

8. A drone positioning device, characterized in that: The drone includes at least two layers of cameras; the visible area of ​​the cameras in each layer decreases as the layers increase; the device includes a first acquisition unit, a first determination unit, a second determination unit, a second acquisition unit and a processing unit, wherein: The first acquisition unit is used to acquire first information of a visible area captured by a first layer of cameras; the first layer of cameras is any layer of cameras in the at least one layer of cameras; The first determining unit is used to determine first positioning information corresponding to the object to be positioned based on the first information and a first preset model; The second determining unit is used to determine a second-layer camera connected to the first-layer camera according to the first positioning information; The second acquisition unit is used to acquire second information of the visible area captured by the second layer camera; The processing unit is used to perform image processing on the second information to obtain second positioning information corresponding to the object to be positioned.

9. A storage medium, characterized in that: The storage medium stores a computer program; when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A drone positioning device, characterized in that: The drone positioning device comprises: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes the steps of the method described in any one of claims 1 to 7.

Images