Ground target position determination method and device for low-altitude unmanned aerial vehicle

Abstract

The invention provides a ground target position determination method and device for a low-altitude unmanned aerial vehicle. The method comprises the following steps: acquiring a plurality of images by using an optical sensor carried by the low-altitude unmanned aerial vehicle, and acquiring pose information when the optical sensor acquires each image; acquiring targets in the images by using a target detection model, and matching the targets in different images to obtain a plurality of images containing the same target; and obtaining position information of the same target based on the pose information corresponding to the plurality of images containing the same target. According to the ground target position determination method and device for the low-altitude unmanned aerial vehicle provided by the invention, the position of the target is estimated based on the pose information during image acquisition, so that the unmanned aerial vehicle can realize the determination of the target position without mounting distance measurement equipment, and the system operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device and storage medium for determining the ground target position of a low-altitude unmanned aerial vehicle. Background Art

[0002] In tasks such as target tracking, personnel rescue, and material delivery, low-altitude unmanned aerial vehicles often need to obtain the precise position of the target in the world coordinate system. However, for scenes with large absolute elevation changes such as mountains, the position of the target is usually difficult to describe with simple latitude and longitude, but requires a position description with elevation information.

[0003] Generally speaking, low-altitude unmanned aerial vehicles can carry ranging sensors such as lidar to obtain altitude information. However, ranging sensors are usually bulky and have high power consumption, which will reduce mobility and mileage, and increase operating costs. Summary of the invention

[0004] The present invention provides a method and device for determining the ground target position of a low-altitude unmanned aerial vehicle, which are used to solve the technical problem of excessively high system operation cost in the prior art.

[0005] In a first aspect, the present invention provides a method for determining a ground target position for a low-altitude unmanned aerial vehicle, comprising the following steps.

[0006] Using an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtaining the position and posture information of each image when the optical sensor collects the image; Acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target; Based on the posture information corresponding to each of the multiple images containing the same target, the position information of the same target is obtained.

[0007] In some embodiments, the acquiring the target in the image using the target detection model and matching the targets in different images includes: Performing target detection on the image using a target detection model to obtain a target detection frame corresponding to each target in the image; Obtain a set of feature points in the target detection frame; Based on the neighborhood range of the target detection frame and the set of feature points in the target detection frame, targets in different images are matched.

[0008] In some embodiments, obtaining a set of feature points in the target detection frame includes: Obtaining the image block corresponding to the target detection frame; Performing feature extraction on the image blocks to obtain a set of feature points corresponding to the image blocks; Determine the feature point set in the target detection frame as the feature point set corresponding to the image block.

[0009] In some embodiments, matching objects in different images based on a neighborhood range of an object detection box and a set of feature points in the object detection box includes: Determine the neighborhood range of the target detection box in the current image; Taking the target detection box existing in the neighborhood range in the subsequent image as a candidate matching box; The feature point set in the candidate matching frame is matched with the feature point set in the target detection frame in the current image.

[0010] In some embodiments, the method further comprises: When the number of successfully matched feature points is higher than a preset threshold, determining that the target in the candidate matching frame and the target in the target detection frame in the current image are the same target; and / or When the number of successfully matched feature points is less than or equal to a preset threshold, it is determined that the target in the candidate matching frame is not the same target as the target in the target detection frame in the current image.

[0011] In some embodiments, the acquiring the position information of the same target based on the pose information corresponding to each of the multiple images containing the same target includes: Based on the posture information corresponding to each of the multiple images containing the same target and the target detection frame of the same target, the position information of the same target in the world coordinate system is obtained by using a multi-eye vision method and a triangulation positioning method.

[0012] In some embodiments, the collecting of multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle includes: Acquire multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle; and / or Optical sensors carried by multiple low-altitude unmanned aerial vehicles are used to obtain multiple images corresponding to the same moment.

[0013] In some embodiments, the step of obtaining the position information of each image captured by the optical sensor includes: Using the positioning sensor carried by the low-altitude unmanned aerial vehicle to obtain the position data of each image captured by the optical sensor in the world coordinate system; The attitude sensor carried by the low-altitude unmanned aerial vehicle is used to obtain the attitude data of each image captured by the optical sensor.

[0014] In a second aspect, the present invention provides a ground target position determination device for a low-altitude unmanned aerial vehicle, comprising the following modules.

[0015] A first acquisition module is used to collect multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle, and obtain the posture information when the optical sensor collects each image; A second acquisition module is used to acquire the target in the image by using the target detection model, and match the targets in different images to obtain multiple images containing the same target; The third acquisition module is used to acquire the position information of the same target based on the posture information corresponding to each of the multiple images containing the same target.

[0016] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the above-described methods for determining the ground target position of a low-altitude unmanned aerial vehicle.

[0017] In a fourth aspect, a non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements any of the above-described methods for determining a ground target position for a low-altitude unmanned aerial vehicle.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for determining the ground target position of a low-altitude unmanned aerial vehicle as described in the first aspect above.

[0019] The method and device for determining the position of a ground target for a low-altitude unmanned aerial vehicle provided by the present invention utilizes an optical sensor carried by the low-altitude unmanned aerial vehicle to collect multiple images, and obtains the posture information when the optical sensor collects each image; utilizes a target detection model to obtain the target in the image, and matches the targets in different images to obtain multiple images containing the same target; based on the posture information corresponding to each of the multiple images containing the same target, obtains the position information of the same target. After target recognition and target matching based on computer vision, the position of the target is estimated based on the posture information when the image is collected, so that the unmanned aerial vehicle can determine the target position without mounting a ranging device, thereby reducing equipment costs, reducing system weight, thereby improving system maneuverability and cruising range, and reducing the total power consumption of the system, thereby reducing the overall system operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is one of the flow charts of the method for determining the ground target position of a low-altitude unmanned aerial vehicle provided by the present invention.

[0022] Figure 2 This is the second flow chart of the method for determining the ground target position of a low-altitude unmanned aerial vehicle provided by the present invention.

[0023] Figure 3 It is a schematic diagram of target feature matching between multiple frame images provided by the present invention.

[0024] Figure 4 It is a schematic diagram of target position estimation based on multi-eye vision provided by the present invention.

[0025] Figure 5 It is a structural schematic diagram of a ground target position determination device for a low-altitude unmanned aerial vehicle provided by the present invention.

[0026] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Figure 1 FIG. 1 is one of the flow charts of the method for determining the ground target position of a low-altitude unmanned aerial vehicle provided by the present invention, such as Figure 1 As shown, the present invention provides a method for determining the ground target position of a low-altitude unmanned aerial vehicle. The method comprises: Step 101: Use an optical sensor carried by a low-altitude unmanned aerial vehicle to capture multiple images, and obtain the position and posture information of each image captured by the optical sensor.

[0029] Specifically, an optical sensor carried or mounted on a low-altitude unmanned aerial vehicle is used to capture multiple images at a fixed frame rate.

[0030] Low-altitude unmanned aerial vehicles are equipped with onboard computing units that are used to transmit images and posture information to cloud computers for processing.

[0031] The optical sensor can be an imaging device such as a visible light camera or a thermal imaging camera. The imaging device should be calibrated with a monocular camera to achieve image dedistortion and obtain internal parameters.

[0032] The images collected by the optical sensor are images taken towards the ground.

[0033] The position information of each image captured by the optical sensor is obtained while the image is captured. The position information includes position information and attitude information. The position information may be position coordinates in a world coordinate system, and the attitude information includes pitch, roll, and yaw angles.

[0034] Step 102: Use the target detection model to obtain the target in the image, and match the targets in different images to obtain multiple images containing the same target.

[0035] Specifically, the trained target detection model is used to identify the target in the image, and the detection box or rectangular box of each target in the image is output. The targets in different images are matched, that is, similarity comparison is performed. If the target similarity is very high, the match is successful, indicating that the targets in the different images being matched are the same target. After traversing all the targets, one or more image sets are obtained, and each image set contains multiple images of the same target.

[0036] The target detection model can be a YOLO series or other target detection network models.

[0037] Step 103: Based on the posture information corresponding to each of the multiple images containing the same target, obtain the position information of the same target.

[0038] Specifically, for each target that has been matched successfully at least once, a set of successfully matched images is counted, and the pose information corresponding to each image in the image set is determined to obtain a pose information set. Based on the pose information set and the pixel position of the target (i.e., the position information obtained by target detection), the position information of the target is estimated.

[0039] The method for determining the position of a ground target for a low-altitude unmanned aerial vehicle provided in an embodiment of the present application uses an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtains the position and posture information when the optical sensor collects each image; uses a target detection model to obtain the target in the image, and matches the targets in different images to obtain multiple images containing the same target; based on the position and posture information corresponding to each of the multiple images containing the same target, obtain the position information of the same target. After target recognition and target matching based on computer vision, the position of the target is estimated based on the position and posture information when the image is collected, so that the unmanned aerial vehicle can determine the target position without mounting a ranging device, reducing equipment costs and system weight, thereby improving system maneuverability and cruising range, and reducing the total power consumption of the system, thereby reducing the overall system operating costs.

[0040] In some embodiments, the method of acquiring multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle includes: Acquire multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle; and / or Optical sensors carried by multiple low-altitude unmanned aerial vehicles are used to obtain multiple images corresponding to the same moment.

[0041] Specifically, the target position may be estimated by using multiple frames of images collected by a single aircraft, or by using multiple frames of images collected by multiple aircraft.

[0042] If a low-altitude unmanned aerial vehicle is used, the optical sensor thereon collects images at different positions or at multiple times to obtain multiple frames of images.

[0043] If multiple low-altitude unmanned aerial vehicles are used, the optical sensors on the multiple low-altitude unmanned aerial vehicles obtain multiple images corresponding to the same moment, that is, each low-altitude unmanned aerial vehicle obtains an image corresponding to that moment, and then these multiple unmanned aerial vehicles need to aggregate the collected images to a certain drone or cloud computer for processing.

[0044] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in the embodiment of the present application can use one low-altitude unmanned aerial vehicle to collect multiple images at different shooting positions or at different times, or can use multiple low-altitude unmanned aerial vehicles to collect multiple images corresponding to the same time, thereby meeting the needs of various scenarios and increasing the flexibility of application.

[0045] In some embodiments, the step of obtaining the position information of each image captured by the optical sensor includes: Using the positioning sensor carried by the low-altitude unmanned aerial vehicle to obtain the position data of each image captured by the optical sensor in the world coordinate system; The attitude sensor carried by the low-altitude unmanned aerial vehicle is used to obtain the attitude data of each image captured by the optical sensor.

[0046] Specifically, the attitude sensor carried by the low-altitude unmanned aerial vehicle is used to obtain the attitude data of each image collected by the optical sensor. The attitude sensor can be a sensor such as a gyroscope, an angular velocity meter or an accelerometer. For example, the attitude data of the imaging device can be obtained by using a rotating gimbal provided by the imaging device.

[0047] The low-altitude UAV is also equipped with a global positioning sensor, which uses the positioning sensor carried by the low-altitude UAV to obtain the position data of the optical sensor in the world coordinate system when collecting each image. Among them, the (global) positioning sensor can provide real-time three-dimensional position information of the UAV in the world coordinate system. The positioning sensor can be a global positioning system (GPS) or real-time dynamic carrier phase differential technology (RTK).

[0048] If multiple low-altitude unmanned aerial vehicles are used, after obtaining the position and posture information when the optical sensor collects each image, the position and posture information of these multiple images are summarized on a certain drone or cloud computer for processing.

[0049] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in the embodiment of the present application uses a global positioning sensor carried by the low-altitude unmanned aerial vehicle to obtain the position data of the low-altitude unmanned aerial vehicle in the world coordinate system, and uses an attitude sensor carried by the low-altitude unmanned aerial vehicle to obtain the attitude data of the optical sensor, thereby realizing the position and attitude data of the aircraft when collecting each image. The operation is simple, the system weight is hardly increased, and the system maneuverability and cruising range are guaranteed.

[0050] In some embodiments, the acquiring the target in the image using the target detection model and matching the targets in different images includes: Performing target detection on the image using a target detection model to obtain a target detection frame corresponding to each target in the image; Obtain a set of feature points in the target detection frame; Based on the neighborhood range of the target detection frame and the set of feature points in the target detection frame, targets in different images are matched.

[0051] Specifically, the target detection model is used to perform target detection on the image, and a target detection box corresponding to each target in the image is output.

[0052] For example, an optical sensor carried by a low-altitude unmanned aerial vehicle or N low-altitude unmanned aerial vehicles is used to collect the images from frame 0 to frame 1. Frame total Frame Image ,in I i Represents the i-th frame image, and N is a positive integer. The target detection model is used to obtain A collection of group target detection boxes ,in Rect i Represents the i-th set of target detection boxes. Each target detection box in each detection box set can be represented by the center point pixel coordinates and the width and height of the rectangle.

[0053] The feature point set in the target detection frame is obtained through feature extraction, and then the targets in different images are matched based on the neighborhood range and feature point set of the target detection frame. That is, for the target detection frames in the same neighborhood range of different images, the corresponding targets are matched by matching the feature point sets therein.

[0054] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in an embodiment of the present application adopts a target detection neural network model to obtain pixel position information of the target in the image (i.e., the target detection frame), and then uses feature extraction and feature matching to match the detected targets in different image frames. Specifically, a neighborhood range of a specific target detection frame is set for target detection frame matching of multiple frames of images, thereby achieving target matching between multiple frames of images, so that the target position can be obtained based on the posture information corresponding to the multiple frames of images matched to the same target.

[0055] In some embodiments, obtaining a set of feature points in the target detection frame includes: Obtaining the image block corresponding to the target detection frame; Performing feature extraction on the image blocks to obtain a set of feature points corresponding to the image blocks; Determine the feature point set in the target detection frame as the feature point set corresponding to the image block.

[0056] Specifically, a set of feature points in the target detection frame is obtained through feature extraction. First, an image block (or sub-image) corresponding to the target detection frame is obtained, and features are extracted from the image block to obtain a corresponding set of feature points, which is the set of feature points in the target detection frame.

[0057] For example, for the target detection box set , according to the range of each target detection frame in its set, the image of the corresponding area is intercepted and the features of its sub-image are extracted. For example, suppose the target detection frame set Include target detection box, from its corresponding image According to the range of each target detection frame, the target sub-image is captured , a total of n sub-images. Then for each sub-image Perform feature extraction to obtain the set of feature points corresponding to each sub-image The feature extraction in this step can be implemented by using a feature algorithm that combines an improved FAST corner detection algorithm and a binary descriptor (Oriented FAST and Rotated BRIEF, ORB) or an operator such as Scale-Invariant Feature Transform (SIFT).

[0058] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in an embodiment of the present application intercepts a sub-image corresponding to a target detection frame and extracts features of the sub-image, thereby converting target matching into matching of feature points in the target detection frame, thereby ensuring matching accuracy.

[0059] In some embodiments, matching objects in different images based on a neighborhood range of an object detection box and a set of feature points in the object detection box includes: Determine the neighborhood range of the target detection box in the current image; Taking the target detection box existing in the neighborhood range in the subsequent image as a candidate matching box; The feature point set in the candidate matching frame is matched with the feature point set in the target detection frame in the current image.

[0060] Specifically, a neighborhood range of a specific target detection box is set, and the target detection box existing in the neighborhood range in the subsequent image is used as a candidate matching box. Then, the feature point set in the candidate matching box is matched with the feature point set in the target detection box in the current image, thereby achieving target matching between images.

[0061] For example, Figure 2 FIG. 2 is a flow chart of a method for determining a ground target position of a low-altitude unmanned aerial vehicle provided by the present invention. Figure 2 As shown, the allowed width neighborhood of the target detection box matching is set to , the height neighborhood is (in and are greater than zero), assuming that the image frame The corresponding target detection box The center point is located at (in and are greater than zero), then in the subsequent images to in Search for the existing target detection box within the range. Figure 3 is a schematic diagram of target feature matching between multiple frame images provided by the present invention, such as Figure 3 As shown, if there is a detection box, the image The corresponding detection box Located on target In the corresponding neighborhood, the set of feature points corresponding to its subgraph With feature point set Perform feature matching.

[0062] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in an embodiment of the present application sets a neighborhood range of a target detection frame, matches a set of feature points in a target detection frame within the neighborhood range in a subsequent image with a set of feature points in a target detection frame in a current image, thereby achieving target matching between multiple frames of images.

[0063] In some embodiments, the method further comprises: When the number of successfully matched feature points is higher than a preset threshold, determining that the target in the candidate matching frame and the target in the target detection frame in the current image are the same target; and / or When the number of successfully matched feature points is less than or equal to a preset threshold, it is determined that the target in the candidate matching frame is not the same target as the target in the target detection frame in the current image.

[0064] Specifically, if the number of successfully matched feature points is higher than a preset threshold, it is considered that the target feature is matched successfully, and the target in the candidate matching frame is the same target as the target detection frame in the current image.

[0065] If the number of successfully matched feature points is less than or equal to the preset threshold, it is considered that the target feature matching fails, and the target in the candidate matching frame is not the same target as the target detection frame in the current image.

[0066] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in the embodiment of the present application uses a preset threshold to determine whether the set of feature points at corresponding positions between multiple frame images matches. The preset threshold can be flexibly set based on experience and specific needs, thereby improving the accuracy of target matching between multiple frame images.

[0067] In some embodiments, the acquiring the position information of the same target based on the pose information corresponding to each of the multiple images containing the same target includes: Based on the posture information corresponding to each of the multiple images containing the same target and the target detection frame of the same target, the position information of the same target in the world coordinate system is obtained by using a multi-eye vision method and a triangulation positioning method.

[0068] Specifically, for each target that has been successfully matched at least once, the image set whose features have been successfully matched is counted, and the three-dimensional position set (i.e., position information) and camera posture set (i.e., posture information) in the world coordinate system corresponding to the image set are obtained. Based on the posture information corresponding to multiple images containing the same target and the target detection frame (i.e., pixel position) of the same target, the multi-eye vision method and triangulation positioning method are used to estimate the position information of the target in the world coordinate system.

[0069] For example, if a goal is l If the feature matching in the frame image is successful, the target is l The pixel coordinates of the center point of the target detection frame of the frame image are used as its pixel position, and then combined with the camera's internal reference information and the position and posture information of the camera in the world coordinate system when the frame image was taken, the camera pinhole imaging model is used to obtain the possible location of the target in the world coordinate system. l Three-dimensional straight lines.

[0070] Figure 4 : is a schematic diagram of target position estimation based on multi-eye vision provided by the present invention, such as Figure 4 As shown, assume that the image is acquired and images The world coordinate system coordinates of the aircraft's camera are and The x, y, and z axes of the camera coordinate system are shown in the figure, where the focus of the camera is and are all located on the z-axis. According to the pinhole imaging model, the target frame and target box The center point of and Lead line and , then the two lines should intersect at the point The location is where the target is.

[0071] Theoretically, this l The frame image corresponds to l The intersection of the two straight lines should be the position of the target in the world coordinate system. In practical applications, due to the existence of various error factors, l The straight lines may not intersect at exactly one point. In this case, the point closest to each non-coplanar straight line can be solved by the least squares method as the position estimate of the world coordinate system where the target is located.

[0072] The method for determining the ground target position of a low-altitude unmanned aerial vehicle provided in the embodiment of the present application can, for targets that have been successfully matched for more than one frame, combine the posture information corresponding to the successfully matched image frames to achieve target position estimation in a multi-eye visual triangulation positioning manner, thereby reducing the dependence of the low-altitude unmanned aerial vehicle on ranging sensors in high-precision target position estimation tasks. The unmanned aerial vehicle can acquire the target position without mounting ranging equipment such as lidar, which can reduce equipment costs, reduce system weight to improve system maneuverability and cruising range, and reduce total system power consumption, thereby reducing system operating costs.

[0073] Figure 5 FIG. 1 is a schematic diagram of a structure of a ground target position determination device for a low-altitude unmanned aerial vehicle provided by the present invention. Figure 5 As shown, the present invention provides a ground target position determination device for a low-altitude unmanned aerial vehicle, comprising a first acquisition module 501, a second acquisition module 502 and a third acquisition module 503.

[0074] The first acquisition module 501 is used to collect multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle, and obtain the position and posture information of each image collected by the optical sensor.

[0075] The second acquisition module 502 is used to acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target.

[0076] The third acquisition module 503 is used to acquire the position information of the same target based on the posture information corresponding to each of the multiple images containing the same target.

[0077] In some embodiments, the second acquisition module 502 includes: A first acquisition unit is used to perform target detection on the image using a target detection model to obtain a target detection frame corresponding to each target in the image; A second acquisition unit, used to acquire a set of feature points in the target detection frame; The matching unit is used to match the targets in different images based on the neighborhood range of the target detection frame and the set of feature points in the target detection frame.

[0078] In some embodiments, the second acquiring unit includes: A first acquisition subunit is used to acquire the image block corresponding to the target detection frame; A second acquisition subunit is used to extract features from the image blocks to obtain a set of feature points corresponding to the image blocks; The first determining subunit is used to determine that the feature point set in the target detection frame is the feature point set corresponding to the image block.

[0079] In some embodiments, the matching unit comprises: A second determination subunit is used to determine the neighborhood range of the target detection frame in the current image; A third determination subunit is used to take the target detection frame existing in the neighborhood range in the subsequent image as a candidate matching frame; The matching subunit is used to match the feature point set in the candidate matching frame with the feature point set in the target detection frame in the current image.

[0080] In some embodiments, the matching unit further comprises: a fourth determination subunit, configured to determine, when the number of successfully matched feature points is greater than a preset threshold, that the target in the candidate matching frame and the target in the target detection frame in the current image are the same target; and / or The fifth determination subunit is used to determine that the target in the candidate matching frame and the target in the target detection frame in the current image are not the same target when the number of successfully matched feature points is less than or equal to a preset threshold.

[0081] In some embodiments, the third acquisition module includes: The third acquisition unit is used to obtain the position information of the same target in the world coordinate system by using a multi-eye vision method and a triangulation positioning method based on the posture information corresponding to each of the multiple images containing the same target and the target detection frame of the same target.

[0082] In some embodiments, the first acquisition module 501 includes: A fourth acquisition unit is used to acquire multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle; and / or The fifth acquisition unit is used to acquire multiple images corresponding to the same moment using optical sensors carried by multiple low-altitude unmanned aerial vehicles.

[0083] In some embodiments, the first acquisition module 501 further includes: A sixth acquisition unit, configured to acquire position data of each image captured by the optical sensor in a world coordinate system by using a positioning sensor carried by the low-altitude unmanned aerial vehicle; The seventh acquisition unit is used to use the attitude sensor carried by the low-altitude unmanned aerial vehicle to acquire the attitude data when the optical sensor captures each image.

[0084] Specifically, the above-mentioned ground target position determination device for low-altitude unmanned aerial vehicles provided by the present invention can implement all the method steps implemented by the above-mentioned ground target position determination method embodiment for low-altitude unmanned aerial vehicles, and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be specifically repeated here.

[0085] It should be noted that the division of units / modules in the above-mentioned embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0086] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 601, a communications interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communications interface 602 and the memory 603 communicate with each other via the communication bus 604. The processor 601 may call the logic instructions in the memory 603 to execute a method for determining a ground target position for a low-altitude unmanned aerial vehicle, the method comprising: Using an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtaining the position and posture information of each image when the optical sensor collects the image; Acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target; Based on the posture information corresponding to each of the multiple images containing the same target, the position information of the same target is obtained.

[0087] In some embodiments, the acquiring the target in the image using the target detection model and matching the targets in different images includes: Performing target detection on the image using a target detection model to obtain a target detection frame corresponding to each target in the image; Obtain a set of feature points in the target detection frame; Based on the neighborhood range of the target detection frame and the set of feature points in the target detection frame, targets in different images are matched.

[0088] In some embodiments, obtaining a set of feature points in the target detection frame includes: Obtaining the image block corresponding to the target detection frame; Performing feature extraction on the image blocks to obtain a set of feature points corresponding to the image blocks; Determine the feature point set in the target detection frame as the feature point set corresponding to the image block.

[0089] In some embodiments, matching objects in different images based on a neighborhood range of an object detection box and a set of feature points in the object detection box includes: Determine the neighborhood range of the target detection box in the current image; Taking the target detection box existing in the neighborhood range in the subsequent image as a candidate matching box; The feature point set in the candidate matching frame is matched with the feature point set in the target detection frame in the current image.

[0090] In some embodiments, the method further comprises: When the number of successfully matched feature points is higher than a preset threshold, determining that the target in the candidate matching frame and the target in the target detection frame in the current image are the same target; and / or When the number of successfully matched feature points is less than or equal to a preset threshold, it is determined that the target in the candidate matching frame is not the same target as the target in the target detection frame in the current image.

[0091] In some embodiments, the acquiring the position information of the same target based on the pose information corresponding to each of the multiple images containing the same target includes: Based on the posture information corresponding to each of the multiple images containing the same target and the target detection frame of the same target, the position information of the same target in the world coordinate system is obtained by using a multi-eye vision method and a triangulation positioning method.

[0092] In some embodiments, the collecting of multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle includes: Acquire multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle; and / or Optical sensors carried by multiple low-altitude unmanned aerial vehicles are used to obtain multiple images corresponding to the same moment.

[0093] In some embodiments, the step of obtaining the position information of each image captured by the optical sensor includes: Using the positioning sensor carried by the low-altitude unmanned aerial vehicle to obtain the position data of each image captured by the optical sensor in the world coordinate system; The attitude sensor carried by the low-altitude unmanned aerial vehicle is used to obtain the attitude data of each image captured by the optical sensor.

[0094] Specifically, the processor 601 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0095] The logic instructions in the memory 603 can be implemented in the form of software functional units and can be stored in a processor-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0096] In some embodiments, a computer program product is further provided, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the ground target position determination method for a low-altitude unmanned aerial vehicle provided in the above-mentioned method embodiments, the method comprising: Using an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtaining the position and posture information of each image when the optical sensor collects the image; Acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target; Based on the posture information corresponding to each of the multiple images containing the same target, the position information of the same target is obtained.

[0097] Specifically, the above-mentioned computer program product provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0098] In some embodiments, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a computer to execute the ground target position determination method for a low-altitude unmanned aerial vehicle provided in the above-mentioned method embodiments, the method comprising: Using an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtaining the position and posture information of each image when the optical sensor collects the image; Acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target; Based on the posture information corresponding to each of the multiple images containing the same target, the position information of the same target is obtained.

[0099] Specifically, the above-mentioned computer-readable storage medium provided by the present invention can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effects. The parts and beneficial effects that are the same as the method embodiments in this embodiment will not be described in detail here.

[0100] It should be noted that the computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0101] It should also be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0102] In the present invention, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0103] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.

[0104] In the present invention, "determine B based on A" means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "determine B based on A and C", "determine B based on A, C and E", "determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B", etc.; for another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "when A meets the first condition, use the first method to determine C, and further determine B based on C", etc.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0109] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for determining the ground target position of a low-altitude unmanned aerial vehicle, characterized in that: include: Using an optical sensor carried by a low-altitude unmanned aerial vehicle to collect multiple images, and obtaining the position and posture information of each image when the optical sensor collects the image; Acquire the target in the image using the target detection model, and match the targets in different images to obtain multiple images containing the same target; Based on the posture information corresponding to each of the multiple images containing the same target, the position information of the same target is obtained.

2. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 1, characterized in that: The method of acquiring the target in the image by using the target detection model and matching the targets in different images includes: Performing target detection on the image using a target detection model to obtain a target detection frame corresponding to each target in the image; Obtain a set of feature points in the target detection frame; Based on the neighborhood range of the target detection frame and the set of feature points in the target detection frame, targets in different images are matched.

3. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 2, characterized in that: The acquiring of a set of feature points in the target detection frame comprises: Obtaining the image block corresponding to the target detection frame; Performing feature extraction on the image blocks to obtain a set of feature points corresponding to the image blocks; Determine the feature point set in the target detection frame as the feature point set corresponding to the image block.

4. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 2, characterized in that: The matching of objects in different images based on the neighborhood range of the object detection frame and the set of feature points in the object detection frame includes: Determine the neighborhood range of the target detection box in the current image; Taking the target detection box existing in the neighborhood range in the subsequent image as a candidate matching box; The feature point set in the candidate matching frame is matched with the feature point set in the target detection frame in the current image.

5. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 4, characterized in that: The method further comprises: When the number of successfully matched feature points is higher than a preset threshold, determining that the target in the candidate matching frame and the target in the target detection frame in the current image are the same target; and / or When the number of successfully matched feature points is less than or equal to a preset threshold, it is determined that the target in the candidate matching frame is not the same target as the target in the target detection frame in the current image.

6. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 1, characterized in that: The acquiring the position information of the same target based on the pose information corresponding to each of the multiple images containing the same target comprises: Based on the posture information corresponding to each of the multiple images containing the same target and the target detection frame of the same target, the position information of the same target in the world coordinate system is obtained by using a multi-eye vision method and a triangulation positioning method.

7. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 1, characterized in that: The method of collecting multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle includes: Acquire multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle; and / or Optical sensors carried by multiple low-altitude unmanned aerial vehicles are used to obtain multiple images corresponding to the same moment.

8. The method for determining the ground target position of a low-altitude unmanned aerial vehicle according to claim 1, characterized in that: The obtaining of the position information of each image captured by the optical sensor includes: Using the positioning sensor carried by the low-altitude unmanned aerial vehicle to obtain the position data of each image captured by the optical sensor in the world coordinate system; The attitude sensor carried by the low-altitude unmanned aerial vehicle is used to obtain the attitude data of each image captured by the optical sensor.

9. A ground target position determination device for a low-altitude unmanned aerial vehicle, characterized in that: include: A first acquisition module is used to collect multiple images using an optical sensor carried by a low-altitude unmanned aerial vehicle, and obtain the posture information when the optical sensor collects each image; A second acquisition module is used to acquire the target in the image by using the target detection model, and match the targets in different images to obtain multiple images containing the same target; The third acquisition module is used to acquire the position information of the same target based on the posture information corresponding to each of the multiple images containing the same target.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for determining the ground target position of a low-altitude unmanned aerial vehicle as described in any one of claims 1 to 8 is implemented.

Images