Image-based positioning method, unmanned aerial vehicle recovery method and device

Abstract

The application discloses an image-based positioning method, a UAV recovery method and device, and relates to the technical field of image processing. The positioning method comprises the following steps: determining a background image based on an original background image without a monitoring target; determining a monitoring image based on an original monitoring image with the monitoring target; wherein the monitoring image and the background image contain the same key attention region, and the key attention region contains a position calibration object; performing filtering processing on the region images of the key attention region in the background image and the monitoring image along a first direction respectively to obtain a first background image and a first monitoring image, and determining a difference image; and determining the physical relative position of the monitoring target and the position calibration object in a preset direction according to the difference image. The application solves the problem that the existing motion target monitoring method cannot quickly position non-uniform motion targets, can quickly position non-uniform motion targets, and can meet the timeliness requirement in the UAV recovery process.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image-based positioning method, a drone recovery method and apparatus. Background Technology

[0002] With the rapid development of edge computing and image processing technologies, image-based localization technology is finding increasingly wider applications in artificial intelligence, autonomous driving, and drones. Common algorithms for monitoring moving targets include inter-frame differencing, the Vibe algorithm, and Gaussian mixture models. However, these algorithms cannot be used directly when the moving target pauses. When the pixel changes within the moving target are small or the target's speed is too fast, inter-frame differencing is prone to detecting holes or segmentation of the target's interior. The Vibe algorithm provides relatively complete monitoring of moving targets but is susceptible to noise and ghosting. Gaussian mixture models require setting appropriate parameters based on the pause time, but in some special cases, the pause time of the moving target is relatively random.

[0003] Therefore, how to quickly locate moving targets with irregular speeds and pauses has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides an image-based localization method, a drone recovery method, and an apparatus, which solves the problem that existing moving target monitoring methods cannot quickly locate non-uniformly moving targets.

[0005] In a first aspect, embodiments of this application provide an image-based positioning method, comprising: performing a differential step based on an original background image without a monitoring target acquired along a preset direction and an original monitoring image containing a monitoring target to obtain a differential image; wherein the differential step comprises: determining a background image based on the original background image without a monitoring target; determining a monitoring image based on the original monitoring image containing a monitoring target; wherein the monitoring image and the background image contain the same key area of ​​interest, and the key area of ​​interest contains a location marker; performing filtering processing along a first direction on the region images of the key area of ​​interest in the background image and the monitoring image respectively to obtain a first background image and a first monitoring image; determining the differential image of the first background image and the first monitoring image; and determining the physical relative position of the monitoring target and the location marker in the preset direction based on the differential image.

[0006] In conjunction with the first aspect, in one possible implementation, before performing the differential step based on the original background image without the monitoring target acquired along a preset direction and the original monitoring image containing the monitoring target, the following steps are included: acquiring multiple original background images of a preset area along the preset direction; wherein the preset area includes a key area of ​​interest; and stopping the acquisition of the original background image when a monitoring target that meets preset conditions appears in the original background image.

[0007] In conjunction with the first aspect, in one possible implementation, determining the background image based on the original background image without the monitoring target includes: taking a preset number of original background images without the monitoring target as a background image group; obtaining a representative background image of the key area of ​​interest based on the background image group; and determining the background image based on the representative background image. The process of obtaining the representative background image includes: dividing the region image of the key area of ​​interest in each original background image in the background image group into a preset number of image blocks; traversing each image block and performing anomaly judgment based on the grayscale values ​​of image blocks at the same position in other original background images; if at least one original background image in the background image group has all image blocks that are normal, then selecting the region image of the key area of ​​interest in one of the original background images as the representative background image; otherwise, selecting the image block with the smallest error among the image blocks corresponding to each position and stitching them together to obtain the representative background image.

[0008] In conjunction with the first aspect, in one possible implementation, determining the background image based on the original background image without the monitoring target further includes: determining multiple representative background images based on multiple background image groups; forming a new background image group by combining at least a portion of the original background images in one background image group with the multiple representative background images; and determining the representative background image in the new background image group as the background image using the process of acquiring the representative background images.

[0009] In conjunction with the first aspect, in one possible implementation, the first direction includes either the horizontal or vertical direction of the region image; when the first direction is horizontal, the filter window size is 1. i; where i represents the length of the filtering window, i is a natural number less than n, and n is the number of horizontal pixels in the region image to be filtered; when the first direction is the vertical direction, the size of the filtering window is j. 1; where j represents the width of the filtering window, j is a natural number less than m, and m is the number of vertical pixels in the region image to be filtered.

[0010] In conjunction with the first aspect, in one possible implementation, determining the physical relative position of the monitoring target and the location marker in the preset direction based on the differential image includes: finding multiple closed region contours in the differential image through contour searching; selecting the region contour with the most internal pixel values ​​and determining the minimum bounding rectangle of the region contour; determining the position information of the monitoring target based on the position information of the minimum bounding rectangle; and determining the physical relative position of the monitoring target and the location marker in the preset direction based on the position information of the monitoring target.

[0011] In conjunction with the first aspect, in one possible implementation, determining the physical relative position of the monitoring target and the location marker in the preset direction based on the differential image further includes: determining a second preset direction that forms a preset angle with the preset direction; performing the differential step based on an original background image without the monitoring target and an original monitoring image containing the monitoring target acquired along the second preset direction to obtain a second differential image; and determining the physical relative position of the monitoring target and the location marker in the second preset direction based on the second differential image.

[0012] Secondly, embodiments of this application provide a method for recovering a drone, comprising: setting initial conditions, including: using the drone as a monitoring target, using the drone nest as a location marker, determining a preset direction relative to the drone nest, and determining a key area of ​​concern based on the location of the drone nest; executing the method described in the first aspect or any possible implementation of the first aspect based on the initial conditions to obtain the physical relative position of the drone and the drone nest; when the physical relative position of the drone and the drone nest meets preset requirements, the drone lands in the drone nest; otherwise, the drone lands at a backup landing point.

[0013] In conjunction with the second aspect, in one possible implementation, the step of the drone landing in the nest when the physical relative position of the drone and the nest meets the preset requirements includes: determining a centerline passing through the center of the nest and perpendicular to the corresponding preset direction; determining one or more vertical distances between the drone and the centerline; and when all the vertical distances are within a preset threshold range, the drone lands in the nest.

[0014] Thirdly, embodiments of this application provide an image-based positioning device, comprising: an execution module, configured to perform a differential step based on an original background image without a monitoring target and an original monitoring image containing a monitoring target acquired along a preset direction, to obtain a differential image; wherein the differential step comprises: a first determination module, configured to determine a background image based on the original background image without a monitoring target; a second determination module, configured to determine a monitoring image based on the original monitoring image containing a monitoring target; wherein the monitoring image and the background image contain the same key area of ​​interest, and the key area of ​​interest contains a location marker; a filtering module, configured to perform filtering processing along a first direction on the area images of the key area of ​​interest in the background image and the monitoring image respectively, to obtain a first background image and a first monitoring image; a differential module, configured to determine the differential image of the first background image and the first monitoring image; and a positioning module, configured to determine the physical relative position of the monitoring target and the location marker in the preset direction based on the differential image.

[0015] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0016] This embodiment of the application, by performing a differential step, can obtain a differential image, thereby quickly determining the physical relative distance between the monitored target and the location marker. By filtering the regional image along a first direction, the high-frequency components of the regional image can be reduced, the visual effects of the original monitoring image can be ignored, and as much directional information as possible can be preserved, thus adapting to changes in the speed of the UAV. This effectively solves the problem that existing moving target monitoring methods cannot quickly locate non-uniformly moving targets, enabling rapid location of moving targets with irregular speeds and pauses. This can then be used to assist in UAV recovery, improving the success rate and safety of UAV recovery, while requiring less computation and meeting the timeliness requirements of the UAV recovery process. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the image-based localization method provided in this application embodiment;

[0019] Figure 2 A flowchart illustrating a method for determining a representative background image provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating the process of determining a background representative image provided in an embodiment of this application;

[0021] Figure 4 Example diagram of the nest provided in the embodiments of this application;

[0022] Figure 5 An example diagram of mean filtering with a 3×3 filtering window size provided in the embodiments of this application;

[0023] Figure 6 An example diagram of filtering processing along a first direction provided in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of an image-based positioning device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0027] Figure 1 This is a flowchart of an image-based localization method provided in an embodiment of this application, including steps 101 to 106. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order of the image-based localization method. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0028] It should be noted that the positioning mentioned in this application refers to the determination of the physical relative position of the monitoring target and the location marker in one or more directional dimensions.

[0029] Step 101: Acquire original background images without monitoring targets and original monitoring images containing monitoring targets along a preset direction. In this embodiment, acquiring original background images without monitoring targets along a preset direction includes: acquiring multiple original background images of a preset area along the preset direction. The preset area includes a key area of ​​interest. When a monitoring target meeting preset conditions appears in the original background image, the acquisition of the original background image is stopped.

[0030] Specifically, the monitoring target in this application is a dynamically defined object (i.e., a moving target that needs to be located or monitored). The focus area is determined based on the position of a location marker within a preset area. The location marker is usually a static object, and a certain direction relative to the location marker is used as the preset direction. For example, the horizontal direction relative to the front of the location marker is used as the preset direction. That is, a monitoring camera is set up horizontally opposite the front of the location marker to capture the original background image / original monitoring image or monitoring video of the horizontal direction of the front of the location marker, and the video frames in the monitoring video are used as the original background image / original monitoring image.

[0031] It should be noted that both the original background image and the original monitoring image are acquired by a monitoring camera that takes pictures along a preset direction. The difference is that the original background image does not show the monitoring target, or shows a part of the monitoring target but is not enough to identify the monitoring target, while the original monitoring image shows the complete monitoring target, or shows a part of the monitoring target but is enough to identify the monitoring target.

[0032] In one embodiment of this application, the application scope of drones in hazardous operation scenarios continues to expand, and the difficulty of drone recovery also increases accordingly. A safe drone recovery process generally consists of three stages: First, the drone quickly enters the monitoring field of view (the deployment area of ​​the drone nest is usually equipped with monitoring cameras), this process lasts 2-3 seconds; second, the drone reaches the designated hovering altitude, this stage lasts 1-3 seconds; third, the drone lands on the drone nest's parking platform. Typically, drones rely mainly on positioning systems for location assistance during missions and recovery. However, in specific environmental areas (such as forests without base station coverage, steep valleys, deep tunnels, etc.), there are limitations, resulting in weak or even lost positioning signals, leading to a failure to locate the drone and ultimately, recovery failure.

[0033] Existing drone nests are typically equipped with surveillance cameras. This application can achieve real-time calculation of drone landing trajectories by relying on existing surveillance cameras, without adding new hardware and while ensuring takeoff and landing safety. This enables timely detection of dangers during drone landing and assists in the safe recovery of drones.

[0034] Specifically, when the drone can receive a positioning signal normally and the positioning data is accurate, the drone can use the positioning data to land on the drone recovery platform. However, if the positioning signal is weak or the positioning data is inaccurate, the drone recovery will fail.

[0035] Therefore, this application utilizes surveillance video captured by cameras deployed near the drone nest to assist in drone retrieval without adding additional hardware. The deployment area of ​​the drone nest is primarily open, and the background scenery varies depending on the actual conditions. When there are trees, flowers, or other vegetation that sways in the wind within the monitoring field of view, the background of the surveillance video will dynamically change. Furthermore, the drone's mission time is mostly concentrated between sunrise and sunset, such as pipeline inspection, forest patrol, and road inspection. During this period, the daytime light changes slowly, which may cause problems such as changes in brightness, changes in light and shadow, reduced pixel count, and increased noise in the surveillance video.

[0036] In this embodiment, a drone is used as the monitoring target, and the drone is exemplarily a vertical take-off and landing (VTOL) drone. A preset direction is determined relative to the direction of the nest; in this application, the horizontal direction of the front of the nest is exemplarily used as the preset direction. A monitoring camera is installed opposite the front of the nest to periodically capture the original background image of the horizontal direction of the front of the nest. Alternatively, a monitoring camera can be used to capture monitoring video, and the video frames therein can be used as the original background image.

[0037] Based on images or videos captured by surveillance cameras in preset directions, multiple original background images without drones and multiple original monitoring images containing drones are obtained.

[0038] In this embodiment, the area directly above the drone nest is designated as the region of focus. For example, if the original background image was captured in a horizontal direction relative to the front of the drone nest, the length of the drone nest is used as the length of the region of focus, and the distance from the drone's hovering height to the top surface of the drone nest is used as the width of the region of focus, thus determining the size of the region of focus. If the original background image was captured in a horizontal direction relative to the left of the drone nest, the width of the drone nest is used as the length of the region of focus, and the distance from the drone's hovering height to the top surface of the drone nest is used as the width of the region of focus, thus determining the size of the region of focus.

[0039] In addition, those skilled in the art can determine the size of the key area of ​​interest based on the performance of the drone, the pixel size of the nest, the resolution of the monitoring video, etc., and this application does not impose any restrictions here.

[0040] The preset monitoring target can be a part or all of the target. When a portion of the target just enters the original background image, the frame difference of several consecutive original background images can be used to determine if a target has entered. This allows for timely termination of the original background image acquisition, preventing interference. Alternatively, when the original background image has a large field of view and the target is small, the acquisition can wait until a significant portion or all of the object has entered, then determine whether the target has entered the field of view based on the object's shape, thus terminating the acquisition of the original background image. This avoids false positives, but ultimately introduces more interference from the original background images.

[0041] Preset conditions can be set according to the specific characteristics of the monitored target. For example, when the monitored target enters the field of view within a preset time period, the judgment can be based on that time period. When the movement trajectory of the monitored target is specific, the judgment can be based on the movement trajectory displayed in several consecutive original background images. The shape of the monitored target can also be used for judgment. Multiple of the above judgment methods can be selected for comprehensive judgment.

[0042] Step 102: Determine the background image based on the original background image without the monitoring target. In this embodiment, the specific implementation method for determining the background image based on the original background image without the monitoring target is as follows: Figure 2 As shown, steps 201 to 206 are included, as detailed below.

[0043] Step 201: Use the original background images without the monitored target for a preset number of frames as the background image group. In this embodiment, the scene in the surveillance video (e.g., changes in lighting, movement of background objects, etc.) may change over time. Therefore, this application periodically updates the background image before the monitored target enters the key area of ​​interest to reflect the real situation of the current scene as much as possible, avoid errors in subsequent judgments, and improve the accuracy and reliability of the background image.

[0044] Specifically, multiple consecutive original background images in which the monitored target appears before the original background image are divided into multiple background image groups according to a preset frame rate. The monitoring camera used in this application has a frame rate of 33 frames per second, so the preset frame rate is set to 33 here.

[0045] Those skilled in the art should recognize that the closer the original background image without the monitored target in the last group of background images is to the original monitoring image containing the monitored target in terms of the time of its capture, the more accurately the background image reflects the true situation of the current scene. Therefore, the preset frame rate set here is not intended to limit the scope of protection of this application, and those skilled in the art can also determine the preset frame rate based on actual needs, monitoring camera performance, etc.

[0046] Step 202: Divide the region of interest in each original background image in the background image group into a preset number of image blocks. In this embodiment, since the region of interest has been determined based on location markers, each original background image contains a region of interest. The region of interest (i.e., the region image) in each original background image in the background image group is then divided into a preset number of image blocks. This application exemplarily employs... The original background image is divided into 9 image blocks based on the desired region of interest. Each background image group then receives the preset number of frames. Preset quantity (i.e., 33) 9 = 297) image blocks.

[0047] Those skilled in the art can also balance the effect with computational efficiency here, employing The division method, or setting the first data to 10, 11, etc., is not intended to limit the scope of protection of this application.

[0048] In addition, the original background image can be converted to grayscale before performing step 202.

[0049] Step 203: Traverse each image block and perform anomaly detection based on the grayscale values ​​of image blocks at the same position in other original background images.

[0050] Specifically, the grayscale value of each image block in the background image group is determined, and a preset number (i.e., 9) grayscale values ​​are obtained for each original background image. The mean and sample standard deviation of the grayscale values ​​of the image blocks corresponding to each position (a total of 9 positions) in the background image group are calculated respectively.

[0051] Furthermore, such as Figure 3 As shown, firstly, the grayscale value of each image block divided in step 202 is calculated, resulting in 297 grayscale values. Then, the image blocks are grouped according to their corresponding positions, i.e., the first image block in the key focus area of ​​each original background image is the first group, the second image block in the key focus area of ​​each original background image is the second group, and so on, dividing the 297 image blocks into 9 groups. The mean (denoted as grayscale mean) and standard deviation (denoted as grayscale deviation) of the grayscale values ​​of the 33 image blocks in each group are calculated, resulting in 9 grayscale means and 9 grayscale deviations.

[0052] This application exemplarily employs the Laida criterion to determine anomalies in the grayscale values ​​of each image block. Specifically, assuming the grayscale values ​​of each image block conform to a normal distribution, it determines whether the difference between the grayscale value and the mean grayscale value of each image block is less than or equal to three times the grayscale deviation. If the difference between the grayscale value and the mean grayscale value of an image block is less than or equal to three times the grayscale deviation, then the grayscale value distribution of that image block is normal. If the difference between the grayscale value and the mean grayscale value of an image block is greater than three times the grayscale deviation, then the grayscale value distribution of that image block is abnormal.

[0053] Those skilled in the art should recognize that the use of the Laida criterion to determine anomalies in the grayscale values ​​of each image block is merely one embodiment of this application and is not intended to limit the scope of protection of this application. Those skilled in the art can also, according to actual needs, determine anomalies in the grayscale values ​​of an image block by whether the difference between the grayscale value and the mean grayscale value is greater than 2 / 1.5 times the grayscale deviation. Alternatively, methods such as the Grubbs criterion or the Dixon criterion can be used to determine anomalies in the grayscale values ​​of image blocks.

[0054] Step 204: Determine whether there exists at least one original background image in the background image group where all image blocks are normal. In this embodiment, based on the abnormal judgment result of the grayscale values ​​of each image block in step 203, it is determined whether there exists at least one original background image in the background image group where all image blocks are normal.

[0055] Specifically, if the grayscale values ​​of at least one original background image in the background image group are normal across a predetermined number (i.e., 9) of the image blocks within its region of interest, then step 205 is executed. If at least one image block within the region of interest of each original background image in the background image group has an abnormal grayscale value, then step 206 is executed.

[0056] Step 205: Select the region image of the key focus area of ​​one of the original background images as the background representative image. In this embodiment, when the grayscale values ​​of at least one original background image divided into a preset number (i.e., 9) of image blocks are normal, an original background image can be directly selected from the background image group, and the region image of the key focus area of ​​the selected original background image can be used as the background representative image.

[0057] Step 206: Select the image block with the smallest error among the image blocks corresponding to each position and stitch them together to obtain the background representative image. In this embodiment, the image block with the smallest difference between the gray value and the mean gray value of the corresponding group among the 9 groups can be used as the stitched image block at that position, thereby determining the 9 stitched image blocks.

[0058] In this embodiment of the application, the gray values ​​of the 33 image blocks in each group can be compared with the gray mean value of the corresponding group, and the image blocks corresponding to the gray values ​​that do not meet the Laida criterion in each group can be eliminated according to the Laida criterion, until the stitched image block at the position corresponding to the image block in each group is determined.

[0059] In the embodiments of this application, the method of removing image blocks with abnormal gray values ​​and then calculating the average gray value of the remaining image blocks for the next round of comparison can also be used to determine the stitched image block at the corresponding position of a certain image block.

[0060] Specifically, taking the first group as an example, if the difference between the grayscale value of the first image block in the first group and the average grayscale value of the remaining 32 image blocks in the first group is greater than 3 times the grayscale deviation, then the grayscale value of that image block is determined to be abnormal, and the image block is removed from the first group. Then, the difference between the grayscale value of the second image block and the average grayscale value of the remaining 31 image blocks is calculated to see if it is greater than 3 times the grayscale deviation. This process continues until an image block's grayscale value is found to be less than or equal to 3 times the average grayscale deviation of the remaining image blocks. In this case, the image block is used as the stitched image block at the corresponding position of the image blocks in the first group. Furthermore, the iteration stops when a stitched image block has not been determined after continuously removing a second number of image blocks. This indicates that the noise in the 33 image blocks in the first group is relatively high. For example, the second number is set to 2.

[0061] In this embodiment, the nine stitched image blocks are stitched together to obtain a representative background image. This application can also use a blurring algorithm to reduce block artifacts at the stitching points. The obtained representative background image can be directly used as the background image.

[0062] It should be noted that this application can also use the area image of the key focus region from the original background image that does not contain the monitored target as the background image directly, or the background image can be obtained using the methods in steps 201 to 206. Using the methods in steps 201 to 206 can improve anti-interference capability and eliminate interference caused by other flying objects, falling objects, and other objects entering the field of view.

[0063] Furthermore, multiple representative background images can be determined based on multiple background image groups. At least a portion of the original background images from one set of background image groups are combined with multiple representative background images to form a new set of background images. The representative background images in the new set of background image groups are then determined using the process of acquiring the representative background images, and these representative images are used as the background images.

[0064] Specifically, this approach aims to capture the background environment before the monitored target enters the field of view as closely as possible to the background environment in the monitored image. Furthermore, by combining several representative background images from the previous period with the latest original background images to form a new set of background images for determining the final representative background image, this approach avoids interference from other objects suddenly entering the field of view at the last moment.

[0065] The original background images mentioned here can be all or part of a selected set of background images. When selecting part of the original background images from the set, original background images without any monitoring targets are preferred to avoid interference from the monitoring targets. When selecting all of the original background images from the set, this application can identify image blocks in the original background images containing the monitoring targets as having grayscale anomalies based on multiple previous representative background images, and thus exclude them. For example, the background image set here is preferably the last set of background images close to the original monitoring images containing the monitoring targets. The multiple original background images selected from the background image set can be continuous or discontinuous. A new background image set is formed based on the selected original background images and the multiple previously determined background representative images.

[0066] In this embodiment, after the UAV receives the nest opening limit signal, the first frame of the monitoring video can be used as the background representative image to determine the background image. Alternatively, a frame from the monitoring video during the UAV's flight from its current position to a hovering height above the nest can be used as the background representative image to determine the background image.

[0067] Step 103: Determine the monitoring image based on the original monitoring image containing the monitoring target. In this embodiment, "containing the monitoring target" should be understood as meaning that the monitoring target can be identified from a portion or all of the monitoring target appearing in the original monitoring image. If a portion of the monitoring target appears in the original monitoring image, but the portion is insufficient to identify it as the monitoring target, it should be understood that the image does not contain the monitoring target. The region image of the key area of ​​interest in one of the multiple original monitoring images containing the monitoring target is taken as the monitoring image.

[0068] Specifically, a subtraction operation can be performed on two consecutive original monitoring images to determine whether the drone has entered the original monitoring image.

[0069] Those skilled in the art should realize that other target recognition methods can also be used to determine whether a monitoring target has entered the current original monitoring image, and this application does not impose any restrictions on this.

[0070] Step 104: Filter the regions of interest in the background image and the monitoring image along the first direction to obtain the first background image and the first monitoring image. In this embodiment, the first direction includes the horizontal or vertical direction of the region image. When the first direction is horizontal, the filtering window size is 1. i. Where i represents the length of the filtering window, i is a natural number less than n, and n is the number of horizontal pixels in the region of the image to be filtered. When the first direction is the vertical direction, the size of the filtering window is j. 1. Where j represents the width of the filtering window, j is a natural number less than m, and m is the number of vertical pixels in the region of the image to be filtered.

[0071] Specifically, the background image and the monitoring image are filtered in the first direction (mean filtering is used here).

[0072] In existing technologies, after moving target detection algorithms (such as frame difference algorithms, background difference algorithms, etc.), smoothing filters (such as mean filtering) with window sizes of 3×3, 5×5, etc., are used to smooth the image, making the visual effect of the algorithm result better. Figure 5 The image shown is an example of mean filtering with a 3×3 filtering window. However, this application focuses on the integrity of a specific positioning direction rather than the visual effect when calculating the physical relative position. Therefore, this application can use a resolution of... The array of two-dimensional images (i.e., the two-dimensional array corresponding to the original background image or the original monitoring image) is regarded as m one-dimensional arrays of 1 row and n columns. Then, mean filtering is performed on these m one-dimensional arrays to preserve image information in the row direction as much as possible and weaken the coupling relationship between rows. Here, m represents the number of vertical pixels in the image of the region to be filtered, and n represents the number of horizontal pixels in the image of the region to be filtered.

[0073] In the process of locating dynamic targets, this application focuses not on the visual effect, but on the integrity of the specific positioning direction. This application uses the mean filter of a one-dimensional array along the first direction to filter the background image and the monitoring image respectively. This can reduce the high-frequency components (such as edges, noise, and details) of the background image and the monitoring image, reduce the sudden changes in gray values, weaken the coupling relationship between different dimensions, preserve the image information of the corresponding dimensions as much as possible, and also help improve the processing efficiency.

[0074] In practical use, the background image and the monitoring image are filtered using a one-dimensional array along the row / column direction (mean filtering is used as an example) to obtain the first background image and the first monitoring image, such as... Figure 6As shown. The filtering process for a one-dimensional array ignores the coupling between rows / columns, preserving more complete wing information. The horizontal integrity of the background / monitoring image determines the reliability of the physical relative position calculation results. For example, the filtering window size of the mean filter here is 1. i or j 1, i represents the length of the filtering window, i is a natural number less than n, n is the number of horizontal pixels in the region image to be filtered, j represents the width of the filtering window, j is a natural number less than m, m is the number of vertical pixels in the region image to be filtered.

[0075] Furthermore, in performing filtering, this application can first convert the background image / monitoring image into grayscale form. Then, a one-dimensional array filtering process is used to perform row / column mean filtering on the background image / monitoring image to obtain the first background image / first monitoring image.

[0076] Step 105: Determine the difference image between the first background image and the first monitoring image. In this embodiment, the obtained first background image and first monitoring image are differentially binarized to obtain the difference image.

[0077] Since this application uses one-dimensional array filtering, the morphological filtering process can be omitted after the first background image and the first monitoring image are differentially binarized. That is, there are no dilation, erosion, opening and closing operations. The default operation not only improves the computational efficiency, but also saves hardware costs.

[0078] Step 106: Determine the physical relative position of the monitoring target and the location marker in a preset direction based on the differential image. In this embodiment, the relative position of the monitoring target and the location marker in the differential image is obtained through the differential image. Based on the relative position and the physical position information and / or physical size information of the location marker, the physical relative position of the monitoring target and the location marker in a preset direction is determined.

[0079] In another specific embodiment of this application, multiple closed region contours are found in the differential image through contour lookup. The region contour with the most internal pixel values ​​is selected, and its minimum bounding rectangle is determined. The position information of the monitoring target is determined based on the position information of the minimum bounding rectangle. Based on the position information of the monitoring target, the physical relative position of the monitoring target and the position calibration object in a preset direction is determined.

[0080] Specifically, contour searching is performed on the differential image to find multiple closed region contours. The contour with the most pixel values ​​among the searched region contours is identified, i.e., the contour with the largest area. The minimum bounding rectangle of the contour with the largest area is determined. The horizontal length of the minimum bounding rectangle represents the movement range of the monitored target in the preset direction from the monitoring viewpoint. The position information of the minimum bounding rectangle is the position information of the monitored target.

[0081] It should be noted that the position information obtained here is the position information of the monitored target in the preset direction (the horizontal direction in front), which can be used to determine the relative positional relationship between the UAV and the position marker in the preset direction.

[0082] In this embodiment of the application, the physical relative position of the UAV and the nest in a preset direction can be determined based on the differential image.

[0083] Specifically, determine the pixel size of the drone nest from the camera's perspective. Determine a centerline passing through the center of the nest and perpendicular to the corresponding preset direction. Based on the drone's position information and the ratio of the actual size of the nest to the pixel size, determine one or more perpendicular distances between the drone and the centerline of the nest.

[0084] Furthermore, given the actual length and width of the parking platform of the nest, its pixel size from the camera's perspective can be calculated using the parameters of the monitoring camera, which will not be elaborated here.

[0085] In this step, such as Figure 4 As shown, the x-axis points along the length of the nest, the y-axis points along the width of the nest, A, B, C, and D represent the four corner points of the nest's landing platform, L represents the actual length of the nest's landing platform, i.e., the actual length of side AB or side CD, O represents the center of the nest, O1 represents the center of side AB, O2 represents the center of side CD, p1 represents the intersection of the bisector of the UAV's smallest bounding rectangle with side AB, and p2 represents the intersection of the bisector of the UAV's smallest bounding rectangle with side CD.

[0086] The physical relative position of the drone and its nest is calculated as follows:

[0087] d1=LX1 / P AB d2=LX2 / P CD In the formula, d1 represents the vertical distance at edge AB, i.e., the actual distance of O1P1; d2 represents the vertical distance at edge CD, i.e., the actual distance of O2P2; L represents the actual length of the docking platform of the nest, i.e., the actual length of edges AB and CD; X1 represents the pixel distance of p1 from the center line O1O2 of the nest in the current frame; X2 represents the pixel distance of p2 from the center line O1O2 of the nest in the current frame; P ABP represents the pixel length of side AB, that is, the pixel distance between the two corner points A and B. CD This represents the pixel length of the CD edge, that is, the pixel distance between the two corner points C and D.

[0088] On the horizontal direction (x-direction) of the drone's front side, the distance from the drone to the center of the drone's nest is between d1 and d2. To ensure the safety of the drone and reduce the false alarm rate, the current relative coordinates of the drone in the horizontal direction are dx = max(d1, d2). If the centerline of the nest is parallel to the centerline of the drone's smallest bounding rectangle, then d1 = d2.

[0089] In this embodiment, the monitoring camera typically lacks depth information. Therefore, to determine the physical relative position of the UAV during landing, a second preset direction forming a preset angle with a preset direction can also be determined. Based on the original background image without the monitoring target and the original monitoring image containing the monitoring target acquired along the second preset direction, a difference step is performed to obtain a second difference image. According to the second difference image, the physical relative position of the monitoring target and the location marker is determined in the second preset direction.

[0090] Specifically, the preset direction or second preset direction refers to the orientation of the monitoring camera. Two monitoring cameras positioned at a preset angle are deployed to capture and calculate the physical relative positions of the drone in different directions.

[0091] Furthermore, a second preset direction is determined that forms a preset angle with the preset direction. Preferably, the preset angle is 90°, meaning the preset direction is perpendicular to the second preset direction. If the preset direction is the horizontal direction of the front of the nest, then the second preset direction can be the horizontal direction of the left / right side of the nest.

[0092] Those skilled in the art should realize that the preset angle between the preset direction and the second preset direction is not necessarily a right angle; it is simply the simplest to calculate when it is a right angle. The preset angle can also be 45°, 30°, etc., and the physical relative distance between the UAV and the nest in the second preset direction (where the preset angle is not a right angle) or in a direction perpendicular to the preset direction can still be calculated using trigonometric functions.

[0093] Substituting the second preset direction into steps 101 to 106, the original background image without the monitoring target and the original monitoring image with the monitoring target are acquired in the second preset direction. Then, a difference step is performed to obtain the second difference image. Finally, based on step 106, the physical relative position of the UAV in the second preset direction is obtained.

[0094] Based on the physical relative positions of the UAV and the nest in two different directions, the two-dimensional coordinates of the plane parallel to the UAV's landing platform relative to the nest can be obtained.

[0095] In this embodiment, after completing its mission, the UAV returns to its home base based on positioning data. Upon reaching the landing altitude, it hovers in preparation for landing. The positioning data is used to determine whether the UAV has reached the hovering altitude. If the UAV has not reached the hovering altitude, steps 101 and 102 are executed until the UAV reaches the hovering altitude. If the UAV reaches the hovering altitude (i.e., enters the original monitoring image), steps 103 to 106 are executed to obtain the physical relative position of the UAV and the nest in a preset direction (or a second preset direction).

[0096] When the physical relative position of the drone and the nest meets preset requirements, the drone lands in the nest. Otherwise, the drone lands at an alternate landing point. Specifically, a centerline is determined that passes through the center of the nest and is perpendicular to the corresponding preset direction (or a second preset direction). One or more vertical distances between the drone and the centerline are determined. When all vertical distances are within preset threshold ranges, the drone lands in the nest. Otherwise, the drone lands at an alternate landing point.

[0097] The preset threshold range here is [-0.3, 0.3] meters. If the physical relative position of the drone and the nest meets the preset threshold range, the drone will land inside the nest. If the physical relative position of the drone and the nest does not meet the preset threshold range, the drone will land at an alternate landing point.

[0098] Furthermore, if the physical relative position between the drone and the nest does not meet the preset threshold range, further judgment can be made. Specifically, based on the next original monitoring image after the current original monitoring image, steps 103 to 106 are executed to redetermine the physical relative distance between the drone and the nest in a preset direction (or a second preset direction), and to determine whether it meets the preset threshold range. If the first threshold physical relative distances obtained based on the first threshold of subsequent original monitoring images do not meet the preset threshold range, the drone will abort its flight and land at the alternate landing point. For example, the first threshold is set to 15.

[0099] In this embodiment, the physical relative distance between the UAV's current position and the center of the nest during landing is the basis for determining whether to land on the nest recovery platform or at the alternate landing point. Therefore, the computational speed of the moving target detection method during landing is required to be high, and its computational load should not be too large. Furthermore, the speed of the vertical takeoff and landing UAV decreases during landing, then decreases again until hovering, and then it continues landing or re-flyes to the alternate landing point based on the calculated physical relative distance. This requires the moving target detection method to adapt to the speed changes of the UAV. Traditional moving target detection methods are generally divided into moving target detection methods based on pixel changes and moving target detection methods based on background modeling. Moving target detection methods based on pixel changes find the change region based on the pixel differences between video frames, mainly including frame difference methods and optical flow methods. However, frame difference methods cannot adapt to stationary or very slow-moving targets, and optical flow methods are sensitive to image noise and have high computational complexity. Background modeling-based moving target detection methods are commonly used for detecting moving targets against static backgrounds. These methods establish a background model of the corresponding scene based on the first N frames of the video and obtain the moving target by comparison. The main methods include Gaussian mixture model and ViBe (full name: Visual Background Extractor, a video background extraction algorithm based on pixel-level modeling). Among them, the Gaussian mixture model has certain limitations in its model assumptions and has a large computational load, while the ViBe algorithm is prone to noise and ghosting.

[0100] Therefore, existing moving target detection methods are insufficient to meet the real-time and accuracy requirements of UAV recovery. This application utilizes multiple background image groups to periodically update the background image, adapting to dynamic scene changes and enabling more accurate determination of the physical relative distance between the UAV and its nest. The computational complexity of the physical relative distance calculation in this application is low, meeting the real-time requirements of UAV recovery. This application employs a one-dimensional array filtering process, ignoring the visual effects of the original monitoring image, thus preserving as much directional information as possible and adapting to changes in UAV speed.

[0101] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0102] like Figure 7As shown in the figure, this application embodiment also provides an image-based positioning device 700. The device includes an execution module 701 and a positioning module 706, wherein the execution module 701 includes a first determination module 702, a second determination module 703, a filtering module 704 and a difference module 705, as detailed below.

[0103] The execution module 701 is used to perform a difference step based on the original background image without the monitoring target acquired along a preset direction and the original monitoring image with the monitoring target to obtain a difference image.

[0104] The first determining module 702 is used to determine the background image based on the original background image that does not contain the monitoring target.

[0105] The second determining module 703 is used to determine a monitoring image based on an original monitoring image containing the monitoring target. The monitoring image and the background image contain the same region of interest, and the region of interest contains location markers.

[0106] The filtering module 704 is used to perform filtering processing along the first direction on the region images of the key areas of interest in the background image and the monitoring image respectively, to obtain the first background image and the first monitoring image.

[0107] The differential module 705 is used to determine the differential image between the first background image and the first monitoring image.

[0108] The positioning module 706 is used to determine the physical relative position of the monitoring target and the location marker in a preset direction based on the differential image.

[0109] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0110] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0111] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0112] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0113] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0114] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0115] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0116] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0117] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. An image-based positioning method, characterized by The method comprises the following steps: performing a difference step based on the original background image without the monitoring target and the original monitoring image with the monitoring target collected along the preset direction to obtain a difference image; wherein the difference step comprises: determining a background image based on the original background image without the monitoring target; determining a monitoring image based on the original monitoring image with the monitoring target; wherein the monitoring image and the background image contain the same key focus area, and the key focus area contains a position calibration object; performing one-dimensional array filtering on a region image of a region of interest in the background image and the monitoring image respectively along a first direction to obtain a first background image and a first monitoring image; wherein the first direction comprises a horizontal direction or a vertical direction of the region image; when the first direction is the horizontal direction, a filter window size is 1 i; wherein i represents a length of the filter window, i is a natural number less than n, n is a horizontal pixel number of the region image to be filtered; when the first direction is the vertical direction, a filter window size is j 1; wherein j represents a width of the filter window, j is a natural number less than m, m is a vertical pixel number of the region image to be filtered. determining a difference image of the first background image and the first monitoring image; determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the difference image.

2. The method of claim 1, wherein, Before performing the difference step based on the original background image without the monitoring target and the original monitoring image with the monitoring target collected along the preset direction, the method comprises the following steps: collecting a plurality of original background images of a preset area along the preset direction; wherein the preset area contains a key focus area; stopping the collection of the original background image when the monitoring target satisfying the preset condition appears in the original background image.

3. The method of claim 1, wherein, The method for determining the background image based on the original background image without the monitoring target comprises the following steps: taking a preset number of original background images without the monitoring target as a background image group; obtaining a background representative image of the key focus area based on the background image group; determining the background image based on the background representative image; wherein the process of obtaining the background representative image comprises the following steps: dividing the area image of the key focus area in each original background image in the background image group into a preset number of image blocks; iterating each image block to perform abnormality judgment based on the gray value of the image block at the same position in other original background images; if each image block corresponding to at least one original background image in the background image group is normal, selecting the area image of the key focus area of one original background image as the background representative image; otherwise, selecting the image block with the smallest error in each position to splice to obtain the background representative image.

4. The method of claim 3, wherein, The method for determining the background image based on the original background image without the monitoring target further comprises the following steps: determining a plurality of background representative images based on a plurality of background image groups; forming a new background image group by combining at least part of the original background images in a background image group with the plurality of background representative images; determining the background representative image in the new background image group as the background image by using the process of obtaining the background representative image.

5. The method of claim 1, wherein, The method for determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the difference image comprises the following steps: finding a plurality of closed area contours in the difference image through contour finding; selecting the area contour with the largest number of internal pixel values and determining the minimum circumscribed rectangle of the area contour; determining the position information of the monitoring target according to the position information of the minimum circumscribed rectangle; determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the position information of the monitoring target.

6. The method of claim 1, wherein, The method for determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the difference image further comprises the following steps: determining a second preset direction with a preset included angle with the preset direction; performing the difference step based on the original background image without the monitoring target and the original monitoring image with the monitoring target collected along the second preset direction to obtain a second difference image; determining the physical relative position of the monitoring target and the location calibration object in the second preset direction according to the second difference image.

7. A method for recovering a drone, characterized by, comprising: setting initial conditions, including: taking a drone as a monitoring target, taking a nest as a location calibration object, determining a preset direction relative to the direction of the nest, and determining a key focus area based on the location of the nest; performing the method of any one of claims 1-6 based on the initial conditions to obtain the physical relative position of the drone and the nest; when the physical relative position of the drone and the nest meets the preset requirements, the drone lands in the nest; otherwise, the drone lands at a backup landing point.

8. The method of claim 7, wherein, the when the physical relative position of the drone and the nest meets the preset requirements, the drone lands in the nest, comprising: determining a center line passing through the center of the nest and perpendicular to the corresponding preset direction; determining one or more vertical distances of the drone from the center line; when the vertical distances are all within a preset threshold range, the drone lands in the nest.

9. An image-based positioning apparatus, characterized by comprising: an execution module for performing a difference step based on an original background image without a monitoring target and an original monitoring image with the monitoring target collected along a preset direction to obtain a difference image; wherein the difference step comprises: a first determination module for determining a background image based on the original background image without the monitoring target; a second determination module for determining a monitoring image based on the original monitoring image with the monitoring target; wherein the monitoring image and the background image contain the same key focus area, and the key focus area contains a location calibration object; The filtering module is configured to perform one-dimensional array filtering along a first direction on a region image of a region of interest in the background image and the monitoring image respectively to obtain a first background image and a first monitoring image; the first direction includes a horizontal direction or a vertical direction of the region image; when the first direction is the horizontal direction, a filtering window size is 1 i; wherein i represents a length of the filtering window, i is a natural number less than n, n is a horizontal pixel number of the region image to be filtered; when the first direction is the vertical direction, a filtering window size is j 1; wherein j represents a width of the filtering window, j is a natural number less than m, m is a vertical pixel number of the region image to be filtered. a difference module for determining the difference image of the first background image and the first monitoring image; a positioning module for determining the physical relative position of the monitoring target and the location calibration object in the preset direction according to the difference image.

Images