Image-based positioning method, unmanned aerial vehicle recycling method and device

Abstract

The invention discloses an image-based positioning method and device, and an unmanned aerial vehicle recovery method and device, and relates to the technical field of image processing, and the positioning method comprises the steps: determining a background image based on an original background image which does not contain a monitoring target; determining a monitoring image based on the original monitoring image containing the monitoring target; wherein the monitoring image and the background image comprise the same focus area, and the focus area comprises a position calibration object; respectively carrying out filtering processing on the area image of the important attention area in the background image and the monitoring image along a first direction to obtain a first background image and a first monitoring image, and determining a difference image of the first background image and the first monitoring image; and according to the difference image, determining the physical relative position of the monitoring target and the position calibration object in the preset direction. The problem that an existing moving target monitoring method cannot quickly position a non-uniform moving target is solved, the non-uniform moving target can be quickly positioned, and then the timeliness requirement in the unmanned aerial vehicle recycling process can be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image-based positioning method, a UAV recovery method and device. BACKGROUND

[0002] With the rapid development of edge computing technology and image processing technology, the application range of image-based positioning technology in the field of artificial intelligence, unmanned driving and unmanned aerial vehicles continues to expand. Common algorithms for monitoring moving targets include inter-frame difference method, Vibe algorithm and Gaussian mixture model. However, when the moving target stops, the inter-frame difference method, Vibe algorithm and the like cannot be directly used. When the internal pixels of the moving target change little or the target moves too fast, the frame difference method is prone to the phenomenon of monitoring the target internal cavity or being segmented into multiple blocks. The ViBe algorithm is relatively complete for monitoring the moving target, but is prone to noise, ghosting and other phenomena; the Gaussian mixture model method needs to set appropriate parameters according to the pause time, but in some special cases, the pause time of the moving target is relatively random.

[0003] Therefore, for moving targets with irregular speed and stop, how to quickly locate becomes a difficult problem to be solved. SUMMARY

[0004] The embodiments of the present application provide an image-based positioning method, a UAV recovery method and device, which solve the problem that the existing moving target monitoring method cannot quickly locate non-uniform speed moving targets.

[0005] In a first aspect, the embodiments of the present application provide an image-based positioning method, comprising: performing a difference step based on an original background image without a monitoring target and an original monitoring image with a monitoring target collected along a preset direction to obtain a difference image; wherein the difference step comprises: determining a background image based on the original background image without the monitoring target; determining a monitoring image based on the original monitoring image with the monitoring target; wherein the monitoring image and the background image contain the same key focus area, and the key focus area contains a position marker; performing filtering processing along a first direction on the region images of the key focus area in the background image and the monitoring image respectively to obtain a first background image and a first monitoring image; determining a difference image of the first background image and the first monitoring image; and determining the physical relative position of the monitoring target and the position marker in the preset direction according to the difference image.

[0006] With reference to the first aspect, in a possible implementation manner, before the step of performing difference based on the original background image without the monitoring target and the original monitoring image with the monitoring target, the method further includes: collecting a plurality of original background images of a preset area along a preset direction; wherein the preset area comprises a key area of interest; and when the monitoring target satisfying a preset condition appears in the original background image, the collection of the original background image is stopped.

[0007] With reference to the first aspect, in a possible implementation manner, the step of determining the background image based on the original background image without the monitoring target comprises: taking a preset number of original background images without the monitoring target as a background image group; obtaining a background representative image of the key area of interest based on the background image group; and determining the background image based on the background representative image; wherein the obtaining process of the background representative image comprises: dividing a region image of the key area of interest in each original background image in the background image group into a preset number of image blocks; traversing each image block, and performing abnormality judgment based on the gray value of the image block at the same position in other original background images; if each image block corresponding to at least one original background image in the background image group is normal, selecting a region image of the key area of interest in one original background image as the background representative image; otherwise, selecting the image block with the smallest error in each position to splice to obtain the background representative image.

[0008] With reference to the first aspect, in a possible implementation manner, the step of determining the background image based on the original background image without the monitoring target further comprises: determining a plurality of background representative images based on a plurality of background image groups; taking at least part of the original background images in a background image group and the plurality of background representative images as a new background image group; and determining the background representative image in the new background image group as the background image by using the obtaining process of the background representative image.

[0009] With reference to the first aspect, in a possible implementation manner, the first direction comprises a horizontal direction or a vertical direction of the region image; when the first direction is the horizontal direction, the size of the filter window is 1 i; wherein i represents the length of the filter window, i is a natural number less than n, and n is the number of horizontal pixels of the region image to be filtered; and when the first direction is the vertical direction, the size of the filter window is j 1; wherein j represents the width of the filter window, j is a natural number less than m, and m is the number of vertical pixels of the region image to be filtered.

[0010] With reference to the first aspect, in a possible implementation manner, the determining, according to the differential image, the physical relative position of the monitoring target and the position calibration object in the preset direction, comprises: finding a plurality of closed region contours in the differential image through contour searching; selecting a region contour with the largest number of internal pixel values, and determining a minimum circumscribed rectangle of the region contour; determining position information of the monitoring target according to position information of the minimum circumscribed rectangle; and determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the position information of the monitoring target.

[0011] With reference to the first aspect, in a possible implementation manner, the determining, according to the differential image, the physical relative position of the monitoring target and the position calibration object in the preset direction, further comprises: determining a second preset direction that forms a preset included angle with the preset direction; performing the difference step based on an original background image without the monitoring target and an original monitoring image with the monitoring target collected along the second preset direction to obtain a second differential image; and determining the physical relative position of the monitoring target and the position calibration object in the second preset direction according to the second differential image.

[0012] The second aspect provides a method for recovering a UAV, comprising: setting initial conditions, including: taking the UAV as a monitoring target, taking a nest as a position calibration object, determining a preset direction relative to the nest, and determining a key attention area based on the position of the nest; performing the method in the first aspect or any possible implementation manner of the first aspect based on the initial conditions to obtain the physical relative position of the UAV and the nest; when the physical relative position of the UAV and the nest meets preset requirements, landing the UAV into the nest; otherwise, landing the UAV into a backup landing point.

[0013] With reference to the second aspect, in a possible implementation manner, the landing the UAV into the nest when the physical relative position of the UAV and the nest meets preset requirements comprises: determining a center line that passes through the center of the nest and is perpendicular to the corresponding preset direction; determining one or more vertical distances between the UAV and the center line; and landing the UAV into the nest when the vertical distances are all within a preset threshold range.

[0014] In a third aspect, an image-based positioning apparatus is provided, comprising: an execution module configured to perform a difference step based on an original background image without a monitoring target and an original monitoring image with the monitoring target collected along a preset direction to obtain a difference image; wherein the difference step comprises: a first determination module configured to determine a background image based on the original background image without the monitoring target; a second determination module configured to determine a monitoring image based on the original monitoring image with the monitoring target; wherein the monitoring image and the background image contain the same key attention region, and the key attention region contains a position calibration object; a filtering module configured to perform filtering processing along a first direction on regional images of the key attention region in the background image and the monitoring image respectively to obtain a first background image and a first monitoring image; a difference module configured to determine a difference image of the first background image and the first monitoring image; and a positioning module configured to determine a physical relative position of the monitoring target and the position calibration object in the preset direction according to the difference image.

[0015] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: By performing the difference step, the difference image can be obtained, and the physical relative distance of the monitoring target and the position calibration object can be quickly determined. By performing the filtering processing along the first direction on the regional images, the high-frequency components of the regional images can be reduced, the visual effects of the original monitoring image can be ignored, and as much directional information as possible can be retained, thereby being able to adapt to the speed change of the unmanned aerial vehicle. The problem that the existing moving target monitoring method cannot quickly position the non-uniform speed moving target is effectively solved, the quick positioning of the moving target with irregular speed and pause can be realized, thereby being able to be used for assisting the unmanned aerial vehicle recovery, the success rate and safety of the unmanned aerial vehicle recovery can be improved, and the calculation amount is small, thereby being able to meet the timeliness requirement in the unmanned aerial vehicle recovery process. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0017] Figure 1 A flowchart of the image-based positioning method provided by the embodiments of the present application is provided. Figure 2 A flowchart of the method for determining a background representative image provided by the embodiments of the present application is provided. Figure 3 A flowchart of the method for determining a background representative image provided by the embodiments of the present application is provided. Figure 4 An example diagram of a nest provided by an embodiment of the present application; Figure 5 An example diagram of mean filtering of a 3x3 filter window size provided by an embodiment of the present application; Figure 6 An example diagram of filtering processing along a first direction provided by an embodiment of the present application; Figure 7 A structural schematic diagram of an image-based positioning device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0019] The following descriptions of some technologies related to the embodiments of the present application are made to help understanding, which should be considered only as exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of clarity and conciseness, the descriptions of some well-known functions and structures are omitted in the following descriptions.

[0020] Figure 1 is a flowchart of an image-based positioning method provided by an embodiment of the present application, including steps 101 to 106. Among them, Figure 1 Only one execution order is shown for the embodiments of the present application, which does not represent the only execution order of the image-based positioning method, and Figure 1 The steps shown can be executed in parallel or in reverse.

[0021] It should be noted that the positioning referred to by the present application refers to the determination of the physical relative position of one or more direction dimensions of the monitoring target and the position marker.

[0022] Step 101: Collecting original background images without monitoring targets and original monitoring images with monitoring targets along a preset direction. In the embodiments of the present application, collecting original background images without monitoring targets along a preset direction includes: collecting multiple original background images of a preset region along a preset direction. Among them, the preset region contains a key area of interest. When a monitoring target that meets the preset condition appears in the original background image, the collection of the original background image is stopped.

[0023] Specifically, the monitoring target in the present application is a dynamic object set by human (i.e. a moving target that needs to be positioned or monitored), and the area of focus is determined according to the position of a position marker in a preset area, which is usually a static object, and a certain direction relative to the position marker is taken as the preset direction. For example, the horizontal direction relative to the front of the position marker is taken as the preset direction. That is, a monitoring camera is arranged in the horizontal direction opposite to the front of the position marker, which is used to shoot the original background image / original monitoring image or monitoring video of the horizontal direction of the front of the position marker, and the video frame in the monitoring video is taken as the original background image / original monitoring image.

[0024] It should be noted that the original background image and the original monitoring image are both collected by the monitoring camera along the preset direction, and the difference lies in that there is no monitoring target in the original background image, or there is a part of the monitoring target but it is not enough to identify the monitoring target, and there is a complete monitoring target in the original monitoring image, or there is a part of the monitoring target but it is enough to identify the monitoring target.

[0025] In an embodiment of the present application, the application range of the unmanned aerial vehicle in the dangerous operation scene continues to expand, and the difficulty of recycling the unmanned aerial vehicle also increases. The safe recycling process of the unmanned aerial vehicle is roughly divided into three stages: the first stage, the unmanned aerial vehicle quickly enters the monitoring field of view (the deployment area of the nest is usually provided with a monitoring camera), and this process lasts for 2-3 seconds; the second stage, the unmanned aerial vehicle reaches the specified hovering height, and this stage lasts for 1-3 seconds; the third stage, the unmanned aerial vehicle lands on the parking platform of the nest. Usually, the unmanned aerial vehicle mainly relies on the assistance of the positioning system to realize positioning during the task process and the recycling process, but there are limitations in specific regional environments (such as forests without base station coverage, steep valley canyons, deep tunnels, etc.), and the positioning signal is weak or even disappears, which leads to the situation that the unmanned aerial vehicle cannot be positioned, resulting in the failure of recycling the unmanned aerial vehicle.

[0026] The deployment area of the existing nest is usually provided with a monitoring camera, and the present application can realize real-time calculation of the landing trajectory of the unmanned aerial vehicle relying on the existing monitoring camera without adding new hardware devices and ensuring the safety of take-off and landing, which can timely discover the danger in the landing process of the unmanned aerial vehicle and assist the safe recycling of the unmanned aerial vehicle.

[0027] Specifically, when the unmanned aerial vehicle can normally receive the positioning signal and the positioning data is not deviated, the unmanned aerial vehicle can use the positioning data to land on the recycling platform of the nest. However, when the positioning signal is weak or the positioning data is deviated, it will lead to the failure of recycling the unmanned aerial vehicle.

[0028] Therefore, the application utilizes the monitoring video captured by the monitoring camera arranged near the nest to assist the recovery of the unmanned aerial vehicle without additional hardware devices. The deployment area of the nest is mainly open, and the background scene of the nest varies depending on the actual situation. When there are trees, flowers, and other vegetation that sway in the wind in the monitoring field of view, the background of the monitoring video will change dynamically. Moreover, the task time of the unmanned aerial vehicle is mostly concentrated in the sunrise to sunset period, such as pipeline inspection, forest inspection, road inspection, etc., during which the daytime illumination changes slowly, which may cause the monitoring video to have problems such as brightness change, light and shadow change, pixel reduction, and increased noise.

[0029] In the embodiments of the application, the unmanned aerial vehicle is taken as the monitoring target, and the unmanned aerial vehicle exemplarily adopts a vertical take-off and landing unmanned aerial vehicle. The preset direction is determined relative to the direction of the nest, and the application exemplarily takes the horizontal direction of the front of the nest as the preset direction. The monitoring camera installed opposite the front of the nest periodically captures the original background image of the horizontal direction of the front of the nest. Here, the monitoring video captured by the monitoring camera can also be used, and the video frame in the monitoring video is taken as the original background image.

[0030] Based on the images or videos captured by the monitoring camera in the preset direction, a plurality of original background images without the unmanned aerial vehicle and a plurality of original monitoring images with the unmanned aerial vehicle are obtained.

[0031] In the embodiments of the application, the area directly above the nest is the key attention area. Exemplarily, if the preset direction of the original background image captured here is the horizontal direction of the front of the nest, the length of the nest is taken as the length of the key attention area, and the distance from the hovering height of the unmanned aerial vehicle to the upper surface of the nest is taken as the width of the key attention area to determine the size of the key attention area. If the preset direction of the original background image captured here is the horizontal direction of the left side of the nest, the width of the nest is taken as the length of the key attention area, and the distance from the hovering height of the unmanned aerial vehicle to the upper surface of the nest is taken as the width of the key attention area to determine the size of the key attention area.

[0032] In addition, those skilled in the art can also determine the size of the key attention area according to the performance of the unmanned aerial vehicle, the pixel size of the nest, the resolution of the monitoring video, etc., which is not limited herein.

[0033] The preset monitoring target can be part or all of the monitoring target. When the monitoring target just partially enters the original background image, the frame difference between several consecutive original background images can be used to determine whether the monitoring target has entered. This allows the acquisition of the original background image to be terminated in a timely manner to avoid interference with the background image. If the original background image has a large field of view and the monitoring target is small, the system can wait until more or all objects have entered the field of view, then determine whether the monitoring target has entered the field of view based on the object's shape, thereby terminating the acquisition of the original background image. This can avoid misjudgments, but the result will be that more original background images will introduce interference.

[0034] Preset conditions can be set based on the specific characteristics of the monitored target. For example, if the monitored target enters the field of view within a preset time period, the detection can be based on that time period. If the monitored target's motion trajectory is specific, the detection can be based on the trajectory shown in several consecutive original background images. The detection can also be based on the shape of the monitored target. Multiple of the above detection methods can be used for comprehensive judgment.

[0035] Step 102: Determine the background image based on the original background image without the monitoring target. In the embodiment of the present application, the specific implementation method of determining the background image based on the original background image without the monitoring target is as follows: Figure 2 As shown, it includes steps 201 to 206, which are specifically as follows.

[0036] Step 201: A preset number of frames of original background images excluding the monitored target are used as a background image group. In this embodiment of the present application, the scene in the surveillance video may change over time (e.g., due to changes in lighting, movement of background objects, etc.). Therefore, this application periodically updates the background image before the monitored target enters the key focus area to best reflect the current scene, avoid errors in subsequent judgments, and improve the accuracy and reliability of the background image.

[0037] Specifically, the multiple consecutive original background images in which the monitored target appears before the original background image are divided into multiple background image groups according to the preset frame number. The frame rate of the surveillance camera used in this application is 33 frames per second, so the preset frame number here is set to 33.

[0038] Those skilled in the art will appreciate that the closer the capture time of the original background image without the monitored target in the final background image group is to the capture time of the original monitoring image with the monitored target, the more accurately the background image reflects the true state of the current scene. Therefore, the preset frame rate set herein does not limit the scope of protection of this application. Those skilled in the art may determine the preset frame rate based on actual needs, surveillance camera performance, and other factors.

[0039] Step 202: dividing the region image of the region of interest in each original background image in the background image group into a preset number of image blocks. In the embodiments of the present application, since the region of interest has been determined based on the position calibration object, each original background image contains the region of interest, and the region of interest (i.e., the region image) in each original background image in the background image group is divided into a preset number of image blocks. The present application exemplarily adopts the division manner of 9, so that the preset number is 9. The region image of the region of interest of the original background image is divided into 9 image blocks, and each background image group obtains a preset number of image blocks , i.e., 33 , 9=297.

[0040] The person skilled in the art can also balance the effect and the calculation efficiency here, adopt the division manner of 10, 11, etc., or set the first data to 10, 11, etc., which is not regarded as a limitation to the protection scope of the present application.

[0041] In addition, before step 202 is performed, the original background image can also be subjected to a grayscale processing.

[0042] Step 203: traversing each image block to perform an abnormality judgment based on the grayscale values of the image blocks at the same position in other original background images.

[0043] Specifically, the grayscale values of each image block in the background image group are determined respectively, and each original background image obtains a preset number (i.e., 9) of grayscale values. The mean value of the grayscale values and the sample standard deviation of the grayscale values of the image blocks corresponding to each position (a total of 9 positions) in the background image group are calculated respectively.

[0044] Further, as shown in FIG. 2B, first, the grayscale values of each image block divided in step 202 are calculated respectively, and 297 grayscale values are obtained. Then, the positions corresponding to the image blocks are taken as groups, i.e., the first image block in the region of interest of each original background image is taken as the first group, the second image block in the region of interest of each original background image is taken as the second group, and so on, 297 image blocks are divided into 9 groups, and the mean value (denoted as the grayscale mean value) and the sample standard deviation (denoted as the grayscale deviation) of the grayscale values of 33 image blocks in each group are calculated respectively, and 9 grayscale mean values and 9 grayscale deviations are obtained. Figure 3

[0045] ​​​The present application exemplarily adopts the Lilliefors criterion to perform abnormality judgment on the gray scale values of each image block, that is, on the basis that the gray scale values of each image block satisfy normal distribution, it is judged whether the difference between the gray scale value of each image block and the gray scale mean value is less than or equal to 3 times the gray scale deviation. If the difference between the gray scale value of each image block and the gray scale mean value is less than or equal to 3 times the gray scale deviation, the gray scale value distribution of the image block is normal. If the difference between the gray scale value of each image block and the gray scale mean value is greater than 3 times the gray scale deviation, the gray scale value distribution of the image block is abnormal.

[0046] The person skilled in the art should realize that the Lilliefors criterion is adopted herein to perform abnormality judgment on the gray scale values of each image block, which is only one embodiment of the present application and does not limit the protection scope of the present application. The person skilled in the art can also perform abnormality judgment on the gray scale values of each image block according to actual needs, that is, whether the difference between the gray scale value of each image block and the gray scale mean value is greater than 2 times / 1.5 times the gray scale deviation. Or the Grubbs criterion, the Dixon criterion and other methods are adopted to perform abnormality judgment on the gray scale values of each image block.

[0047] Step 204: judging whether each image block corresponding to at least one original background image in the background image group is normal. In the embodiment of the present application, based on the abnormality judgment result of the gray scale value of each image block in step 203, it is judged whether each image block corresponding to at least one original background image in the background image group is normal.

[0048] Specifically, if the gray scale values of the preset number (i.e. 9) of image blocks divided in the key attention region of at least one original background image in the background image group are all normal, step 205 is executed. If the gray scale values of at least one image block exist in the image blocks divided in the key attention region of each original background image in the background image group, step 206 is executed.

[0049] Step 205: selecting the region image of the key attention region of one original background image as the background representative image. In the embodiment of the present application, when the gray scale values of the preset number (i.e. 9) of image blocks divided in the key attention region of at least one original background image in the background image group are all normal, one original background image can be directly selected from the background image group, and the region image of the key attention region of the selected original background image is selected as the background representative image.

[0050] Step 206: selecting the image block with the minimum error in each position corresponding image block to perform splicing to obtain the background representative image. In the embodiment of the present application, the image block with the minimum difference between the gray scale value and the gray scale mean value of the corresponding group in the 9 groups is selected as the splicing image block at the position, so as to determine the 9 splicing image blocks.

[0051] In the embodiments of the present application, the gray scale values of the 33 image blocks in each group can also be compared with the corresponding gray scale mean value of the group respectively, and the image block corresponding to the gray scale value that does not meet the Rayleigh criterion is removed according to the Rayleigh criterion, until the image block at the position corresponding to each group is determined.

[0052] In the embodiments of the present application, the gray scale mean value of the remaining image blocks can be calculated after removing the image blocks with abnormal gray scale values, and the method of comparing the gray scale mean value of the remaining image blocks in the next round can be used to determine the image block at the position corresponding to the image block.

[0053] Specifically, taking the first group as an example, if the difference between the gray scale value of the first image block in the first group and the gray scale mean value of the remaining 32 image blocks in the first group is greater than 3 times the gray scale deviation, it is determined that the gray scale value of the image block is abnormal, the image block is removed from the first group, and then it is determined whether the difference between the gray scale value of the second image block and the gray scale mean value of the remaining 31 image blocks is greater than 3 times the gray scale deviation, and so on, until the difference between the gray scale value of an image block and the gray scale mean value of the remaining image blocks is less than or equal to 3 times the gray scale deviation, and the image block is determined as the image block at the position corresponding to the first group image block. Moreover, when the second data image blocks are continuously removed and the image block is not determined, the iteration is stopped, which indicates that the noise of the 33 image blocks in the first group is large. The second data is exemplarily set to 2.

[0054] In the embodiments of the present application, the obtained 9 image blocks are spliced to obtain a background representative image. The application can also use a fuzzy algorithm to weaken the blocking effect at the splicing position. The obtained background representative image can be directly used as a background image.

[0055] It should be noted that the application can also directly use the region image of the key attention region of the original background image without the monitoring target as the background image, or use the method in steps 201 to 206 to obtain the background image. The method of steps 201 to 206 can improve the anti-interference performance and exclude the interference caused by other flying objects, falling objects and other objects entering the field of view.

[0056] In addition, a plurality of background representative images can also be determined based on a plurality of background image groups. At least part of the original background images in a group of background image groups and a plurality of background representative images form a new background image group. The background representative images in the new background image group are determined by the acquisition process of the background representative images, and are used as background images.

[0057] Specifically, on one hand, the background environment before the monitoring target enters the field of view can be obtained as much as possible, so that it is most similar to the background environment in the monitoring image. On the other hand, the previous multiple background representative images and the latest several original background images are taken as a new background image group to determine the final background representative image, so that the sudden entry of other interference objects into the field of view at the last moment can be avoided.

[0058] The at least part of the original background images can be all of the selected background image group or part of the original background images. When part of the original background images in the background image group is selected, the original background images without the monitoring target are preferentially selected to avoid the interference of the monitoring target on the background images. When all of the original background images in the background image group are selected, the image blocks in the original background images containing the monitoring target can be determined as grayscale abnormities based on the previous multiple background representative images, so as to be excluded. Exemplarily, the background image group is preferably the last group of background images close to the original monitoring image containing the monitoring target. The multiple original background images selected from the background image group can be continuous or discontinuous. The selected original background images and the previous multiple background representative images form a new background image group.

[0059] In the embodiment of the application, the first frame of the monitoring video after the UAV receives the uncapping limit signal of the nest can be taken as the background representative image to determine the background image. A frame in the monitoring video during the process that the UAV flies from the current position to the hovering height above the nest can also be taken as the background representative image to determine the background image.

[0060] Step 103: determining the monitoring image based on the original monitoring image containing the monitoring target. In the embodiment of the application, the original monitoring image containing the monitoring target should be understood as the partial or whole monitoring target that can be determined from the original monitoring image. If the partial monitoring target appears in the original monitoring image, but the partial monitoring target is not enough to determine that it is the monitoring target, it should be understood that the image does not contain the monitoring target. The region image of the key attention region of one of the multiple original monitoring images containing the monitoring target is taken as the monitoring image.

[0061] Specifically, the subtraction operation can be performed on the two continuous original monitoring images to determine whether the UAV enters the original monitoring image.

[0062] Those skilled in the art should realize that other target recognition methods can also be used to determine whether the monitoring target enters the current original monitoring image, which is not limited in the application.

[0063] Step 104: filtering the region images of the key attention region in the background image and the monitoring image respectively along a first direction to obtain a first background image and a first monitoring image. In the embodiments of the present application, the first direction includes a horizontal direction or a vertical direction of the region image. When the first direction is the horizontal direction, the size of the filtering window is 1 i. Wherein i represents the length of the filtering window, i is a natural number less than n, and n is the horizontal pixel number of the region image to be filtered. When the first direction is the vertical direction, the size of the filtering window is j 1. Wherein j represents the width of the filtering window, j is a natural number less than m, and m is the vertical pixel number of the region image to be filtered.

[0064] Specifically, the background image and the monitoring image are filtered along the first direction (here, mean filtering is used).

[0065] In the prior art, after the motion target detection algorithm (such as frame difference algorithm, background difference algorithm, etc.), a smoothing filter (such as mean filter) with a window size of 3x3, 5x5, etc. is used for image smoothing, so that the visual effect of the algorithm result is better, such as Figure 5 , which is an example of mean filtering with a 3x3 filtering window size. But the present application pays attention to the integrity of a specific positioning direction, not the visual effect. Therefore, the present application can regard the two-dimensional array of the image (i.e. the two-dimensional array corresponding to the original background image or the original monitoring image) with a resolution of as m one-dimensional arrays with 1 row and n columns, and then perform mean filtering on the m one-dimensional arrays to retain the image information in the row direction as much as possible and weaken the coupling relationship between rows. Wherein m represents the vertical pixel number of the region image to be filtered, and n represents the horizontal pixel number of the region image to be filtered.

[0066] In the positioning process of the dynamic target, the present application pays attention to the integrity of a specific positioning direction, not the visual effect. The present application performs filtering on the background image and the monitoring image respectively by using mean filtering of one-dimensional arrays along the first direction, which can reduce the high-frequency components (such as edges, noises, details) of the background image and the monitoring image, reduce the sudden changes of the gray values therein, weaken the coupling relationship between different dimensions, retain the image information in the corresponding dimension as much as possible, and also help to improve the processing efficiency.

[0067] In actual use, the background image and the monitoring image are filtered along the row / column direction to obtain a first background image and a first monitoring image, such as Figure 6The filtering process of the one-dimensional array ignores the coupling relationship between rows and columns, and can retain more complete wing information. The integrity of the background image / monitoring image in the horizontal direction determines the reliability of the calculation result of the physical relative position. Exemplarily, the size of the filtering window of the mean filter is 1 i or j 1, i represents the length of the filtering window, i is a natural number less than n, n is the horizontal pixel number of the region image to be filtered, j represents the width of the filtering window, j is a natural number less than m, m is the vertical pixel number of the region image to be filtered.

[0068] In addition, when performing the filtering process, the background image / monitoring image can be first converted into a gray-scale image form. Then, the filtering process of the one-dimensional array is adopted to perform mean filtering on the background image / monitoring image by rows and columns, to obtain a first background image / first monitoring image.

[0069] Step 105: determining a difference image of the first background image and the first monitoring image. In the embodiment of the present application, the first background image and the first monitoring image obtained are subjected to difference binarization to obtain a difference image.

[0070] Since the filtering process of the one-dimensional array is adopted, the morphological filtering process can be omitted after the difference binarization of the first background image and the first monitoring image, that is, there is no operation of expansion, corrosion, opening and closing, and the default operation not only improves the calculation efficiency, but also saves the hardware cost.

[0071] Step 106: determining the physical relative position of the monitoring target and the position calibration object in the preset direction according to the difference image. In the embodiment of the present application, the relative position of the monitoring target and the position calibration object in the difference image is obtained through the difference image, and based on the relative position and the physical position information and / or the physical size information of the position calibration object, the physical relative position of the monitoring target and the position calibration object in the preset direction is determined.

[0072] In another specific embodiment of the present application, a plurality of closed region contours are found in the difference image through contour finding. The region contour with the largest number of internal pixel values is selected, and the minimum circumscribed rectangle of the region contour is determined. The position information of the monitoring target is determined according to the position information of the minimum circumscribed rectangle. The physical relative position of the monitoring target and the position calibration object in the preset direction is determined according to the position information of the monitoring target.

[0073] Specifically, a contour search is performed on the differential image to identify multiple closed area contours. The contour with the most pixel values ​​among the identified area contours is then counted, representing the contour with the largest area. The minimum bounding rectangle of the contour with the largest area is determined. The horizontal length of the minimum bounding rectangle represents the motion range of the monitored target in the preset direction within the monitoring field of view. The location of the minimum bounding rectangle is the location of the monitored target.

[0074] It should be noted that the position information obtained here is the position information of the monitoring target in the preset direction (horizontal direction of the front), which can be used to determine the relative position relationship between the drone and the position calibration object in the preset direction.

[0075] In an embodiment of the present application, the physical relative position of the drone and the nest in a preset direction can be determined based on the differential image.

[0076] Specifically, the pixel size of the nest in the camera's field of view is determined. A centerline passing through the nest's center and perpendicular to a corresponding preset direction is determined. Based on the drone's position information and the ratio of the nest's actual size to its pixel size, one or more vertical distances between the drone and the nest's centerline are determined.

[0077] Furthermore, given the actual length and width of the parking platform of the machine nest, its pixel size under the camera's viewing angle can be calculated using the parameters of the monitoring camera, which will not be repeated here.

[0078] In this step, if Figure 4 As shown in the figure, the x-axis points to the length direction of the machine nest, the y-axis points to the width direction of the machine nest, A, B, C, and D represent the four corner points of the parking platform of the machine nest, L represents the actual length of the parking platform of the machine nest, that is, the actual length of the AB side or the CD side, O represents the center of the machine nest, O1 represents the center of the AB side, O2 represents the center of the CD side, p1 represents the intersection point of the median line of the minimum circumscribed rectangle of the UAV and the AB side, and p2 represents the intersection point of the median line of the minimum circumscribed rectangle of the UAV and the CD side.

[0079] The physical relative position of the drone and the nest is calculated as follows: d1=LX1 / P AB , d2=LX2 / P CD , where d1 represents the vertical distance at the AB side, i.e., the real distance of O1P1, d2 represents the vertical distance at the CD side, i.e., the real distance of O2P2, L represents the actual length of the parking platform of the machine nest, i.e., the actual length of the AB side and the CD side, X1 represents the pixel distance from p1 to the center line O1O2 of the machine nest in the current frame, X2 represents the pixel distance from p2 to the center line O1O2 of the machine nest in the current frame, P AB Indicates the pixel length of the AB side, that is, the pixel distance between the two corner points A and B, PCD Pixel length of CD side, i.e. pixel distance between two corner points of C and D.

[0080] In the horizontal direction (i.e. x direction) of the front face of the nest, the distance from the UAV to the center of the nest is between d1 and d2. To ensure the safety of the UAV and reduce the false alarm rate, the relative coordinate of the current UAV in the horizontal direction is dx = max(d1, d2). If the center line of the nest is parallel to the center line of the minimum circumscribed rectangle of the UAV, then d1 = d2.

[0081] In the embodiments of the present application, the monitoring camera generally does not have depth information, so for the physical relative position of the UAV during landing, the present application can also determine a second preset direction which forms a preset angle with the preset direction. Based on the original background image without the monitoring target and the original monitoring image with the monitoring target collected along the second preset direction, a difference step is performed to obtain a second difference image. According to the second difference image, the physical relative position of the monitoring target and the position calibration object in the second preset direction is determined.

[0082] Specifically, along the preset direction or the second preset direction refers to the orientation direction of the monitoring camera. By deploying two monitoring cameras at a preset angle, the physical relative position of the UAV in different directions is captured and calculated.

[0083] Further, a second preset direction which forms a preset angle with the preset direction is determined. The preferred preset angle is 90°, i.e. the preset direction is perpendicular to the second preset direction. If the above embodiment is used, the horizontal direction of the front face of the nest is the preset direction, and then the horizontal direction of the left / right face of the nest can be the second preset direction.

[0084] Those skilled in the art should be aware that the preset angle between the preset direction and the second preset direction is not necessarily a right angle, but the calculation is the simplest in the case of a right angle. The preset angle can also be 45°, 30°, etc., and through trigonometric functions, the physical relative distance of the UAV and the nest in the second preset direction (the preset angle is not a right angle) or the direction which forms a right angle with the preset direction can still be calculated.

[0085] The second preset direction is substituted into steps 101 to 106, the original background image without the monitoring target and the original monitoring image with the monitoring target in the second preset direction are collected, then a difference step is performed to obtain a second difference image, and then the physical relative position of the UAV in the second preset direction is obtained based on step 106 According to the physical relative positions of the UAV and the nest in two different directions, the two-dimensional coordinates of the plane parallel to the parking platform of the UAV relative to the nest can be obtained.

[0086] In the embodiments of the present application, after the UAV finishes the task and returns according to the positioning data, hovers to prepare for landing after reaching the landing altitude point, and determines whether the UAV reaches the hovering height at this time according to the positioning data. If the UAV does not reach the hovering height, steps 101 and 102 are continuously executed until the UAV reaches the hovering height. If the UAV reaches the hovering height (i.e., enters the original monitoring image), steps 103 to 106 are executed to obtain the physical relative position of the UAV and the nest in the preset direction (or the second preset direction).

[0087] When the physical relative position of the UAV and the nest meets the preset requirement, the UAV lands in the nest. Otherwise, the UAV lands in the backup landing point. Specifically, a center line passing through the center of the nest and perpendicular to the corresponding preset direction (or the second preset direction) is determined. One or more perpendicular distances of the UAV to the center line are determined. When the perpendicular distances are all within the preset threshold range, the UAV lands in the nest. Otherwise, the UAV lands in the backup landing point.

[0088] The preset threshold range here is [-0.3, 0.3] meters, and if the physical relative position of the UAV and the nest meets the preset threshold range, the UAV lands in the nest. If the physical relative position of the UAV and the nest does not meet the preset threshold range, the UAV lands in the backup landing point.

[0089] In addition, if the physical relative position of the UAV and the nest does not meet the preset threshold range, further judgment can also be performed. Specifically, steps 103 to 106 are performed based on the next original monitoring image of the current original monitoring image, the physical relative distance of the UAV and the nest in the preset direction (or the second preset direction) is re-determined, and it is determined whether it meets the preset threshold range. If the first threshold value of the first threshold value of the physical relative distance obtained based on the first threshold value of the original monitoring image after the current original monitoring image does not meet the preset threshold range, the UAV flies again and lands in the backup landing point. Exemplarily, the first threshold value is set to 15.

[0090] In the embodiments of the present application, the physical relative distance of the current position information of the UAV during the landing process relative to the center of the nest is the basis for selecting the landing of the nest recovery platform or the landing of the alternate landing point, and therefore the calculation speed of the moving target detection method during the landing process has a higher requirement, and the calculation amount thereof should not be too large. Moreover, the UAV during the landing process changes from fast to slow, and then from slow to hovering, and then continues to land or fly back to the alternate landing point according to the calculated physical relative distance, which requires the moving target detection method to adapt to the speed change of the UAV. The traditional moving target detection method is generally divided into a moving target detection method based on pixel change and a moving target detection method based on background modeling. The moving target detection method based on pixel change finds the change area according to the pixel difference between video frames, mainly including frame difference method, optical flow method and the like, but the frame difference method cannot adapt to the target that is static or moves at a very slow speed, and the optical flow method is sensitive to image noise and has high calculation complexity. The moving target detection method based on background modeling is a common detection method for moving targets in a static background, which establishes a background model of the corresponding scene according to the first N frames of video, and obtains the moving target by comparison, mainly including Gaussian mixture model method, ViBe (full name: Visual Background Extractor, a video background extraction algorithm based on pixel-level modeling) algorithm and the like, wherein the model assumption of the Gaussian mixture model method has certain limitations and the calculation amount is large, and the ViBe algorithm is prone to noise, ghosting and the like.

[0091] Therefore, it can be seen that the existing moving target detection method cannot adapt to the real-time and accuracy requirements in the UAV recovery process. The background images can be periodically updated by the plurality of background image groups, so as to adapt to the dynamic change of the scene and more accurately confirm the physical relative distance of the UAV from the nest. The calculation complexity of the physical relative distance is low, and the real-time requirement of the UAV recovery can be adapted. The one-dimensional array filtering processing is adopted, the visual effect of the original monitoring image is ignored, the direction information can be retained as much as possible, and then the speed change of the UAV can be adapted.

[0092] Although the present application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. When the device or client product is executed in practice, the method order shown in the embodiments or the drawings can be executed in sequence or in parallel (for example, in a parallel processor or multi-thread processing environment).

[0093] As Figure 7As shown, the embodiment of the present application further provides an image-based positioning device 700. The device includes: an execution module 701 and a positioning module 706, wherein the execution module 701 includes a first determination module 702, a second determination module 703, a filtering module 704 and a difference module 705, as follows.

[0094] The execution module 701 is used to perform a difference step based on the original background image without the monitoring target and the original monitoring image with the monitoring target collected along a preset direction to obtain a differential image.

[0095] The first determining module 702 is configured to determine a background image based on an original background image that does not contain a monitoring target.

[0096] The second determining module 703 is used to determine a monitoring image based on the original monitoring image containing the monitoring target, wherein the monitoring image and the background image contain the same focus area, and the focus area contains a position calibration object.

[0097] The filtering module 704 is configured to perform filtering processing along a first direction on the background image and the regional image of the focused area in the monitoring image, respectively, to obtain a first background image and a first monitoring image.

[0098] The difference module 705 is used to determine a difference image between the first background image and the first monitoring image.

[0099] The positioning module 706 is used to determine the physical relative position of the monitoring target and the position calibration object in a preset direction based on the differential image.

[0100] Some modules in the apparatus described herein may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0101] The devices or modules described in the above application embodiments can be implemented by computer chips or physical devices, or by products with certain functions. For ease of description, the above devices are described separately by function in various modules. When implementing the embodiments of this application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0102] The methods, apparatuses or modules described in the present application can be implemented in a computer readable program code manner. The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code (for example, software or firmware) executable by the (micro)processor, logic gates, switches, Application Specific Integrated Circuit (ASIC), programmable logic controller and embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in a pure computer readable program code manner, the same function can also be implemented by logically programming the method steps in the form of logic gates, switches, Application Specific Integrated Circuit, programmable logic controller and embedded microcontroller. Therefore, such a controller can be considered as a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both a software module implementing the method and a structure within the hardware component.

[0103] The embodiments of the present application further provide a device, which comprises: a processor; a memory for storing processor executable instructions; and the processor executes the executable instructions to implement the method according to the embodiments of the present application.

[0104] In addition, the functional modules in the various embodiments of the present application can be integrated in one processing module, or each module can exist independently, or two or more modules can be integrated in one module.

[0105] The storage medium described above includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD) or Memory Card. The memory can be used to store computer program instructions.

[0106] Those skilled in the art can clearly understand the application by the description of the foregoing embodiments. The technical solutions of the application can be implemented by means of software and necessary hardware. Based on such an understanding, the technical solutions of the application can be embodied in the form of a software product or in the form of data migration. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, or an optical disk, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a mobile terminal, a server, or a network device) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments. The whole or part of the application can be used in many general or special computer system environments or configurations. For example, personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, small computers, large computers, distributed computing environments including any of the above systems or devices, and the like.

[0108] The foregoing embodiments are only used to illustrate the technical solutions of the application, rather than limit the application; although the application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the scope of the technical solutions of the application.

Claims

1. An image-based positioning method, characterized in that: include: A differential image is obtained by performing a differential step based on an original background image not containing the monitoring target and collected along a preset direction and an original monitoring image containing the monitoring target; wherein the differential step includes: determining a background image based on an original background image that does not contain a monitoring target; Determining a monitoring image based on an original monitoring image containing a monitoring target; wherein the monitoring image and the background image contain the same focus area, and the focus area contains a position calibration object; Performing filtering processing along a first direction on the background image and the regional image of the focused area in the monitoring image respectively to obtain a first background image and a first monitoring image; determining a difference image between the first background image and the first monitoring image; The physical relative position of the monitoring target and the position calibration object in the preset direction is determined according to the differential image.

2. The method according to claim 1, characterized in that Before performing the difference step based on the original background image without the monitoring target and the original monitoring image with the monitoring target acquired along the preset direction, the method includes: Collecting multiple original background images of a preset area along a preset direction; wherein the preset area includes a focus area; When a monitoring target that meets a preset condition appears in the original background image, the acquisition of the original background image is stopped.

3. The method according to claim 1, characterized in that The determining of the background image based on the original background image without the monitoring target includes: A preset number of frames of original background images without monitoring targets are used as a background image group; Obtaining a background representative image of the focused area based on the background image group; determining a background image based on the background representative image; The process of obtaining the background representative image includes: Dividing a region image of a focus area in each original background image in the background image group into a preset number of image blocks; Traverse each image block and make anomaly judgment based on the grayscale value of the image block at the same position in other original background images; If all image blocks corresponding to at least one original background image in the background image group are normal, then selecting a region image of a focus area of ​​one of the original background images as a background representative image; Otherwise, the image block with the smallest error among the image blocks corresponding to each position is selected for splicing to obtain the background representative image.

4. The method according to claim 3, characterized in that The determining of the background image based on the original background image without the monitoring target further includes: determining a plurality of background representative images based on the plurality of background image groups; Combining at least part of the original background images in a set of background image groups with the multiple background representative images into a new background image group; The background representative image in the new background image group is determined as the background image by utilizing the background representative image acquisition process.

5. The method according to claim 1, wherein The first direction includes a horizontal direction or a vertical direction of the area image; When the first direction is the horizontal direction, the filter window size is 1 i; where i represents the length of the filter window, i is a natural number less than n, and n is the number of horizontal pixels in the area image to be filtered; When the first direction is the longitudinal direction, the filter window size is j 1; where j represents the width of the filter window, j is a natural number less than m, and m is the number of vertical pixels in the area image to be filtered.

6. The method according to claim 1, characterized in that Determining the physical relative position of the monitoring target and the position calibration object in the preset direction based on the differential image includes: Finding a plurality of closed area contours in the differential image by contour searching; Select the area outline with the largest number of internal pixel values ​​and determine the minimum circumscribed rectangle of the area outline; Determine the location information of the monitoring target according to the location information of the minimum circumscribed rectangle; The physical relative position of the monitoring target and the position calibration object in the preset direction is determined according to the position information of the monitoring target.

7. The method according to claim 1, characterized in that The determining of the physical relative position of the monitoring target and the position calibration object in the preset direction based on the differential image further includes: Determining a second preset direction that forms a preset angle with the preset direction; Based on the original background image without the monitoring target and the original monitoring image with the monitoring target acquired along the second preset direction, performing the subtraction step to obtain a second differential image; The physical relative position of the monitoring target and the position calibration object in the second preset direction is determined according to the second differential image.

8. A method for recovering a drone, characterized in that: include: Setting initial conditions, including: using the drone as the monitoring target, the drone nest as the position calibration object, determining the preset direction relative to the drone nest, and determining the key focus area based on the location of the drone nest; Executing the method according to any one of claims 1 to 7 based on the initial conditions to obtain the physical relative position of the UAV and the machine nest; When the physical relative position of the drone and the nest meets the preset requirements, the drone lands in the nest; Otherwise, the drone lands at the alternate landing point.

9. The method according to claim 8, characterized in that When the physical relative position of the UAV and the nest meets the preset requirements, the UAV lands in the nest, including: Determine the center line passing through the center of the machine nest and perpendicular to the corresponding preset direction; determining one or more vertical distances of the drone from the centerline; When the vertical distances are all within a preset threshold range, the drone lands in the nest.

10. An image-based positioning device, characterized in that: include: An execution module is configured to perform a difference step based on an original background image not containing a monitoring target and an original monitoring image containing a monitoring target acquired along a preset direction to obtain a difference image; wherein the difference step includes: A first determining module is configured to determine a background image based on an original background image that does not contain a monitoring target; A second determining module is configured to determine a monitoring image based on an original monitoring image containing a monitoring target; wherein the monitoring image and the background image contain the same focus area, and the focus area contains a position calibration object; a filtering module, configured to perform filtering processing along a first direction on the background image and the regional image of the focused area in the monitoring image, respectively, to obtain a first background image and a first monitoring image; a difference module, configured to determine a difference image between the first background image and the first monitoring image; A positioning module is used to determine the physical relative position of the monitoring target and the position calibration object in the preset direction based on the differential image.

Images