Dynamic and precise positioning of Hummingbird UAV based on multi-view fusion

Abstract

The present invention relates to an unmanned aerial vehicle, and more specifically, to a hummingbird unmanned aerial vehicle with dynamic and precise positioning based on multi-view fusion. It includes a format unification processing module and a multi-image fusion module. The present invention combines a plurality of images in a unified format through a multi-image fusion module, removes the overlapping contents therein according to the image content information, obtains the characteristic points of the monitoring field, and generates a fused image as the monitoring characteristic image of the current monitoring area. It can not only obtain images of the monitoring location at different angles and provide a plurality of characteristic points as a reference for later monitoring personnel, but also obtain a partial full view of the current monitoring location through the fused image, generate a range characteristic image, further improve the reference effect of the monitoring point, and avoid the occurrence of reference point failure caused by the overlap of some characteristic points.

Description

Technical Field

[0001] The present invention relates to an unmanned aerial vehicle, in particular to a hummingbird unmanned aerial vehicle with dynamic and precise positioning based on multi-view fusion. Background Art

[0002] The Hummingbird UAV is a small drone with a body similar to that of a hummingbird. It can perform reconnaissance and surveillance missions in complex environments such as cities and forests.

[0003] When traditional hummingbird drones are exploring roads, they need to provide multiple feature points to obtain information on new roads. For example, they need to use some special markers for positioning and processing to obtain real-time images to provide reference points for later monitoring personnel. However, the terrain of some road areas is similar and the image information obtained is basically the same. At this time, the reference points provided are too poor and can easily mislead monitoring personnel.

[0004] In order to address the above problems, there is an urgent need for a hummingbird drone with dynamic and precise positioning based on multi-view fusion. Summary of the invention

[0005] The purpose of the present invention is to provide a hummingbird drone with dynamic and precise positioning based on multi-view fusion to solve the problems raised in the above background technology.

[0006] To achieve the above purpose, a dynamic and precise positioning hummingbird UAV based on multi-view fusion is provided, including an image monitoring module, a monitoring instrument positioning and identification module, a multi-image content acquisition module, a format unification processing module and a multi-image fusion module;

[0007] The image monitoring module monitors images at different angles through a flight device equipped with different monitoring instruments to obtain image information;

[0008] The monitoring instrument positioning and identification module locates and numbers each monitoring instrument in combination with the position of the different monitoring instruments on the flight equipment;

[0009] The multi-image content acquisition module combines the image information fed back by the current monitoring instrument and acquires the position information of the current monitoring instrument according to the label corresponding to the monitoring instrument and the monitoring angle;

[0010] The format unification processing module performs format recognition on the image information fed back by different monitoring instruments, formulates a unified format, performs unified processing on the images of different formats, and generates multiple images of unified format;

[0011] The multi-image fusion module combines multiple images in a unified format, removes overlapping content according to image content information, obtains feature points of the monitoring area, and generates a fused image as a monitoring feature image of the current monitoring area;

[0012] The flying device includes a drone body, which includes a frame for carrying various monitoring instruments, a pair of brackets are arranged on both sides of the frame, an impeller for providing flight power is arranged on the top of the bracket, and a support pad is arranged on the bottom of the bracket. The monitoring instrument includes a monitoring device, which includes a connecting frame, a side panel and a camera, an angle adjustment device is connected between the camera and the side panel, the monitoring instrument positioning and identification module performs positioning processing on each of the cameras in combination with their installation positions, and the monitoring instrument positioning acquisition module identifies the current monitoring angle by identifying the offset angle of the angle adjustment device.

[0013] As a further improvement of the technical solution, the method for generating multiple images in a unified format by the format unification processing module includes the following steps:

[0014] S1, transmitting each fed-back image to an image format converter;

[0015] S2, identifying the format of the current image through an image format converter;

[0016] S3. Plan the unified format images in advance, compare the image formats with the images fed back, convert the images that have not been successfully compared into the unified format, and generate multiple unified format images.

[0017] As a further improvement of the technical solution, the method for generating a fused image by the multi-image fusion module comprises the following steps:

[0018] S10, obtaining multiple uniform format image information contents;

[0019] S20, determining a common monitoring area of ​​different images, and cutting the common monitoring area through an image format converter;

[0020] S30, determining a combination order of the fused images in combination with the labels of the monitoring instruments corresponding to the images;

[0021] S40, fusing multiple images in a unified format according to the combination order through an image format converter to generate a fused image.

[0022] As a further improvement of the present technical solution, the angle adjustment device includes a connecting plate, a connecting gear and a servo motor. A side groove is opened on the side of the side plate, the connecting gear is located at the inner end of the side groove, the connecting plate is fixed on the side of the camera, the connecting gear is sleeved on the inner side of the connecting plate, and the servo motor maintains a coaxial connection with the side of the connecting gear.

[0023] As a further improvement of the present technical solution, a scraper plate is provided at the top of the connecting gear, and the scraper plate includes a bottom shaft and a driving gear coaxially connected to the bottom shaft, the driving gear is meshingly connected to the connecting gear, and a cleaning assembly is provided at the top of the bottom shaft. When the servo motor drives the camera to shift upward, the cleaning assembly will be driven to shift closer to the camera position through the driving gear meshingly connected to the connecting gear.

[0024] As a further improvement of the technical solution, the cleaning assembly includes a pair of clamps, the two clamps are respectively arranged at the two ends of the bottom shaft, and the top ends of the two clamps are connected to arc scrapers, and a plurality of burrs are arranged in an array on the inner side of the arc scrapers.

[0025] As a further improvement of the present technical solution, a connecting frame is connected to the side surface of the top end of the side plate, a fixed arc plate is obliquely arranged at the edge of one side of the connecting frame, a mounting groove is opened on the inner side of the fixed arc plate, the cross-sectional size of the mounting groove is consistent with the cross-sectional size of the arc scraper, and a number of spikes are arranged in an array on both sides of the inner end of the mounting groove.

[0026] As a further improvement of the technical solution, partitions are provided on both sides of the bottom end of the side panel, and a gap is reserved between the two partitions for the camera to freely shuttle.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. In the dynamic and precise positioning Hummingbird UAV based on multi-view fusion, multiple images in a unified format are combined through a multi-image fusion module. According to the image content information, the overlapping content is eliminated, the characteristic points of the monitoring field are obtained, and a fused image is generated. As the monitoring characteristic image of the current monitoring area, it can not only obtain images of the monitoring location at different angles and provide multiple characteristic points as a reference for later monitoring personnel, but also obtain a partial picture of the current monitoring location through the fused image, generate a range characteristic image, further improve the reference effect of the monitoring point, and avoid the occurrence of reference point failure caused by the overlap of some characteristic points.

[0029] 2. In the hummingbird drone with dynamic and precise positioning based on multi-view fusion, the camera lens is self-cleaned by means of a scraper ring. When arriving at the monitoring location, the camera is first driven upward by the servo motor. At this time, the connecting gear will drive the driving gear meshing with it to rotate in the opposite direction, and adjust the cleaning component to be offset closer to the camera position until the cleaning component contacts the camera lens, and the stains adhered during the flight are scraped off to maintain the cleanliness of the camera lens and ensure the shooting effect.

[0030] 3. In the hummingbird drone with dynamic and precise positioning based on multi-view fusion, the self-cleaning function of burrs is realized by setting a fixed arc plate. When the camera rotates downward, the arc scraper is driven upward by the driving gear and inserted into the inner end of the mounting groove on the side of the fixed arc plate, and the two sides of the arc scraper are scraped by the thorns set at the inner end of the mounting groove. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall structure of the present invention;

[0032] Figure 2 This is a schematic diagram of the structure of the drone body of the present invention;

[0033] Figure 3 This is a structural breakdown diagram of the monitoring device of the present invention;

[0034] Figure 4 For the present invention Figure 3 A local enlarged view of point A;

[0035] Figure 5 For the present invention Figure 3 A partial enlarged view of point B;

[0036] Figure 6 It is a schematic diagram of the structure of the scraper plate of the present invention;

[0037] Figure 7 It is the overall system connection block diagram of the present invention;

[0038] Figure 8 A flow chart of a method for generating multiple images in a unified format according to the present invention;

[0039] Fig. 9 The figure is a flow chart of the method for generating a fused image according to the present invention.

[0040] The meaning of each number in the figure is:

[0041] 10. UAV body; 110. frame; 120. support pad; 130. impeller;

[0042] 20. Monitoring equipment; 210. Connecting frame; 220. Side plate; 221. Side groove; 230. Camera; 231. Connecting plate; 232. Connecting gear; 240. Scraping plate; 241. Burr; 242. Bottom shaft; 243. Driving gear; 250. Fixed arc plate; 251. Mounting groove; 252. Burr. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0045] See also Figure 1-Figure 9 As shown, a dynamic and precise positioning hummingbird UAV based on multi-view fusion is provided, including an image monitoring module, a monitoring instrument positioning and identification module, a multi-image content acquisition module, a format unified processing module and a multi-image fusion module;

[0046] The image monitoring module monitors images at different angles through flying equipment equipped with different monitoring instruments to obtain image information;

[0047] The monitoring instrument location identification module locates and identifies each monitoring instrument based on the location of the different monitoring instruments on the flight equipment, and labels them;

[0048] The multi-image content acquisition module combines the image information fed back by the current monitoring instrument and obtains the location information of the current monitoring instrument according to the label corresponding to the monitoring instrument and the monitoring angle;

[0049] The format unification processing module identifies the format of image information fed back by different monitoring instruments, formulates a unified format, performs unified processing on images of different formats, and generates multiple images of unified format;

[0050] The multi-image fusion module combines multiple images in a unified format, removes overlapping content according to the image content information, obtains the feature points of the monitoring area, and generates a fused image as the monitoring feature image of the current monitoring area.

[0051] The flying device includes a drone body 10, which includes a frame 110 for carrying various monitoring instruments. A pair of brackets are arranged on both sides of the frame 110. An impeller 130 for providing flight power is arranged at the top of the bracket, and a support pad 120 is arranged at the bottom of the bracket. The monitoring instrument includes a monitoring device 20, which includes a connecting frame 210, a side panel 220 and a camera 230. An angle adjustment device is connected between the camera 230 and the side panel 220. The monitoring instrument positioning and recognition module performs positioning processing on each camera 230 in combination with its installation position. The monitoring instrument positioning acquisition module adjusts the offset angle of the device through the identification angle to identify the current monitoring angle.

[0052] When in use, during the road survey, first start each impeller 130, drive the frame 110 to move through the impeller 130, and drive the frame 110 to fly along the survey road for survey. When arriving at the predetermined monitoring location, the current position of the frame 110 is suspended and fixed by the impeller 130, and the offset angle of the camera 230 is controlled by the angle adjustment device to obtain image information of the current monitoring location. At the same time, the obtained image information and the offset angle of the camera 230 are fed back to the multi-image content acquisition module. The multi-image content acquisition module combines the monitoring instrument positioning and recognition module to perform positioning processing on each monitoring instrument, obtains the mark of the camera 230 of the current feedback monitoring location, and then transmits the monitoring images fed back by each camera 230 to the format system in turn. The unified processing module recognizes the format of image information fed back by different monitoring instruments through the format unification processing module, formulates a unified format, performs unified processing on images of different formats, and generates multiple images of unified format. Subsequently, the multiple images of unified format are combined through the multi-image fusion module, and the overlapping contents are eliminated according to the image content information to obtain the characteristic points of the monitoring field and generate a fused image as the monitoring characteristic image of the current monitoring area. It can not only obtain images of the monitoring location at different angles and provide multiple characteristic points as a reference for later monitoring personnel, but also obtain a partial picture of the current monitoring location through the fused image, generate a range characteristic image, and further improve the reference effect of the monitoring point to avoid the occurrence of reference point failure caused by the overlap of some characteristic points.

[0053] Since the parameters such as exposure, white balance, and focus of different monitoring instruments at different monitoring points will change due to different positions, the format of the image information fed back from shooting will also be different. When performing image fusion later, it is necessary to ensure that the format of each image is consistent in order to perform effective fusion. In addition, the method for the format unification processing module to generate multiple images in a unified format includes the following steps:

[0054] S1, transmitting each fed-back image to an image format converter;

[0055] S2, identifying the format of the current image through an image format converter;

[0056] S3. Plan the unified format images in advance, compare the image formats with the images fed back, convert the images that have not been successfully compared into the unified format, and generate multiple unified format images.

[0057] Therefore, in the process of image processing, it is first necessary to transmit each feedback image to the image format converter, and use the image format converter to identify the format of the current image, that is, the width, height, spacing, etc. of the picture, and plan a unified format image in advance, and compare the image format with each feedback image. The image that has successfully matched is used directly, and the image that has not been matched successfully is converted to a unified format to generate multiple unified format images for later image fusion processing.

[0058] Furthermore, the method for generating a fused image by the multi-image fusion module includes the following steps:

[0059] S10, obtaining multiple uniform format image information contents;

[0060] S20, determining a common monitoring area of ​​different images, and cutting the common monitoring area through an image format converter;

[0061] S30, determining a combination order of the fused images in combination with the labels of the monitoring instruments corresponding to the images;

[0062] S40, fusing multiple images in a unified format according to the combination order through an image format converter to generate a fused image.

[0063] During specific use, during the image fusion process of multiple unified format images, due to the different monitoring angles of each monitoring instrument, its monitoring range is easy to overlap, resulting in image overlap in the fused image. At this time, it is necessary to obtain the information content of multiple unified format images, that is, the distribution of special points in each image, such as the appearance of feature points in a certain image (trees, rocks and slopes different from the previous monitoring points), and this feature point appears in the images fed back by different monitoring instruments, indicating that the two monitoring images have overlapped. At this time, the common monitoring area is cropped through the image format converter to obtain an image of a single feature point, and the combination order of the fused image is determined in combination with the number of the monitoring instrument corresponding to each image, and the image format converter is used to fuse multiple unified format images according to the combination order to generate a fused image.

[0064] Furthermore, the angle adjustment device includes a connecting plate 231, a connecting gear 232 and a servo motor. A side groove 221 is opened on the side of the side plate 220, and the connecting gear 232 is located at the inner end of the side groove 221. The connecting plate 231 is fixed to the side of the camera 230, and the connecting gear 232 is sleeved on the inner side of the connecting plate 231. The servo motor and the side of the connecting gear 232 are coaxially connected. When the angle of the camera 230 is adjusted to obtain image information at different angles of the monitoring location, the connecting gear 232 is driven to rotate by the servo motor, and the connecting gear 232 drives the camera 230 to shift the angle through the connecting plate 231. The multi-image content acquisition module obtains the transmission direction and number of transmission circles of the servo motor to obtain the monitoring angle of the current camera 230, which serves as a reference for later positioning the monitoring image position.

[0065] Since the lens of the camera 230 is exposed to the outside for a long time and is exploring in an unfamiliar road section, the lens is easily exposed to dust or obstructions in the air during flight, resulting in stains adhering to the lens, forming a shooting obstruction, and affecting the quality of the captured image. Specifically, a scraper plate 240 is arranged on the top of the connecting gear 232. The scraper plate 240 includes a bottom shaft 242 and a driving gear 243 coaxially connected to the bottom shaft 242. The driving gear 243 is meshedly connected to the connecting gear 232, and a cleaning component is arranged on the top of the bottom shaft 242. When the servo motor drives the camera 230 to deflect upward, the cleaning component is cleaned by the connecting gear 232. The driving gear 243 that is meshed with the connecting gear 232 will drive the cleaning component to shift toward the position close to the camera 230, and the camera 230 will perform the self-cleaning function of the lens through the scraper ring plate 240. When arriving at the monitoring location, the camera 230 will first be driven to rotate upward by the servo motor. At this time, the connecting gear 232 will drive the driving gear 243 that is meshed with it to rotate in the opposite direction, and adjust the cleaning component to shift toward the position close to the camera 230 until the cleaning component contacts the camera 230 lens, and scrape off the stains adhered during the flight, maintain the cleanliness of the camera 230 lens, and ensure the shooting effect.

[0066] In addition, the cleaning component includes a pair of clamps, and the two clamps are respectively arranged at the two ends of the bottom shaft 242, and the top of the two clamps are connected to an arc scraper, and a plurality of burrs 241 are arranged in an array on the inner side of the arc scraper. When the bottom shaft 242 drives the clamps and the arc scraper and the camera 230 approach each other, the inner side of the arc scraper will gradually approach the position of the camera 230 lens until the two are in contact. At this time, the burrs 241 arranged on the inner side of the arc scraper scrape off the stains adhered to the camera 230 lens under the action of friction, so as to avoid the stains from forming an obstruction that affects the later shooting effect.

[0067] Furthermore, a connecting frame 210 is connected to the top side of the side plate 220, and a fixed arc plate 250 is obliquely arranged at the edge of one side of the connecting frame 210. A placement groove 251 is provided on the inner side of the fixed arc plate 250. The cross-sectional size of the placement groove 251 is consistent with the cross-sectional size of the arc scraper, and a plurality of thorns 252 are arranged in an array on both sides of the inner end of the placement groove 251. After the arc scraper performs a long-term stain scraping work, the burrs 241 arranged on the inner wall will accumulate stains, and the accumulation of stains will accumulate over time. The movement will exceed the load of the burr 241, and the scraping effect of the burr 241 will be significantly reduced. In order to ensure the scraping and cleaning effect of the burr 241, the self-cleaning function of the burr 241 is realized by setting a fixed arc plate 250. When the camera 230 rotates downward, the arc scraper is driven upward by the driving gear 243 and inserted into the inner end of the receiving groove 251 on the side of the fixed arc plate 250. The protrusions 252 set at the inner end of the receiving groove 251 scrape the two sides of the arc scraper.

[0068] Furthermore, partitions are provided on both sides of the bottom end of the side plate 220, and a gap is reserved between the two partitions for the camera 230 to move freely. When the position of the monitoring location needs to be adjusted, the servo motor drives the camera 230 to shift downward and move to the inner end of the gap formed between the two partitions, and the arc-shaped scraper just extends into the inner end of the placement groove 251 for cleaning, thereby reducing dust adhesion during operation, further reducing subsequent cleaning work, and increasing the service life of the camera 230 and the scraper ring plate 240.

[0069] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. Dynamic and precise positioning of Hummingbird UAV based on multi-view fusion, characterized by: It includes image monitoring module, monitoring instrument positioning and identification module, multi-image content acquisition module, format unification processing module and multi-image fusion module; The image monitoring module monitors images at different angles through a flight device equipped with different monitoring instruments to obtain image information; The monitoring instrument positioning and identification module locates and numbers each monitoring instrument in combination with the position of the different monitoring instruments on the flight equipment; The multi-image content acquisition module combines the image information fed back by the current monitoring instrument and acquires the position information of the current monitoring instrument according to the label corresponding to the monitoring instrument and the monitoring angle; The format unification processing module performs format recognition on the image information fed back by different monitoring instruments, formulates a unified format, performs unified processing on the images of different formats, and generates multiple images of unified format; The multi-image fusion module combines multiple images in a unified format, removes overlapping content according to image content information, obtains feature points of the monitoring area, and generates a fused image as a monitoring feature image of the current monitoring area; The flying device comprises an unmanned aerial vehicle body (10), the unmanned aerial vehicle body (10) comprises a frame (110) for carrying each monitoring instrument, a pair of brackets are arranged on both sides of the frame (110), an impeller (130) for providing flight power is arranged at the top of the bracket, and a support pad (120) is arranged at the bottom of the bracket, the monitoring instrument comprises a monitoring device (20), the monitoring device (20) comprises a connecting frame (210), a side plate (220) and a camera (230), an angle adjustment device is connected between the camera (230) and the side plate (220), the monitoring instrument positioning and identification module performs positioning processing on each camera (230) in combination with its installation position, and the monitoring instrument positioning acquisition module identifies the current monitoring angle by identifying the offset angle of the angle adjustment device.

2. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 1 is characterized by: The method for generating a plurality of images in a unified format by the format unification processing module comprises the following steps: S1, transmitting each fed-back image to an image format converter; S2, identifying the format of the current image through an image format converter; S3. Plan the unified format images in advance, compare the image formats with the images fed back, convert the images that have not been successfully compared into the unified format, and generate multiple unified format images.

3. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 1 is characterized in that: The method for generating a fused image by the multi-image fusion module comprises the following steps: S10, obtaining multiple uniform format image information contents; S20, determining a common monitoring area of ​​different images, and cutting the common monitoring area through an image format converter; S30, determining a combination order of the fused images in combination with the labels of the monitoring instruments corresponding to the images; S40, fusing multiple images in a unified format according to the combination order through an image format converter to generate a fused image.

4. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 1, characterized in that: The angle adjustment device comprises a connecting plate (231), a connecting gear (232) and a servo motor; a side groove (221) is provided on the side of the side plate (220); the connecting gear (232) is located at the inner end of the side groove (221); the connecting plate (231) is fixed to the side of the camera (230); the connecting gear (232) is sleeved on the inner side of the connecting plate (231); and the servo motor is coaxially connected to the side of the connecting gear (232).

5. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 4 is characterized in that: A scraper plate (240) is arranged at the top of the connecting gear (232), and the scraper plate (240) comprises a bottom shaft (242) and a driving gear (243) coaxially connected to the bottom shaft (242), the driving gear (243) is meshingly connected to the connecting gear (232), and a cleaning component is arranged at the top of the bottom shaft (242), and when the servo motor drives the camera (230) to deflect upward, the driving gear (243) meshingly connected to the connecting gear (232) will drive the cleaning component to deflect toward a position close to the camera (230).

6. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 5, characterized in that: The cleaning assembly comprises a pair of clamping plates, the two clamping plates are respectively arranged at the two ends of the bottom shaft (242), and the top ends of the two clamping plates are connected to arc-shaped scrapers, and a plurality of burrs (241) are arranged in an array on the inner side of the arc-shaped scrapers.

7. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 6, characterized in that: The top side of the side plate (220) is connected to a connecting frame (210), and a fixed arc plate (250) is obliquely arranged at the edge of one side of the connecting frame (210). A placement groove (251) is provided on the inner side of the fixed arc plate (250), and the cross-sectional size of the placement groove (251) is consistent with the cross-sectional size of the arc-shaped scraper, and a plurality of thorns (252) are arranged in an array on both sides of the inner end of the placement groove (251).

8. The dynamic and precise positioning hummingbird drone based on multi-view fusion according to claim 7, characterized in that: Partition plates are provided on both sides of the bottom end of the side plate (220), and a gap is reserved between the two partition plates for the camera (230) to freely shuttle.

Images