Real-time splicing method for aerial photography images of unmanned aerial vehicle

Abstract

The invention relates to the technical field of image processing, and discloses a real-time splicing method for aerial images of unmanned aerial vehicles, which comprises the following steps: acquiring aerial images of all unmanned aerial vehicles, positioning data and monitoring data of various environmental factors, classifying the aerial images and the positioning data to form a data set, and splicing the same horizontal line and the same horizontal line by a real-time splicing module according to an equal interval, the linear distance between two unmanned aerial vehicles farthest in the east-west direction is partitioned, the linear distance between two unmanned aerial vehicles farthest in the south-north direction in the same horizontal line is partitioned, then the partitions form a basic network, the method is suitable for real-time changing flight formation, real-time splicing and networking are more accurate, and a real-time splicing module performs real-time splicing and networking according to an interference data set. Potential environmental interference is monitored in real time, then aerial images are spliced, the interference degree of environmental factors on the aerial images is judged, corresponding correction measures are taken, flexible correction is achieved, and the adaptability to the environment is good.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a real-time stitching method for unmanned aerial vehicle aerial photography images. Background Art

[0002] Drone photography has the advantages of high efficiency, flexibility and low cost, and is widely used in surveying and mapping, exploration, emergency and disaster relief. Drone aerial photogrammetry technology has greatly reduced the cost of traditional aerial photogrammetry. Drones are usually equipped with small cameras and sensors, which can achieve high-resolution imaging effects. They can also flexibly adjust the flight altitude and path according to the terrain and measurement requirements, prepare and deploy in a shorter time, and make flight time and mission planning more flexible. The high-resolution images taken by drones can automatically extract the features of objects through software, and perform more accurate three-dimensional modeling and measurement, reducing the need for a large number of ground control points in traditional methods. Drone photogrammetry is not only suitable for large-scale geographic surveying tasks, but can also be widely used in small and medium-sized projects, such as farmland monitoring, building measurement, mining area exploration, etc. Because drones have a low flight altitude and flexible maneuverability, they can perform refined ground collection. Many areas that cannot be efficiently covered by traditional aerial surveys can be easily completed by drones.

[0003] At present, the traditional UAV aerial photography image stitching method needs to wait for the aerial photography mission to be completed before post-processing, which cannot meet the real-time image stitching needs. In addition, in the case of high wind speed, strong light, low visibility and abnormal temperature, it is difficult to guarantee the stitching quality of aerial images and the operating efficiency of the flight formation, and the environmental adaptability is poor. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a real-time stitching method for UAV aerial photography images, which has the advantages of more accurate real-time stitching networking, flexible correction and good environmental adaptability, and solves the problem that traditional UAV aerial photography image stitching methods cannot meet real-time requirements and have poor environmental adaptability.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] According to a first aspect of the present invention, a method for real-time stitching of drone aerial photography images is provided. The method comprises:

[0009] The image dataset is collected by connecting drones through the network. The image dataset includes the aerial images of all drones. The expression of the image dataset is {Y1 s 、Y2 s 、Y3s 、...、YN s}, Y1 s To YN s are the aerial images of the first to the Nth drones, respectively, and s represents the specific time of obtaining a single drone aerial image; the drones are connected through the network to collect the positioning data set, and the positioning data set includes the positioning data of all drones. The expression of the positioning data set is {D1 m 、D2 m 、D3 m ,...,DN m}, D1 m To DN m are the positioning data of the first to the Nth UAV, respectively. The positioning data is the GPS coordinates. m represents the specific time of obtaining the positioning data of a single UAV. The environmental data set is collected through the network connection big data platform. The environmental data set includes monitoring data of various environmental factors. The expression of the environmental data set is {FS t , GZ t , NJ t , WD t}, FS t Indicates the wind speed value of the drone’s environment, GZ t Indicates the light intensity of the drone’s environment, NJ t Indicates the visibility of the drone’s environment, WD t represents the temperature of the environment where the drone is located, and t represents the specific time of obtaining environmental factors;

[0010] According to the positioning data set, the corresponding basic network Jcw is constructed and generated; through the fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY, combined with the environmental data set, the corresponding interference data group Grsj is analyzed and generated;

[0011] According to the image data set and the basic network Jcw, all the aerial images of the UAV are stitched together in real time to generate the corresponding stitching data group Pjsj. Then, combined with the interference data group Grsj, the interference degree of various environmental factors on the aerial images of the UAV is judged, and corresponding correction measures are taken.

[0012] Further: The construction process of the basic network Jcw is as follows:

[0013] According to the positioning data set, the positioning data of the two drones with the farthest distance in the east-west direction on the same horizontal line are selected and marked as Dd m (x d ,y d ) and Dx m (x x ,y x ), where Ddm (x d ,y d ) represents the GPS coordinates of the easternmost drone on the same horizontal line, Dx m (x x ,y x ) represents the GPS coordinates of the westernmost drone on the same horizontal line, and then the positioning data of the two drones with the farthest distance in the north-south direction on the same horizontal line are selected and marked as Dn m (x n ,y n ) and Db m (x b ,y b ), where Dn m (x n ,y n ) represents the GPS coordinates of the southernmost drone on the same horizontal line, Db m (x b ,y b ) represents the GPS coordinates of the northernmost drone on the same horizontal line, Dd m (x d ,y d ), Dx m (x x ,y x )、Dn m (x n ,y n ) and Db m (x b ,y b ), the specific time m for obtaining the drone positioning data is the same;

[0014] Calculate the straight-line distance A between the two UAVs that are farthest apart in the east-west direction on the same horizontal line. The calculation formula is as follows:

[0015]

[0016] Calculate the straight-line distance B between the two UAVs that are farthest apart in the north-south direction on the same horizontal line. The calculation formula is as follows:

[0017]

[0018] Based on the same horizontal line, the straight-line distance A between the two UAVs farthest from each other in the east-west direction and the straight-line distance B between the two UAVs farthest from each other in the north-south direction, the basic network Jcw is constructed, and its expression is as follows:

[0019]

[0020] In the formula, c represents the equal spacing interval, and according to the equal spacing interval, the straight-line distance A between the two UAVs farthest apart in the east-west direction on the same horizontal line is divided into The straight-line distance B between the two UAVs with the longest distance in the north-south direction on the same horizontal line is divided into Segment separator, Indicated by Segment separation and The segments are separated into virtual grids on the same horizontal line, which is the basic network Jcw.

[0021] Further: the calculation process of the interference data group Grsj is as follows:

[0022]

[0023] In the formula, maxFY represents the maximum value of the wind speed threshold, minFY represents the minimum value of the wind speed threshold, maxFY ≥ FS t ≥minFY means comparing the wind speed value of the drone's environment with the wind speed threshold, maxGY means the maximum value of the light threshold, minGY means the minimum value of the light threshold, maxGY≥GZ t ≥minGY means comparing the light intensity of the drone's environment with the light threshold, maxNY means the maximum value of the visibility threshold, minNY means the minimum value of the visibility threshold, maxNY≥NJ t ≥minNY means comparing the visibility of the drone's environment with the visibility threshold, maxWY means the maximum value of the temperature threshold, minWY means the minimum value of the temperature threshold, maxWY≥WD t ≥minWY means comparing the temperature of the drone's environment to the temperature threshold.

[0024] Further: the calculation process of the spliced ​​data set Pjsj is as follows:

[0025] Extract the aerial image of the i-th UAV in the image dataset, and mark the area of ​​the i-th UAV aerial image as YM i ;

[0026] Calculate the coverage area FM based on the basic network Jcw Jcw , and its calculation formula is as follows:

[0027] FM Jcw =A×B

[0028] According to the screen area YM of the i-th UAV aerial image i and coverage area FM Jcw , calculate the scaling ratio BL, the calculation formula is as follows:

[0029]

[0030] According to the scaling ratio BL, all the aerial images of the UAV are stitched together to generate the corresponding stitching data set Pjsj, which is calculated as follows:

[0031]

[0032] In the formula, BL×YM i Indicates that the area of ​​the i-th drone aerial image is enlarged according to the scaling ratio. It means that the picture area of ​​all drone aerial images is enlarged according to the zoom ratio, which is the stitching data group.

[0033] Further: In the interference data group Grsj, the wind speed value FS of the environment where the drone is located t When the wind speed threshold FY is exceeded, it indicates that the UAV flight is unstable and the image clarity of the spliced ​​data set Pjsj is unstable, so the UAV's flight altitude is reduced and the flight speed is slowed down.

[0034] Further: In the interference data group Grsj, the light intensity GZ of the environment where the drone is located t When the illumination threshold GY is exceeded, it indicates that the drone lens has a glare problem, and the image content of the spliced ​​data group Pjsj is missing. The exposure value of the drone aerial image is reduced and the aerial photography angle is adjusted. In the interference data group Grsj, the visibility NJ of the drone environment is t When the visibility is lower than the visibility threshold NY, it means that the image resolution of the spliced ​​data set Pjsj decreases, the contrast of the drone aerial image is enhanced and the flight altitude is reduced.

[0035] Further: In the interference data group Grsj, the temperature WD of the environment where the drone is located t When the temperature threshold WY is exceeded, it indicates that the drone lens has a defocus problem, the image clarity of the stitched data group Pjsj is unstable, and the focal length of the drone lens is reduced and the lens aperture is increased.

[0036] According to a second aspect of the present invention, a real-time stitching system for drone aerial photography is provided. The system comprises:

[0037] Data acquisition module and real-time splicing module;

[0038] The data acquisition module is composed of an image data unit, a positioning data unit and an environment data unit. The image data unit is connected to a drone through a network to collect an image data set, and the image data set includes the aerial images of all drones. The positioning data unit is connected to a drone through a network to collect a positioning data set, and the positioning data set includes the positioning data of all drones. The environment data unit is connected to a big data platform through a network to collect an environment data set, and the environment data set includes monitoring data of various environmental factors.

[0039] The real-time stitching module consists of a formation analysis unit, an environmental analysis unit and a stitching management unit. The formation analysis unit constructs and generates a corresponding basic network Jcw based on the positioning data set. The environmental analysis unit is provided with a fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY, and then combines the environmental data set to analyze and generate a corresponding interference data group Grsj. The stitching management unit stitches the aerial images of all drones in real time based on the image data set and the basic network Jcw to generate a corresponding stitching data group Pjsj, and then combines the interference data group Grsj to determine the degree of interference of various environmental factors on the drone aerial images, and take corresponding correction measures.

[0040] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the method is implemented when the processor executes the program.

[0041] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method when executed by a processor.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention connects the drones and the big data platform through a data acquisition module network, obtains the aerial images of all drones, the positioning data of all drones and the monitoring data of various environmental factors, and classifies them into image data sets, positioning data sets and environmental data sets. The real-time stitching module divides the straight-line distance between the two drones farthest in the east-west direction on the same horizontal line into partitions according to the positioning data sets, and divides the straight-line distance between the two drones farthest in the north-south direction on the same horizontal line into partitions at equal intervals, and then forms a virtual grid on the same horizontal line with the partitions, constructs and generates the corresponding basic network Jcw, which is suitable for real-time changing flight formations and quickly determines the collaborative relationship between the drones, thereby improving the efficiency and accuracy of image stitching, and making the real-time stitching network more accurate.

[0044] 2. The present invention sets a fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY through a real-time stitching module, and then combines the environmental data set to analyze and generate a corresponding interference data group Grsj, monitors potential environmental interference that affects the quality of aerial photography in real time, and then stitches the aerial images of all drones in real time to generate a corresponding stitching data group Pjsj. If the aerial images of a single drone in the basic network Jcw cannot provide valid data, the scaling ratio of adjacent drones can be adjusted in a targeted manner, and the invalid aerial images can be replaced with the aerial images of adjacent drones to ensure seamless connection of panoramic images and improve image continuity. The real-time stitching module determines the degree of interference of various environmental factors on the drone aerial images based on the interference data group Grsj, and performs corresponding correction measures to avoid stitching errors caused by factors such as wind speed, light, visibility, etc., to ensure that high-quality stitching results can be obtained in complex environments, and the ability to flexibly correct the environment is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart of a method for real-time stitching of drone aerial photography images according to an embodiment of the present invention is shown;

[0046] Figure 2 A block diagram of a real-time stitching system for drone aerial photography images according to an embodiment of the present invention is shown;

[0047] Figure 3 A block diagram of an exemplary electronic device capable of implementing embodiments of the present invention is shown. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Since traditional drone aerial photography image stitching methods require post-processing after the aerial photography mission is completed, they cannot meet the real-time image stitching needs. In addition, in conditions of high wind speed, strong light, low visibility and abnormal temperature, it is difficult to guarantee the stitching quality of aerial images and the operating efficiency of the flight formation, and the environmental adaptability is poor.

[0050] See also Figure 1 , a real-time stitching method for unmanned aerial photography images, comprising:

[0051] S101, collecting an image dataset through a network connection with a drone, wherein the image dataset includes aerial images of all drones, and the expression of the image dataset is {Y1s 、Y2 s 、Y3 s 、...、YN s}, Y1 s To YN s are the aerial images of the first to the Nth drones, respectively, and s represents the specific time of obtaining a single drone aerial image; the drones are connected through the network to collect the positioning data set, and the positioning data set includes the positioning data of all drones. The expression of the positioning data set is {D1 m 、D2 m 、D3 m ,...,DN m}, D1 m To DN m are the positioning data of the first to the Nth UAV, respectively. The positioning data is the GPS coordinates. m represents the specific time of obtaining the positioning data of a single UAV. The environmental data set is collected through the network connection big data platform. The environmental data set includes monitoring data of various environmental factors. The expression of the environmental data set is {FS t , GZ t , NJ t , WD t}, FS t Indicates the wind speed value of the drone’s environment, GZ t Indicates the light intensity of the drone’s environment, NJ t Indicates the visibility of the drone’s environment, WD t represents the temperature of the environment where the drone is located, and t represents the specific time of obtaining environmental factors;

[0052] S102, construct and generate a corresponding basic network Jcw according to the positioning data set; analyze and generate a corresponding interference data group Grsj by combining a fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY with the environmental data set;

[0053] S103, based on the image data set and the basic network Jcw, all the drone aerial images are stitched together in real time to generate the corresponding stitching data group Pjsj, and then combined with the interference data group Grsj, the interference degree of various environmental factors on the drone aerial images is determined, and corresponding correction measures are taken.

[0054] See also Figure 2 , a real-time stitching system for drone aerial photography images, including a data acquisition module and a real-time stitching module;

[0055] The data acquisition module consists of an image data unit, a positioning data unit, and an environmental data unit. The image data unit connects to the drone through the network to collect image data sets. The image data set includes all the aerial images of the drone. The expression of the image data set is {Y1 s 、Y2 s 、Y3 s 、...、YN s}, Y1 s To YN s are the aerial images of the first to the Nth drones, respectively. s represents the specific time of obtaining a single drone aerial image. According to the unified time axis, the aerial image data is collected in real time and continuously, which effectively solves the problem of data delay and ensures the timeliness and consistency of the images in the subsequent stitching process.

[0056] The positioning data unit collects positioning data sets through network connection to UAVs. The positioning data sets include the positioning data of all UAVs. The expression of positioning data sets is {D1 m 、D2 m 、D3 m ,...,DN m}, D1 m To DN m They are the positioning data of the first to the Nth drones, and the positioning data is the GPS coordinates. m represents the specific time of obtaining the positioning data of a single drone, which ensures the synchronous collection of the positioning data of all drones, provides basic data for the subsequent construction of the basic network Jcw, and can splice out a more realistic flight trajectory;

[0057] The environmental data unit collects environmental data sets through the network connection big data platform. The environmental data sets include monitoring data of various environmental factors. The expression of the environmental data set is {FS t , GZ t , NJ t , WD t}, FS t Indicates the wind speed value of the drone’s environment, GZ t Indicates the light intensity of the drone’s environment, NJ t Indicates the visibility of the drone’s environment, WD t It indicates the temperature of the environment where the drone is located, and t indicates the specific time of obtaining environmental factors, so as to grasp the environmental conditions during flight in real time, which is convenient for subsequent interference analysis and correction;

[0058] The real-time splicing module consists of a formation analysis unit, an environment analysis unit, and a splicing management unit. The formation analysis unit generates the corresponding basic network Jcw based on the positioning data set. The construction process is as follows:

[0059] S11. Based on the positioning data set, the positioning data of the two UAVs with the farthest distance in the east-west direction on the same horizontal line are selected and marked as Dd m (x d ,y d ) and Dx m (x x ,y x ), where Dd m (x d ,y d ) represents the GPS coordinates of the easternmost drone on the same horizontal line, Dx m (x x ,y x ) represents the GPS coordinates of the westernmost drone on the same horizontal line, and then the positioning data of the two drones with the farthest distance in the north-south direction on the same horizontal line are selected and marked as Dn m (x n ,y n ) and Db m (x b ,y b ), where Dn m (x n ,y n ) represents the GPS coordinates of the southernmost drone on the same horizontal line, Db m (x b ,y b ) represents the GPS coordinates of the northernmost drone on the same horizontal line, Dd m (x d ,y d ), Dx m (x x ,y x )、Dn m (x n ,y n ) and Db m (x b ,y b ), the specific time m for obtaining the drone positioning data is the same;

[0060] S12. Calculate the straight-line distance A between the two UAVs that are farthest apart in the east-west direction on the same horizontal line. The calculation formula is as follows:

[0061]

[0062] S13. Calculate the straight-line distance B between the two UAVs that are farthest apart in the north-south direction on the same horizontal line. The calculation formula is as follows:

[0063]

[0064] S14. Based on the straight-line distance A between the two UAVs farthest from each other in the east-west direction and the straight-line distance B between the two UAVs farthest from each other in the north-south direction on the same horizontal line, construct the basic network Jcw, which is expressed as follows:

[0065]

[0066] In the formula, c represents the equal spacing interval, and according to the equal spacing interval, the straight-line distance A between the two UAVs farthest apart in the east-west direction on the same horizontal line is divided into The straight-line distance B between the two UAVs with the longest distance in the north-south direction on the same horizontal line is divided into Segment separator, Indicated by Segment separation and The virtual grid of the same horizontal line is formed by segment separation, which is the basic network Jcw. It is suitable for real-time changing flight formations and quickly judges the coordination relationship between drones, thereby improving the efficiency and accuracy of image stitching, and making real-time stitching networking more accurate;

[0067] The environmental analysis unit is set with a fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY, and then combined with the environmental data set to analyze and generate the corresponding interference data group Grsj. The calculation process is as follows:

[0068]

[0069] In the formula, maxFY represents the maximum value of the wind speed threshold, minFY represents the minimum value of the wind speed threshold, maxFY ≥ FS t ≥minFY means comparing the wind speed value of the drone's environment with the wind speed threshold, maxGY means the maximum value of the light threshold, minGY means the minimum value of the light threshold, maxGY≥GZ t ≥minGY means comparing the light intensity of the drone's environment with the light threshold, maxNY means the maximum value of the visibility threshold, minNY means the minimum value of the visibility threshold, maxNY≥NJ t ≥minNY means comparing the visibility of the drone's environment with the visibility threshold, maxWY means the maximum value of the temperature threshold, minWY means the minimum value of the temperature threshold, maxWY≥WD t ≥minWY means comparing the temperature of the drone's environment to the temperature threshold, and monitoring the potential environmental interference that affects the quality of aerial photography in real time, which helps to adjust the image acquisition and stitching strategies in real time later;

[0070] The stitching management unit stitches all the aerial images of the drones in real time according to the image data set and the basic network Jcw to generate the corresponding stitching data set Pjsj. The calculation process is as follows:

[0071] S21, extract the aerial image of the i-th UAV in the image dataset, and mark the area of ​​the i-th UAV aerial image as YM i ;

[0072] S22. Calculate the coverage area FM based on the basic network Jcw Jcw , and its calculation formula is as follows:

[0073] FM Jcw =A×B

[0074] S23, according to the screen area YM of the i-th drone aerial image i and coverage area FM Jcw , calculate the scaling ratio BL, the calculation formula is as follows:

[0075]

[0076] S24, stitching all the aerial images of the drones according to the scaling ratio BL, and generating a corresponding stitching data set Pjsj, the calculation formula of which is as follows:

[0077]

[0078] In the formula, BL×YM i Indicates that the area of ​​the i-th drone aerial image is enlarged according to the scaling ratio. Indicates that the screen area of ​​all drone aerial images is enlarged according to the zoom ratio, which is the stitching data group. In actual use, if the aerial image of a single drone in the basic network Jcw cannot provide valid data, the zoom ratio of adjacent drones can be adjusted specifically, and the invalid aerial image can be replaced with the aerial image of the adjacent drone to ensure the seamless connection of the panoramic image and improve the coherence of the image;

[0079] The stitching management unit determines the interference degree of various environmental factors on the drone aerial images according to the interference data group Grsj, and takes corresponding correction measures. In the interference data group Grsj, the wind speed value FS of the environment where the drone is located t When the wind speed threshold FY is exceeded, it means that the drone is unstable in flight, the image clarity of the spliced ​​data set Pjsj is unstable, the drone's flight altitude is reduced and the flight speed is slowed down, interfering with the light intensity GZ of the drone's environment in the data set Grsj. tWhen the illumination threshold GY is exceeded, it indicates that the drone lens has a glare problem, and the image content of the spliced ​​data group Pjsj is missing. The exposure value of the drone aerial image is reduced and the aerial photography angle is adjusted, which interferes with the visibility NJ of the drone environment in the data group Grsj. t When the visibility is lower than the visibility threshold NY, it means that the image resolution of the spliced ​​data set Pjsj decreases, the contrast of the drone aerial image is enhanced and the flight altitude is reduced, which interferes with the temperature WD of the drone environment in the data set Grsj. t When the temperature threshold WY is exceeded, it indicates that the drone lens is out of focus and the image clarity of the stitched data set Pjsj is unstable. The focal length of the drone lens is reduced and the lens aperture is increased to avoid stitching errors caused by factors such as wind speed, light, and visibility. This ensures that high-quality stitching results can be obtained in complex environments, and the ability to flexibly correct environmental adaptability is good.

[0080] The invention also provides an electronic device and a readable storage medium.

[0081] Figure 3 A schematic block diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0082] The electronic device includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM 302 or a computer program loaded from a storage unit 308 into a RAM 303. In the RAM 303, various programs and data required for the operation of the electronic device can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0083] A number of components in the electronic device are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0084] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the real-time stitching method of drone aerial photography images. For example, in some embodiments, the real-time stitching method of drone aerial photography images may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 302 and / or a communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the real-time stitching method of drone aerial photography images described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the real-time stitching method of drone aerial photography images in any other appropriate manner (for example, by means of firmware).

[0085] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of the present invention, a readable storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A readable storage medium may be a machine-readable signal medium or a machine-readable storage medium. A readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. More specific examples of readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0089] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0090] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time stitching method for drone aerial photography, characterized in that: include: The image dataset is collected by connecting drones through the network. The image dataset includes the aerial images of all drones. The expression of the image dataset is {Y1 s 、Y2 s 、Y3 s 、...、YN s }, Y1 s To YN s are the aerial images of the first to the Nth drones, respectively, and s represents the specific time of obtaining a single drone aerial image; The positioning data set is collected by connecting the drones through the network. The positioning data set includes the positioning data of all drones. The expression of the positioning data set is {D1 m 、D2 m 、D3 m ,...,DN m }, D1 m To DN m are the positioning data of the first to the Nth UAV, respectively. The positioning data is the GPS coordinates. m represents the specific time of obtaining the positioning data of a single UAV. The environmental data set is collected through the network connection big data platform. The environmental data set includes monitoring data of various environmental factors. The expression of the environmental data set is {FS t , GZ t , NJ t , WD t }, FS t Indicates the wind speed value of the drone’s environment, GZ t Indicates the light intensity of the drone’s environment, NJ t Indicates the visibility of the drone’s environment, WD t represents the temperature of the environment where the drone is located, and t represents the specific time of obtaining environmental factors; According to the positioning data set, the corresponding basic network Jcw is constructed and generated; through the fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY, combined with the environmental data set, the corresponding interference data group Grsj is analyzed and generated; According to the image data set and the basic network Jcw, all the aerial images of the UAV are stitched together in real time to generate the corresponding stitching data group Pjsj. Then, combined with the interference data group Grsj, the interference degree of various environmental factors on the aerial images of the UAV is judged, and corresponding correction measures are taken.

2. The real-time stitching method of drone aerial photography according to claim 1, characterized in that: The construction process of the basic network Jcw is as follows: According to the positioning data set, the positioning data of the two drones with the farthest distance in the east-west direction on the same horizontal line are selected and marked as Dd m (x d ,y d ) and Dx m (x x ,y x ), where Dd m (x d ,y d ) represents the GPS coordinates of the easternmost drone on the same horizontal line, Dx m (x x ,y x ) represents the GPS coordinates of the westernmost drone on the same horizontal line, and then the positioning data of the two drones with the farthest distance in the north-south direction on the same horizontal line are selected and marked as Dn m (x n ,y n ) and Db m (x b ,y b ), where Dn m (x n ,y n ) represents the GPS coordinates of the southernmost drone on the same horizontal line, Db m (x b ,y b ) represents the GPS coordinates of the northernmost drone on the same horizontal line, Dd m (x d ,y d ), Dx m (x x ,y x )、Dn m (x n ,y n ) and Db m (x b ,y b ), the specific time m for obtaining the drone positioning data is the same; Calculate the straight-line distance A between the two UAVs that are farthest apart in the east-west direction on the same horizontal line. The calculation formula is as follows: Calculate the straight-line distance B between the two UAVs that are farthest apart in the north-south direction on the same horizontal line. The calculation formula is as follows: Based on the same horizontal line, the straight-line distance A between the two UAVs farthest from each other in the east-west direction and the straight-line distance B between the two UAVs farthest from each other in the north-south direction, the basic network Jcw is constructed, and its expression is as follows: In the formula, c represents the equal spacing interval, and according to the equal spacing interval, the straight-line distance A between the two UAVs farthest apart in the east-west direction on the same horizontal line is divided into The straight-line distance B between the two UAVs with the longest distance in the north-south direction on the same horizontal line is divided into Segment separator, Indicated by Segment separation and The segments are separated into virtual grids on the same horizontal line, which is the basic network Jcw.

3. The real-time stitching method of drone aerial photography according to claim 2, characterized in that: The calculation process of the interference data group Grsj is as follows: In the formula, maxFY represents the maximum value of the wind speed threshold, minFY represents the minimum value of the wind speed threshold, maxFY ≥ FS t ≥minFY means comparing the wind speed value of the drone's environment with the wind speed threshold, maxGY means the maximum value of the light threshold, minGY means the minimum value of the light threshold, manGY≥GZ t ≥minGY means comparing the light intensity of the drone's environment with the light threshold, maxNY means the maximum value of the visibility threshold, minNY means the minimum value of the visibility threshold, maxNY≥NJ t ≥minNY means comparing the visibility of the drone's environment with the visibility threshold, maxWY means the maximum value of the temperature threshold, minWY means the minimum value of the temperature threshold, maxWY≥WD t ≥minWY means comparing the temperature of the drone's environment to the temperature threshold.

4. The real-time stitching method of drone aerial photography according to claim 3 is characterized by: The calculation process of the spliced ​​data set Pjsj is as follows: Extract the aerial image of the i-th UAV in the image dataset, and mark the area of ​​the i-th UAV aerial image as YM i ; Calculate the coverage area FM based on the basic network Jcw Jcw , and its calculation formula is as follows: FM Jcw =A×B According to the screen area YM of the i-th UAV aerial image i and coverage area FM jcw , calculate the scaling ratio BL, the calculation formula is as follows: According to the scaling ratio BL, all the aerial images of the UAV are stitched together to generate the corresponding stitching data set Pjsj, which is calculated as follows: In the formula, BL×YM i Indicates that the area of ​​the i-th drone aerial image is enlarged according to the scaling ratio. It means that the picture area of ​​all drone aerial images is enlarged according to the zoom ratio, which is the stitching data group.

5. The real-time stitching method of drone aerial photography according to claim 4, characterized in that: In the interference data group Grsj, the wind speed value FS of the environment where the drone is located t When the wind speed threshold FY is exceeded, it indicates that the UAV flight is unstable and the image clarity of the spliced ​​data set Pjsj is unstable, so the UAV's flight altitude is reduced and the flight speed is slowed down.

6. The real-time stitching method of drone aerial photography according to claim 5, characterized in that: In the interference data group Grsj, the light intensity GZ of the environment where the drone is located t When the illumination threshold GY is exceeded, it indicates that the drone lens has a glare problem, and the image content of the spliced ​​data group Pjsj is missing. The exposure value of the drone aerial image is reduced and the aerial photography angle is adjusted. In the interference data group Grsj, the visibility NJ of the drone environment is t When the visibility is lower than the visibility threshold NY, it means that the image resolution of the spliced ​​data set Pjsj decreases, the contrast of the drone aerial image is enhanced and the flight altitude is reduced.

7. The real-time stitching method of drone aerial photography according to claim 6, characterized in that: In the interference data group Grsj, the temperature WD of the environment where the drone is located t When the temperature threshold WY is exceeded, it indicates that the drone lens has a defocus problem, the image clarity of the stitched data group Pjsj is unstable, and the focal length of the drone lens is reduced and the lens aperture is increased.

8. A real-time stitching system for drone aerial photography, characterized in that: include: Data acquisition module and real-time splicing module; The data acquisition module is composed of an image data unit, a positioning data unit and an environment data unit. The image data unit is connected to a drone through a network to collect an image data set, and the image data set includes the aerial images of all drones. The positioning data unit is connected to a drone through a network to collect a positioning data set, and the positioning data set includes the positioning data of all drones. The environment data unit is connected to a big data platform through a network to collect an environment data set, and the environment data set includes monitoring data of various environmental factors. The real-time stitching module consists of a formation analysis unit, an environmental analysis unit and a stitching management unit. The formation analysis unit constructs and generates a corresponding basic network Jcw based on the positioning data set. The environmental analysis unit is provided with a fixed range of wind speed threshold FY, light threshold GY, visibility threshold NY and temperature threshold WY, and then combines the environmental data set to analyze and generate a corresponding interference data group Grsj. The stitching management unit stitches the aerial images of all drones in real time based on the image data set and the basic network Jcw to generate a corresponding stitching data group Pjsj, and then combines the interference data group Grsj to determine the degree of interference of various environmental factors on the drone aerial images, and take corresponding correction measures.

9. An electronic device, characterized in that: include: at least one processor; A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images