Field panorama synthesis method and system based on unmanned aerial vehicle low-altitude cruise shot video and OpenCV + GAN model

Abstract

The invention discloses a field panorama synthesis method based on a low-altitude cruise shot video of an unmanned aerial vehicle and an OpenCV + GAN model. According to the method, a construction site video is shot through an unmanned aerial vehicle, pictures are read and stored frame by frame by using OpenCV, the pictures are used as target domain pictures for CycleGAN neural network training and spliced into a panorama with low quality, the panorama is segmented into a plurality of pictures through OpenCV to serve as source domain pictures, and the source domain pictures and the target domain pictures are used for training the CycleGAN neural network. After training is completed, in practical application, the unmanned aerial vehicle performs low-altitude cruise shooting of a shot video, synthesizes a panorama with poor quality by using OpenCV, performs panorama partitioning, inputs a trained CycleGAN neural network to generate high-quality panorama partitioning, and finally generates a high-quality construction site panorama through OpenCV edge splicing. The feasible method for generating the high-quality panorama through the low-altitude cruise video of the unmanned aerial vehicle is achieved, and the method is suitable for the fields of building construction, engineering supervision, safety monitoring and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing and image synthesis of low-altitude cruise shooting by unmanned aerial vehicles, and in particular to a construction site panoramic image synthesis method and system based on low-altitude cruise shooting video of unmanned aerial vehicles and an OpenCV+GAN model. Background Art

[0002] In the fields of construction, engineering supervision, and safety monitoring, high-quality panoramic images of construction sites are of great significance for project management and decision-making. Traditional panoramic image synthesis methods usually rely on ground-based shooting equipment, which has problems such as limited viewing angles and difficulty in shooting. In recent years, with the development of drone technology, low-altitude cruising shooting by drones has become an effective means of obtaining panoramic images of construction sites. However, the quality of videos and images taken by drones is unstable, and the stitching error is large, which directly affects the quality of the final generated panorama. Summary of the invention

[0003] In order to solve the problem that the quality of videos and images taken by existing drones is unstable, resulting in large stitching errors and affecting the quality of the final panorama, the present invention provides a construction site panorama synthesis method based on drone low-altitude cruising video and OpenCV+GAN model. Through the combination of OpenCV+GAN model, the quality of the panorama is significantly improved, the stitching error is reduced, and more accurate and reliable visual information is provided for project management.

[0004] According to one aspect of the present invention, a method for synthesizing a panoramic image of a scene based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model is provided, comprising: Obtain construction site videos shot by drones at low altitudes; Preprocess the acquired video data based on OpenCV to obtain preprocessed image blocks; The pre-processed image blocks are input into the trained GAN neural network model to output the construction site panoramic image blocks; wherein the training of the GAN neural network model includes: Get construction site videos shot by drones; Use OpenCV to process the video and generate several construction site images as target domain images; splicing the target domain images to form a preliminary panoramic image of the construction site, and segmenting the preliminary panoramic image to form a source domain image; A training set is constructed according to the target domain images and the source domain images, the GNA neural network model is trained, and the trained GNA neural network model is output.

[0005] As a further technical solution, the acquired video data is preprocessed based on OpenCV, including: Based on OpenCV, the construction site video is read frame by frame and saved to obtain several construction site pictures; Perform OpenCV stitching on the saved pictures of the construction site to form a preliminary panoramic picture of the construction site; The preliminary panoramic image of the construction site is segmented using OpenCV to form preliminary panoramic image blocks of the construction site.

[0006] As a further technical solution, OpenCV segmentation is performed on the preliminary construction site panorama, further comprising: The size of the segmented panorama blocks is made the same as the size of the source domain images during GAN neural network model training.

[0007] As a further technical solution, when OpenCV stitching is performed on the saved construction site pictures, it also includes: The stitching algorithm based on feature point matching is used for OpenCV stitching to form a preliminary panoramic view of the construction site.

[0008] As a further technical solution, when performing OpenCV segmentation on the preliminary construction site panoramic image, it also includes: Use uniform segmentation or content-based intelligent segmentation algorithm for OpenCV segmentation.

[0009] As a further technical solution, after obtaining the construction site panoramic image blocks output by the GAN neural network model, it also includes: Edge detection and alignment techniques are used to perform OpenCV edge stitching to form the final panoramic view of the construction site.

[0010] As a further technical solution, after the final panoramic view of the construction site is formed, it also includes: The final panoramic view of the construction site is transmitted and stored in real time.

[0011] As a further technical solution, the construction site video captured by the drone at low altitude cruising is also available: Get the geographic location information of the drone when shooting the video.

[0012] As a further technical solution, the method further includes: The final construction site panorama is evaluated for quality, including resolution, color reproduction and / or detail clarity.

[0013] According to one aspect of the present invention, a system for synthesizing a panoramic image of a scene based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model is provided, comprising: The data acquisition module is used to obtain the construction site video taken by the drone at low altitude; The preprocessing module is used to preprocess the acquired video data based on OpenCV to obtain preprocessed image blocks; A panoramic image processing module is used to input the pre-processed image blocks into the trained GAN neural network model and output the construction site panoramic image blocks; wherein the training of the GAN neural network model includes: Get construction site videos shot by drones; Use OpenCV to process the video and generate several construction site images as target domain images; splicing the target domain images to form a preliminary panoramic image of the construction site, and segmenting the preliminary panoramic image to form a source domain image; A training set is constructed according to the target domain images and the source domain images, the GNA neural network model is trained, and the trained GNA neural network model is output.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses OpenCV to read, splice and segment the video shot by a drone at low altitude cruising, obtains construction site panoramic image blocks with poor quality, and then inputs the construction site panoramic image blocks with poor quality into a trained GAN neural network model to obtain construction site panoramic image blocks with excellent quality, and then obtains a construction site panoramic image with excellent quality based on OpenCV edge stitching, which solves the problems of unstable quality and large stitching error of drone-shot videos in the prior art, improves the quality of the panorama, reduces the stitching error, and provides more accurate and reliable visual information for fields such as building construction, project supervision and safety monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction is given below to the drawings used in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 It is a flow chart of a method for synthesizing a panoramic image of a construction site based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model according to an embodiment of the present invention; Figure 2This is a schematic diagram of some pictures exported from a video shot by a drone at low altitude cruising according to an embodiment of the present invention; Figure 3 is a schematic diagram of a low-quality panoramic image stitched by OpenCV according to an embodiment of the present invention; Figure 4 is a schematic diagram of a high-quality panoramic image converted by a trained CycleGAN neural network according to an embodiment of the present invention; FIG5 is a schematic diagram showing a partial comparison between a low-quality panoramic image and a high-quality panoramic image according to an embodiment of the present invention, wherein (a) is a low-quality panoramic image and (b) is a high-quality panoramic image. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention are arbitrarily combined with each other to form a new technical solution. This combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that this combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0018] The embodiment of the present invention provides a method for synthesizing a panoramic image of a construction site based on a video shot by a low-altitude cruise drone and an OpenCV+GAN model, aiming to solve the problems of unstable quality and large stitching error of videos shot by drones in the prior art, thereby improving the quality of the panoramic image of the construction site and providing more accurate and reliable visual information for fields such as construction, project supervision and safety monitoring.

[0019] In the technical solution provided by the embodiment of the present invention, first, a video shot by a drone is obtained; then, video processing, stitching of low-quality panoramas and image segmentation are performed based on OpenCV; then, the segmented low-quality image blocks are input into a trained CycleGAN neural network to output high-quality panorama blocks; finally, a high-quality panorama is generated by edge stitching.

[0020] In the embodiment of the present invention, a drone is used to cruise at low altitude at the construction site to shoot video, and the flight altitude does not exceed 100 meters to ensure the clarity and resolution of the video. The drone is equipped with a GPS positioning system to ensure that the geographical location information of the video is accurate, which is convenient for later data processing and analysis.

[0021] After obtaining the video shot by the drone, use OpenCV to read the video frame by frame and save it as an image. The part of OpenCV used for video processing includes but is not limited to reading, saving, image stitching and image segmentation of video frames. In addition, the video shot by the drone is preprocessed, such as denoising and contrast enhancement, to improve the efficiency and quality of subsequent processing.

[0022] Furthermore, OpenCV is used to stitch the saved images into a low-quality panorama. A stitching algorithm based on feature point matching is used in the stitching process to improve the stitching accuracy and speed.

[0023] After stitching the low-quality panoramas, OpenCV is used to segment the low-quality panoramas into multiple images. Uniform segmentation or content-based intelligent segmentation algorithms are used to ensure the independence and integrity of each image.

[0024] In actual applications, drones are cruising at low altitudes to shoot videos, and OpenCV is used to synthesize poor quality panoramas, which are then divided into blocks. The block size is the same as the source domain image size for training the CycleGAN neural network to ensure the consistency of the data format input into the CycleGAN neural network.

[0025] Furthermore, the segmented multiple pictures are input into the trained CycleGAN neural network to generate high-quality panoramic image blocks. Batch processing is used in the generation process to improve the efficiency of generating high-quality panoramic image blocks.

[0026] After obtaining high-quality panorama blocks, OpenCV edge stitching is used to generate high-quality construction site panorama. Edge detection and alignment technology are used in the stitching process to ensure the accuracy and continuity of the stitching.

[0027] Preferably, the embodiment of the present invention also includes real-time transmission and storage, which is used to transmit and store the generated high-quality panoramic image in real time to facilitate remote monitoring and historical data analysis.

[0028] Preferably, the embodiment of the present invention also includes a quality assessment for performing a quality assessment on the generated high-quality panoramic image, wherein the assessment indicators include but are not limited to resolution, color reproduction, detail clarity, etc., to ensure that the ultimately generated panoramic image meets the requirements of the application scenario.

[0029] See also Figure 1 The method for synthesizing a panoramic image of a construction site based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model provided by an embodiment of the present invention comprises the following steps: S100, drone video S110, drone selection: Choose a drone with a high-definition camera and a stable flight system, such as the DJI Mavic series. The maximum flight altitude of the drone should not exceed 100 meters to ensure the clarity and resolution of the video.

[0030] S120, shooting path planning: Through the pre-set flight path, the overlap rate between adjacent paths should be greater than 60%, which is conducive to the preliminary synthesis of the panorama by OpenCV and ensures that the drone covers the entire construction site to avoid missing important areas.

[0031] S130, GPS positioning: The drone is equipped with a GPS positioning system to record the geographical location information of the video, which is convenient for later data processing and analysis. Some pictures exported from the video shot by the drone at low altitude cruising are as follows Figure 2 shown.

[0032] S200, OpenCV video processing S210, video reading and saving: Use OpenCV's cv2.VideoCapture function to read the video shot by the drone, read it frame by frame and save it as an image. The saved image format is JPEG.

[0033] S220, video preprocessing: preprocess the read video frames, including denoising, contrast enhancement, etc., to improve the efficiency and quality of subsequent processing. Preprocessing methods include Gaussian blur, histogram equalization, etc.

[0034] S300, panorama with low stitching quality S310, Image stitching algorithm: Use OpenCV's cv2.Stitcher class for image stitching. During the stitching process, a stitching algorithm based on feature point matching, such as SIFT or ORB, is used to improve the stitching accuracy and speed.

[0035] S320, stitching parameter optimization: adjust stitching parameters, such as exposure compensation, cropping area, etc., to optimize the stitching effect.

[0036] In step S300, when OpenCV is used to stitch together low-quality panoramic images, a stitching algorithm based on feature point matching is used to improve stitching accuracy and speed. Figure 3 shown.

[0037] S400, Image Segmentation S410, evenly dividing: evenly dividing the stitched panoramic image into a plurality of images, and the size of each image is the same as the size of the image saved in S210.

[0038] S420, Smart Segmentation: Intelligent segmentation is performed based on the content of the panorama to ensure that each image contains important structure and detail information.

[0039] In the step S400, when the low-quality panoramic image is segmented into multiple images through OpenCV, a uniform segmentation or content-based intelligent segmentation algorithm is used to ensure the independence and integrity of each image.

[0040] S500, training CycleGAN neural network S510, data preparation: the segmented images are used as source domain images, and the original images taken by the drone are used as target domain images. During the training process of the CycleGAN neural network, the number ratio of source domain images and target domain images is 1:1 and the image sizes are the same to ensure the training effect of the model.

[0041] S520, model training: Use the PyTorch framework to build the CycleGAN neural network model. The Adam optimizer is used in the training process. The loss functions include generative adversarial loss, cycle consistency loss, and identity mapping loss.

[0042] S530, training parameters: set parameters such as learning rate, batch size, number of training rounds, etc. to ensure the training effect of the model.

[0043] S600, video processing in action S610, video synthesis: In practical applications, drones shoot videos while cruising at low altitudes, and use OpenCV to synthesize panoramas of poor quality.

[0044] S620, panorama segmentation: evenly segment the synthesized panorama into multiple images, each image having the same size as the source domain image for training the CycleGAN neural network. When OpenCV is used for panorama segmentation in actual applications, the size of the segment is the same as the size of the source domain image for training the CycleGAN neural network to ensure the consistency of the data format input into the CycleGAN neural network.

[0045] S700, generate high quality panorama tiles S710, batch processing: input the divided panoramic image into the trained CycleGAN neural network to generate high-quality panoramic image blocks. Batch processing is used in the generation process to improve the generation efficiency.

[0046] S720, Block Optimization: Optimize the generated high-quality panorama blocks to ensure the clarity and detail information of each block.

[0047] S800, edge stitching to generate high-quality panoramas S810, edge detection: Use OpenCV's cv2.Canny function to perform edge detection and extract edge information between blocks.

[0048] S820, alignment technology: Use OpenCV's cv2.findHomography function for alignment to ensure continuity and accuracy between blocks.

[0049] S830, final stitching: The optimized high-quality panorama is divided into blocks and edge stitched to generate the final high-quality panorama. The high-quality panorama converted by the trained CycleGAN neural network is shown in the figure below. Figure 4 A partial comparison between a low-quality panorama and a high-quality panorama is shown in FIG5 .

[0050] In the process of generating high-quality construction site panoramas through OpenCV edge stitching, edge detection and alignment technology are used to ensure the accuracy and continuity of stitching. The generated high-quality panoramas can be used in the fields of building construction, project supervision and safety monitoring to improve the management level and safety of the construction site.

[0051] S900, real-time transmission and storage S910, real-time transmission: The generated high-quality panoramic images are transmitted to the central server in real time via the network for remote monitoring.

[0052] S920, storage management: Store high-quality panoramic images in the cloud or local server, supporting historical data analysis and backtracking.

[0053] S1000, Quality Assessment S1100 Evaluation indicators: Evaluate the quality of the generated high-quality panoramas. Evaluation indicators include resolution, color reproduction, detail clarity, etc.

[0054] S1200, evaluation method: Use indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index) for evaluation to ensure that the final generated panorama meets the requirements of the application scenario.

[0055] The implementation basis of each embodiment of the present invention is realized by programmed processing through a device with a processor function. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, on the basis of the above embodiments, an embodiment of the present invention provides a live panoramic image synthesis system based on a drone low-altitude cruising video and an OpenCV+GAN model, which is used to execute the live panoramic image synthesis method based on a drone low-altitude cruising video and an OpenCV+GAN model in the above method embodiment.

[0056] The system includes: a data acquisition module, which is used to acquire a construction site video shot by a drone during low-altitude cruising; a preprocessing module, which is used to preprocess the acquired video data based on OpenCV to obtain preprocessed image blocks; a panoramic image processing module, which is used to input the preprocessed image blocks into a trained GAN neural network model and output construction site panoramic image blocks; wherein the training of the GAN neural network model includes: acquiring a construction site video shot by a drone during cruising; processing the video using OpenCV to form a number of construction site images as target domain images; splicing a preliminary panoramic image of the construction site based on the target domain images, and segmenting the preliminary panoramic image to form a source domain image; constructing a training set based on the target domain images and the source domain images, training the GNA neural network model, and outputting the trained GNA neural network model.

[0057] The on-site panoramic image synthesis system based on the video shot by the low-altitude cruise of the UAV and the OpenCV+GAN model provided by the embodiment of the present invention aims at the problem that the quality of the videos and images shot by the existing UAV is unstable, resulting in large stitching errors and affecting the quality of the final panoramic image. By adopting the above-mentioned several modules and combining the OpenCV+GAN model, the quality of the panoramic image is significantly improved, the stitching error is reduced, and more accurate and reliable visual information is provided for project management.

[0058] It should be noted that the system embodiment provided by the present invention is used to implement the methods in the above method embodiment as well as the methods in other method embodiments provided by the present invention. The only difference is that the corresponding functional modules are set. The principle is basically the same as the principle of the above system embodiment provided by the present invention. As long as the technical personnel in this field refer to the specific technical solutions in other method embodiments on the basis of the above system embodiment, obtain the corresponding technical means and the technical solutions composed of these technical means by combining the technical features, and on the premise of ensuring the practicality of the technical solutions, improve the modules in the above system embodiment to obtain the corresponding system class embodiments, which are used to implement the methods in other method class embodiments. For example: Based on the content of the above system embodiment, as a preferred embodiment, in the on-site panoramic image synthesis system based on drone low-altitude cruise shooting video and OpenCV+GAN model provided in the embodiment of the present invention, the preprocessing module is also used to execute the following instructions: Based on OpenCV, the construction site video is read frame by frame and saved to obtain several construction site pictures; Perform OpenCV stitching on the saved pictures of the construction site to form a preliminary panoramic picture of the construction site; The preliminary panoramic image of the construction site is segmented using OpenCV to form preliminary panoramic image blocks of the construction site.

[0059] Based on the content of the above system embodiment, as a preferred embodiment, in the on-site panoramic image synthesis system based on drone low-altitude cruise shooting video and OpenCV+GAN model provided in the embodiment of the present invention, the preprocessing module is also used to execute the following instructions: The size of the segmented panorama blocks is made the same as the size of the source domain images during GAN neural network model training.

[0060] Based on the content of the above system embodiment, as a preferred embodiment, in the on-site panoramic image synthesis system based on drone low-altitude cruise shooting video and OpenCV+GAN model provided in the embodiment of the present invention, the preprocessing module is also used to execute the following instructions: The stitching algorithm based on feature point matching is used for OpenCV stitching to form a preliminary panoramic view of the construction site.

[0061] Based on the content of the above system embodiment, as a preferred embodiment, in the on-site panoramic image synthesis system based on drone low-altitude cruise shooting video and OpenCV+GAN model provided in the embodiment of the present invention, the preprocessing module is also used to execute the following instructions: Use uniform segmentation or content-based intelligent segmentation algorithm for OpenCV segmentation.

[0062] Based on the content of the above system embodiment, as a preferred embodiment, the on-site panoramic image synthesis system based on the low-altitude cruise shooting video of the drone and the OpenCV+GAN model provided in the embodiment of the present invention, the panoramic image processing module is also used to execute the following instructions: Edge detection and alignment techniques are used to perform OpenCV edge stitching to form the final panoramic view of the construction site.

[0063] Based on the content of the above system embodiment, as a preferred embodiment, the on-site panoramic image synthesis system based on the low-altitude cruise shooting video of the drone and the OpenCV+GAN model provided in the embodiment of the present invention also includes: The transmission and storage module is used for real-time transmission and storage of the final panoramic view of the construction site.

[0064] Based on the content of the above system embodiment, as a preferred embodiment, in the on-site panoramic image synthesis system based on drone low-altitude cruise shooting video and OpenCV+GAN model provided in the embodiment of the present invention, the data acquisition module is also used to execute the following instructions: Get the geographic location information of the drone when shooting the video.

[0065] Based on the content of the above system embodiment, as a preferred embodiment, the on-site panoramic image synthesis system based on the low-altitude cruise shooting video of the drone and the OpenCV+GAN model provided in the embodiment of the present invention also includes: The quality assessment module is used to assess the quality of the final construction site panoramic image, and the assessment indicators include resolution, color reproduction and / or detail clarity.

[0066] The system embodiments described above are merely illustrative, wherein the units described as separate components are or are not physically separated, and the components shown as units are or are not physical units, located in one place, or distributed on multiple network units. Some or all of the modules are selected according to actual conditions to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement without creative effort.

[0067] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0071] In summary, the present invention provides a method for synthesizing a panoramic view of a construction site based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model. The method uses a drone to shoot a video of the construction site, uses OpenCV to read and save the pictures frame by frame, and these pictures will be used as target domain pictures for training the CycleGAN neural network, spliced ​​into a low-quality panoramic view, and then divided into multiple pictures as source domain pictures by OpenCV, and the CycleGAN neural network is trained together with the target domain pictures. After the training is completed, in actual applications, the drone is used to shoot a video at low altitude cruising, and a panorama of poor quality is synthesized using OpenCV, and the panorama is divided into blocks, and the trained CycleGAN neural network is input to generate high-quality panoramic blocks, and finally a high-quality construction site panorama is generated by edge stitching through OpenCV. The present invention realizes a feasible method for generating high-quality panoramas through drone low-altitude cruising videos, which is suitable for fields such as construction, engineering supervision and safety monitoring.

[0072] The terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, for example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to the steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for synthesizing a panoramic image of a scene based on a video shot by a drone at low altitude cruising and an OpenCV+GAN model, characterized in that: include: Obtain construction site videos shot by drones at low altitudes; Preprocess the acquired video data based on OpenCV to obtain preprocessed image blocks; The pre-processed image blocks are input into the trained GAN neural network model to output the construction site panoramic image blocks; wherein the training of the GAN neural network model includes: Get construction site videos shot by drones; Use OpenCV to process the video and generate several construction site images as target domain images; splicing the target domain images to form a preliminary panoramic image of the construction site, and segmenting the preliminary panoramic image to form a source domain image; A training set is constructed according to the target domain images and the source domain images, the GNA neural network model is trained, and the trained GNA neural network model is output.

2. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 1, characterized in that: Preprocess the acquired video data based on OpenCV, including: Based on OpenCV, the construction site video is read frame by frame and saved to obtain several construction site pictures; Perform OpenCV stitching on the saved pictures of the construction site to form a preliminary panoramic picture of the construction site; The preliminary panoramic image of the construction site is segmented using OpenCV to form preliminary panoramic image blocks of the construction site.

3. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 2, characterized in that: The OpenCV segmentation of the preliminary construction site panorama also includes: The size of the segmented panorama blocks is made the same as the size of the source domain images during GAN neural network model training.

4. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 2, characterized in that: When OpenCV stitching is performed on the saved construction site pictures, it also includes: The stitching algorithm based on feature point matching is used for OpenCV stitching to form a preliminary panoramic view of the construction site.

5. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 1, characterized in that: When performing OpenCV segmentation on the preliminary construction site panorama, it also includes: Use uniform segmentation or content-based intelligent segmentation algorithm for OpenCV segmentation.

6. According to claim 1, the method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model is characterized in that: After obtaining the construction site panorama block output by the GAN neural network model, it also includes: Edge detection and alignment techniques are used to perform OpenCV edge stitching to form the final panoramic view of the construction site.

7. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 6, characterized in that: After the final panoramic view of the construction site is formed, it also includes: The final panoramic view of the construction site is transmitted and stored in real time.

8. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 1, characterized in that: Get construction site videos shot by drones at low altitudes, including: Get the geographic location information of the drone when shooting the video.

9. The method for synthesizing a panoramic image of a scene based on a drone low-altitude cruise shooting video and an OpenCV+GAN model according to claim 1, characterized in that: The method further comprises: The final construction site panorama is evaluated for quality, including resolution, color reproduction and / or detail clarity.

10. A live panoramic image synthesis system based on drone low-altitude cruise video and OpenCV+GAN model, characterized in that: include: The data acquisition module is used to obtain the construction site video taken by the drone at low altitude; The preprocessing module is used to preprocess the acquired video data based on OpenCV to obtain preprocessed image blocks; A panoramic image processing module is used to input the pre-processed image blocks into the trained GAN neural network model and output the construction site panoramic image blocks; wherein the training of the GAN neural network model includes: Get construction site videos shot by drones; Use OpenCV to process the video and generate several construction site images as target domain images; splicing the target domain images to form a preliminary panoramic image of the construction site, and segmenting the preliminary panoramic image to form a source domain image; A training set is constructed according to the target domain images and the source domain images, the GNA neural network model is trained, and the trained GNA neural network model is output.

Images