Method and system for complementing sky area of panorama of unmanned aerial vehicle

Abstract

The embodiment of the invention discloses a sky area completion method and system for a panorama of an unmanned aerial vehicle. According to the technical scheme provided by the embodiment of the invention, the panoramic image shot by the unmanned aerial vehicle is acquired, and the sky missing area of the panoramic image is detected; extracting a target sky area from the panoramic image, cutting the target sky area into a plurality of sky sub-blocks, and pasting the sky sub-blocks to the sky missing area at fixed intervals; and performing edge restoration on each sky sub-block in the sky missing area, and outputting a sky complementation image corresponding to the panoramic image based on the sky missing area after edge restoration. By adopting the technical means, the target sky area is cut into the plurality of sky sub-blocks and the sky sub-blocks are pasted to the sky missing area, and then the edges of the sky sub-blocks are repaired, so that the display effect of the panoramic image is guaranteed, complex operations such as manual calculation or algorithm calculation are avoided, the completion efficiency of the sky area of the panoramic image is improved, and the user experience is improved. And the universality of sky area completion operation is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and particularly to a method and system for filling the sky area of a drone panorama image. Background Art

[0002] At present, with the rapid development of drone technology, more and more drones are applied to aerial photography tasks. Especially in large-scale shooting, panoramic images have become an important form of drone shooting materials. When a drone conducts aerial photography, the sky in some areas of the panoramic image often appears missing due to limitations such as flight angle and lens coverage. Therefore, usually after generating the panoramic image, a manual repair method or a rule-based algorithm is used to fill the sky area of the panoramic image to avoid missing information in the panoramic image and ensure the display effect of the panoramic image.

[0003] However, the manual repair method is relatively complex to operate and takes a long time to fill. When using a rule-based algorithm to fill the sky area, the computational amount is relatively large, the operating efficiency is relatively low, and the algorithm requires a large amount of training data, which is not suitable for large-scale applications. Summary of the Invention

[0004] The embodiments of the present application provide a method and system for filling the sky area of a drone panorama image, which can improve the filling efficiency of the sky area of the panorama image, enhance the universality of the sky area filling operation, and solve the technical problem of the cumbersome and time-consuming process of filling the sky area of the panorama image.

[0005] In a first aspect, the embodiments of the present application provide a method for filling the sky area of a drone panorama image, including: Obtain a panoramic image captured by a drone, and detect the sky missing area in the panoramic image; Extract a target sky area from the panoramic image, cut the target sky area into multiple sky sub-blocks, and paste each sky sub-block into the sky missing area at a fixed interval; Perform edge repair on each sky sub-block in the sky missing area, and output a sky filling image corresponding to the panoramic image based on the sky missing area after edge repair.

[0006] Further, performing edge repair on each sky sub-block in the sky missing area includes: Determine an edge mask area based on the fixed interval of each sky sub-block in the sky missing area; Input the edge mask area and the panoramic image pasted with the sky sub-blocks into an artificial intelligence generation content model based on deep learning, and perform sky content filling on the edge mask area based on the artificial intelligence generation content model.

[0007] Further, determining an edge mask region based on the fixed intervals of each sky sub-block in the sky missing region includes: Determining a sub-block interval region based on the fixed intervals of each sky sub-block in the sky missing region; Determining the specified edge regions of each sky sub-block, and determining the edge mask region based on the sub-block interval region and the sub-block interval region.

[0008] Further, outputting a sky completion image corresponding to the panoramic image based on the sky missing region after edge repair includes: Covering the sky missing region after edge repair to the corresponding position of the panoramic image to generate a sky completion image corresponding to the panoramic image.

[0009] Further, before detecting the sky missing region of the panoramic image, it further includes: Performing normalization processing on the panoramic image with the original size according to the set size; Correspondingly, after outputting a sky completion image corresponding to the panoramic image based on the sky missing region after edge repair, it further includes: Adjusting the size of the sky completion image to the original size of the panoramic image.

[0010] Further, detecting the sky missing region of the panoramic image includes: Scanning the pixel information row by row along the set direction starting from the specified position of the panoramic image, and determining the sky missing region of the panoramic image based on the scanned blank pixel information.

[0011] Further, determining the sky missing region of the panoramic image based on the scanned blank pixel information includes: Determining the first row of pixels where all pixel information is blank as the bottom boundary based on the scanned blank pixel information, and determining the sky missing region of the panoramic image based on the bottom boundary and the scanned blank pixel information row by row along the set direction.

[0012] In a second aspect, an embodiment of the present application provides a sky region completion system for an unmanned aerial vehicle panoramic image, including: A detection module, configured to obtain a panoramic image captured by an unmanned aerial vehicle and detect the sky missing region of the panoramic image; A pasting module, configured to extract a target sky region from the panoramic image, cut the target sky region into multiple sky sub-blocks, and paste each sky sub-block into the sky missing region at a fixed interval; A repair module, configured to perform edge repair on each sky sub-block in the sky missing region, and output a sky completion image corresponding to the panoramic image based on the sky missing region after edge repair.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and one or more processors; the memory is configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method for completing the sky area of the drone panoramic image as described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the method for completing the sky area of the drone panoramic image as described in the first aspect when executed by a computer processor.

[0015] In the embodiment of the present application, a panoramic image captured by a drone is obtained, and a sky missing area of the panoramic image is detected; a target sky area is extracted from the panoramic image, the target sky area is cut into multiple sky sub-blocks, and each sky sub-block is pasted into the sky missing area at a fixed interval; edge repair is performed on each sky sub-block in the sky missing area, and a sky completion image corresponding to the panoramic image is output based on the sky missing area after edge repair. By adopting the above technical means, by cutting the target sky area into multiple sky sub-blocks and pasting them into the sky missing area, and then performing edge repair on the sky sub-blocks, while ensuring the display effect of the panoramic image, complex operations such as manual or algorithm calculation are avoided, and the efficiency of completing the sky area of the panoramic image is improved. And it can adapt to the completion of the sky area of different panoramic images, and improve the universality of the sky area completion operation. Description of the Drawings

[0016] Figure 1 is a flowchart of a method for completing the sky area of a drone panoramic image provided in Embodiment 1 of the present application; Figure 2 is a schematic diagram of the sky missing in the panoramic image in Embodiment 1 of the present application; Figure 3 is a schematic diagram of pasting sky sub-blocks in the sky missing area in Embodiment 1 of the present application; Figure 4 is a flowchart of edge repair of sky sub-blocks in Embodiment 1 of the present application; Figure 5 is a schematic diagram of the edge mask area in Embodiment 1 of the present application; Figure 6 is a schematic diagram of the sky completion image in Embodiment 1 of the present application; Figure 7 is a schematic structural diagram of a system for completing the sky area of a drone panoramic image provided in Embodiment 2 of the present application; Figure 8It is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present application. Detailed implementation manners

[0017] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further describes the specific embodiments of the present application in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. Additionally, it should be noted that for the convenience of description, only the parts related to the present application are shown in the drawings, rather than all the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0018] Embodiment 1: Figure 1 A flowchart of a method for filling in the sky area of a drone panoramic image provided in Embodiment 1 of the present application is given. The method for filling in the sky area of the drone panoramic image provided in this embodiment can be executed by a device for filling in the sky area of the drone panoramic image. The device for filling in the sky area of the drone panoramic image can be implemented in software and / or hardware. The device for filling in the sky area of the drone panoramic image can be composed of two or more physical entities, or can be composed of one physical entity. Generally, the device for filling in the sky area of the drone panoramic image can be a processing device such as a drone image processing system, a computer, an image processing server, etc.

[0019] The following takes an image processing server as an example of the main body for executing the method for filling in the sky area of the drone panoramic image for description. Refer to Figure 1 , the method for filling in the sky area of the drone panoramic image specifically includes: S110. Obtain a panoramic image captured by the drone, and detect the sky missing area of the panoramic image.

[0020] When filling in the drone panoramic image in the present application, through intelligent analysis and generation models, the sky missing area in the panoramic image is automatically identified and repaired. By automatically processing the images captured by the drone, the sky part can be efficiently filled in, and the generated sky image seamlessly connects with the original image, thereby greatly improving the effect and speed of image repair and reducing the need for manual intervention.

[0021] A panoramic image is an image that can cover a wide viewing angle (such as horizontal, vertical, or 360 degrees) through special shooting or image stitching techniques. Such images are usually composed of multiple adjacent photos stitched together to show a complete and seamless view. The image contains missing parts of the sky caused by factors such as the aerial shooting angle, the lens coverage, or the overlapping areas during the stitching process. Therefore, for a panoramic image obtained by stitching drone shots, it is first necessary to automatically identify the areas in the image that do not cover the complete sky due to shooting angle limitations or insufficient stitching algorithms. These areas are usually located at the top edge of the image and appear as invalid areas filled with specific colors (such as pure black or pure white) or missing content. By accurately scanning this missing area, the missing sky area for sky completion can be obtained.

[0022] Optionally, before detecting the missing sky area of the panoramic image, it further includes: Normalize the panoramic image of the original size according to the set size; For a panoramic image taken by a drone, it is first necessary to standardize the image so that subsequent processing can be efficiently and consistently repaired. In this step, first perform size normalization on the image and adjust it to the set size (such as 1920x1080 pixels) to ensure that the image has a unified resolution before processing. It should be noted that the set size is smaller than the original size of the panoramic image. By reducing the size, the algorithm complexity of subsequent processing is simplified, and the processing inconsistency caused by different image sizes can also be avoided. After normalization, extract the height and width of the image to determine the specific repair area in the subsequent steps. Through this process, it can be ensured that the image has consistency in subsequent processing, facilitating the quick positioning of the missing area and laying a foundation for the subsequent automatic detection and repair of the sky area.

[0023] Furthermore, detecting the missing sky area of the panoramic image includes: Scan the pixel information row by row along the set direction starting from the specified position of the panoramic image, and determine the missing sky area of the panoramic image based on the scanned blank pixel information.

[0024] Based on the above-normalized panoramic image, the detection of the missing sky area can be carried out. When detecting the missing sky area, first, start from the specified position of the processed image and scan the pixel information of the panoramic image row by row along the set direction. It can be understood that since the sky area of the panoramic image is generally at the top, to reduce the scanning calculation amount, this application starts from the middle of the panoramic image and scans row by row upward to check the pixel information of each row. According to the actual detection requirements, it can also start from one-third of the vertical position of the panoramic image and scan along the direction of the sky area. This application does not make a fixed limit on the specific pixel information scanning method and will not elaborate here.

[0025] By performing a line-by-line scan, it is possible to accurately identify whether there is valid pixel data in the image, especially the missing areas in the sky part. It can be understood that since the pixel values in the missing part of the sky are zero or close to zero, the missing area of the sky in the panoramic image can be determined through the blank pixel information scanned.

[0026] Specifically, determining the missing area of the sky in the panoramic image based on the scanned blank pixel information includes: Determining the first row of pixels with all blank pixel information as the bottom boundary based on the scanned blank pixel information, and determining the missing area of the sky in the panoramic image based on the bottom boundary and the blank pixel information scanned row by row in a set direction.

[0027] In the scanning process of this application, if all the pixels in a certain row are blank (that is, the pixel value is zero or close to zero, indicating that there is no valid image information in this area), it is determined that this row is at the bottom boundary of the missing area of the sky. Then, through continuous scanning, the upper boundary of the missing area of the sky is further determined. Once it is detected that the sky area is blank, the specific position and boundary of this area can be calibrated. It can be understood that since the panoramic images taken by drones usually have a consistent shooting height and the heights of the missing areas are roughly equal, the range of the missing area of the sky can be accurately located according to this characteristic, which is convenient for the execution of subsequent repair steps. This line-by-line scanning method can ensure the detection of the missing area of the sky with high precision and provide accurate area information for the next sky repair. Refer to Figure 2 and by performing a line-by-line scan, it can be determined that the top black area in the panoramic image is the missing area of the sky, so as to perform subsequent accurate and efficient sky filling operations.

[0028] Optionally, the missing area of the sky can also be accurately located by analyzing the pixel values of the image edge (identifying the preset filling color mark), performing color threshold segmentation by combining the characteristics that the sky is located at the top and has relatively uniform color (such as setting the blue and brightness ranges in the HSV space), or using the transparency channel (Alpha channel) or stitching mask information of the image. By automatically completing the positioning of the missing area, the cumbersome manual search and annotation process is completely replaced, thus significantly improving the efficiency. At the same time, this method based on simple image analysis and preset rules has a small amount of calculation and does not require training data, laying a precise foundation for subsequent rapid processing.

[0029] S120. Extract the target sky area from the panoramic image, cut the target sky area into multiple sky sub-blocks, and paste each sky sub-block into the missing area of the sky at a fixed interval.

[0030] Further, based on the panoramic image with the sky missing area already determined above, the present application selects an area with clear texture and natural color from the existing and valid sky part in the same panoramic image as the target sky area. Optionally, the target sky area can be an adjacent area such as below or on both sides of the sky missing area to ensure that the subsequent filling effect is natural enough. In addition, the size of the target sky area can also be set according to actual needs. According to actual needs, the target sky area can also be the real sky area image information at the corresponding position captured by other cameras. The present application does not make a fixed limitation on the selection of the target sky area and will not elaborate here.

[0031] Further, the target sky area is cut into multiple sky sub - blocks with a fixed size, and then each sky sub - block is fixedly pasted to each position of the sky missing area. Optionally, if the target sky area comes from an adjacent area such as below or on both sides of the sky missing area, when pasting the sky sub - blocks, the sky sub - blocks can be pasted to the nearest position on the sky missing area according to their distance from the sky missing area in the panoramic image. And a certain interval is left between sub - blocks until no more sub - blocks can be placed in the sky missing area. It can be understood that since the left and right sides in the panoramic picture are connected, the left edge of the left - most sub - block and the right edge of the right - most sub - block are connected and there is no interval, ensuring that there is no splitting phenomenon when the panoramic picture after sky filling is viewed in the panoramic viewer and avoiding splicing traces. In addition, according to actual needs, the interval between sub - blocks can be a vertical interval or both vertical and horizontal intervals. The present application does not make a fixed limitation on the specific pasting method and will not elaborate here.

[0032] Refer to Figure 3 , by dividing the target sky area into multiple smaller rectangular or square image blocks (sky sub - blocks). These sub - blocks are pasted and copied into the previously detected sky missing area at a preset fixed interval (vertical direction), thereby obtaining Figure 3 the initial filling effect of the sky missing area shown. It can be understood that using the existing sky information in the image for copying and pasting has extremely low computational complexity and far exceeds the speed of manual drawing or complex generation models. And cutting the large sky into sub - blocks and pasting them with intervals effectively avoids obvious repeated textures caused by whole - block copying (such as the repeated appearance of the same clouds). The gaps between sub - blocks naturally break the pattern repeatability, reserve space for the subsequent fusion step, and at the same time introduce subtle randomness visually, making the generated sky texture closer to reality. This strategy makes full use of the internal resources of the image, without the need for external data or model training, ensuring the universality and efficiency of the method, especially suitable for large - scale applications.

[0033] S130. Perform edge repair on each sky sub-block in the sky missing area, and output the sky completion image of the corresponding panoramic image based on the sky missing area after edge repair.

[0034] Based on the sky missing area where the sky sub-blocks have been pasted as described above, it is necessary to eliminate the rough traces left by the previous paste to achieve seamless fusion. In this application, the artificial intelligence generated content model (AIGC) is used to perform edge repair on each sky sub-block to obtain the final sky completion image. AIGC refers to using technologies such as deep learning and generative adversarial networks (GANs) to generate image content that conforms to the original image or other samples through an AI model. AIGC can create new content, such as images, music, text, etc., by analyzing existing data. There are many ways for the artificial intelligence generated content model to generate content, and this application does not make a fixed limitation on this.

[0035] By automatically identifying the sky missing area, cropping the original target sky area and filling the sky missing area, the surface repair is transformed into line repair, thereby reducing the performance dependence on the image generation model, increasing the time cost of image repair, and ensuring the natural and seamless connection of the repair effect, making the sky completion method have a wide range of application prospects. This method can be widely applied to fields such as drone aerial photography, panoramic image processing, video production, etc., providing a new solution for the development of image processing technology. It can process panoramic images taken by drones, automatically identify the sky missing area, and repair these areas through AIGC technology to ensure that the repair effect is natural and visually consistent. This technology is especially suitable for the repair of large-scale aerial photography materials, especially in the splicing of complex panoramic images taken by drones, and has a wide range of application prospects.

[0036] Optionally, performing edge repair on each sky sub-block in the sky missing area includes: S1301. Determine the edge mask area based on the fixed intervals of each sky sub-block in the sky missing area; S1302. Input the edge mask area and the panoramic image pasted with sky sub-blocks into the artificial intelligence generated content model based on deep learning, and perform sky content filling on the edge mask area based on the artificial intelligence generated content model.

[0037] In this application, based on the sky missing area where the sky sub-blocks are pasted, sub-block edge repair is performed. An image mask is made according to the interval part between sub-blocks to obtain the edge mask area. An image mask refers to marking or covering a specific area in an image through binary processing. The mask area is usually white or black, and the marked area is defined by an algorithm or manually. The mask can help the image processing algorithm operate in the specified area, such as repair, replacement, or deformation.

[0038] The edge mask region is the part that needs to be edge-repaired between each sky sub-block. Among them, the edge mask region is determined based on the fixed interval of each sky sub-block in the sky missing region, including: Determine the sub-block interval region based on the fixed interval of each sky sub-block in the sky missing region; Determine the specified edge region of each sky sub-block, and determine the edge mask region based on the sub-block interval region and the sub-block interval region.

[0039] The edge mask region is as Figure 5 shown. The edge mask region of this application, in addition to the sub-block interval region of each sky sub-block, also includes the specified edge region of each sky sub-block, so that when performing edge repair subsequently, the connection between edges is more natural.

[0040] Among them, based on this edge mask region, the artificial intelligence generated content model (AIGC) can use the SimpleLama model based on deep learning. By analyzing the edge mask region and combining the panoramic image of the pasted sky sub-block, the model can automatically fill the content of the sky part in the edge mask region, so as to output the sky-completed image corresponding to the panoramic image. This process makes full use of the understanding of image content by AI technology. By cropping the sky region of the panoramic image, it ensures that the generated sky region is consistent with the original image in terms of style and tone. By filling the sub-image, it restricts the free play of AIGC technology, turning the repair (surface) work of the entire sky region into the repair (line) of the sub-block interval region of the sky region, making the sky completion work faster and better.

[0041] Optionally, when edge repairing the missing sky area, edge repairing operations can also be performed on the edge contours of each pasted sky sub-block itself and the fixed interval gaps retained between the sub-blocks (i.e., the blank areas not covered by the pasting). For the sub-block edges, slight feathering or blurring can be used to soften the boundaries; for the gap areas, content-based (especially based on the diffusion principle, such as the fast marching method FMM) image repair algorithms are mainly applied. Using the color and texture information of the known pixels around the gap (i.e., the edge pixels of the pasted sub-blocks), reasonable color gradient information is gradually filled into the gap through the diffusion mechanism, ensuring that the texture and color changes of the filled area are smoothly transitioned and naturally coherent with the surrounding environment, thereby improving the visual quality of the final result. And effectively eliminate the fragmented artificial traces caused by the sub-block boundary lines and interval gaps, so that the completed sky area is visually presented as a continuous and natural whole. The repair algorithm ensures that the transition inside the completed area and between the completed area and the valid area of the original image (such as the edge of the ground scene) is smooth and seamless. At the same time, since the repair operation only acts on the relatively narrow sub-block edges and gap areas, rather than the entire large missing area, the amount of calculation is significantly lower than direct global repair. Ultimately, the repaired missing sky area is seamlessly integrated with the valid part of the original panorama, outputting a panoramic image with a complete sky and natural visual effects.

[0042] Based on the edge-repaired sky missing area, the sky completion image corresponding to the panoramic image is output, including: The sky missing area after edge repair is covered to the corresponding position of the panoramic image to generate a sky completion image corresponding to the panoramic image.

[0043] Finally, through image processing technology, the sky missing area after edge repair is synthesized with the original panoramic image, and the repaired sky missing area is seamlessly embedded in the panoramic image without changing the pixel value of the non-sky area, avoiding the loss of picture details when the image size is reduced or enlarged. At this point, the entire repair process is completed and the sky completion image corresponding to the panoramic image is output.

[0044] In addition, after outputting a sky completion image corresponding to the panoramic image based on the sky missing area after edge repair, it also includes: Resize the sky completion image to the original size of the panorama image.

[0045] Finally, the generated sky completion image is resized to ensure that the restored sky completion image is consistent with the original size of the panoramic image. In particular, it is necessary to ensure that there is no abrupt boundary between the restoration of the sky area and the stitching of the panorama. Then the restored sky completion image is output, in which the sky part has been perfectly completed by the AIGC technology and naturally merged with the rest of the image.

[0046] As described above, by obtaining the panoramic image captured by the drone, the sky missing area of the panoramic image is detected; the target sky area is extracted from the panoramic image, the target sky area is cut into multiple sky sub-blocks, and each sky sub-block is pasted into the sky missing area at a fixed interval; the edges of each sky sub-block in the sky missing area are repaired, and the sky completion image corresponding to the panoramic image is output based on the sky missing area after edge repair. By adopting the above technical means, by cutting the target sky area into multiple sky sub-blocks and pasting them into the sky missing area, and then repairing the edges of the sky sub-blocks, the display effect of the panoramic image is ensured while avoiding complex operations such as manual or algorithm calculation, and the completion efficiency of the sky area of the panoramic image is improved. And it can adapt to the sky area completion of different panoramic images and improve the universality of the sky area completion operation.

[0047] Embodiment 2: Based on the above embodiment, Figure 7 FIG. is a schematic structural diagram of a sky area completion system for a drone panoramic image provided in Embodiment 2 of the present application. Refer to Figure 7 , the sky area completion system for a drone panoramic image provided in this embodiment specifically includes: a detection module 21, a pasting module 22, and a repair module 23.

[0048] Among them, the detection module 21 is used to obtain the panoramic image captured by the drone and detect the sky missing area of the panoramic image; The pasting module 22 is used to extract the target sky area from the panoramic image, cut the target sky area into multiple sky sub-blocks, and paste each sky sub-block into the sky missing area at a fixed interval; The repair module 23 is used to repair the edges of each sky sub-block in the sky missing area and output the sky completion image corresponding to the panoramic image based on the sky missing area after edge repair.

[0049] Specifically, repairing the edges of each sky sub-block in the sky missing area includes: Determining an edge mask area based on the fixed interval of each sky sub-block in the sky missing area; Inputting the edge mask area and the panoramic image pasted with the sky sub-blocks into an artificial intelligence generation content model based on deep learning, and filling the sky content in the edge mask area based on the artificial intelligence generation content model.

[0050] Specifically, determining the edge mask area based on the fixed interval of each sky sub-block in the sky missing area includes: Determining a sub-block interval area based on the fixed interval of each sky sub-block in the sky missing area; Determine the specified edge regions of each sky sub-block, and determine the edge mask region based on the sub-block interval region and the sub-block interval region.

[0051] Specifically, based on the sky missing region after edge repair, output the sky completion image corresponding to the panoramic image, including: Cover the sky missing region after edge repair to the corresponding position of the panoramic image to generate the sky completion image corresponding to the panoramic image.

[0052] Specifically, before detecting the sky missing region of the panoramic image, it also includes: Standardize the panoramic image with the original size according to the set size; Correspondingly, after outputting the sky completion image corresponding to the panoramic image based on the sky missing region after edge repair, it also includes: Adjust the size of the sky completion image to the original size of the panoramic image.

[0053] Specifically, detecting the sky missing region of the panoramic image includes: Start scanning the pixel information row by row from the specified position of the panoramic image along the set direction, and determine the sky missing region of the panoramic image based on the scanned blank pixel information.

[0054] Specifically, determining the sky missing region of the panoramic image based on the scanned blank pixel information includes: Determine the first row of pixels where all pixel information is blank based on the scanned blank pixel information as the bottom boundary, and determine the sky missing region of the panoramic image based on the bottom boundary and the scanned blank pixel information row by row along the set direction.

[0055] Above, by obtaining the panoramic image taken by the drone, detecting the sky missing region of the panoramic image; extracting the target sky region from the panoramic image, cutting the target sky region into multiple sky sub-blocks, and pasting each sky sub-block into the sky missing region at a fixed interval; performing edge repair on each sky sub-block in the sky missing region, and outputting the sky completion image corresponding to the panoramic image based on the sky missing region after edge repair. By adopting the above technical means, by cutting the target sky region into multiple sky sub-blocks and pasting them into the sky missing region, and then performing edge repair on the sky sub-blocks, the display effect of the panoramic image is guaranteed while avoiding complex operations such as manual or algorithm calculation, and the efficiency of completing the sky region of the panoramic image is improved. And it can adapt to the sky region completion of different panoramic images, and improve the universality of the sky region completion operation.

[0056] The sky region completion system of the drone panoramic image provided in the second embodiment of the present application can be used to execute the sky region completion method of the drone panoramic image provided in the first embodiment, and has the corresponding functions and beneficial effects.

[0057] Embodiment 3: Embodiment 3 of this application provides an electronic device. Refer to Figure 8 , the electronic device includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The number of processors in the electronic device can be one or more, and the number of memories in the electronic device can be one or more. The processor, memory, communication module, input device, and output device of the electronic device can be connected through a bus or other means.

[0058] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the sky area completion method of the drone panoramic image in any embodiment of this application (for example, the detection module, pasting module, and repair module in the sky area completion system of the drone panoramic image). The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory can further include memories remotely set relative to the processor, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and their combinations.

[0059] The communication module is used for data transmission.

[0060] The processor executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, implements the above-mentioned sky area completion method of the drone panoramic image.

[0061] The input device can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device can include display devices such as a display screen.

[0062] The above-provided electronic device can be used to execute the sky area completion method of the drone panoramic image provided in the above Embodiment 1, and has corresponding functions and beneficial effects.

[0063] Embodiment 4: An embodiment of the present application further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute a method for completing the sky area of a drone panoramic image when executed by a computer processor. The method for completing the sky area of the drone panoramic image includes: obtaining a panoramic image captured by the drone, and detecting the sky missing area of the panoramic image; extracting a target sky area from the panoramic image, cutting the target sky area into multiple sky sub-blocks, and pasting each sky sub-block into the sky missing area at a fixed interval; performing edge repair on each sky sub-block in the sky missing area, and outputting a sky-completed image corresponding to the panoramic image based on the sky missing area after edge repair.

[0064] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks or tape drives; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media located in different locations (such as in different computer systems connected via a network). The storage medium may store program instructions (such as specifically implemented as a computer program) executable by one or more processors.

[0065] Certainly, for a storage medium containing computer-executable instructions provided by an embodiment of the present application, the computer-executable instructions are not limited to the method for completing the sky area of the drone panoramic image as described above, and may also execute related operations in the method for completing the sky area of the drone panoramic image provided by any embodiment of the present application.

[0066] The system, storage medium, and electronic device for completing the sky area of the drone panoramic image provided in the above embodiments can execute the method for completing the sky area of the drone panoramic image provided by any embodiment of the present application. For technical details not described in detail in the above embodiments, reference may be made to the method for completing the sky area of the drone panoramic image provided by any embodiment of the present application.

[0067] The above are only the preferred embodiments of the present application and the technical principles applied. The present application is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions that can be made by those skilled in the art will not depart from the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments. Without departing from the concept of the present application, it may also include more other equivalent embodiments, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for completing the sky area of a panoramic image of an unmanned aerial vehicle, characterized in that, Including: Obtain a panoramic image captured by a drone, and detect a sky missing area of the panoramic image; Extract a target sky area from the panoramic image, crop the target sky area into multiple sky sub-blocks, and paste each of the sky sub-blocks into the sky missing area at a fixed interval; Perform edge repair on each of the sky sub-blocks in the sky missing area, and output a sky completion image corresponding to the panoramic image based on the sky missing area after edge repair.

2. The method for completing the sky area of the panoramic view of the unmanned aerial vehicle according to claim 1, characterized in that, The performing edge repair on each of the sky sub-blocks in the sky missing area includes: Determine an edge mask area based on the fixed interval of each of the sky sub-blocks in the sky missing area; Input the edge mask area and the panoramic image where the sky sub-blocks are pasted into an artificial intelligence generated content model based on deep learning, and perform sky content filling on the edge mask area based on the artificial intelligence generated content model.

3. The method for completing the sky area of the panoramic image of the drone according to claim 2, characterized in that, The determining an edge mask area based on the fixed interval of each of the sky sub-blocks in the sky missing area includes: Determine a sub-block interval area based on the fixed interval of each of the sky sub-blocks in the sky missing area; Determine a specified edge area of each of the sky sub-blocks, and determine an edge mask area based on the sub-block interval area and the sub-block interval area.

4. The method for completing the sky area of the panoramic image of the unmanned aerial vehicle according to claim 1, wherein The outputting a sky completion image corresponding to the panoramic image based on the sky missing area after edge repair includes: Cover the sky missing area after edge repair to the corresponding position of the panoramic image to generate a sky completion image corresponding to the panoramic image.

5. The method for complementing the sky area of the panoramic view of the drone according to claim 1, characterized in that, Before the detecting the sky missing area of the panoramic image, it further includes: Perform standardization processing on the panoramic image with the original size according to a set size; Correspondingly, after the outputting a sky completion image corresponding to the panoramic image based on the sky missing area after edge repair, it further includes: Adjust the size of the sky completion image to the original size of the panoramic image.

6. The method for completing the sky area of the panoramic view of the drone according to any one of claims 1-5, characterized in that, The detecting the sky missing area of the panoramic image includes: Scan pixel information row by row along a set direction starting from a specified position of the panoramic image, and determine the sky missing area of the panoramic image based on the scanned blank pixel information.

7. The method for completing the sky area of the panoramic view of the unmanned aerial vehicle according to claim 6, wherein The determining the sky missing area of the panoramic image based on the scanned blank pixel information includes: Determine the first row of pixels where all pixel information is blank as the bottom boundary based on the scanned blank pixel information, and determine the sky missing area of the panoramic image based on the bottom boundary and the scanned blank pixel information row by row along the set direction.

8. A sky area completion system for a panoramic view of an unmanned aerial vehicle, characterized in that, Including: A detection module, configured to obtain a panoramic image captured by a drone, and detect a sky missing area of the panoramic image; A pasting module, configured to extract a target sky area from the panoramic image, crop the target sky area into multiple sky sub-blocks, and paste each of the sky sub-blocks into the sky missing area at a fixed interval; A repair module for edge-repairing each sky sub-block in the sky missing area and outputting a sky completion image corresponding to the panoramic image based on the sky missing area after edge repair.

9. An electronic device, characterized in that, Comprising: A memory and one or more processors; The memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for sky area completion of a drone panoramic image according to any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the method for sky area completion of a drone panoramic image according to any one of claims 1-7 when executed by a computer processor.

Images