An unmanned aerial vehicle real scene modeling quality enhancement method based on edge computing

Abstract

The present application relates to the technical field of computer vision and digital image processing, and particularly relates to a UAV real scene modeling quality enhancement method fusing edge computing. The content comprises: collecting original images shot in the UAV flight process, and performing preprocessing to obtain preprocessed gray images; based on the preprocessed gray images, a multi-factor representation construction mechanism oriented to modeling contribution perception is introduced to construct a geometric coverage factor and a structure feature factor; based on the geometric coverage factor and the structure feature factor, a modeling value prediction score of the preprocessed gray images is calculated; a perception quality constraint correction mechanism is introduced to correct the modeling value prediction score to obtain a corrected modeling value prediction score; and based on the corrected modeling value prediction score, images are screened for subsequent three-dimensional real scene modeling. The technical problem of single image screening standard and no collaborative constraint mechanism established between modeling contribution evaluation and perception quality is solved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and digital image processing technology, and in particular to a method for enhancing the quality of UAV real-scene modeling by incorporating edge computing. Background Technology

[0002] With the rapid development of UAV aerial surveying and 3D reconstruction technologies, UAV reality modeling has been widely applied in fields such as urban modeling, smart parks, digital twins, and geographic information mapping. In practical applications, UAVs typically continuously acquire a large number of high-resolution images during flight and upload them to a central server for 3D reconstruction processing. However, due to the continuous flight path and short shooting intervals of UAVs, there is a large amount of overlapping viewpoints and spatial coverage between adjacent images, resulting in a large number of redundant images. If all images are used for 3D reconstruction, it will not only significantly increase the data transmission pressure and storage burden but also increase the consumption of central computing resources and reduce the overall modeling efficiency.

[0003] To reduce the data size, images are usually screened before uploading. Common screening methods are mainly based on image perception quality indicators, such as image sharpness, contrast or texture complexity, and only high-quality images are retained. In the process of 3D modeling, the modeling value of an image depends not only on the imaging quality, but also on its spatial location differences, shooting perspective differences and its ability to complete key structural regions. Therefore, the above-mentioned technologies still have the following shortcomings: (1) The image screening criteria are singular, mainly based on perception quality indicators, and lack quantitative representation of image spatial coverage differences and structural completion capabilities, making it difficult to accurately reflect the actual contribution of images in 3D modeling; (2) No collaborative constraint mechanism between modeling contribution evaluation and perception quality has been established, which easily leads to the mistaken retention of low-quality images or the mistaken removal of key structural images, resulting in a deviation between modeling quality and screening results.

[0004] Therefore, in order to address the above shortcomings, there is an urgent need to provide a method for enhancing the quality of UAV reality modeling by incorporating edge computing. Summary of the Invention

[0005] This invention provides a method for enhancing the quality of UAV real-scene modeling by integrating edge computing, in order to solve the technical problems of single image selection criteria and lack of a collaborative constraint mechanism between modeling contribution evaluation and perception quality.

[0006] The present invention provides a method for enhancing the quality of UAV reality modeling by incorporating edge computing, comprising the following steps:

[0007] S1. Collect raw images captured during the flight of the UAV and preprocess them to obtain preprocessed grayscale images; based on the preprocessed grayscale images, introduce a multi-factor representation construction mechanism for modeling contribution perception to construct geometric coverage factors and structural feature factors;

[0008] S2. Based on the geometric coverage factor and structural feature factor, calculate the modeling value prediction score of the preprocessed grayscale image; introduce a perceptual quality constraint correction mechanism to correct the modeling value prediction score, and obtain the corrected modeling value prediction score; select images based on the corrected modeling value prediction score for subsequent 3D real scene modeling.

[0009] Preferably, S1 specifically includes:

[0010] Based on the metadata of the original image, a vector of acquisition location and unit shooting direction is constructed; the metadata of the original image includes geographic coordinates and shooting direction.

[0011] Preferably, S1 specifically includes:

[0012] In the process of implementing the multi-factor representation construction mechanism for modeling contribution perception, a sliding window is introduced, and a historical grayscale image set is constructed by combining historical preprocessed grayscale images; based on the current preprocessed grayscale image and the historical preprocessed grayscale image, the geometric coverage factor is calculated by combining the acquisition location and the unit shooting direction vector.

[0013] Preferably, S1 specifically includes:

[0014] In the implementation of the multi-factor representation construction mechanism for modeling contribution perception, the edge response intensity and the number of corner points within a unit area of ​​the preprocessed grayscale image are calculated and normalized respectively to obtain the normalized value of the edge response intensity and the normalized value of the number of corner points within a unit area of ​​the preprocessed grayscale image. The normalized values ​​of the edge response intensity and the number of corner points within a unit area of ​​the preprocessed grayscale image are then weighted and fused to obtain the structural feature factor.

[0015] Preferably, S2 specifically includes:

[0016] The geometric coverage factor and structural feature factor are normalized separately to obtain normalized geometric coverage factor and normalized structural feature factor; the normalized geometric coverage factor and normalized structural feature factor are then weighted and fused to obtain the modeling value prediction score of the preprocessed grayscale image.

[0017] Preferably, S2 specifically includes:

[0018] In the process of implementing the perceptual quality constraint correction mechanism, an image perceptual quality factor is introduced to exponentially weaken the modeling value prediction score of the preprocessed grayscale image, resulting in the corrected modeling value prediction score.

[0019] Preferably, S2 specifically includes:

[0020] The image perception quality factor is constructed by incorporating the average brightness and sharpness of the preprocessed grayscale image.

[0021] Preferably, S2 specifically includes:

[0022] The revised modeling value prediction scores are sorted in descending order, and a retention ratio parameter is set; based on the retention ratio parameter, images are screened.

[0023] The beneficial effects of the technical solution of the present invention are:

[0024] 1. This invention proposes a multi-factor representation construction mechanism for modeling contribution perception. By constructing geometric coverage factors and structural feature factors, it quantifies the modeling contribution of images in two key dimensions: spatial coverage difference and structural information completion capability. This can effectively distinguish between viewpoint redundant images and images that play a key role in 3D structural completion, realizing the transformation from single-perception quality evaluation to multi-dimensional modeling contribution characterization, and laying a reliable data representation foundation for enhancing the quality of 3D real scene modeling.

[0025] 2. This invention proposes a perceptual quality constraint correction mechanism. By introducing the mean of sharpness and brightness, an image perceptual quality factor is constructed, which exponentially weakens the modeling value prediction score. While preserving the contribution of image geometric coverage and structural completion, it effectively suppresses the interference of blurred images and abnormally exposed images on the scoring results, and avoids low imaging quality images being misjudged as high-value images due to geometric differences. This improves the stability and robustness of the edge-side screening results, and ensures that the final screened image has both modeling structural contribution and imaging usability. Attached Figure Description

[0026] Figure 1 This is a flowchart of a method for enhancing the quality of UAV real-scene modeling by incorporating edge computing, as described in this invention. Detailed Implementation

[0027] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0029] The following description, in conjunction with the accompanying drawings, details a specific scheme for enhancing the quality of UAV real-scene modeling by incorporating edge computing, provided by this invention.

[0030] See attached document Figure 1 The diagram illustrates a flowchart of a method for enhancing the quality of UAV reality modeling by incorporating edge computing, according to an embodiment of the present invention. The method includes the following steps:

[0031] S1. Collect raw images captured during the flight of the UAV and preprocess them to obtain preprocessed grayscale images; based on the preprocessed grayscale images, introduce a multi-factor representation construction mechanism for modeling contribution perception to construct geometric coverage factors and structural feature factors;

[0032] The first image captured during the drone's flight The metadata of the original frame image includes: geographic coordinates and shooting direction; the geographic coordinates are the longitude, latitude, and altitude of the UAV when it captured the image, acquired through the UAV navigation system, and converted into three-dimensional coordinates in a unified three-dimensional Cartesian coordinate system using existing geographic coordinate transformation methods, i.e., the acquisition location. The unit is meters; the shooting direction includes pitch angle, roll angle, and yaw angle, which are collected by the UAV attitude sensor and converted into a three-dimensional shooting direction vector using existing Euler angle attitude calculation methods, and then normalized to obtain a unit shooting direction vector. The first The image resolution of the original frame image is ,in Image width, Image height;

[0033] For the collected first Preprocessing of the original image frame: First, the existing weighted average method is used to process the first frame. Convert the original image of frame to the first frame Frame grayscale image, and for the first The grayscale image of the frame is subjected to noise suppression using existing median filtering methods, resulting in the noise-suppressed image of the [frame name missing]. A grayscale image of the first frame; then, the noise-suppressed first frame... The grayscale image of the first frame is enhanced using an existing contrast-limited adaptive histogram equalization algorithm to obtain the preprocessed first frame. Frame grayscale image; the acquisition location With unit shooting direction vector As image geometric metadata, it does not participate in the pixel-level image preprocessing process and retains its original data format; the original image, the preprocessed grayscale image and the corresponding metadata are stored in the UAV real-scene image database.

[0034] To address the issue that existing image selection methods in 3D reality modeling rely solely on perceptual quality indicators while neglecting the contribution of images in terms of spatial coverage differences and structural completion capabilities, a multi-factor representation construction mechanism oriented towards perceptual modeling contribution is proposed. By constructing geometric coverage factors and structural feature factors, the modeling contribution of preprocessed grayscale images in the dimensions of spatial coverage and structural completion is quantitatively represented. The specific process is as follows:

[0035] The introduced length is The sliding window will display the current... Preprocessed before frame The set of grayscale images constitutes a historical grayscale image set, denoted as . ,in , represents the index of the historical preprocessed grayscale image within the sliding window; based on the current preprocessed image, the index of the previous image... The first frame grayscale image and the first frame after historical preprocessing The geometric coverage factor is calculated by considering the degree of geometric difference between the shooting direction and the acquisition location of the frame grayscale image. The formula is as follows: , in, This is the first preprocessed step. The geometric coverage factor of a frame grayscale image measures the degree of difference in geometric spatial coverage between the current preprocessed grayscale image and the set of historical preprocessed grayscale images. The larger the value, the more different the shooting direction and acquisition location are, and the more likely it is to cover previously unmodeled areas, thus contributing more to the 3D real scene modeling. , which represents the number of frames of the preprocessed grayscale image within the sliding window; , represents the index of the historical preprocessed grayscale image within the sliding window; , indicating the current preprocessed th The first frame grayscale image and the first frame after historical preprocessing The similarity of shooting directions between grayscale images of frames; the closer the value is to 1, the more similar the shooting directions are. For the current preprocessed number The unit shooting direction vector of a grayscale image frame; For the first time after historical preprocessing The unit shooting direction vector of a grayscale image frame; Through linear transformation Mapped to Within the range; , indicating the current preprocessed th The first frame grayscale image and the first frame after historical preprocessing The relative proximity of the acquisition locations of the frame grayscale image in three-dimensional space; the larger the value, the closer the acquisition locations. Indicates the current preprocessed number of... The first frame grayscale image and the first frame after historical preprocessing Euclidean distance between the acquisition locations of a grayscale image frame, in meters; For the current preprocessed number The acquisition location of the grayscale image frame; For the first time after historical preprocessing The acquisition location of the grayscale image frame; , is the spatial distance adjustment factor, in meters, and is set by professional technicians;

[0036] Furthermore, calculate the current preprocessed... The edge response intensity and corner count per unit area of ​​the grayscale image are normalized using the maximum and minimum values ​​of edge response intensity and corner count of the historical preprocessed grayscale images within a sliding window, and then weighted and fused to calculate the structural feature factor, as shown in the following formula: , in, For the current preprocessed number The structural feature factor of the frame grayscale image is used to measure the current preprocessed frame. In 3D reality modeling, grayscale images provide the ability to complete key structures such as building boundaries and object outlines. The larger the value, the greater the contribution. , , respectively, are the edge response weights and corner density weights, satisfying This was obtained through cross-validation. For the current preprocessed number Width of the grayscale image frame; For the current preprocessed number The height of the frame grayscale image; Indicates the current preprocessed number of... The area of ​​a frame of grayscale image; , indicating the current preprocessed th Edge response intensity of a frame grayscale image For the current preprocessed number The pixel positions of a frame grayscale image. , These represent the current preprocessed number of... Frame grayscale image at pixel position The horizontal and vertical gradients at a given point are calculated using existing Sobel operator methods. Indicates the current preprocessed number of... Edge response intensity per unit area of ​​a grayscale image frame; Indicates the current preprocessed number of... The normalized value of the edge response intensity per unit area in the grayscale image of a frame. If there is a case where the denominator is zero, the normalization result is set to 0 by default to avoid numerical abnormalities (the same applies below). , These represent the minimum and maximum values ​​of edge response intensity per unit area within the historical grayscale image of the sliding window, respectively; , represents the index of the historical preprocessed grayscale image within the sliding window; , indicating if pixel position The corner response value is greater than the threshold. If the value is true, the pixel is considered a corner point, and the indicator function is set to 1; otherwise, it is set to 0. This is an indicator function used to determine pixel position. Is the corner response value greater than the threshold? ; Indicates the current preprocessed number of... Frame grayscale image at pixel position The corner response value at the location is obtained by calculating using the existing Harris algorithm. The threshold for determining the Harris response was obtained through cross-validation. Indicates the current preprocessed number of... The number of corner points per unit area in a grayscale image frame; Indicates the current preprocessed number of... The normalized value of the number of corner points per unit area in a grayscale image frame; , These represent the minimum and maximum values ​​of the number of corner points per unit area in the historical grayscale image within the sliding window, respectively.

[0037] The above formula proposes a multi-factor representation construction mechanism for modeling contribution perception. By constructing geometric coverage factors and structural feature factors, it quantifies the modeling contribution of preprocessed grayscale images in two key dimensions: spatial coverage difference and structural information completion capability. It can effectively distinguish between viewpoint redundant images and images that play a key role in 3D structural completion, realizing the transformation from single-perception quality evaluation to multi-dimensional modeling contribution characterization, and laying a reliable data representation foundation for enhancing the quality of 3D real scene modeling.

[0038] S2. Based on the geometric coverage factor and structural feature factor, calculate the modeling value prediction score of the preprocessed grayscale image; introduce a perceptual quality constraint correction mechanism to correct the modeling value prediction score, and obtain the corrected modeling value prediction score; select images based on the corrected modeling value prediction score for subsequent 3D real scene modeling.

[0039] The current preprocessed number Geometric coverage factor of a frame grayscale image The maximum and minimum geometric coverage factors of the preprocessed historical grayscale images within the sliding window are normalized using the min-max normalization method to obtain the normalized geometric coverage factor. ; the current preprocessed number Structural feature factors of a frame grayscale image The maximum and minimum structural feature factors of the preprocessed grayscale images within the sliding window are normalized using the min-max normalization method to obtain the normalized structural feature factors. ; Normalized geometric cover factor Normalized structural feature factors The weighted fusion is performed according to the importance coefficients of geometric coverage and structural features to calculate the preprocessed i-th... Modeling value prediction score of grayscale images The value ranges of the geometric coverage importance weight coefficient and the structural feature importance weight coefficient are as follows: The sum of these inequalities equals 1, obtained through cross-validation.

[0040] Furthermore, to effectively suppress the impact of quality issues such as blur and brightness deviation on the 3D reality modeling effect, a perceptual quality constraint correction mechanism is proposed. This mechanism constructs an image perceptual quality factor by introducing the mean brightness and sharpness. Based on this factor, the preprocessed image... The modeling value prediction score of the grayscale image is subjected to exponential attenuation correction to obtain the corrected modeling value prediction score, as shown in the following formula: , in, Indicates the first preprocessed... Corrected modeling value prediction score for grayscale images; Indicates the first preprocessed... Modeling value prediction score for grayscale images; , indicating the first after preprocessing Perceptual quality factor of a grayscale image frame; , indicating the first after preprocessing The sharpness loss of a grayscale image frame is related to its resolution. Higher resolution results in less sharpness loss and a larger perceptual quality factor. The smaller the reduction correction; Indicates the current preprocessed number of... The sharpness of a frame of grayscale image is calculated by measuring the sharpness of the first frame after preprocessing. The Laplacian variance of the grayscale values ​​of all pixels in a frame grayscale image is obtained; the larger the value, the clearer the image. , These represent the minimum and maximum values ​​of the historical grayscale image sharpness index within the sliding window, respectively. Indicates the first [number] after historical preprocessing The sharpness of a grayscale image frame. , represents the index of the grayscale image after historical preprocessing within the sliding window; , indicating the first after preprocessing The average brightness of the frame grayscale image is calculated by preprocessing the first frame. The average grayscale value of all pixels in a frame grayscale image is obtained. To represent the first after preprocessing The ideal brightness value for a frame grayscale image is usually the ideal neutral brightness, such as 128; Indicates the first preprocessed step The absolute brightness deviation between the mean brightness value and the ideal brightness value of a frame of grayscale image is used to measure whether the preprocessed grayscale image is overexposed or underexposed. A larger value indicates a lower image quality and a lower image perception quality factor. The greater the reduction correction; , indicating the preprocessed first The absolute brightness deviation of the frame grayscale image is normalized; , which are the sharpness loss weighting coefficient and the brightness deviation weighting coefficient, respectively, obtained through cross-validation;

[0041] The above formula proposes a perceptual quality constraint correction mechanism. By introducing the mean brightness and sharpness, an image perceptual quality factor is constructed, which exponentially weakens the modeling value prediction score. While preserving the contribution of image geometric coverage and structural completion, it can effectively suppress the interference of blurred images and abnormally exposed images on the scoring results, and avoid low imaging quality images being misjudged as high-value images due to geometric differences. This improves the stability and robustness of the edge-side screening results, and ensures that the final screened image has both modeling structural contribution and imaging usability.

[0042] Preprocessed within the sliding window Corrected modeling value prediction score of grayscale image Sort by numerical value from largest to smallest, and calculate the corrected modeling value prediction score within the sliding window. The percentage of images greater than or equal to its mean is set as the retention ratio parameter. The corrected modeling value prediction score ranking is selected from the top. The original image corresponding to the preprocessed grayscale image is used for subsequent 3D reality modeling, thereby improving the modeling quality.

[0043] In summary, a method for enhancing the quality of UAV reality modeling by incorporating edge computing has been developed.

[0044] The order of the embodiments is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0045] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for enhancing the quality of UAV real-scene modeling by incorporating edge computing, characterized in that, Includes the following steps: S1. Collect raw images captured during the flight of the UAV and preprocess them to obtain preprocessed grayscale images; introduce a multi-factor representation construction mechanism for modeling contribution perception, and construct a historical grayscale image set by introducing a sliding window and combining it with historical preprocessed grayscale images. Based on the current preprocessed grayscale image and the historical preprocessed grayscale images, combined with the acquisition location and unit shooting direction vector, the geometric coverage factor is calculated; By calculating the edge response intensity and the number of corner points per unit area of ​​the preprocessed grayscale image, and then normalizing them respectively, we can obtain the normalized values ​​of the edge response intensity and the number of corner points per unit area of ​​the preprocessed grayscale image. The normalized values ​​of edge response intensity per unit area and the normalized values ​​of the number of corner points per unit area in the preprocessed grayscale image are weighted and fused to obtain the structural feature factor. S2. Calculate the modeling value prediction score of the preprocessed grayscale image based on the geometric coverage factor and structural feature factor; A perceptual quality constraint correction mechanism is introduced. By constructing an image perceptual quality factor, the modeling value prediction score of the preprocessed grayscale image is exponentially weakened and corrected to obtain the corrected modeling value prediction score. Images are selected based on the corrected modeling value prediction score for subsequent 3D real-scene modeling; the image perception quality factor is constructed based on the mean brightness and sharpness of the preprocessed grayscale image.

2. The method for enhancing the quality of UAV real-scene modeling by incorporating edge computing as described in claim 1, characterized in that, S1 specifically includes: Based on the metadata of the original image, a vector of acquisition location and unit shooting direction is constructed; the metadata of the original image includes geographic coordinates and shooting direction.

3. The method for enhancing the quality of UAV real-scene modeling by incorporating edge computing as described in claim 1, characterized in that, S2 specifically includes: The geometric coverage factor and structural feature factor are normalized separately to obtain normalized geometric coverage factor and normalized structural feature factor; the normalized geometric coverage factor and normalized structural feature factor are then weighted and fused to obtain the modeling value prediction score of the preprocessed grayscale image.

4. The method for enhancing the quality of UAV real-scene modeling by incorporating edge computing as described in claim 1, characterized in that, S2 specifically includes: The revised modeling value prediction scores are sorted in descending order, and a retention ratio parameter is set; based on the retention ratio parameter, images are screened.

Images