Real-time updating method of three-dimensional geographic information based on multi-feature fusion
By chunking processing of the surface top view image and precise setting of the fog removal intensity parameters, the problem of poor fog removal effect under the influence of heavy fog is solved, and the accuracy and image quality of three-dimensional map construction and geographic information update are improved.
Patent Information
- Application Number
- CN202411162299.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-23
AI Technical Summary
During the construction of a three-dimensional map with multiple features fusion, the surface top view image affected by heavy fog weather has poor clarity, resulting in poor fog removal effect. It is difficult for the existing fog removal algorithm to accurately select the fog removal intensity parameters, resulting in information loss or excessive deal with.
By blocking the surface top view image, the clarity index of each block image is determined, the defog intensity parameter range is accurately set, and the block image is defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defog-defo Update three-dimensional geographic information.
It improves the accuracy of the fog removal effect and the retention of image information, enhances the accuracy of three-dimensional map construction and geographic information update, and ensures image quality and information integrity.
Smart Images

Figure CN118689890B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image data processing, and in particular to a real-time updating method of three-dimensional geographic information based on multi-feature fusion in the technical field of image data processing. Background Art
[0002] Multi-feature fused three-dimensional geographic information refers to a method of building and updating geographic spatial models by integrating multiple data sources and feature information such as satellite images, aerial images, lidar data, and terrain models. This method not only improves the accuracy and comprehensiveness of geographic information, but also supports multiple application areas such as refined urban planning, real-time updating and monitoring, as well as environmental protection and resource management. The accuracy of collected information has an important impact on the accuracy of information when building three-dimensional maps. When collecting visual image information of the surface, the surface overhead image may be disturbed by foggy weather, resulting in poor clarity of the surface overhead image, which in turn leads to poor accuracy when subsequently using the surface overhead image to extract information and build three-dimensional maps.
[0003] In the process of building a three-dimensional map using multi-feature data fusion, it is necessary to defog the surface visual images collected in foggy weather to improve their display clarity. When using the Dark Channel Prior (DCP) to defog the surface overhead image, since the surface overhead image is a three-dimensional surface image, the height of each position in the image may be different, and the degree of interference from the fog is different. If the defogging intensity parameter is too small, the defogging effect will be poor; if the defogging intensity parameter is too large, it may cause excessive processing of the image, loss of details and brightness and other information. Summary of the invention
[0004] In order to solve the technical problem of poor defogging effect on the collected three-dimensional surface images in the prior art, the purpose of the present invention is to provide a real-time updating method of three-dimensional geographic information based on multi-feature fusion, and the technical scheme adopted is as follows:
[0005] In a first aspect, an embodiment of the present invention provides a method for real-time updating of three-dimensional geographic information based on multi-feature fusion, the method comprising:
[0006] Obtaining top-view images of the surface;
[0007] Performing block processing on the surface bird's-eye view image to obtain a plurality of block images;
[0008] Determining a defogging intensity parameter range based on the clarity indexes of the plurality of block images;
[0009] Performing defogging processing on the plurality of block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the plurality of block images;
[0010] Based on the defogging target images corresponding to the multiple block images, the three-dimensional geographic information corresponding to the surface bird's-eye view image is updated.
[0011] In combination with the first aspect, in some possible implementations, the performing block processing on the surface overhead image to obtain a plurality of block images includes:
[0012] Determining a grayscale image of the surface overhead image;
[0013] Determining a block parameter based on the grayscale value of the grayscale image and the grayscale value of the surface overhead image;
[0014] The grayscale image and the surface overhead image are divided into blocks using the block parameters to obtain the multiple block images.
[0015] In the above method, the surface overhead view image is converted into a grayscale image, and the grayscale image and the surface overhead view image are blocked according to the blocking parameters of the surface overhead view image, so that a plurality of blocked images can be obtained accurately and quickly, so as to accurately analyze the dehazing intensity parameter range of the image through the plurality of blocked images.
[0016] In combination with the first aspect, in some possible implementations, determining the block parameter based on the grayscale value of the grayscale image and the grayscale value of the surface overhead image includes:
[0017] Determine the difference between the maximum value and the minimum value in the grayscale value of the top view image;
[0018] The preset image segmentation hyperparameters are used to adjust the difference and the mean of the grayscale values of the grayscale image to obtain the segmentation parameters.
[0019] In the above method, the difference between the maximum grayscale value and the minimum grayscale value of the surface overhead image is combined with the mean grayscale value of the grayscale image, so that the block parameters can be obtained more accurately to facilitate image block segmentation.
[0020] In combination with the first aspect above, in some possible implementations, the method further includes:
[0021] Clustering each of the plurality of block images to obtain a plurality of cluster clusters;
[0022] Based on the multiple clusters, determining the gradient distribution characteristics of each block image;
[0023] Based on the grayscale value of each block image and the gradient distribution characteristics, a clarity index of each block image is determined.
[0024] In the above manner, by combining the grayscale value of each block image with the gradient distribution characteristics of the block image, the clarity index of the block image can be accurately analyzed, so as to further determine the defogging intensity parameter range through the clarity index.
[0025] In combination with the first aspect, in some possible implementations, determining the clarity index of each block image based on the grayscale value of each block image and the gradient distribution feature includes:
[0026] Based on the grayscale value of each block image, determine the grayscale value and average grayscale value of any pixel in each block image;
[0027] The difference between the grayscale value of any pixel and the average grayscale value, as well as the gradient distribution feature, are fused to obtain a clarity index of each block image.
[0028] In the above manner, by calculating the difference between the gradient distribution characteristics and the grayscale value of the block image, the clarity index of the block image can be quickly analyzed.
[0029] In combination with the first aspect, in some possible implementations, determining the defogging intensity parameter range based on the clarity indexes of the multiple block images includes:
[0030] Determining a minimum value of a defogging intensity parameter based on the clarity index and a preset defogging intensity control range constant;
[0031] Determining a maximum value of a defogging intensity parameter based on the preset defogging intensity control range constant;
[0032] The defogging intensity parameter range is obtained based on the minimum defogging intensity parameter value and the maximum defogging intensity parameter value.
[0033] In the above manner, by combining the clarity index of the block image with the preset defogging intensity control range parameter, the maximum and minimum values of the defogging intensity parameter can be accurately determined, thereby obtaining the defogging intensity parameter range of the block image.
[0034] In combination with the first aspect, in some possible implementations, performing defogging on the multiple block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the multiple block images includes:
[0035] Performing multiple defogging operations on each block image based on the defogging intensity parameter range to obtain multiple defogging initial images;
[0036] Determining defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images;
[0037] Based on the defogging effect evaluation values of the multiple defogging initial images, a defogging target image corresponding to each block image is determined in the multiple defogging initial images.
[0038] In the above manner, by comparing the defogging effect evaluation values of a plurality of defogging initial images, an image with the largest defogging effect evaluation value is selected from the plurality of defogging initial images, thereby obtaining the defogging target image.
[0039] In combination with the first aspect above, in some possible implementations, determining the defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images includes:
[0040] Determine a clarity mean value of the clarity indicators corresponding to the multiple defogging initial images;
[0041] Based on the clarity index of each defogging initial image in the multiple defogging initial images and the clarity mean, a defogging effect evaluation value of each defogging initial image is determined.
[0042] In the above method, by first calculating the clarity index and the clarity mean of each defogging initial image, the defogging effect evaluation value of each defogging initial image can be quickly and accurately analyzed, so as to screen out the defogging target image with better defogging effect from each defogging initial image.
[0043] In combination with the first aspect, in some possible implementations, updating the three-dimensional geographic information corresponding to the surface bird's-eye view image based on the defogging target images corresponding to the multiple block images includes:
[0044] splicing the defogging target images corresponding to the multiple block images according to the position order corresponding to the multiple block images to obtain a restored image;
[0045] Based on the restored image, the three-dimensional geographic information corresponding to the surface bird's-eye view image is updated.
[0046] In the above manner, by splicing the defogging target images with better defogging effects, a restored image with higher image quality can be obtained, thereby making the updated three-dimensional geographic information more accurate.
[0047] In combination with the first aspect above, in some possible implementations, updating the three-dimensional geographic information corresponding to the surface bird's-eye view image based on the restored image includes:
[0048] Extracting geometric features and temporal features from the restored image;
[0049] Performing multi-feature fusion on the geometric features and the temporal features to obtain fused features;
[0050] Based on the fused features, the three-dimensional geographic information corresponding to the surface bird's-eye view image is updated.
[0051] In the above method, multi-feature fusion is performed on the restored image with good image quality to update the three-dimensional geographic information of the surface bird's-eye view image, thereby improving the accuracy of constructing a three-dimensional map and updating the three-dimensional geographic information.
[0052] In a second aspect, a device for real-time updating of three-dimensional geographic information based on multi-feature fusion is provided, the device comprising:
[0053] An acquisition module is used to acquire a bird's-eye view image of the surface;
[0054] A block division module is used to perform block processing on the surface overhead image to obtain a plurality of block images;
[0055] A determination module, configured to determine a defogging intensity parameter range based on the clarity indexes of the plurality of block images;
[0056] A defogging module, configured to perform defogging processing on the plurality of block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the plurality of block images;
[0057] An updating module is used to update the three-dimensional geographic information corresponding to the surface bird's-eye view image based on the defogging target images corresponding to the multiple block images.
[0058] According to a third aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method according to the first aspect or any possible implementation of the first aspect.
[0059] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.
[0060] The present invention has the following beneficial effects: after obtaining a top-view image of a surface, the top-view image of the surface is processed in blocks to obtain a plurality of block images, and then the range of defogging intensity parameters is determined based on the clarity index of the plurality of block images; in this way, after the top-view image of the surface is processed in blocks, the range of defogging intensity parameters of the block images can be calculated according to each block image, thereby improving the accuracy of calculating the range of defogging intensity parameters, and facilitating the selection of more appropriate defogging intensity parameters for different block images. Then, the plurality of block images are defogged based on the range of defogging intensity parameters to obtain defogging target images corresponding to the plurality of block images, and the three-dimensional geographic information corresponding to the top-view image of the surface is updated based on the defogging target images corresponding to the plurality of block images. In this way, by defogging the block images with an appropriate range of defogging intensity parameters, the obtained defogging target images can better retain image information and clarity, thereby making the three-dimensional geographic information corresponding to the updated top-view image of the surface more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0062] Figure 1 It is a schematic diagram of the implementation flow of the method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention;
[0063] Figure 2 It is another implementation flow diagram of the method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention;
[0064] Figure 3 It is a schematic diagram of an application scenario of a method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention;
[0065] Figure 4 It is another implementation flow diagram of the method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention;
[0066] Figure 5 It is another implementation flow diagram of the method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention;
[0067] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0068] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a real-time update method of three-dimensional geographic information based on multi-feature fusion proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0069] Among them, in the description of the embodiments of the present invention, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" refers to two or more than two.
[0070] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.
[0071] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0072] The following specifically describes the application scenarios of the real-time updating method of three-dimensional geographic information based on multi-feature fusion provided by the present invention in conjunction with the accompanying drawings. Figure 1 , which shows a flow chart of a method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention, the method comprising:
[0073] 101. Collect and pre-process the surface top-view images through drones and other equipment.
[0074] 102, obtaining the block parameters of the image, and performing block processing on the image.
[0075] 103, obtaining a clarity index of each block image and obtaining a defogging intensity parameter range.
[0076] 104 , obtaining a defogging target image according to the defogging intensity parameter range of each block image, so as to obtain a plurality of defogging target images.
[0077] Here, the image with the best defogging effect (ie, the defogging target image) is obtained according to the defogging intensity parameter range of each block image, so as to obtain the image after the defogging operation of all the block images, that is, the defogging target image corresponding to all the block images.
[0078] 105, constructing a three-dimensional map of the defogging target image and updating the three-dimensional geographic information.
[0079] In an embodiment of the present invention, when the collected surface overhead image is defogging, since the image is a top-view image of the surface, and different positions in the surface overhead image are at different distances from the acquisition device. The degree of interference by the fog may be different. In order to better defog the surface overhead image, the clarity and details are better preserved. First, the grayscale histogram of the surface overhead image is analyzed, and the surface overhead image and the grayscale image of the surface overhead image are processed in blocks. Then, each block image is analyzed to obtain the grayscale gradient distribution characteristics of the pixel points of each block image. Finally, the gradient distribution characteristics of the block image are further analyzed to obtain the clarity index; and the clarity index of each block image is analyzed to obtain the defogging intensity parameter range; and each block image is defogged through the defogging intensity parameter range to obtain the best image after defogging of each block image, so as to obtain the defogging fused image. Thereby improving the accuracy of subsequent three-dimensional map construction and geographic information data update using the defogging fused image.
[0080] The following is a detailed description of the specific scheme of the real-time updating method of three-dimensional geographic information based on multi-feature fusion provided by the present invention in conjunction with the accompanying drawings. Figure 2 , which shows a flow chart of a method for real-time updating of three-dimensional geographic information based on multi-feature fusion provided by an embodiment of the present invention, the method comprising:
[0081] 201, obtain a bird's-eye view image of the surface.
[0082] Here, in the process of building a three-dimensional map using multi-feature data fusion, it is necessary to perform defogging on the surface visual images collected in foggy weather. Since the surface overhead image is a three-dimensional surface image, the height of each position in the image may be different, and the degree of interference from the fog is different. If the defogging intensity parameter is too small, the defogging effect will be poor, and if the defogging intensity parameter is too large, it may cause excessive processing of the image and loss of details and brightness. Therefore, when defogging the image, it is necessary to select a suitable defogging intensity parameter to ensure that the processed image information and clarity are better retained when defogging the surface overhead image.
[0083] In some possible implementations, first, geographic data is acquired through a variety of sensors, and data preprocessing is performed to ensure quality and consistency. Then, geometric features such as normals and curvatures are extracted from the data, and semantic information is extracted through deep learning or image processing, while considering the temporal characteristics of time changes. The feature fusion stage integrates data from different sources into a unified model and adjusts feature weights based on accuracy. The change detection algorithm identifies changes in geographic information and updates the database or model in real time. When using multi-feature data to construct a three-dimensional map, the collected surface visual image is interfered by heavy fog, which may affect the accuracy of obtaining information on the surface overhead view image during the subsequent construction and update of map information. Therefore, it is necessary to perform defogging operations on the collected images. Use drones or other equipment to collect surface visual images, obtain surface overhead views, and grayscale images of surface overhead views. Among them, the grayscale image of the surface overhead view image and the grayscale image after defogging, such as Figure 3 As shown, image 301 represents a grayscale image of a top-view image of the surface, and image 302 represents a grayscale image after defogging.
[0084] 202, performing block processing on the surface bird's-eye view image to obtain a plurality of block images.
[0085] Here, firstly, the block parameters of the surface top view image are obtained, and the surface top view image and the grayscale image are processed by block processing to obtain multiple block images; then, each block image is analyzed to obtain the clarity index of each block image, and the defogging intensity parameter range is obtained; finally, the image with the best defogging effect is obtained according to the defogging intensity parameter range of each block image, and then the defogging target image after all images are defogged is obtained. In some possible implementations, the above step 202 can be performed by Figure 4 The steps shown achieve:
[0086] 401, determine a grayscale image of the surface overhead image.
[0087] Here, the surface overhead view image is gray-scale converted to obtain a gray-scale image of the surface overhead view image.
[0088] 402 : Determine a block parameter based on the grayscale value of the grayscale image and the grayscale value of the surface overhead image.
[0089] Here, the mean value of the grayscale value of the grayscale image and the grayscale value of the surface overhead image are combined to obtain the grayscale image of the surface overhead image. In the process of real-time updating and fusion of three-dimensional geographic information, it is usually necessary to use drones or high-precision equipment to collect visual information of the surface in order to extract features such as buildings, vegetation, and landforms. At the same time, accurate three-dimensional data such as ground elevation and building height are collected using equipment such as laser radar, and three-dimensional stereo maps are constructed using data collected by other equipment using modeling technology. In the process of constructing a three-dimensional stereo map, the accuracy of the collected data is the key to the accuracy of the three-dimensional stereo map construction. In the process of constructing a three-dimensional stereo map, the collected surface visual image is more susceptible to natural environmental factors. For example, when obtaining surface visual information in foggy weather, due to the influence of foggy weather, the obtained image has a hazy feeling, which may affect the accuracy of three-dimensional map drawing, thereby affecting the accuracy of three-dimensional map construction. Therefore, it is necessary to defog the images collected on foggy days.
[0090] In the process of collecting images of the surface using drones and other equipment, since the surface is a three-dimensional structure, when looking down at the surface overhead image, objects with higher heights such as buildings are closer to the acquisition equipment, so the imaging equipment is closer to it, and the fog blocks and loses less light, so the image clarity at this location may be clearer than that at a lower location; that is, when collecting three-dimensional surface images, the degree of haze influence on different locations of the surface overhead image may be different. When using the defogging algorithm to defog the surface overhead image, the degree of haze influence on different locations in the image is different, so the size of the parameters controlling the defogging intensity may also be different, so as to ensure that the locations with different haze influence degrees are appropriately defogged, thereby ensuring the overall defogging effect of the image. Therefore, in order to more accurately defog the surface overhead image, the overall haze of the surface overhead image is first evaluated, and then the surface overhead image is adaptively divided into blocks to facilitate the subsequent better removal of haze at different locations of the surface overhead image. Then, when dividing the image into blocks, first obtain the grayscale histogram of the surface top view image, and obtain the number of extreme points, the maximum grayscale value and the minimum grayscale value, and the grayscale mean of the pixel points in the grayscale image of the surface top view image. Then, analyze the grayscale value to obtain the block parameters.
[0091] In some possible implementations, first, the difference between the maximum and minimum values in the grayscale value of the surface top view image is determined. Then, the difference and the mean of the grayscale value of the grayscale image are adjusted using a preset image segmentation hyperparameter to obtain the segmentation parameter. The segmentation parameter of the surface top view image is shown in formula (1):
[0092] (1);
[0093] Among them, C represents the block parameter of the surface top view image; N represents the number of extreme points of the grayscale histogram of the surface top view image. The smaller the value, the more uniform the grayscale distribution of the current image, and the image details may be blurred. In order to better preserve the details when defogging the image, the more blocks are used, so that the local information analysis is more accurate during the defogging process. , They respectively represent the maximum and minimum values of the grayscale histogram of the surface overhead image. It represents the average gray value of the pixels in the grayscale image of the surface top view. S represents the image block hyperparameter, and the value here is 16 (this value can be an empirical value). represents an exponential function with a natural constant as base; Ceiling sign. It represents the difference between the maximum and minimum values of the grayscale histogram of the surface overhead image. The difference represents the grayscale distribution range of the grayscale image pixels of the surface overhead image. The smaller the difference is, the closer the color of the surface overhead image is as a whole, and some details may not be prominent enough, so more blocks are needed to analyze its local changes. It indicates the difference between the gray value 255 and the average gray value of the pixel in the grayscale image of the surface top view. The smaller the value, the whiter the overall color of the current grayscale image is, and the more serious the haze may be. When performing block defogging for three-dimensional surface imaging, more block parameters may be required. In this way, by combining the difference between the maximum and minimum grayscale values of the surface top view image with the mean grayscale value of the grayscale image, the block parameters can be obtained more accurately to facilitate image segmentation.
[0094] 403 , using the block parameters to block the grayscale image and the surface overhead image to obtain the multiple block images.
[0095] Here, the block parameters are obtained through the above, and then the current surface top view image and grayscale image are divided into Thus, by converting the surface overhead image into a grayscale image, the grayscale image and the surface overhead image are divided into blocks according to the block parameters of the surface overhead image, so that multiple block images can be accurately and quickly obtained, so as to accurately analyze the defogging intensity parameter range of the image through the multiple block images.
[0096] 203 : Determine a defogging intensity parameter range based on the clarity indicators of the plurality of block images.
[0097] Here, after obtaining multiple block images, the clarity index of each block image is further calculated. Through the above block processing of the surface overhead image and the grayscale image of the surface overhead image, when defogging each block of the image, it is necessary to analyze the block image, select the appropriate defogging intensity parameter, and defog it. For the blocks with poor display effects in the blocks, it may be severely affected by haze and have poor clarity, and the defogging intensity may be large. For the blocks with good display effects in the blocks, it may be less affected by haze, and when defogging them, the defogging intensity parameter may be small. Therefore, when defogging each block, its clarity is first estimated and calculated.
[0098] In some possible implementations, first, cluster each of the multiple block images to obtain multiple cluster clusters; for example, perform K-means clustering on each block image to obtain a cluster cluster of the block image. Then, based on the multiple cluster clusters, determine the gradient distribution characteristics of each block image.
[0099] Here, first, the Sobel operator is used to calculate the gradient amplitude of each pixel in the block image; then, the gradient amplitudes obtained for each block pixel are sorted in ascending order, so that each block image has a pixel gradient amplitude sequence. For the gradient amplitude sequence in any block image, the K-means clustering algorithm is used to cluster the gradient amplitude sequence of the block image into two categories; and the two clusters are marked as cluster a and cluster b from left to right. Through the analysis of the cluster clusters, the calculation formula for one of the measurement indicators of the clarity of the block image is shown in formula (2):
[0100] (2);
[0101] in, Represents the gradient distribution characteristics of the i-th block image. Indicates the number of pixels contained in cluster a in the i-th block image; They respectively represent the number of pixels contained in cluster b in the i-th block image; represents the mean value of the gradient amplitude of the pixels in cluster b in the i-th block image; represents the mean value of the gradient amplitude of the pixels in cluster a in the ith block image; c represents a constant parameter to prevent the denominator from being 0, and the value here is 0.001 (empirical value). It represents the ratio between the number of pixels in cluster b and the number of pixels in cluster a in the i-th block image. The larger the value, the more pixels belonging to details such as edges in the block, and the larger the clarity feature index of the block. It represents the ratio of the mean gradient amplitude of the pixels in cluster b to the mean gradient amplitude of the pixels in cluster a in the i-th block image. The larger the value, the more prominent the pixels belonging to details such as edges in the block image are, and the higher the clarity of the block image may be. Therefore, The product of indicates that the more obvious the gradient distribution characteristics of the pixels in the block image are, the higher the display clarity of the block image may be.
[0102] Finally, based on the grayscale value of each block image and the gradient distribution feature, the clarity index of each block image is determined.
[0103] In some possible implementations, the clarity index of the block image can be determined by the following process:
[0104] First, based on the grayscale value of each block image, the grayscale value and the average grayscale value of any pixel in each block image are determined. Then, the difference between the grayscale value of any pixel and the average grayscale value, as well as the gradient distribution feature, are fused to obtain the clarity index of each block image. In this way, by calculating the difference between the gradient distribution feature and the grayscale value of the block image, the clarity index of the block image can be quickly analyzed.
[0105] Here, when the block image is relatively clear, the color difference between the pixels in the block image is also relatively large, because the colors of different pixels may represent the color difference of different objects or different positions of the same object. When the block image is seriously affected by haze, the block image will show an unclear and hazy feeling. At this time, the color difference of the pixels in the block image may be affected by the haze, resulting in a small color difference between the pixels in the block image. Therefore, the calculation formula for the clarity in the block image is shown in formula (3):
[0106] (3);
[0107] in, Represents the clarity index in the i-th block image; represents the linear normalization function; Represents the gray value of the Lth pixel in the i-th block image; Represents the average gray value of the pixels in the i-th block image; Indicates the number of pixels in the block; It represents the gradient distribution characteristic value in the i-th block image. The larger the value is, the more obvious the gradient distribution of the pixels in the block image is, and the higher the clarity of the block image may be. The mean of the sum of the absolute values of the difference between the grayscale value of each pixel in the i-th block image and the average grayscale value of the pixels in the block, that is, the degree of difference in the color of the pixels in the block. The larger the value, the clearer the current block image may be. In this way, by combining the grayscale value of each block image with the gradient distribution characteristics of the block image, the clarity index of the block image can be accurately analyzed, so as to further determine the defogging intensity parameter range through the clarity index. Through the above process, the clarity index of each block image is obtained. When each block image is defogged, the intensity of defogging is more related to the degree to which the block image is affected by haze. When the block image is less clear, it means that it may be more affected by haze. Then, when it is defogged, the defogging intensity may be greater. Therefore, the clarity of each block image can roughly determine its defogging intensity.
[0108] In some possible implementations, the above step 203 may be implemented by the following steps 231 to 233 (not shown):
[0109] 231. Determine a minimum value of a defogging intensity parameter based on the clarity index and a preset defogging intensity control range constant.
[0110] For example, when defogging is performed on each block image, the defogging intensity parameter range is expressed as: , where R represents the defogging intensity control range constant, which is 2 here. Represents the clarity index in the i-th block image. This is the minimum value of the dehazing intensity parameter.
[0111] 232. Determine a maximum value of a defogging intensity parameter based on the preset defogging intensity control range constant.
[0112] in, This is the maximum value of the dehazing intensity parameter.
[0113] 233. Obtain the defogging intensity parameter range based on the minimum defogging intensity parameter value and the maximum defogging intensity parameter value.
[0114] Here, the minimum value of the defogging intensity parameter and the maximum value of the defogging intensity parameter are combined to obtain the defogging intensity parameter range. In this way, by combining the clarity index of the block image with the preset defogging intensity control range parameter, the maximum value and the minimum value of the defogging intensity parameter can be accurately determined, thereby obtaining the defogging intensity parameter range of the block image.
[0115] 204 , performing defogging processing on the plurality of block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the plurality of block images.
[0116] Here, after obtaining the defogging intensity parameter range of each block image, defogging is performed on each block image multiple times according to the defogging intensity parameter range, and a defogging target image with the best defogging effect is selected from multiple defogging initial images. In some possible implementations, the above step 204 can be performed by Figure 5 The steps shown achieve:
[0117] 501 , performing multiple defogging operations on each block image based on the defogging intensity parameter range to obtain multiple defogging initial images.
[0118] Here, the defogging intensity parameter range of each block image is obtained through the above process. When performing defogging enhancement on any block image, the defogging operation is first performed with the minimum value of the defogging intensity parameter range corresponding to the block image, and then the defogging intensity parameter of the block is traversed from small to large with a step size of 0.02 to obtain multiple defogging images, that is, multiple defogging initial images and the clarity index of the image obtained after each defogging are obtained.
[0119] 502 : Determine defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images.
[0120] Here, by determining the average clarity of the clarity indexes corresponding to the multiple defogging initial images, and based on the clarity index of each defogging initial image in the multiple defogging initial images and the average clarity, the defogging effect evaluation value of each defogging initial image is determined. In this way, by first calculating the clarity index and the average clarity of each defogging initial image, the defogging effect evaluation value of each defogging initial image can be quickly and accurately analyzed, so as to screen out the defogging target image with better defogging effect from each defogging initial image.
[0121] In some possible implementations, after obtaining multiple defogging initial images and corresponding clarity indicators, the defogging effect evaluation value of each defogging initial image is calculated, as shown in formula (4):
[0122] (4);
[0123] in, It represents the defogging effect evaluation value of the defogging initial image obtained after the rth defogging of the i-th block image; Represents the clarity index of the i-th block image after the r-th defogging (that is, the clarity index of the r-th defogging initial image corresponding to the i-th block image); It indicates the number of images obtained by performing the dehazing operation on the i-th block image within the dehazing intensity parameter range. It indicates the mean value of the image display clarity index (i.e., the clarity mean) obtained by performing multiple dehazing operations on the i-th block image. It represents the ratio between the clarity index of the i-th block image after r-th defogging and the average value of the image display clarity index obtained by multiple defogging of the block image. The larger the value is, the better the r-th defogging effect of the block is likely to be, that is, the higher the defogging effect evaluation value is.
[0124] 503 : Based on the defogging effect evaluation values of the multiple defogging initial images, determine a defogging target image corresponding to each block image in the multiple defogging initial images.
[0125] Here, when the rth dehazed initial image When the value of is the largest, the defogging initial image is retained. The same operation is performed on the remaining block images. After that, the defogging target image after defogging each block image is restored into a whole image in the order of position, so that the image after defogging the top view image of the surface can be obtained, that is, the restored image. In this way, by comparing the defogging effect evaluation values of multiple defogging initial images, the image with the largest defogging effect evaluation value is selected from the multiple defogging initial images, and the defogging target image can be obtained.
[0126] 205. Based on the defogging target images corresponding to the multiple block images, update the three-dimensional geographic information corresponding to the surface bird's-eye view image.
[0127] Here, first, the defogging target images corresponding to the multiple block images are spliced according to the position order corresponding to the multiple block images to obtain a restored image; wherein the restored image is the image restored after the defogging target images are spliced, and the image information and image details of the original surface overhead image can be well preserved in the restored image. Then, based on the restored image, the three-dimensional geographic information corresponding to the surface overhead image is updated; wherein, the three-dimensional geographic information corresponding to the surface overhead image is updated by performing three-dimensional modeling on the restored image. In this way, by splicing the defogging target images with better defogging effect, a restored image with higher image quality can be obtained, thereby making the updated three-dimensional geographic information more accurate.
[0128] In some possible implementations, first, geometric features and temporal features are extracted from the restored image; then, the geometric features and the temporal features are multi-feature fused to obtain fused features, for example, the geometric features and the temporal features are spliced to obtain fused features; finally, based on the fused features, the three-dimensional geographic information corresponding to the surface overhead image is updated. Here, geometric features, such as normals and curvatures, are extracted from the spliced restored image. Information of the surface overhead image is obtained by deep learning or image feature extraction, while considering temporal features of time changes. In the feature fusion stage, data from different sources are integrated into a unified model, and a large-area three-dimensional stereo map is constructed for the fused features using three-dimensional modeling technology (such as point cloud data processing, surface reconstruction, etc.). The weight of the acquired data features is adjusted according to the accuracy. The change of geographic information is identified by using a change detection algorithm, and the database or three-dimensional geographic model is updated in real time to update the three-dimensional geographic information corresponding to the surface overhead image. In this way, by performing multi-feature fusion on restored images with better image quality to update the three-dimensional geographic information of the surface bird's-eye view image, the accuracy of constructing a three-dimensional map and updating the three-dimensional geographic information can be improved.
[0129] An embodiment of the present invention provides a real-time updating device for three-dimensional geographic information based on multi-feature fusion, and the device includes: an acquisition module, used to acquire a surface overhead view image; a blocking module, used to block the surface overhead view image to obtain a plurality of block images; a determination module, used to determine a defogging intensity parameter range based on clarity indicators of the plurality of block images; a defogging module, used to defog the plurality of block images based on the defogging intensity parameter range to obtain defogged target images corresponding to the plurality of block images; and an updating module, used to update the three-dimensional geographic information corresponding to the surface overhead view image based on the defogging target images corresponding to the plurality of block images.
[0130] In some possible implementations, the blocking module is also used to determine a grayscale image of the surface overhead view image; determine blocking parameters based on the grayscale values of the grayscale image and the grayscale values of the surface overhead view image; and use the blocking parameters to block the grayscale image and the surface overhead view image to obtain the multiple block images.
[0131] In some possible implementations, the blocking module is also used to determine the difference between the maximum and minimum values in the grayscale value of the top view image; using preset image blocking hyperparameters, the difference and the mean of the grayscale value of the grayscale image are adjusted to obtain the blocking parameters.
[0132] In some possible implementations, the determination module is further used to cluster each of the multiple block images to obtain multiple cluster clusters; determine the gradient distribution characteristics of each block image based on the multiple cluster clusters; and determine the clarity index of each block image based on the grayscale value of each block image and the gradient distribution characteristics.
[0133] In some possible implementations, the determination module is also used to determine the grayscale value and average grayscale value of any pixel in each block image based on the grayscale value of each block image; the difference between the grayscale value of any pixel and the average grayscale value, and the gradient distribution feature are merged to obtain the clarity index of each block image.
[0134] In some possible implementations, the determination module is also used to determine the minimum value of the defogging intensity parameter based on the clarity index and a preset defogging intensity control range constant; determine the maximum value of the defogging intensity parameter based on the preset defogging intensity control range constant; and obtain the defogging intensity parameter range based on the minimum value of the defogging intensity parameter and the maximum value of the defogging intensity parameter.
[0135] In some possible implementations, the defogging module is further used to perform multiple defogging operations on each block image based on the defogging intensity parameter range to obtain multiple defogging initial images; determine the defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images; and determine the defogging target image corresponding to each block image in the multiple defogging initial images based on the defogging effect evaluation values of the multiple defogging initial images.
[0136] In some possible implementations, the defogging module is further used to determine a clarity mean of clarity indicators corresponding to the multiple defogging initial images; and determine a defogging effect evaluation value of each defogging initial image based on the clarity indicator of each defogging initial image in the multiple defogging initial images and the clarity mean.
[0137] In some possible implementations, the update module is also used to splice the defogging target images corresponding to the multiple block images according to the position order corresponding to the multiple block images to obtain a restored image; based on the restored image, update the three-dimensional geographic information corresponding to the surface bird's-eye view image.
[0138] In some possible implementations, the updating module is further used to extract geometric features and temporal features from the restored image; perform multi-feature fusion on the geometric features and the temporal features to obtain fused features; and update the three-dimensional geographic information corresponding to the surface overhead view image based on the fused features.
[0139] Optionally, the transmission medium may be a wired link (such as but not limited to coaxial cable, optical fiber and digital subscriber line (DSL), etc.) or a wireless link (such as but not limited to wireless Fidelity (WIFI), Bluetooth and mobile device network, etc.). It should be noted that the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the shopping guide device provided in the above embodiment belongs to the same concept as the image extension method embodiment. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0140] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, the electronic device 600 includes: a memory 601, a processor 602, and a computer program 603 stored in the memory 601 and running on the processor 602, wherein when the processor 602 executes the computer program 603, the device can execute any of the three-dimensional geographic information real-time update methods based on multi-feature fusion described above. In addition, an embodiment of the present invention also protects a device, which may include a memory and a processor, wherein an executable program code is stored in the memory, and the processor is used to call and execute the executable program code to execute the three-dimensional geographic information real-time update method based on multi-feature fusion provided by an embodiment of the present invention.
[0141] This embodiment can divide the functional modules of the device according to the above method example. For example, it can correspond to each functional module, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation. In the case of dividing each module corresponding to each function, the device may also include a signal uploading module, a determination module, an adjustment module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.
[0142] It should be understood that the device provided in this embodiment is used to execute the above-mentioned real-time update method of three-dimensional geographic information based on multi-feature fusion, so the same effect as the above-mentioned implementation method can be achieved. In the case of an integrated unit, the device may include a processing module and a storage module. Among them, when the device is applied to a device, the processing module can be used to control and manage the actions of the device. The storage module can be used to support the device to execute mutual program codes, etc. Among them, the processing module can be a processor or a controller, which can implement or execute various exemplary logical boxes, modules and circuits described in conjunction with the disclosure of the present invention. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module can be a memory.
[0143] In addition, the device provided by the embodiment of the present invention may be specifically a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute the real-time update method of three-dimensional geographic information based on multi-feature fusion provided by the above embodiment. This embodiment also provides a computer-readable storage medium, in which a computer program code is stored, and when the computer program code is run on a computer, the computer executes the above-mentioned related method steps to implement the real-time update method of three-dimensional geographic information based on multi-feature fusion provided by the above embodiment.
[0144] The present embodiment also provides a computer program product, when the computer program product is run on a computer, the computer executes the above-mentioned related steps to implement the real-time update method of three-dimensional geographic information based on multi-feature fusion provided in the above embodiment. Among them, the device, computer-readable storage medium, computer program product or chip provided in the present embodiment are all used to execute the corresponding method provided above, so the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here. Through the description of the above implementation mode, the technicians in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by the present invention, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiment described above is only schematic, for example, the division of modules or units is only a logical function division, and there can be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.
[0145] It should be noted that the sequence of the above-mentioned embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-tasking and parallel processing are also possible or may be advantageous. The various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. The above content is only a specific implementation method of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention.
Claims
1. A real-time updating method of three-dimensional geographic information based on multi-feature fusion, characterized in that: The method comprises: Obtaining top-view images of the surface; Performing block processing on the surface bird's-eye view image to obtain a plurality of block images; The step of performing block processing on the surface overhead image to obtain a plurality of block images includes: Determining a grayscale image of the surface overhead image; Determining a block parameter based on the grayscale value of the grayscale image and the grayscale value of the surface overhead image; Using the block parameters to block the grayscale image and the surface overhead image to obtain the multiple block images; The determining of the block parameters based on the grayscale value of the grayscale image and the grayscale value of the surface overhead image includes: Determine the difference between the maximum value and the minimum value in the grayscale value of the surface top view image; Using a preset image segmentation hyperparameter, adjusting the difference and the mean of the grayscale value of the grayscale image to obtain the segmentation parameter; Clustering each of the plurality of block images to obtain a plurality of cluster clusters; Based on the multiple clusters, determining the gradient distribution characteristics of each block image; Determining a clarity index of each of the segmented images based on the grayscale value of each segmented image and the gradient distribution feature; The step of clustering each of the plurality of block images to obtain a plurality of cluster clusters; and determining the gradient distribution feature of each block image based on the plurality of cluster clusters includes: The Sobel operator is used to calculate the gradient amplitude of each pixel in the block image, and the obtained gradient amplitudes are sorted in ascending order. Each block image has a pixel gradient amplitude sequence; for the gradient amplitude sequence in any block image, the K-means clustering algorithm is used to cluster the gradient amplitude sequence of the block image into two categories; and the two clusters are marked as cluster a and cluster b from left to right; the gradient distribution feature calculation formula of the block image is: in, Represents the gradient distribution characteristics of the i-th block image; Indicates the number of pixels contained in cluster a in the i-th block image; They respectively represent the number of pixels contained in cluster b in the i-th block image; represents the mean value of the gradient amplitude of the pixels in cluster b in the i-th block image; represents the mean value of the gradient amplitude of the pixels in cluster a in the i-th block image; c represents a constant parameter; The determining of the clarity index of each block image based on the grayscale value of each block image and the gradient distribution feature includes: Based on the grayscale value of each block image, determine the grayscale value and average grayscale value of any pixel in each block image; fuse the difference between the grayscale value of any pixel and the average grayscale value, and the gradient distribution feature to obtain the clarity index of each block image; Determining a defogging intensity parameter range based on the clarity indexes of the plurality of block images; Performing defogging processing on the plurality of block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the plurality of block images; Based on the defogging target images corresponding to the multiple block images, updating the three-dimensional geographic information corresponding to the surface bird's-eye view image; The step of determining the defogging intensity parameter range based on the clarity indexes of the plurality of block images comprises: Determining a minimum value of a defogging intensity parameter based on the clarity index and a preset defogging intensity control range constant; Determining a maximum value of a defogging intensity parameter based on the preset defogging intensity control range constant; Based on the minimum value of the defogging intensity parameter and the maximum value of the defogging intensity parameter, obtaining the defogging intensity parameter range; The defogging process is performed on the plurality of block images based on the defogging intensity parameter range to obtain defogging target images corresponding to the plurality of block images, including: Performing multiple defogging operations on each block image based on the defogging intensity parameter range to obtain multiple defogging initial images; Determining defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images; Based on the defogging effect evaluation values of the multiple defogging initial images, a defogging target image corresponding to each block image is determined in the multiple defogging initial images.
2. The method for real-time updating of three-dimensional geographic information based on multi-feature fusion according to claim 1 is characterized in that: The step of determining the defogging effect evaluation values of the multiple defogging initial images based on the clarity indicators corresponding to the multiple defogging initial images comprises: Determine a clarity mean value of the clarity indicators corresponding to the multiple defogging initial images; Based on the clarity index of each defogging initial image in the multiple defogging initial images and the clarity mean, a defogging effect evaluation value of each defogging initial image is determined.
3. The real-time updating method of three-dimensional geographic information based on multi-feature fusion according to claim 1 is characterized in that: The updating of the three-dimensional geographic information corresponding to the surface bird's-eye view image based on the defogging target image corresponding to the multiple block images includes: splicing the defogging target images corresponding to the multiple block images according to the position order corresponding to the multiple block images to obtain a restored image; Based on the restored image, the three-dimensional geographic information corresponding to the surface bird's-eye view image is updated.
4. The real-time updating method of three-dimensional geographic information based on multi-feature fusion according to claim 3 is characterized in that: The updating of the three-dimensional geographic information corresponding to the surface bird's-eye view image based on the restored image includes: Extracting geometric features and temporal features from the restored image; Performing multi-feature fusion on the geometric features and the temporal features to obtain fused features; Based on the fused features, the three-dimensional geographic information corresponding to the surface bird's-eye view image is updated.
