Highway construction multi-source data fusion processing system based on cloud platform

Abstract

The invention relates to the technical field of multi-source data fusion, in particular to a road construction multi-source data fusion processing system based on a cloud platform, which comprises a processor and a memory, the processor executes a computer program stored in the memory so as to realize the following steps: acquiring a pixel point replacement number corresponding to each gray value in a gray value set corresponding to each target sub-block and a corresponding replacement gray value, and replacing the pixel point gray value in each target sub-block according to the replacement gray value and the pixel point replacement number corresponding to each gray value in the gray value set corresponding to each target sub-block to obtain a high-resolution image after replacement is completed. And replacing the first principal component of the multispectral image with the two-dimensional matrix of the replaced high-resolution image, and performing PCA inverse transformation on all principal components of the replaced multispectral image to obtain a fused image. And the fusion effect can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion, and particularly relates to a multi-source data fusion processing system for highway construction based on a cloud platform. Background Art

[0002] Since the multi-source data fusion in highway construction is crucial for improving the positioning accuracy, construction efficiency, construction quality, etc. of highway construction, currently, multi-source data fusion is required in highway construction. Usually, during the highway construction process, remote sensing images of the highway construction section need to be collected. The remote sensing images generally include high-resolution images and multi-spectral images. Then, the obtained high-resolution images and multi-spectral images are fused to obtain fused remote sensing image data with both spatial resolution and spectral information. That is, subsequently, by analyzing the completed fused images, it is crucial for aspects such as construction positioning and construction straightness.

[0003] In the prior art, traditional PCA transformation technology is usually used to fuse high-resolution images and multi-spectral images. However, during the process of using PCA transformation technology to fuse high-resolution images and multi-spectral images, the traditional histogram matching method may reduce the accuracy of important data on the high-resolution image or cause the loss of important data on the high-resolution image. For example, after fusion, the accuracy of data related to highway construction on the high-resolution image may be low or data related to highway construction may be lost. That is, the traditional histogram matching method will result in a poor fusion effect. When the fusion effect is poor, it will have a negative impact on aspects such as highway construction efficiency and construction quality. Therefore, how to improve the fusion effect of high-resolution images and multi-spectral images has become an urgent problem to be solved. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a multi-source data fusion processing system for highway construction based on a cloud platform. The specific technical solution adopted is as follows: An embodiment of the present invention provides a multi-source data fusion processing system for highway construction based on a cloud platform, including a processor and a memory. The processor executes the computer program stored in the memory to implement the following steps: During the highway construction process, obtain high-resolution images and multi-spectral images of the construction section; Use windows of different sizes to block the high-resolution image, obtain the evaluation values of the blocking effects of each window, and obtain each target sub-block on the high-resolution image according to the evaluation values of the blocking effects; Use the region growing algorithm to separately segment each of the target sub - blocks to obtain each segmentation region in the target sub - blocks. Based on the gray - scale values of the neighborhood pixels of each pixel point in each target sub - block and each segmentation region in each target sub - block, obtain the importance degree characterization value of each target sub - block; Perform principal component analysis on the multi - spectral image to obtain the principal components of the multi - spectral image, and based on the first principal component of the multi - spectral image, obtain the corresponding matching sub - blocks for each target sub - block; Based on the cumulative histogram of each target sub - block and the importance degree characterization value of each target sub - block, obtain the number of pixel points to be replaced corresponding to each gray - scale value in the gray - scale value set corresponding to each target sub - block. Based on the cumulative probability value on the cumulative histogram of each target sub - block and the cumulative probability value on the cumulative histogram of the matching sub - block, obtain the replacement gray - scale value corresponding to each gray - scale value in the gray - scale value set corresponding to each target sub - block. The gray - scale value set corresponding to the target sub - block is composed of all types of gray - scale values that appear in the target sub - block. Based on the replacement gray - scale value and the number of pixel points to be replaced corresponding to each gray - scale value in the gray - scale value set corresponding to each target sub - block, replace the pixel - point gray - scale values in each target sub - block to obtain the high - resolution image after replacement. Replace the first principal component of the multi - spectral image with the two - dimensional matrix of the high - resolution image after replacement, and perform inverse PCA transformation on all principal components of the replaced multi - spectral image to obtain the fused image.

[0005] Beneficial effects: The present invention first obtains high-resolution images and multi-spectral images of the construction section, and divides the high-resolution images into blocks using windows of different sizes to obtain the evaluation values of the block effects of each window. According to the evaluation values of the block effects, each target sub-block on the high-resolution image is obtained; then, the region growing algorithm is used to segment each target sub-block respectively to obtain each segmentation region in the target sub-block, and according to the gray values of the neighborhood pixels of each pixel point in each target sub-block and each segmentation region in each target sub-block, the characterization value of the importance degree of each target sub-block is obtained; then, principal component analysis is performed on the multi-spectral image to obtain the principal components of the multi-spectral image, and according to the first principal component of the multi-spectral image, the corresponding matching sub-blocks of each target sub-block are obtained; then, according to the cumulative histogram of each target sub-block and the characterization value of the importance degree of each target sub-block, the number of pixel points corresponding to each gray value in the gray value set corresponding to each target sub-block is obtained. According to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, the replacement gray value corresponding to each gray value in the gray value set corresponding to each target sub-block is obtained. According to the replacement gray value corresponding to each gray value in the gray value set corresponding to each target sub-block and the number of pixel point replacements, the gray values of the pixel points in each target sub-block are replaced to obtain the high-resolution image after replacement. The first principal component of the multi-spectral image is replaced with the two-dimensional matrix of the high-resolution image after replacement, and inverse PCA transformation is performed on all the principal components of the replaced multi-spectral image to obtain the fused image. Moreover, based on the number of pixel point replacements and the replacement gray values determined by the characterization value of the importance degree and the cumulative histogram, the present invention can not only combine multi-spectral information but also retain the accuracy of the information related to highway construction on the high-resolution image as much as possible during the image fusion process, so as to achieve the purpose of improving the fusion effect. Description of the Drawings

[0006] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0007] Figure 1 It is a flowchart of a method for fusing multi-source data in highway construction based on a cloud platform according to the present invention; Figure 2 It is a schematic diagram of dividing the high-resolution image into blocks. Detailed Embodiments

[0008] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the embodiments of the present invention.

[0009] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0010] This embodiment provides a multi-source data fusion processing system for highway construction based on a cloud platform, including a processor and a memory. The processor executes the computer program stored in the memory to implement a multi-source data fusion processing method for highway construction based on a cloud platform, as Figure 1 shown. The multi-source data fusion processing method for highway construction based on a cloud platform includes the following steps: Step S001, during the highway construction process, obtain high-resolution images and multi-spectral images of the construction section.

[0011] This embodiment mainly improves the fusion effect of high-resolution images and multi-spectral images by optimizing or improving the matching method of the cumulative histogram. That is, during the fusion process of this embodiment, by optimizing or improving the matching method of the cumulative histogram, not only can multi-spectral information be combined, but also the accuracy of the information related to highway construction on the high-resolution image can be retained as much as possible, thereby improving the fusion effect. And this embodiment mainly fuses the remote sensing images collected during the highway construction process. Therefore, this embodiment needs to first obtain remote sensing images next. The remote sensing images include high-resolution images and multi-spectral images.

[0012] During the highway construction process, this embodiment will use an unmanned aerial vehicle (UAV) to obtain remote sensing images of the highway construction section, and remote sensing image acquisition needs to be performed at each monitoring moment. Since the UAV in this embodiment is equipped with a high-resolution camera and a multi-spectral camera, the remote sensing images collected by the UAV in this embodiment include high-resolution images and multi-spectral images. The high-resolution images and multi-spectral images belong to remote sensing images. That is, at any acquisition moment, high-resolution images and multi-spectral images of the highway construction section can be collected by the UAV. For the sake of easy understanding, the high-resolution images and multi-spectral images obtained at any acquisition moment will be used for fusion processing in the following of this embodiment. In addition, the image acquisition frequency of the camera needs to be set by the implementer according to the actual situation. For example, it can be set that the cameras perform acquisitions every 1 second, and in this embodiment, it is required that the acquisition moments of the high-resolution camera and the multi-spectral camera are consistent.

[0013] It should be noted that during the highway construction process, in order to ensure the positioning accuracy, construction efficiency, construction quality, etc. of the highway, it is usually necessary to collect remote sensing images of the construction section at regular intervals. As the construction progresses, the subsequent collected remote sensing images will include images of the constructed section and the unconstructed section.

[0014] Therefore, through the above process, this embodiment can obtain high-resolution images and multi-spectral images that need to be fused. In addition, it should be noted that the collected high-resolution images and multi-spectral images will be transmitted to the cloud platform and then undergo a series of processing and analysis.

[0015] Step S002: Divide the high-resolution image into blocks using windows of different sizes to obtain the evaluation values of the block effects of each window. According to the evaluation values of the block effects, obtain each target sub-block on the high-resolution image.

[0016] Since currently in the process of fusing high-resolution images and multi-spectral images using PCA transformation technology, the traditional histogram matching method may reduce the accuracy of important data on the high-resolution image or cause the loss of important data on the high-resolution image. For example, after fusion, it may lead to a lower accuracy of data related to highway construction on the high-resolution image or the loss of data related to highway construction, that is, the traditional histogram matching method will result in a poor fusion effect. When the fusion effect is poor, it will have a negative impact on aspects such as highway construction efficiency and construction quality. And the traditional histogram matching method means directly matching the cumulative histogram of the complete high-resolution image with the cumulative histogram of the complete multi-spectral image. In order to improve the fusion effect of high-resolution images and multi-spectral images in this embodiment, this embodiment will subsequently provide a new histogram matching method, that is, this embodiment will subsequently improve or optimize the histogram matching method to improve the fusion effect, that is to say, this embodiment will improve or optimize the histogram matching method so that important data accuracy can be retained as much as possible during the fusion process of remote sensing images.

[0017] Based on the above analysis, it can be seen that in this embodiment, next, the high-resolution image will be divided into blocks using windows of different sizes to obtain the evaluation values of the block effects of each window, and then according to the evaluation values of the block effects of each window, each target sub-block on the high-resolution image will be obtained, and the target sub-block is the basis for implementing the improvement or optimization of histogram matching.

[0018] Based on the above analysis, it can be seen that in this embodiment, it is necessary to first determine the size interval to be selected. And in specific applications, the implementer needs to set the minimum and maximum values of the size interval to be selected according to the actual situation. For example, in this embodiment, the minimum value can be set to 2, and the maximum value can be set to one-fourth of the length of the high-resolution image. Then, all integer values in the size interval to be selected are obtained and are all recorded as window size values. After that, the set constructed by all the window size values is recorded as the window size value set. Then, the windows corresponding to each window size value in the window size value set are obtained. And if the z-th window size value in the window size value set is x, then the length and width of the window corresponding to the z-th window size value are also x.

[0019] After obtaining the windows corresponding to each window size value in the window size value set, the high-resolution image is respectively segmented using the windows corresponding to each window size value in the window size value set, and the block effect evaluation values corresponding to the windows of each window size value in the window size value set are obtained. In addition, for the sake of easy understanding, this embodiment will describe the process of obtaining the block effect evaluation value of the window q corresponding to any window size value, that is, the process of obtaining the block effect evaluation value of the window corresponding to the window q is as follows: First, the high-resolution image is evenly and non-overlappingly segmented using the window q, and the results obtained from the segmentation are all recorded as the sub-blocks to be determined on the high-resolution image under the window q. And the process of giving an example of evenly and non-overlappingly segmenting the high-resolution image using the window q is as follows: First, a window with the size of the window q is obtained at the upper leftmost corner of the high-resolution image and is recorded as the sliding window corresponding to the window q. Then, the sliding window corresponding to the window q slides in the order from top to bottom and from left to right. All the windows obtained on the high-resolution image after the sliding is completed are recorded as the sub-blocks to be determined on the high-resolution image under the window q. And the sliding step of the sliding window corresponding to the window q during sliding is the length or width of the window q. The length and width of the window q are the same. If the length and width of the window q are 3, then the schematic diagram of evenly and non-overlappingly segmenting the high-resolution image using the window q is as Figure 2 shown, and Figure 2 the smallest square on it represents a pixel point, and Figure 2 the area divided by the thicker black border on it is the sub-block to be determined obtained after evenly and non-overlappingly segmenting the high-resolution image using the window q. And since the length or width of the high-resolution image may not be an integer multiple of the length or width of the window q, the size of the sub-block to be determined near the edge of the high-resolution image may be smaller than the size of the window q. For example, Figure 2 the area of the two sub-blocks to be determined on the rightmost side on it is relatively small compared to Figure 2 the other sub-blocks to be determined on it.

[0020] After obtaining each to-be-determined sub-block on the high-resolution image under window q, the concentration characterization values corresponding to each to-be-determined sub-block on the high-resolution image under window q are obtained. Then, the average value of the concentration characterization values corresponding to all to-be-determined sub-blocks on the high-resolution image under window q is calculated and denoted as the block effect evaluation value of window q. And the larger the concentration characterization value corresponding to the to-be-determined sub-block, the more concentrated the gray value distribution of the pixel points in the corresponding to-be-determined sub-block, and it also indicates that the area corresponding to the to-be-determined sub-block is more likely to be an environmental area or a highway construction area. And when the concentration characterization value corresponding to the to-be-determined sub-block is larger, it also indicates that the block effect evaluation value under window q is larger. And the larger the block effect evaluation value under window q, it indicates that when using window q to block the high-resolution image, the highway construction area and the background area can be separated to a large extent, so as to be able to retain the high-resolution image in the subsequent process.

[0021] In addition, the specific process of obtaining the concentration characterization values corresponding to each to-be-determined sub-block on the high-resolution image under the above window q is as follows: For any to-be-determined sub-block on the high-resolution image under window q: First, count all the types of gray values that appear in the to-be-determined sub-block, and denote the set composed of all types of gray values that appear in the to-be-determined sub-block as the first set. For example, if the types of gray values that appear in the to-be-determined sub-block are only 8 and 19, then the subsequent first set is composed of gray values 8 and 19. Then, count the frequencies of each gray value in the first set that appear in the to-be-determined sub-block, and denote the square value of the frequency of each gray value in the first set that appears in the to-be-determined sub-block as the frequency characterization value of the corresponding gray value. Then, obtain the sum of the frequency characterization values of all gray values in the first set and denote it as the first eigenvalue corresponding to the to-be-determined sub-block. Obtain the result of subtracting the minimum gray value in the to-be-determined sub-block from the maximum gray value in the to-be-determined sub-block and denote it as the second eigenvalue corresponding to the to-be-determined sub-block. Finally, denote the product of the first eigenvalue corresponding to the to-be-determined sub-block and its corresponding second eigenvalue as the concentration characterization value corresponding to the to-be-determined sub-block. And it should be noted that the pixel gray value in the to-be-determined sub-block is the pixel gray value obtained by performing gray value processing on the corresponding to-be-determined sub-block. In addition, the larger the first eigenvalue and the second eigenvalue corresponding to the to-be-determined sub-block, the fewer the types of gray values in the to-be-determined sub-block, which can indicate that there are more pixel points with the same gray value in the to-be-determined sub-block, and thus it also indicates that the concentration of the pixel gray value distribution in the to-be-determined sub-block is better.

[0022] Therefore, through the above method, this embodiment can obtain the evaluation values of the block effect of the windows corresponding to each window size value in the window size value set. After obtaining the evaluation values of the block effect, the target sub-blocks on the high-resolution image are obtained according to the evaluation values of the block effect of the windows corresponding to each window size value. The specific process of obtaining the target sub-blocks is as follows: Among the evaluation values of the block effect corresponding to all windows, the window corresponding to the maximum evaluation value of the block effect is selected as the optimal window, and all sub-blocks obtained by evenly and non-overlappingly dividing the high-resolution image using the optimal window are recorded as the target sub-blocks on the high-resolution image. The method of evenly and non-overlappingly dividing the high-resolution image using any window is the same as the method of evenly and non-overlappingly dividing the high-resolution image using window q described above, so it will not be described in detail here.

[0023] Therefore, through the above process, this embodiment obtains each target sub-block on the high-resolution image.

[0024] Step S003: Use the region growing algorithm to separately segment each of the target sub-blocks to obtain each segmentation region in the target sub-blocks. According to the gray values of the neighboring pixels of each pixel point in each of the target sub-blocks and each segmentation region in each of the target sub-blocks, obtain the characterization value of the importance degree of each of the target sub-blocks.

[0025] In this embodiment, the characterization value of the importance degree of the target sub-blocks will be obtained next. The characterization value of the importance degree is an important parameter for subsequent cumulative histogram matching and is also an important parameter for realizing the precision of retaining important data. The important data refers to the data related to highway construction on the high-resolution image. When obtaining the characterization value of the importance degree, it is necessary to first use the region growing algorithm to separately segment each of the target sub-blocks to obtain each segmentation region in each of the target sub-blocks. The result of segmenting the target sub-blocks using the region growing algorithm is an important parameter for subsequently obtaining the characterization value of the importance degree of the corresponding target sub-blocks. Then, in this embodiment, the specific process of using the region growing algorithm to separately segment each of the target sub-blocks to obtain each segmentation region in each of the target sub-blocks is as follows: For any target sub-block, first, initial seed points are placed in the target sub-block, and then the initial seed points in the target sub-block start to perform region growing. During the growing process, if there are no other seed points that meet the growing conditions within the neighborhood range of all seed points, the growth is immediately stopped, and all the growing regions obtained at the end of the growth are recorded as the segmentation regions in the target sub-block. The neighborhood range of the seed point refers to the region range formed by the eight-neighborhood of the seed point and all the pixel points in the target sub-block, which is called the neighborhood range of the corresponding seed point. Additionally, when the absolute value of the gray-level difference between a certain pixel point within the neighborhood range of a seed point and the seed point is less than the preset gray-level difference threshold, it is determined that there are other seed points that meet the growing conditions within the neighborhood range of the seed point. And in the case where the initial seed points and the growing conditions are known, the process of region growing is a well-known technology, so it will not be described in detail in this embodiment.

[0026] In specific applications, the implementer needs to set the preset gray-level difference threshold according to the actual situation. For example, in this embodiment, the preset gray-level difference threshold can be set to 3.

[0027] Therefore, through the above process, this embodiment can obtain all the segmentation regions in each target sub-block. After obtaining all the segmentation regions in each target sub-block, the importance characterization value of each target sub-block is obtained according to the gray-level values of the neighborhood pixels of each pixel point in each target sub-block and each segmentation region in each target sub-block. And for the sake of easy understanding, the following will describe the process of obtaining the importance characterization value of target sub-block a as an example. That is, the process of obtaining the importance characterization value of target sub-block a is as follows: First, according to the gray-level values of the neighborhood pixels of each pixel point in target sub-block a, the neighborhood distribution characterization value corresponding to each pixel point in target sub-block a is obtained. Then, according to the neighborhood distribution characterization values corresponding to all the pixel points in target sub-block a and the gray-level values of all the pixel points in target sub-block a, the gray-level distribution feature value corresponding to target sub-block a is obtained. And since the area around the road is generally a natural environment composed of various terrains such as trees, grasslands or hillsides, while the road area or the area of the road under construction is artificially built, so relatively speaking, compared with the area around the road, the gray-level distribution of the road area or the area of the road under construction is more uniform and shows a high-brightness feature. That is, the artificially built areas such as road buildings are gray-level uniform and high-brightness, while the background natural environment has various terrains such as trees, grasslands or hillsides, with low brightness generated by reflecting natural light and relatively poor uniformity. And because the larger the gray-level distribution feature value corresponding to target sub-block a, the higher the gray-level uniformity within target sub-block a and the more obvious the high-brightness feature within target sub-block a. Therefore, the larger the gray-level distribution feature value corresponding to target sub-block a, the greater the possibility that the area corresponding to target sub-block a belongs to an artificially built area such as a road building.

[0028] In this embodiment, the specific process of obtaining the neighborhood distribution characterization value corresponding to each pixel point in the target sub-block a according to the gray values of the neighborhood pixel points of each pixel point in the target sub-block a is as follows: For any pixel point b in the target sub-block a: First, denote all the pixel points located in the target sub-block a and belonging to the eight-neighborhood of the pixel point b as the neighborhood pixel point set corresponding to the pixel point b. Then, obtain the neighborhood difference corresponding to each pixel point in the neighborhood pixel point set corresponding to the pixel point b. The neighborhood difference corresponding to the c-th pixel point in the neighborhood pixel point set corresponding to the pixel point b refers to the absolute value of the gray difference between the c-th pixel point and the pixel point b. Denote the set constructed by all the neighborhood differences corresponding to the pixel points in the neighborhood pixel point set corresponding to the pixel point b as the neighborhood difference set corresponding to the pixel point b. Then, calculate the sum of all the neighborhood differences in the neighborhood difference set corresponding to the pixel point b and use it as the comprehensive neighborhood difference of the pixel point b. Finally, denote the reciprocal of the result obtained by adding the comprehensive neighborhood difference of the pixel point b and a preset first constant as the neighborhood distribution characterization value of the pixel point b. And in specific applications, the implementer needs to set the preset first constant according to the actual situation. The role of the preset first constant here is to prevent the denominator from being 0. For example, in this embodiment, the preset first constant can be set to 1.

[0029] In this embodiment, the specific process of obtaining the gray distribution feature value corresponding to the target sub-block a according to the neighborhood distribution characterization values corresponding to all the pixel points in the target sub-block a and the gray values of all the pixel points in the target sub-block a is as follows: Obtain the gray mean value of all the pixel points in the target sub-block a and denote it as the first gray mean value. Obtain the sum of the neighborhood distribution characterization values of all the pixel points in the target sub-block a and denote it as the comprehensive distribution characterization value. Denote the normalized value of the result obtained by adding the comprehensive distribution characterization value and the first gray mean value as the gray distribution feature corresponding to the target sub-block a, and the gray distribution feature corresponding to the target sub-block a is , where Norm() is the normalization function, is the result obtained by adding the comprehensive distribution characterization value and the first gray mean value, and the larger the comprehensive distribution characterization value and the first gray mean value are, the larger the gray distribution feature value corresponding to the target sub-block a is.

[0030] Next, obtain the minimum bounding rectangles of each segmented region in the target sub-block a, and obtain the region feature values corresponding to each segmented region in the target sub-block a according to the areas of the minimum bounding rectangles of each segmented region in the target sub-block a; then obtain the mean value of the region feature values corresponding to all the segmented regions in the target sub-block a, and denote it as the region feature mean value. After that, use the normalization function Norm() to normalize the region feature mean value, and denote the result obtained by the normalization process as the spatial distribution feature value corresponding to the target sub-block a; and since the shape distribution characteristics of the highway construction area tend to be rectangles with edges and corners compared to the natural environment around the highway, while the shape distribution characteristics of the natural environment around the highway generally present irregular characteristics compared to the highway construction area, so in this embodiment, the larger the spatial distribution feature value corresponding to the target sub-block a, the greater the possibility that the region corresponding to the target sub-block a belongs to man-made construction areas such as highway construction.

[0031] In this embodiment, the specific process of obtaining the region feature values corresponding to each segmented region in the target sub-block a according to the areas of the minimum bounding rectangles of each segmented region in the target sub-block a is as follows: For the d-th segmented region in the target sub-block a: Denote the minimum bounding rectangle of the d-th segmented region as the rectangle to be analyzed, and denote the area of the rectangle to be analyzed as the area to be analyzed. Denote the ratio of the total number of pixel points in the d-th segmented region to the area to be analyzed as the region feature value corresponding to the d-th segmented region. The larger the region feature value corresponding to the d-th segmented region, the higher the filling rate of the rectangle to be analyzed, which indicates that the d-th segmented region has more characteristics of rectangular distribution, and further indicates that the d-th segmented region is more likely to belong to man-made construction areas such as highway construction.

[0032] Finally, obtain the mean value of the gray-scale distribution feature value corresponding to the target sub-block a and the spatial distribution feature value corresponding to the target sub-block a, and denote it as the importance characterization value of the target sub-block a; and the larger the importance characterization value of the target sub-block a, the greater the possibility that the region corresponding to the target sub-block a belongs to man-made construction areas such as highway construction. And the greater the possibility that the region corresponding to the target sub-block a belongs to man-made construction areas such as highway construction, then when performing matching later, the data accuracy in the target sub-block a should be retained more.

[0033] Therefore, through the above process, this embodiment can obtain the importance characterization value of each target sub-block.

[0034] Step S004: Perform principal component analysis on the multi-spectral image to obtain the principal components of the multi-spectral image, and obtain the matching sub-blocks corresponding to the respective target sub-blocks according to the first principal component of the multi-spectral image.

[0035] Immediately following this embodiment, principal component analysis is performed on the multi-spectral image, and based on the analysis results, the principal components of the multi-spectral image are obtained. The process of performing principal component analysis on the multi-spectral image is a well-known technology, so it will not be described in detail here. Then, according to the first principal component of the multi-spectral image, the matching sub-blocks corresponding to each target sub-block are obtained. The first principal component of the multi-spectral image is a two-dimensional matrix. In this embodiment, the specific process of obtaining the matching sub-blocks corresponding to each target sub-block is as follows: First, the first principal component is evenly and non-overlappingly divided into blocks using the optimal window, and all the sub-blocks obtained from the even and non-overlapping division are denoted as the sub-blocks to be assigned on the first principal component. The specific process of evenly and non-overlappingly dividing the first principal component using the optimal window is the same as the process of evenly and non-overlappingly dividing the high-resolution image using the optimal window described above, so it will not be described in this embodiment. Also, since the size of each image constituting the multi-spectral image is the same as the size of the high-resolution image, the size of the first principal component of the multi-spectral image is the same as the size of the two-dimensional matrix formed by the high-resolution image. That is to say, the first principal component is a two-dimensional matrix, and the size of the two-dimensional matrix corresponding to the first principal component is the same as the size of the two-dimensional matrix formed by the pixel gray values on the high-resolution image. Therefore, on the first principal component, sub-blocks to be assigned with the same positions as each target sub-block can be obtained, and they are denoted as the matching sub-blocks corresponding to the corresponding target sub-blocks. That is, the position of the data in the matching sub-block corresponding to any target sub-block on the first principal component is the same as the position of all the pixel points in this target sub-block on the high-resolution image. If the center point of a certain target sub-block is the pixel point at the i-th row and j-th column position on the high-resolution image, then the central data in the matching sub-block corresponding to this target sub-block is the data at the i-th row and j-th column position on the first principal component.

[0036] Therefore, through the above process, this embodiment can obtain the matching sub-blocks corresponding to each target sub-block. The matching sub-blocks are mainly used for the subsequent histogram matching process, and in this embodiment, each target sub-block is matched separately.

[0037] Step S005: According to the cumulative histograms of the respective target sub-blocks and the importance degree characterization values of the respective target sub-blocks, obtain the number of pixel points to be replaced corresponding to each gray value in the set of gray values corresponding to the respective target sub-blocks. According to the cumulative probability values on the cumulative histograms of the respective target sub-blocks and the cumulative probability values on the cumulative histogram of the matching sub-block, obtain the replacement gray values corresponding to each gray value in the set of gray values corresponding to the respective target sub-blocks. The set of gray values corresponding to a target sub-block is composed of all types of gray values that appear in the target sub-block. According to the replacement gray values corresponding to each gray value in the set of gray values corresponding to the respective target sub-blocks and the number of pixel points to be replaced, replace the pixel point gray values in the respective target sub-blocks to obtain the high-resolution image after replacement. Replace the first principal component of the multi-spectral image with the two-dimensional matrix of the high-resolution image after replacement, and perform an inverse PCA transformation on all the principal components of the multi-spectral image after replacement to obtain the fused image.

[0038] In this embodiment, the histogram of the target sub-block and the histogram of the corresponding matching sub-block of the target sub-block are then obtained. The abscissa value of the histogram of the target sub-block is the gray value, and the ordinate value is the frequency of the corresponding gray value appearing in the corresponding target sub-block. The abscissa value of the histogram of the corresponding matching sub-block of the target sub-block is the data in the matching sub-block, and the ordinate value is the frequency of the different numerical data in the matching sub-block appearing in the corresponding matching sub-block. Then, according to the histogram of the target sub-block and the histogram of the corresponding matching sub-block of the target sub-block, the cumulative histogram of the target sub-block and the cumulative histogram of the corresponding matching sub-block of the target sub-block are respectively obtained. The process of obtaining the cumulative histogram is well-known and will not be described in detail here.

[0039] After obtaining the cumulative histogram, the set of gray values corresponding to each target sub-block is then obtained. The set of gray values corresponding to a target sub-block is composed of all types of gray values that appear in the corresponding target sub-block. Then, according to the cumulative histograms of the respective target sub-blocks and the importance degree characterization values of the respective target sub-blocks, obtain the number of pixel points to be replaced corresponding to each gray value in the set of gray values corresponding to the respective target sub-blocks. The specific process of obtaining the number of pixel points to be replaced corresponding to each gray value in the set of gray values corresponding to the respective target sub-blocks is as follows: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: first, count the frequency of occurrence of the grayscale value f in the target sub-block a, and record it as the quantity index value corresponding to the grayscale value f; then round up the value obtained by subtracting the importance representation value of the target sub-block a from the preset first constant and multiplying the result by the quantity index value corresponding to the grayscale value f, and record it as the number of pixel replacements corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a; and the larger the importance representation value of the target sub-block a, the greater the possibility that the area corresponding to the target sub-block a belongs to a man-made construction area such as a highway construction, so the data accuracy in the target sub-block a must be retained when matching, so the larger the importance representation value of the target sub-block a, the smaller the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to the target sub-block a.

[0040] Therefore, according to the above process, this embodiment can obtain the number of pixel replacements corresponding to each gray value in the gray value set corresponding to each target sub-block.

[0041] Then, according to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block corresponding to the target sub-block, the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, and the specific process of obtaining the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: first, on the cumulative histogram of the target sub-block a, obtain the cumulative probability value of the grayscale value f, and record it as the cumulative probability value to be analyzed; then, on the cumulative histogram of the matching sub-block corresponding to the target sub-block a, obtain the horizontal coordinate value corresponding to the cumulative probability value closest to the cumulative probability value to be analyzed, and record it as the replacement grayscale value corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a; and if the number of cumulative probability values ​​closest to the cumulative probability value to be analyzed is greater than 1, then select the horizontal coordinate value corresponding to any cumulative probability value from all the cumulative probability values ​​closest to the cumulative probability value to be analyzed.

[0042] Therefore, this embodiment can obtain the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block according to the above process.

[0043] After obtaining the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, the grayscale value of the pixel in each target sub-block is replaced according to the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, and a high-resolution image after replacement is obtained, thereby completing the histogram matching process; and in this embodiment, according to the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, the specific process of replacing the grayscale value of the pixel in each target sub-block to obtain the high-resolution image after replacement is as follows: For any target sub-block a: first, the grayscale value set corresponding to the target sub-block a is recorded as set G, and the replacement set corresponding to each grayscale value in set G is obtained, and then the grayscale value of the pixel point in the replacement set corresponding to each grayscale value in set G is replaced with the replacement grayscale value corresponding to the corresponding grayscale value in set G, and the target sub-block a after the replacement is recorded as the replaced target sub-block of the target sub-block a, that is, the grayscale values ​​of all pixels in the replacement set corresponding to the vth grayscale value in set G are replaced with the replacement grayscale value corresponding to the vth grayscale value in set G; then the image composed of the replaced target sub-blocks of all target sub-blocks on the high-resolution image is recorded as the high-resolution image after the replacement, and the yth replaced target sub-block on the high-resolution image after the replacement is the replaced target sub-block of the yth target sub-block on the high-resolution image.

[0044] In this embodiment, the method for obtaining the replacement set corresponding to each gray value in the set G is: For any grayscale value s in the set G, the number of pixel replacements corresponding to the grayscale value s in the set G is recorded as M. In the target sub-block a, M pixels with grayscale value s are randomly selected, and the set constructed by the randomly selected M pixels with grayscale value s is recorded as the replacement set corresponding to the grayscale value s.

[0045] In addition, it should be noted that, relative to the existing histogram matching process, the difference between this embodiment and the existing one is that this embodiment does not match the histogram of the complete image, but first divides the image into blocks and matches the histogram of each block. In this embodiment, the grayscale values ​​of all pixels in the block are not replaced, but the number of replacements is determined according to the size of the importance representation value corresponding to each block. Moreover, this matching method in this embodiment can not only combine multi-spectral information in the process of image fusion, but also retain the accuracy of highway construction-related information on the high-resolution image as much as possible, thereby improving the fusion effect.

[0046] After obtaining the high-resolution image after replacement, the two-dimensional matrix of the high-resolution image after replacement is obtained, denoted as the target two-dimensional matrix, and the first principal component of the multi-spectral image is replaced with the target two-dimensional matrix. Then, inverse PCA transformation is performed on all principal components of the multi-spectral image after replacement, and the image obtained after the inverse PCA transformation is the fused image. The process of inverse PCA transformation is a well-known technology, so it will not be described in detail here.

[0047] So far, this embodiment has completed the fusion of the multi-spectral image and the high-resolution image.

[0048] In summary, this embodiment first obtains the high-resolution image and the multi-spectral image, and divides the high-resolution image into blocks using windows of different sizes to obtain the evaluation values of the block effects of each window. According to the evaluation values of the block effects, each target sub-block on the high-resolution image is obtained. Then, the region growing algorithm is used to segment each target sub-block respectively to obtain each segmentation region in the target sub-block, and according to the gray values of the neighborhood pixels of each pixel point in each target sub-block and each segmentation region in each target sub-block, the characterization value of the importance degree of each target sub-block is obtained. After that, principal component analysis is performed on the multi-spectral image to obtain the principal components of the multi-spectral image, and according to the first principal component of the multi-spectral image, the matching sub-block corresponding to each target sub-block is obtained. Then, according to the cumulative histogram of each target sub-block and the characterization value of the importance degree of each target sub-block, the number of pixel points corresponding to each gray value in the gray value set corresponding to each target sub-block is obtained. According to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, the replacement gray value corresponding to each gray value in the gray value set corresponding to each target sub-block is obtained. According to the replacement gray value corresponding to each gray value in the gray value set corresponding to each target sub-block and the number of pixel point replacements, the gray values of the pixel points in each target sub-block are replaced to obtain the high-resolution image after replacement. The first principal component of the multi-spectral image is replaced with the two-dimensional matrix of the high-resolution image after replacement, and inverse PCA transformation is performed on all principal components of the multi-spectral image after replacement to obtain the fused image. And the number of pixel point replacements and the replacement gray values determined by this embodiment according to the characterization value of the importance degree and the cumulative histogram can enable not only the combination of multi-spectral information but also the retention of the accuracy of the information related to highway construction on the high-resolution image as much as possible during the image fusion process, so as to achieve the purpose of improving the fusion effect.

[0049] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A highway construction multi-source data fusion processing system based on a cloud platform, comprising a processor and a memory, characterized in that: The processor executes the computer program stored in the memory to implement the following steps: During highway construction, high-resolution images and multispectral images of the construction section are obtained; Using windows of different sizes to divide the high-resolution image into blocks, obtaining a block effect evaluation value of each window, and obtaining each target sub-block on the high-resolution image according to the block effect evaluation value; Using a region growing algorithm to segment each of the target sub-blocks, respectively, to obtain each segmented region in the target sub-block, and obtaining an importance representation value of each of the target sub-blocks according to the grayscale value of a neighboring pixel point of each pixel point in each of the target sub-blocks and each segmented region in each of the target sub-blocks; Performing principal component analysis on the multispectral image to obtain principal components of the multispectral image, and obtaining matching sub-blocks corresponding to each target sub-block according to a first principal component of the multispectral image; According to the cumulative histogram of each target sub-block and the importance representation value of each target sub-block, the number of pixel point replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, and according to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, the grayscale value set corresponding to the target sub-block is composed of all types of grayscale values ​​appearing in the target sub-block, and according to the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block and the number of pixel point replacements, the grayscale value of the pixel point in each target sub-block is replaced to obtain a high-resolution image after replacement, and the first principal component of the multispectral image is replaced with the two-dimensional matrix of the high-resolution image after replacement, and all principal components of the replaced multispectral image are subjected to PCA inverse transformation to obtain a fused image.

2. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1 is characterized in that: The method for obtaining the block effect evaluation value of each window includes: For any window: use the window to divide the high-resolution image into uniform and non-overlapping blocks, and record the results of the block division as the sub-blocks to be determined on the high-resolution image under the window; obtain the centrality representation value corresponding to each sub-block to be determined, and record the average of the centrality representation values ​​corresponding to all sub-blocks to be determined on the high-resolution image under the window as the evaluation value of the block division effect of the window.

3. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 2 is characterized in that: The method for obtaining the centrality representation value corresponding to the sub-block to be determined includes: For any sub-block to be determined: all grayscale value types appearing in the sub-block to be determined are counted, and a set consisting of all grayscale value types appearing in the sub-block to be determined is recorded as a first set, the frequency of occurrence of each grayscale value in the first set in the sub-block to be determined is counted, and the square value of the frequency of occurrence of each grayscale value in the first set in the sub-block to be determined is recorded as the frequency representation value of the corresponding grayscale value, and the cumulative sum of the frequency representation values ​​of all grayscale values ​​in the first set is recorded as the first eigenvalue corresponding to the sub-block to be determined; the difference between the maximum grayscale value in the sub-block to be determined and the minimum grayscale value in the sub-block to be determined is recorded as the second eigenvalue corresponding to the sub-block to be determined; The product of the first eigenvalue and the second eigenvalue is recorded as the centrality representation value corresponding to the sub-block to be determined.

4. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1, characterized in that: The method for acquiring each target sub-block on the high-resolution image comprises: Among the blocking effect evaluation values ​​corresponding to all windows, the window corresponding to the largest blocking effect evaluation value is selected as the optimal window, and all sub-blocks obtained by uniformly and non-overlappingly blocking the high-resolution image using the optimal window are recorded as target sub-blocks on the high-resolution image.

5. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1, characterized in that: The method for obtaining the importance representation value of each target sub-block includes: For any target sub-block a: Obtaining a neighborhood distribution representation value corresponding to each pixel point in the target sub-block a; The cumulative sum of the neighborhood distribution characterization values ​​corresponding to all pixels in the target sub-block a is recorded as the comprehensive distribution characterization value, and the normalized value of the result obtained by adding the grayscale mean value of all pixels in the target sub-block a to the comprehensive distribution characterization value is recorded as the grayscale distribution characteristic value corresponding to the target sub-block a; Obtaining the regional feature values ​​corresponding to each segmented area in the target sub-block a, the regional feature value corresponding to the d-th segmented area in the target sub-block a is the ratio of the total number of pixels in the d-th segmented area to the area of ​​the rectangle to be analyzed, and the rectangle to be analyzed is the minimum circumscribed rectangle of the d-th segmented area; recording the normalized value of the mean of the regional feature values ​​corresponding to all the segmented areas in the target sub-block a as the spatial distribution feature value corresponding to the target sub-block a; The average of the grayscale distribution characteristic value and the spatial distribution characteristic value is used as the importance representation value of the target sub-block a.

6. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 5 is characterized in that: The method for obtaining the neighborhood distribution representation value corresponding to each pixel point in the target sub-block a includes: For any pixel point b in the target sub-block a: Obtain a neighborhood difference value set corresponding to the pixel point b, wherein the cth neighborhood difference value in the neighborhood difference value set corresponding to the pixel point b is the absolute value of the grayscale difference between the cth pixel point in the neighborhood pixel point set corresponding to the pixel point b and the pixel point b, and the neighborhood pixel point set corresponding to the pixel point b refers to a set consisting of all pixels located in the target sub-block a and belonging to the eight neighborhoods of the pixel point b; The cumulative sum of all neighborhood difference values ​​in the neighborhood difference value set corresponding to the pixel point b is recorded as the comprehensive neighborhood difference value of the pixel point b, and the reciprocal of the result obtained by adding the comprehensive neighborhood difference value of the pixel point b and a preset first constant is recorded as the neighborhood distribution characterization value of the pixel point b.

7. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 4 is characterized in that: The method for obtaining the matching sub-blocks corresponding to each target sub-block includes: Using an optimal window to divide the first principal component into even, non-overlapping blocks, and recording all obtained sub-blocks as sub-blocks to be allocated; On the first principal component, sub-blocks to be allocated having the same positions as those of the respective target sub-blocks are obtained and recorded as matching sub-blocks corresponding to the corresponding target sub-blocks.

8. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1, characterized in that: The method for obtaining the number of pixel replacements includes: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: count the frequency of occurrence of the grayscale value f in the target sub-block a, and record it as the quantity index value corresponding to the grayscale value f; round the value of the result obtained by multiplying the quantity index value by the importance representation value of the target sub-block a from a preset first constant, and record it as the number of pixel point replacements corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a.

9. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1, characterized in that: The method for obtaining the replacement grayscale value includes: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: on the cumulative histogram of the target sub-block a, obtain the cumulative probability value of the grayscale value f, and record it as the cumulative probability value to be analyzed; on the cumulative histogram of the matching sub-block corresponding to the target sub-block a, obtain the horizontal coordinate value corresponding to the cumulative probability value closest to the cumulative probability value to be analyzed, and record it as the replacement grayscale value corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a.

10. The highway construction multi-source data fusion processing system based on cloud platform as claimed in claim 1, characterized in that: The method of replacing the pixel grayscale values ​​in each target sub-block according to the replacement grayscale values ​​corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block and the number of pixel replacements to obtain a high-resolution image after the replacement, comprises: For any target sub-block a: record the grayscale value set corresponding to the target sub-block a as set G, obtain a replacement set corresponding to each grayscale value in the set G, replace the grayscale value of the pixel point in the replacement set corresponding to each grayscale value in the set G with the replacement grayscale value corresponding to the corresponding grayscale value in the set G, and record the target sub-block a after replacement as the replaced target sub-block of the target sub-block a; Recording an image composed of the replaced target sub-blocks of all target sub-blocks on the high-resolution image as a high-resolution image after replacement; The method for obtaining the replacement set corresponding to each gray value in the set G includes: For any grayscale value s in the set G, the number of pixel replacements corresponding to the grayscale value s in the set G is recorded as M, and in the target sub-block a, M pixel points with grayscale values ​​of s are randomly selected, and the set constructed by the randomly selected M pixel points with grayscale values ​​of s is recorded as the replacement set corresponding to the grayscale value s.

Images