Highway construction multi-source data fusion processing system based on cloud platform

Abstract

The present invention relates to the technical field of multi-source data fusion, and more particularly to a multi-source data fusion processing system for highway construction based on a cloud platform. The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the following steps: obtaining the number of pixel replacements and the corresponding replacement grayscale values corresponding to each grayscale value in a grayscale value set corresponding to each target sub-block; replacing the pixel grayscale values in each target sub-block according to the replacement grayscale values and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block to obtain a high-resolution image after the replacement; replacing the first principal component of a multispectral image with a two-dimensional matrix of the high-resolution image after the replacement; and performing an inverse PCA transform on all principal components of the replaced multispectral image to obtain a fused image. The present invention can also improve fusion effects.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-source data fusion, and in particular to a multi-source data fusion processing system for highway construction based on a cloud platform. Background Art

[0002] Since multi-source data fusion in highway construction is crucial for improving highway construction positioning accuracy, construction efficiency, construction quality, etc., multi-source data fusion is currently needed in highway construction. Usually, during the highway construction process, remote sensing images of the highway construction section need to be collected. Remote sensing images generally include high-resolution images and multispectral images. The acquired high-resolution images and multispectral images are then fused to obtain fused remote sensing image data with both spatial resolution and spectral information. That is, the fused image is subsequently analyzed, which is crucial for construction positioning, construction straight lines, etc.

[0003] In the existing technology, traditional PCA transformation technology is usually used to fuse high-resolution images with multispectral images. However, in the process of using PCA transformation technology to fuse high-resolution images with multispectral images, the traditional histogram matching method may reduce the accuracy of important data on the high-resolution image or cause the loss of important data on the high-resolution image. For example, after fusion, the accuracy of data related to highway construction on the high-resolution image may be low or the data related to highway construction may be lost. That is, the traditional histogram matching method will result in poor fusion effect, and when the fusion effect is poor, it will have a negative impact on highway construction efficiency, construction quality and other aspects. Therefore, how to improve the effect of fusing high-resolution images with multispectral images has become an urgent problem to be solved. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a multi-source data fusion processing system for highway construction based on a cloud platform. The technical solutions adopted are as follows:

[0005] One embodiment of the present invention provides a cloud platform-based highway construction multi-source data fusion processing system, comprising a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the following steps:

[0006] During highway construction, high-resolution and multispectral images of the construction section are acquired;

[0007] Using windows of different sizes to divide the high-resolution image into blocks, obtaining a block effect evaluation value of each window, and obtaining each target sub-block on the high-resolution image according to the block effect evaluation value;

[0008] Segmenting each target sub-block using a region growing algorithm to obtain each segmented region in the target sub-block, and obtaining an importance representation value of each target sub-block according to the grayscale value of a neighboring pixel point of each pixel point in each target sub-block and each segmented region in each target sub-block;

[0009] Performing principal component analysis on the multispectral image to obtain principal components of the multispectral image, and obtaining matching sub-blocks corresponding to the respective target sub-blocks based on a first principal component of the multispectral image;

[0010] According to the cumulative histogram of each target sub-block and the importance representation value of each target sub-block, the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, and according to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, the grayscale value set corresponding to the target sub-block is composed of all types of grayscale values appearing in the target sub-block, and according to the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block and the number of pixel replacements, the grayscale values of the pixel points in the target sub-block are replaced to obtain a high-resolution image after replacement, the first principal component of the multispectral image is replaced with the two-dimensional matrix of the high-resolution image after replacement, and all principal components of the replaced multispectral image are subjected to inverse PCA transformation to obtain a fused image.

[0011] Beneficial effects: The present invention first obtains a high-resolution image and a multispectral image of the construction section, and uses windows of different sizes to block the high-resolution image, obtains the blocking effect evaluation value of each window, and obtains each target sub-block on the high-resolution image according to the blocking effect evaluation value; then uses the region growing algorithm to segment each target sub-block respectively to obtain each segmented area in the target sub-block, and obtains the importance representation value of each target sub-block according to the grayscale value of the neighborhood pixel point of each pixel point in each target sub-block and each segmented area in each target sub-block; then performs principal component analysis on the multispectral image to obtain the principal component of the multispectral image, and obtains the matching sub-block corresponding to each target sub-block according to the first principal component of the multispectral image; then according to the accumulation of each target sub-block The histogram and the importance representation value of each target sub-block are used to obtain the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block. The replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained based on the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block. The grayscale values of the pixels in each target sub-block are replaced based on the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block to obtain a high-resolution image after replacement. The first principal component of the multispectral image is replaced with a two-dimensional matrix of the high-resolution image after replacement. All principal components of the replaced multispectral image are subjected to an inverse PCA transform to obtain a fused image. The present invention determines the number of pixel replacements and the replacement grayscale value based on the importance representation value and the cumulative histogram, so that during the image fusion process, multispectral information can be combined while preserving the accuracy of highway construction-related information on the high-resolution image as much as possible, thereby achieving the purpose of improving the fusion effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 This is a flow chart of a multi-source data fusion processing method for highway construction based on a cloud platform of the present invention;

[0014] Figure 2 Schematic diagram of high-resolution image segmentation. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field fall within the scope of protection of the embodiments of the present invention.

[0016] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0017] This embodiment provides a highway construction multi-source data fusion processing system based on a cloud platform, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a highway construction multi-source data fusion processing method based on a cloud platform, such as Figure 1 As shown, the cloud platform-based highway construction multi-source data fusion processing method includes the following steps:

[0018] Step S001: During the highway construction process, high-resolution images and multispectral images of the construction section are acquired.

[0019] This embodiment mainly improves the effect of fusing high-resolution images and multispectral images by optimizing or improving the matching method of the cumulative histogram. That is, during the fusion process, this embodiment optimizes or improves the matching method of the cumulative histogram, which not only combines multispectral information but also retains the accuracy of highway construction-related information on the high-resolution image as much as possible, thereby improving the fusion effect. This embodiment mainly fuses remote sensing images collected during the highway construction process. Therefore, this embodiment needs to first obtain remote sensing images, which include high-resolution images and multispectral images.

[0020] During the highway construction process, this embodiment will use drones to obtain remote sensing images of the highway construction section, and remote sensing images need to be collected at each monitoring moment. Since the drone in this embodiment will be equipped with a high-resolution camera and a multispectral camera, the remote sensing images collected by the drone in this embodiment include high-resolution images and multispectral images. The high-resolution images and multispectral images are remote sensing images, that is, at any collection moment, high-resolution images and multispectral images of the highway construction section can be collected by the drone. For ease of understanding, this embodiment will subsequently perform fusion processing on the high-resolution images and multispectral images obtained at any collection moment; in addition, the frequency of camera image collection needs to be set by the implementer according to actual conditions. For example, the camera can be set to collect images once every 1 second, and in this embodiment, the collection moments of the high-resolution camera and the multispectral camera are required to be consistent.

[0021] It should also be noted that during the highway construction process, in order to ensure the positioning accuracy, construction efficiency, construction quality, etc. of the highway construction, it is usually necessary to collect remote sensing images of the construction section at regular intervals. As the construction progresses, the subsequent remote sensing images collected will include images of the constructed sections and images of the unconstructed sections.

[0022] Therefore, this embodiment can obtain high-resolution images and multispectral images that need to be fused through the above process; in addition, it should be noted that the collected high-resolution images and multispectral images will be transmitted to the cloud platform, and then a series of processing and analysis will be carried out.

[0023] Step S002 : dividing the high-resolution image into blocks using windows of different sizes, obtaining a block effect evaluation value of each window, and obtaining each target sub-block on the high-resolution image based on the block effect evaluation value.

[0024] In the current process of using PCA transformation technology to fuse high-resolution images with multispectral images, the traditional histogram matching method may reduce the accuracy of important data on the high-resolution image or cause the loss of important data on the high-resolution image. For example, after fusion, the accuracy of data related to highway construction on the high-resolution image may be low or the data related to highway construction may be lost. That is, the traditional histogram matching method will result in poor fusion effect. When the fusion effect is poor, it will have a negative impact on highway construction efficiency, construction quality, etc., and the traditional histogram matching method refers to directly matching the cumulative histogram of the complete high-resolution image with the cumulative histogram of the complete multispectral image. In order to improve the effect of fusing high-resolution images with multispectral images, this embodiment will subsequently provide a new histogram matching method. That is, this embodiment will subsequently improve the fusion effect by improving or optimizing the histogram matching method. In other words, this embodiment will subsequently improve or optimize the histogram matching method so that the accuracy of important data of remote sensing images can be retained as much as possible during the fusion process.

[0025] Based on the above analysis, this embodiment will first use windows of different sizes to block the high-resolution image to obtain the blocking effect evaluation value of each window, and then obtain each target sub-block on the high-resolution image based on the blocking effect evaluation value of each window. The target sub-block is the basis for achieving histogram matching improvement or optimization.

[0026] Based on the above analysis, it can be seen that this embodiment needs to first determine the size interval to be selected, and in specific applications, the implementer needs to set the minimum and maximum values of the size interval to be selected according to actual conditions. For example, in this embodiment, the minimum value can be set to 2, and the maximum value can be set to one-quarter of the length of the high-resolution image. Then, all integer values in the size interval to be selected are obtained and recorded as window size values. After that, all constructed sets of window size values are recorded as a window size value set. Then, the windows corresponding to each window size value in the window size value set are obtained. If the zth window size value in the window size value set is x, then the length and width of the window corresponding to the zth window size value are also x.

[0027] After obtaining the windows corresponding to the respective window size values in the window size value set, the high-resolution image is divided into blocks using the windows corresponding to the respective window size values in the window size value set, and a block effect evaluation value of the window corresponding to the respective window size values in the window size value set is obtained. In addition, for ease of understanding, this embodiment will describe the process of obtaining the block effect evaluation value of window q corresponding to any window size value, that is, the process of obtaining the block effect evaluation value of the window corresponding to window q is:

[0028] First, use window q to perform uniform non-overlapping block division on the high-resolution image, and record the results of the block division as the sub-blocks to be determined on the high-resolution image under window q; and the example process of using window q to perform uniform non-overlapping block division on the high-resolution image is as follows: first, obtain a window of size window q in the upper left corner of the high-resolution image, and record it as the sliding window corresponding to window q, and then slide the sliding window corresponding to window q in order from top to bottom and from left to right, and all windows on the high-resolution image obtained after the sliding is completed are recorded as the sub-blocks to be determined on the high-resolution image under window q, and the sliding step length of the sliding window corresponding to window q when sliding is the length or width of window q, and the length and width of window q are the same. If the length and width of window q are 3, then the schematic diagram of using window q to perform uniform non-overlapping block division on the high-resolution image is as shown below. Figure 2 shown, and Figure 2 The smallest square on represents a pixel, and Figure 2 The area divided by the thicker black border is the sub-block to be determined after the high-resolution image is evenly divided into non-overlapping blocks using window q. Since the length or width of the high-resolution image may not be an integer multiple of the length or width of window q, the size of the sub-block to be determined near the edge of the high-resolution image may be smaller than the size of window q, such as Figure 2 The areas of the two undetermined sub-blocks on the right are relative to Figure 2 Compared with other undetermined sub-blocks on the

[0029] After obtaining each undetermined sub-block on the high-resolution image under window q, the centrality representation value corresponding to each undetermined sub-block on the high-resolution image under window q is obtained, and then the average of the centrality representation values corresponding to all undetermined sub-blocks on the high-resolution image under window q is calculated and recorded as the blocking effect evaluation value of window q. The larger the centrality representation value corresponding to the undetermined sub-block, the more concentrated the grayscale value distribution of the pixel points in the corresponding undetermined sub-block, and the more likely the area corresponding to the undetermined sub-block is to be an environmental area or a highway construction area. The larger the centrality representation value corresponding to the undetermined sub-block, the larger the blocking effect evaluation value under window q. The larger the blocking effect evaluation value under window q, the more the highway construction area and the background area can be separated to a greater extent when the high-resolution image is blocked using window q, so that the high-resolution image can be retained later.

[0030] In addition, the specific process of obtaining the centrality representation value corresponding to each to-be-determined sub-block on the high-resolution image under the above window q is as follows:

[0031] For any undetermined sub-block on the high-resolution image under window q: first, count the types of all grayscale values appearing in the undetermined sub-block, and record the set consisting of grayscale values of all categories appearing in the undetermined sub-block as the first set. For example, if the grayscale value types appearing in the undetermined sub-block are only 8 and 19, then the subsequent first set is composed of grayscale values 8 and 19; then count the frequency of each grayscale value in the first set appearing in the undetermined sub-block, and record the square value of the frequency of each grayscale value in the first set appearing in the undetermined sub-block as the frequency representation value of the corresponding grayscale value; then obtain the cumulative sum of the frequency representation values of all grayscale values in the first set, and record it as the first eigenvalue corresponding to the undetermined sub-block, and obtain the undetermined sub-block. The result obtained by subtracting the minimum grayscale value in the sub-block to be determined from the maximum grayscale value in the determined sub-block is recorded as the second eigenvalue corresponding to the sub-block to be determined; finally, the product of the first eigenvalue corresponding to the sub-block to be determined and the second eigenvalue corresponding to the sub-block to be determined is recorded as the concentration representation value corresponding to the sub-block to be determined; and it should be noted that the grayscale value of the pixel in the sub-block to be determined is the grayscale value of the pixel obtained by grayscale value processing of the corresponding sub-block to be determined; in addition, the larger the first eigenvalue corresponding to the sub-block to be determined and the second eigenvalue corresponding to the sub-block to be determined, the fewer the grayscale value types in the sub-block to be determined, thereby indicating that there are more pixels with the same grayscale value in the sub-block to be determined, and further indicating that the concentration of the grayscale value distribution of the pixels in the sub-block to be determined is better.

[0032] Therefore, this embodiment can obtain the blocking effect evaluation value of the window corresponding to each window size value in the window size value set in the above manner. After obtaining the blocking effect evaluation value, each target sub-block on the high-resolution image is obtained according to the blocking effect evaluation value of the window corresponding to each window size value, and the specific acquisition process of the target sub-block is: among the blocking effect evaluation values corresponding to all windows, the window corresponding to the largest blocking effect evaluation value is selected as the optimal window, and all sub-blocks obtained by uniformly and non-overlappingly blocking the high-resolution image using the optimal window are recorded as target sub-blocks on the high-resolution image, and the method of uniformly and non-overlappingly blocking the high-resolution image using any window is the same as the above-mentioned method of uniformly and non-overlappingly blocking the high-resolution image using window q, so it will not be described in detail.

[0033] Therefore, this embodiment obtains each target sub-block on the high-resolution image through the above process.

[0034] Step S003, using the region growing algorithm to segment each target sub-block respectively to obtain each segmented region in the target sub-block, and obtaining the importance representation value of each target sub-block according to the grayscale value of the neighborhood pixel point of each pixel point in each target sub-block and each segmented region in each target sub-block.

[0035] This embodiment will then obtain the importance representation value of the target sub-block, and the importance representation value is an important parameter for subsequent cumulative histogram matching and an important parameter for retaining the accuracy of important data. The important data refers to data related to highway construction on the high-resolution image. In order to obtain the importance representation value, it is necessary to first use the region growing algorithm to segment each target sub-block separately to obtain each segmented region in each target sub-block. The result of segmenting the target sub-block using the region growing algorithm is an important parameter for subsequently obtaining the importance representation value of the corresponding target sub-block. Therefore, in this embodiment, the specific process of segmenting each target sub-block using the region growing algorithm to obtain each segmented region in each target sub-block is as follows:

[0036] For any target sub-block, an initial seed point is first placed in the target sub-block, and then the initial seed point in the target sub-block starts to perform region growth. During the growth process, if there are no other seed points that meet the growth conditions in the neighborhood range of all seed points, the growth is stopped immediately, and all growth areas obtained after the growth is completed are recorded as segmentation areas in the target sub-block. The neighborhood range of the seed point refers to the area range composed of the eight neighborhoods belonging to the seed point and all pixel points of the target sub-block, which is called the neighborhood range of the corresponding seed point; in addition, when the absolute value of the grayscale difference between a pixel point in the neighborhood range of a seed point and the seed point is less than the preset grayscale difference threshold, it is determined that there are other seed points that meet the growth conditions in the neighborhood range of the seed point; and when the initial seed point and the growth conditions are known, the process of region growing is a well-known technology, so this embodiment will not be described in detail.

[0037] In specific applications, the implementer needs to set the preset grayscale difference threshold according to actual conditions. For example, in this embodiment, the preset grayscale difference threshold can be set to 3.

[0038] Therefore, this embodiment can obtain all the segmented areas in each target sub-block through the above process. After obtaining all the segmented areas in each target sub-block, the importance representation value of each target sub-block is obtained according to the grayscale value of the neighboring pixel points of each pixel point in each target sub-block and each segmented area in each target sub-block. For ease of understanding, this embodiment will take the process of obtaining the importance representation value of target sub-block a as an example to describe, that is, the process of obtaining the importance representation value of target sub-block a is:

[0039] First, based on the grayscale values of the neighboring pixels of each pixel in the target sub-block a, the neighborhood distribution representation value corresponding to each pixel in the target sub-block a is obtained. Then, based on the neighborhood distribution representation values corresponding to all pixels in the target sub-block a and the grayscale values of all pixels in the target sub-block a, the grayscale distribution characteristic value corresponding to the target sub-block a is obtained. Moreover, since the area around the highway is generally a natural environment composed of various terrains such as trees, grasslands, or hillsides, while the highway area or the highway area under construction is artificially constructed, the grayscale distribution of the highway area or the highway area under construction is more uniform and exhibits a bright feature compared to the area around the highway. That is, the grayscale of artificially constructed areas such as highway buildings is uniform and bright, while the background natural environment has various terrains such as trees, grasslands, or hillsides, and the brightness generated by reflected natural light is low and the uniformity is relatively poor. Moreover, since the larger the grayscale distribution characteristic value corresponding to the target sub-block a, the higher the grayscale uniformity in the target sub-block a and the more obvious the bright feature in the target sub-block a, the larger the grayscale distribution characteristic value corresponding to the target sub-block a, the more likely it is that the area corresponding to the target sub-block a belongs to an artificially constructed area such as a highway building.

[0040] In this embodiment, the specific process of obtaining the neighborhood distribution representation value corresponding to each pixel in the target sub-block a according to the grayscale value of the neighborhood pixel of each pixel in the target sub-block a is as follows:

[0041] For any pixel point b in the target sub-block a: first, all the pixels located in the target sub-block a and belonging to the eight neighborhoods of pixel point b are recorded as the neighborhood pixel point set corresponding to pixel point b, and then the neighborhood difference corresponding to each pixel point in the neighborhood pixel point set corresponding to pixel point b is obtained, and the neighborhood difference corresponding to the c-th pixel point in the neighborhood pixel point set corresponding to pixel point b refers to the absolute value of the grayscale difference between the c-th pixel point and pixel point b, and the neighborhood difference corresponding to all pixels in the neighborhood pixel point set corresponding to pixel point b is calculated. The constructed set is recorded as the neighborhood difference set corresponding to pixel point b, and then the cumulative sum of all neighborhood differences in the neighborhood difference set corresponding to pixel point b is calculated and used as the comprehensive neighborhood difference of pixel point b. Finally, the reciprocal of the result obtained by adding the comprehensive neighborhood difference of pixel point b to the preset first constant is recorded as the neighborhood distribution representation value of pixel point b; and in specific applications, the implementer needs to set the preset first constant according to actual conditions, and the role of the preset first constant here is to prevent the denominator from being 0. For example, in this embodiment, the preset first constant can be set to 1.

[0042] In this embodiment, according to the neighborhood distribution representation values corresponding to all pixels in the target sub-block a and the grayscale values of all pixels in the target sub-block a, the specific process of obtaining the grayscale distribution characteristic value corresponding to the target sub-block a is as follows: obtaining the grayscale mean of all pixels in the target sub-block a and recording it as the first grayscale mean, obtaining the cumulative sum of the neighborhood distribution representation values of all pixels in the target sub-block a and recording it as the comprehensive distribution representation value, adding the comprehensive distribution representation value to the first grayscale mean, and recording the normalized value of the result obtained as the grayscale distribution feature corresponding to the target sub-block a, and the grayscale distribution feature corresponding to the target sub-block a is , Norm() is the normalization function, It is the result obtained by adding the comprehensive distribution representation value and the first grayscale mean, and the larger the comprehensive distribution representation value and the first grayscale mean, the larger the grayscale distribution characteristic value corresponding to the target sub-block a.

[0043] Next, the minimum bounding rectangle of each segmented area in the target sub-block a is obtained, and the regional feature value corresponding to each segmented area in the target sub-block a is obtained based on the area of the minimum bounding rectangle of each segmented area in the target sub-block a. Then, the mean of the regional feature values corresponding to all the segmented areas in the target sub-block a is obtained and recorded as the regional feature mean. Thereafter, the regional feature mean is normalized using the normalization function Norm(), and the result obtained by the normalization is recorded as the spatial distribution feature value corresponding to the target sub-block a. Moreover, since the shape distribution characteristics of the highway construction area are more inclined to be a rectangular with sharp corners than the natural environment around the highway, and the shape distribution characteristics of the natural environment around the highway are generally irregular compared to the highway construction area, in this embodiment, when the spatial distribution feature value corresponding to the target sub-block a is larger, it indicates that the area corresponding to the target sub-block a is more likely to belong to an artificial construction area such as a highway construction area.

[0044] In this embodiment, the specific process of obtaining the region feature value corresponding to each segmented region in the target sub-block a according to the area of the minimum circumscribed rectangle of each segmented region in the target sub-block a is as follows:

[0045] For the d-th segmented area in the marker block a: the minimum circumscribed rectangle of the d-th segmented area is recorded as the rectangle to be analyzed, and the area of the rectangle to be analyzed is recorded as the area to be analyzed. The ratio of the total number of pixels in the d-th segmented area to the area to be analyzed is recorded as the regional characteristic value corresponding to the d-th segmented area. The larger the regional characteristic value corresponding to the d-th segmented area, the higher the filling rate of the rectangle to be analyzed, and the more rectangular distribution characteristics the d-th segmented area has, the greater the possibility that the d-th segmented area belongs to a man-made construction area such as a highway building.

[0046] Finally, the mean of the grayscale distribution eigenvalue corresponding to the target sub-block a and the spatial distribution eigenvalue corresponding to the target sub-block a is obtained and recorded as the importance representation value of the target sub-block a. The greater the importance representation value of the target sub-block a, the greater the possibility that the area corresponding to the target sub-block a belongs to an artificially constructed area such as a highway building. The greater the possibility that the area corresponding to the target sub-block a belongs to an artificially constructed area such as a highway building, the more important it is to retain the data accuracy in the target sub-block a during subsequent matching.

[0047] Therefore, this embodiment can obtain the importance representation value of each target sub-block through the above process.

[0048] Step S004 : performing principal component analysis on the multispectral image to obtain principal components of the multispectral image, and obtaining matching sub-blocks corresponding to the respective target sub-blocks based on the first principal component of the multispectral image.

[0049] Next, this embodiment performs principal component analysis on the multispectral image, and obtains the principal components of the multispectral image based on the analysis results. The process of performing principal component analysis on the multispectral image is a well-known technology and is therefore not described in detail. Then, based on the first principal component of the multispectral image, the matching sub-blocks corresponding to each target sub-block are obtained. The first principal component of the multispectral image is a two-dimensional matrix. In this embodiment, the specific process of obtaining the matching sub-blocks corresponding to each target sub-block is as follows:

[0050] First, the first principal component is uniformly divided into non-overlapping blocks using the optimal window, and all sub-blocks obtained by the uniform non-overlapping block division are recorded as sub-blocks to be allocated on the first principal component; and the specific process of uniformly dividing the first principal component into non-overlapping blocks using the optimal window is the same as the above-mentioned process of uniformly dividing the high-resolution image into non-overlapping blocks using the optimal window, so it will not be described again in this embodiment; and since the size of each image constituting the multispectral image is consistent with the size of the high-resolution image, the size of the first principal component of the multispectral image is consistent with the size of the two-dimensional matrix formed by the high-resolution image, that is, the first principal component is a two-dimensional matrix, and The two-dimensional matrix corresponding to the first principal component is consistent in size with the two-dimensional matrix formed by the grayscale values of the pixels on the high-resolution image; therefore, on the first principal component, the sub-blocks to be allocated that are the same as the positions of the target sub-blocks can be obtained and recorded as the matching sub-blocks corresponding to the corresponding target sub-blocks, that is, the position of the data in the matching sub-block corresponding to any target sub-block on the first principal component is consistent with the positions of all the pixels in the target sub-block on the high-resolution image. If the center point of a target sub-block is the pixel at the i-th row and j-th column on the high-resolution image, then the center data in the matching sub-block corresponding to the target sub-block is the data at the i-th row and j-th column on the first principal component.

[0051] Therefore, this embodiment can obtain the matching sub-block corresponding to each target sub-block through the above process, and the matching sub-block is mainly used in the subsequent histogram matching process. In addition, this embodiment performs separate matching on each target sub-block.

[0052] Step S005: obtaining the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block based on the cumulative histogram of each target sub-block and the importance representation value of each target sub-block; obtaining the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block based on the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block; the grayscale value set corresponding to the target sub-block is composed of all types of grayscale values appearing in the target sub-block; replacing the pixel grayscale values in each target sub-block based on the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block to obtain a high-resolution image after replacement; replacing the first principal component of the multispectral image with the two-dimensional matrix of the high-resolution image after replacement; and performing an inverse PCA transform on all principal components of the replaced multispectral image to obtain a fused image.

[0053] This embodiment then obtains a histogram of the target sub-block and a histogram of the matching sub-block corresponding to the target sub-block, wherein the abscissa value of the histogram of the target sub-block is the grayscale value, and the ordinate value is the frequency of occurrence of the corresponding grayscale value in the corresponding target sub-block. The abscissa value of the histogram of the matching sub-block corresponding to the target sub-block is the data in the matching sub-block, and the ordinate value is the frequency of occurrence of data of different values in the matching sub-block in the corresponding matching sub-block. Then, based on the histogram of the target sub-block and the histogram of the matching sub-block corresponding to the target sub-block, a cumulative histogram of the target sub-block and a cumulative histogram of the matching sub-block corresponding to the target sub-block are respectively obtained. The process of obtaining the cumulative histogram is well known and thus will not be described in detail.

[0054] After obtaining the cumulative histogram, the grayscale value set corresponding to each target sub-block is obtained, and the grayscale value set corresponding to the target sub-block is composed of all types of grayscale values appearing in the corresponding target sub-block. Then, based on the cumulative histogram of each target sub-block and the importance representation value of each target sub-block, the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained. The specific process of obtaining the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is as follows:

[0055] For any grayscale value f in the grayscale value set corresponding to any target sub-block a: first, count the frequency of the grayscale value f appearing in the target sub-block a, and record it as the quantity index value corresponding to the grayscale value f; then round up the result obtained by subtracting the importance representation value of the target sub-block a from the preset first constant and multiplying it by the quantity index value corresponding to the grayscale value f, and record it as the number of pixel replacements corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a; and the larger the importance representation value of the target sub-block a, the more likely it is that the area corresponding to the target sub-block a belongs to an artificially constructed area such as a highway construction, so the data accuracy in the target sub-block a must be retained during matching, so the larger the importance representation value of the target sub-block a, the smaller the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to the target sub-block a.

[0056] Therefore, according to the above process, this embodiment can obtain the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block.

[0057] Then, according to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block corresponding to the target sub-block, the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained. The specific process of obtaining the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is as follows:

[0058] For any grayscale value f in the grayscale value set corresponding to any target sub-block a: first, on the cumulative histogram of the target sub-block a, obtain the cumulative probability value of the grayscale value f, and record it as the cumulative probability value to be analyzed; then, on the cumulative histogram of the matching sub-block corresponding to the target sub-block a, obtain the horizontal coordinate value corresponding to the cumulative probability value closest to the cumulative probability value to be analyzed, and record it as the replacement grayscale value corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a; and if the number of cumulative probability values closest to the cumulative probability value to be analyzed is greater than 1, then select the horizontal coordinate value corresponding to any cumulative probability value from all the cumulative probability values closest to the cumulative probability value to be analyzed.

[0059] Therefore, this embodiment can obtain the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block according to the above process.

[0060] After obtaining the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, the grayscale values of the pixels in each target sub-block are replaced according to the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, thereby obtaining a high-resolution image after the replacement, thereby completing the histogram matching process. In this embodiment, the specific process of replacing the grayscale values of the pixels in each target sub-block according to the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block to obtain a high-resolution image after the replacement is as follows:

[0061] For any target sub-block a: first, the grayscale value set corresponding to the target sub-block a is recorded as set G, and the replacement set corresponding to each grayscale value in set G is obtained. Then, the grayscale value of the pixel point in the replacement set corresponding to each grayscale value in set G is replaced with the replacement grayscale value corresponding to the corresponding grayscale value in set G, and the target sub-block a after the replacement is recorded as the replaced target sub-block of the target sub-block a, that is, the grayscale value of all the pixel points in the replacement set corresponding to the vth grayscale value in set G is replaced with the replacement grayscale value corresponding to the vth grayscale value in set G; then, the image composed of the replaced target sub-blocks of all target sub-blocks on the high-resolution image is recorded as the high-resolution image after replacement, and the yth replaced target sub-block on the high-resolution image after replacement is the replaced target sub-block of the yth target sub-block on the high-resolution image.

[0062] In this embodiment, the method for obtaining the replacement set corresponding to each grayscale value in the set G is:

[0063] For any grayscale value s in the set G, the number of pixel replacements corresponding to the grayscale value s in the set G is recorded as M. In the target sub-block a, M pixels with grayscale values of grayscale value s are randomly selected, and the set constructed by the randomly selected M pixels with grayscale value s is recorded as the replacement set corresponding to the grayscale value s.

[0064] In addition, it should be noted that, relative to the existing histogram matching process, the difference between this embodiment and the existing one is that this embodiment does not match the histogram of the complete image, but first divides the image into blocks and matches the histogram of each block. In this embodiment, the grayscale values of all pixels in the block are not replaced, but the number of replacements is determined based on the size of the importance representation value corresponding to each block. Moreover, this matching method in this embodiment can not only combine multispectral information in the process of image fusion, but also retain the accuracy of highway construction-related information on the high-resolution image as much as possible, thereby improving the fusion effect.

[0065] After obtaining the high-resolution image after replacement, the two-dimensional matrix of the high-resolution image after replacement is obtained and recorded as the target two-dimensional matrix. The first principal component of the multispectral image is replaced with the target two-dimensional matrix. Then, all principal components of the replaced multispectral image are subjected to an inverse PCA transformation, and the images obtained after the inverse PCA transformation are fused. The process of the inverse PCA transformation is a well-known technology and will not be described in detail.

[0066] So far, this embodiment has completed the fusion of multispectral images and high-resolution images.

[0067] In summary, this embodiment first obtains a high-resolution image and a multispectral image, and uses windows of different sizes to block the high-resolution image, obtains the blocking effect evaluation value of each window, and obtains each target sub-block on the high-resolution image according to the blocking effect evaluation value; then uses the region growing algorithm to segment each target sub-block respectively to obtain each segmented area in the target sub-block, and obtains the importance representation value of each target sub-block according to the grayscale value of the neighborhood pixel point of each pixel point in each target sub-block and each segmented area in each target sub-block; then performs principal component analysis on the multispectral image to obtain the principal component of the multispectral image, and obtains the matching sub-block corresponding to each target sub-block according to the first principal component of the multispectral image; and then obtains the matching sub-block corresponding to each target sub-block according to the cumulative histogram of each target sub-block. The image and the importance representation value of each target sub-block are used to obtain the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block. Based on the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, a replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained. Based on the replacement grayscale value and the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block, the grayscale values of the pixels in each target sub-block are replaced to obtain a high-resolution image after replacement. The first principal component of the multispectral image is replaced with a two-dimensional matrix of the high-resolution image after replacement. An inverse PCA transform is performed on all principal components of the replaced multispectral image to obtain a fused image. Furthermore, the number of pixel replacements and the replacement grayscale values determined based on the importance representation value and the cumulative histogram in this embodiment enable, during the image fusion process, the multispectral information to be combined while preserving the accuracy of highway construction-related information in the high-resolution image as much as possible, thereby achieving an improved fusion effect.

[0068] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A highway construction multi-source data fusion processing system based on a cloud platform, comprising a processor and a memory, characterized in that: The processor executes the computer program stored in the memory to implement the following steps: During highway construction, high-resolution and multispectral images of the construction section are acquired; Using windows of different sizes to divide the high-resolution image into blocks, obtaining a block effect evaluation value of each window, and obtaining each target sub-block on the high-resolution image according to the block effect evaluation value; Segmenting each target sub-block using a region growing algorithm to obtain each segmented region in the target sub-block, and obtaining an importance representation value of each target sub-block according to the grayscale value of a neighboring pixel point of each pixel point in each target sub-block and each segmented region in each target sub-block; Performing principal component analysis on the multispectral image to obtain principal components of the multispectral image, and obtaining matching sub-blocks corresponding to the respective target sub-blocks based on a first principal component of the multispectral image; According to the cumulative histogram of each target sub-block and the importance representation value of each target sub-block, the number of pixel replacements corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, and according to the cumulative probability value on the cumulative histogram of each target sub-block and the cumulative probability value on the cumulative histogram of the matching sub-block, the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block is obtained, the grayscale value set corresponding to the target sub-block is composed of all types of grayscale values appearing in the target sub-block, and according to the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block and the number of pixel replacements, the grayscale values of the pixel points in the target sub-block are replaced to obtain a high-resolution image after replacement, the first principal component of the multispectral image is replaced with the two-dimensional matrix of the high-resolution image after replacement, and all principal components of the replaced multispectral image are subjected to inverse PCA transformation to obtain a fused image.

2. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method for obtaining the block effect evaluation value of each window includes: For any window: using the window to perform uniform non-overlapping block division on the high-resolution image, and recording the results of the block division as each undetermined sub-block on the high-resolution image under the window; obtaining the centrality representation value corresponding to each undetermined sub-block, and recording the average of the centrality representation values corresponding to all undetermined sub-blocks on the high-resolution image under the window as the block division effect evaluation value of the window; The method for obtaining the centrality representation value corresponding to the sub-block to be determined includes: for any sub-block to be determined: Count all grayscale value types appearing in the sub-block to be determined, and record the set consisting of all grayscale value types appearing in the sub-block to be determined as a first set, count the frequency of each grayscale value in the first set appearing in the sub-block to be determined, and record the square value of the frequency of each grayscale value in the first set appearing in the sub-block to be determined as the frequency representation value of the corresponding grayscale value, and record the cumulative sum of the frequency representation values of all grayscale values in the first set as the first eigenvalue corresponding to the sub-block to be determined; record the difference between the maximum grayscale value in the sub-block to be determined and the minimum grayscale value in the sub-block to be determined as the second eigenvalue corresponding to the sub-block to be determined; and record the product of the first eigenvalue and the second eigenvalue as the concentration representation value corresponding to the sub-block to be determined.

3. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method for acquiring each target sub-block on the high-resolution image includes: Among the blocking effect evaluation values corresponding to all windows, the window corresponding to the largest blocking effect evaluation value is selected as the optimal window, and all sub-blocks obtained by uniformly and non-overlappingly blocking the high-resolution image using the optimal window are recorded as target sub-blocks on the high-resolution image.

4. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method for obtaining the importance representation value of each target sub-block includes: For any target sub-block a: Obtaining a neighborhood distribution representation value corresponding to each pixel point in the target sub-block a; The cumulative sum of the neighborhood distribution representation values corresponding to all pixels in the target sub-block a is recorded as the comprehensive distribution representation value, and the normalized value of the result obtained by adding the grayscale mean value of all pixels in the target sub-block a to the comprehensive distribution representation value is recorded as the grayscale distribution characteristic value corresponding to the target sub-block a; Obtaining regional characteristic values corresponding to each segmented area in the target sub-block a, where the regional characteristic value corresponding to the d-th segmented area in the target sub-block a is the ratio of the total number of pixels in the d-th segmented area to the area of the rectangle to be analyzed, and the rectangle to be analyzed is the minimum circumscribed rectangle of the d-th segmented area; recording the normalized value of the mean of the regional characteristic values corresponding to all segmented areas in the target sub-block a as the spatial distribution characteristic value corresponding to the target sub-block a; The mean of the grayscale distribution characteristic value and the spatial distribution characteristic value is used as the importance representation value of the target sub-block a.

5. The cloud platform-based highway construction multi-source data fusion processing system according to claim 4, characterized in that: The method for obtaining the neighborhood distribution representation value corresponding to each pixel point in the target sub-block a includes: For any pixel point b in the target sub-block a: Obtain a neighborhood difference value set corresponding to the pixel point b, where the cth neighborhood difference value in the neighborhood difference value set corresponding to the pixel point b is the absolute value of the grayscale difference between the cth pixel point in the neighborhood pixel point set corresponding to the pixel point b and the pixel point b, and the neighborhood pixel point set corresponding to the pixel point b is a set consisting of all pixels located in the target sub-block a and belonging to the eight neighborhoods of the pixel point b; The cumulative sum of all neighborhood differences in the neighborhood difference value set corresponding to the pixel point b is recorded as the comprehensive neighborhood difference value of the pixel point b, and the reciprocal of the result obtained by adding the comprehensive neighborhood difference value of the pixel point b to the preset first constant is recorded as the neighborhood distribution representation value of the pixel point b.

6. The cloud platform-based highway construction multi-source data fusion processing system according to claim 3, characterized in that: The method for obtaining the matching sub-blocks corresponding to the target sub-blocks includes: Using an optimal window to divide the first principal component into even, non-overlapping blocks, and recording all the obtained sub-blocks as sub-blocks to be allocated; On the first principal component, sub-blocks to be allocated having the same positions as the respective target sub-blocks are obtained and recorded as matching sub-blocks corresponding to the corresponding target sub-blocks.

7. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method for obtaining the number of pixel replacements includes: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: count the frequency of the grayscale value f appearing in the target sub-block a, and record it as the quantity index value corresponding to the grayscale value f; round the value of the result obtained by multiplying the importance representation value of the target sub-block a by a preset first constant and then multiplying it by the quantity index value, and record it as the number of pixel replacements corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a.

8. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method for obtaining the replacement grayscale value includes: For any grayscale value f in the grayscale value set corresponding to any target sub-block a: on the cumulative histogram of the target sub-block a, obtain the cumulative probability value of the grayscale value f, and record it as the cumulative probability value to be analyzed; on the cumulative histogram of the matching sub-block corresponding to the target sub-block a, obtain the horizontal coordinate value corresponding to the cumulative probability value closest to the cumulative probability value to be analyzed, and record it as the replacement grayscale value corresponding to the grayscale value f in the grayscale value set corresponding to the target sub-block a.

9. The cloud platform-based highway construction multi-source data fusion processing system according to claim 1, characterized in that: The method of replacing the grayscale values of pixels in each target sub-block according to the replacement grayscale value corresponding to each grayscale value in the grayscale value set corresponding to each target sub-block and the number of pixel replacements to obtain a high-resolution image after the replacement, comprising: For any target sub-block a: record the grayscale value set corresponding to the target sub-block a as set G, obtain a replacement set corresponding to each grayscale value in set G, replace the grayscale value of the pixel point in the replacement set corresponding to each grayscale value in set G with the replacement grayscale value corresponding to the corresponding grayscale value in set G, and record the target sub-block a after the replacement as the replaced target sub-block of the target sub-block a; Recording an image composed of the replaced target sub-blocks of all target sub-blocks on the high-resolution image as a high-resolution image after replacement; The method for obtaining the replacement set corresponding to each grayscale value in the set G includes: For any grayscale value s in the set G, the number of pixel replacements corresponding to the grayscale value s in the set G is recorded as M. In the target sub-block a, M pixel points with grayscale values of grayscale value s are randomly selected, and the set constructed by the randomly selected M pixel points with grayscale value s is recorded as the replacement set corresponding to the grayscale value s.

Images