Remote sensing image mosaic method, device, equipment and storage medium

Abstract

The present disclosure proposes a remote sensing image mosaicking method, apparatus, device and storage medium, the method comprising: determining a corresponding image mosaicking mode based on an application scenario of a current mosaicking task; obtaining an image data source under the determined image mosaicking mode, screening the image data source, and filtering out image data for image mosaicking from the image data source; constructing mosaicking lines based on the image data using a mosaicking line construction strategy associated with the image mosaicking mode; and mosaicking the image data according to the constructed mosaicking lines, thereby realizing automated operation of remote sensing image mosaicking for different scenarios.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a remote sensing image mosaic method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of sensor technology and remote sensing data processing methods, remote sensing imagery data has gradually adopted the characteristics of "three mores" (multi-platform, multi-sensor, and multi-temporal), "four highs" (high spectral resolution, high spatial resolution, high temporal resolution, and high radiometric resolution), and global coverage. Due to its inherent wide coverage and rich information content, remote sensing imagery data has been widely used in industries such as emergency response, surveying and mapping, land administration, forestry, and national defense, with remarkable success. The most prominent application of remote sensing imagery is providing high-precision regional orthophoto mosaics for various industries. Currently, my country has achieved full coverage of national 1:50,000 basic geographic data and 2-meter orthophoto data, with image resolutions reaching 0.5 meters or even higher in some areas. Due to limitations in data acquisition methods, data immutability (providing geospatial information that is as up-to-date as possible) cannot meet the needs of national economic development. Therefore, the dynamic update model of geographic data, exemplified by high-resolution remote sensing imagery, has become an important method for image mosaicking to meet practical application requirements.

[0003] With the rapid development of Earth observation technology, remote sensing data acquisition methods have significantly increased in variety, quantity, and capabilities. However, the imaging conditions of various satellite / aerospace platforms are complex, and different application scenarios have different requirements for the accuracy and timeliness of mosaic products. Consequently, the processing methods for mosaicking tasks are also different. This leads to differences in the production process of image mosaicking and the inability to achieve automated operation. Summary of the Invention

[0004] In view of this, the present disclosure proposes a remote sensing image mosaicking method, apparatus, device and storage medium, which can realize the automated operation of remote sensing image mosaicking.

[0005] According to one aspect of the present disclosure, a remote sensing image mosaic method is provided, comprising:

[0006] Determine the corresponding image mosaic mode based on the application scenario of the current mosaic task;

[0007] In the determined image mosaic mode, obtaining an image data source, and screening the image data source to select image data for image mosaic from the image data source;

[0008] Based on the image data, constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode;

[0009] The image data is mosaicked according to the constructed mosaic lines.

[0010] In a possible implementation, the image mosaic mode includes at least one of regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic, and emergency update mosaic.

[0011] In a possible implementation, when filtering the image data source, different filtering methods are correspondingly set in different image mosaic modes;

[0012] The screening method includes at least one of screening based on screening conditions and screening based on screening models;

[0013] Wherein, when data screening is performed based on the screening conditions, the screening conditions include: at least one of imaging conditions, image conditions and task constraints;

[0014] When data screening is performed based on the screening model, the screening model includes at least one of a new area task screening model and a similar task screening model;

[0015] The new region screening model is constructed according to the screening conditions.

[0016] In a possible implementation, constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode includes:

[0017] When the image mosaic mode is regional high-precision mosaic, high-precision update mosaic, or emergency update mosaic, mosaic lines are constructed using a mosaic line construction strategy based on a morphological method;

[0018] When the image mosaic mode is regional emergency mosaic, mosaic lines are constructed using a mosaic line construction strategy based on synonymous points.

[0019] In a possible implementation, after the image data for image mosaicking is screened out from the image data source, the method further includes: performing pre-processing on the screened image data;

[0020] Among them, different image mosaic modes are matched with corresponding data preprocessing methods.

[0021] In a possible implementation, after mosaicking the image data according to the constructed mosaicking lines, the method further includes: performing color processing on the mosaicked image data.

[0022] In a possible implementation, color processing is performed on the mosaicked image data, including:

[0023] When the image mosaic mode is regional high-precision mosaic or regional emergency mosaic, color uniformity processing is performed without a base map;

[0024] When the image mosaic mode is high-precision update mosaic or emergency update mosaic, color uniformity processing is performed based on the base map.

[0025] According to a second aspect of the present disclosure, a remote sensing image mosaicking device is provided, comprising:

[0026] The mosaic mode acquisition module is used to determine the corresponding image mosaic mode based on the application scenario of the current mosaic task;

[0027] A data screening model is used to obtain an image data source under the determined image mosaic mode, and to screen the image data source to select image data for image mosaicking from the image data source;

[0028] A mosaic line construction module, configured to construct mosaic lines based on the image data using a mosaic line construction strategy associated with the image mosaic mode;

[0029] The mosaic module is used to perform mosaic processing on the image data according to the constructed mosaic lines.

[0030] According to a third aspect of the present disclosure, a remote sensing image mosaicking device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above method when executing the executable instructions.

[0031] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0032] In this disclosure, based on the application scenario of the current mosaicking task, a corresponding image mosaicking mode is determined; image data is screened and mosaicking lines are constructed within the determined image mosaicking mode, thereby completing the mosaicking process of the image data. Since the image mosaicking method is the same under the same image mosaicking mode, the automated operation of remote sensing image mosaicking can be achieved by determining the image mosaicking mode.

[0033] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic flow chart showing a method for mosaicking remote sensing images according to an embodiment of the present disclosure is provided;

[0035] Figure 2 A schematic flow chart showing a method for constructing an inlay line according to an embodiment of the present disclosure is shown;

[0036] Figure 3 A schematic diagram illustrating obtaining overlapping areas of valid areas according to an embodiment of the present disclosure;

[0037] Figure 4 A schematic diagram showing an initial mosaic reference line according to an embodiment of the present disclosure;

[0038] Figure 5 A schematic diagram showing the overlapping area segmentation result according to an embodiment of the present disclosure;

[0039] Figure 6 A schematic diagram showing a mosaic line extraction result according to an embodiment of the present disclosure;

[0040] Figure 7 A schematic diagram showing a complete inlaid line according to an embodiment of the present disclosure;

[0041] Figure 8 A schematic flow chart showing a method for constructing an inlay line according to another embodiment of the present disclosure;

[0042] Figure 9 A schematic diagram illustrating the effective area range of the overlap zone according to an embodiment of the present disclosure;

[0043] Figure 10 A schematic diagram illustrating extraction of geographic entity element information in a valid overlapping area according to an embodiment of the present disclosure is shown;

[0044] Figure 11 A schematic diagram showing an image update edge determination result according to an embodiment of the present disclosure is shown;

[0045] Figure 12 A schematic diagram of region of interest segmentation according to an embodiment of the present disclosure is shown;

[0046] Figure 13 A schematic diagram of generating updated tessellated line segments according to an embodiment of the present disclosure is shown;

[0047] Figure 14 A schematic diagram showing a mosaic line search algorithm structure based on an improved GraphCut according to an embodiment of the present disclosure;

[0048] Figure 15 FIG2 shows a schematic diagram of generating updated mosaic line segments according to another embodiment of the present disclosure;

[0049] Figure 16 A schematic block diagram showing a remote sensing image mosaicking device according to an embodiment of the present disclosure is shown;

[0050] Figure 17A schematic block diagram of a remote sensing image mosaicking device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0051] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0052] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0053] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0054] <Method Example>

[0055] Figure 1 FIG. 1 is a schematic flow chart showing a method for mosaicking remote sensing images according to an embodiment of the present disclosure. Figure 1 As shown, the remote sensing image mosaic method includes steps S110-S140.

[0056] S110: Determine a corresponding image mosaic mode based on an application scenario of the current mosaic task.

[0057] The mosaicking task involves stitching together two or more remote sensing images to create a mosaicked image. Application scenarios require the mosaicked image generated by the mosaicking task. These applications include surveying and mapping, national economic development, land and resources surveys, ecological and environmental monitoring, regional rescue operations, large-scale disaster monitoring, basic surveying and mapping basemap updates, local military operations, and emergency rescue.

[0058] Different application scenarios have different requirements for mosaicked images. For example, for applications such as surveying and mapping, national economic development, land and resources surveys, and ecological and environmental monitoring, mosaicking tasks are required to generate mosaicked images covering a large area. These mosaicked images must meet the following requirements: high accuracy, moderate timeliness, on-demand temporal requirements, and high product quality. For applications such as regional rescue and large-scale disaster monitoring, mosaicking tasks are required to generate mosaicked images covering a large area. These mosaicked images must meet the following requirements: moderate accuracy, maximum timeliness, high temporal requirements, and moderate product quality. For applications such as basic surveying and mapping basemap updates and local military operations, mosaicking tasks are required to update remote sensing imagery for a local area. These mosaicked images must meet the following requirements: high accuracy, high timeliness, maximum temporal requirements, and high product quality. For applications such as emergency rescue, mosaicking tasks are required to update remote sensing imagery for a local area. These mosaicked images must meet the following requirements: high accuracy, high timeliness, moderate temporal requirements, and high product quality.

[0059] In order to meet the needs of different application scenarios for mosaicked images, it is necessary to select the image mosaicking mode corresponding to the application scenario for image mosaicking.

[0060] In a possible implementation, the image mosaic mode includes at least one of regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic, and emergency update mosaic.

[0061] In a possible implementation, the mapping relationship between the application scenario and the image mosaic mode may be as shown in Table 1. When the application scenario of the mosaic task is obtained, the corresponding image mosaic mode may be determined by querying the mapping relationship table.

[0062] Table 1

[0063]

[0064] S120 , in the determined image mosaic mode, obtaining an image data source, and screening the image data source to select image data for image mosaic from the image data source.

[0065] In different image mosaic modes, the selected image data source is also different. In an embodiment where the image mosaic modes include regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic, and emergency update mosaic, the mapping relationship between the different image mosaic modes and the selected image data source can be shown in Table 1.

[0066] In the regional high-precision mosaic mode, you can choose historical remote sensing image data as the image data source, or you can choose the latest remote sensing image data acquired in real time as the image data source. There is no specific limitation here.

[0067] In the regional emergency mosaic mode, when the historical remote sensing image data meets the requirements of the application scenario, the historical remote sensing image data is selected as the image data source for rapid image mosaicking; when the historical remote sensing image data does not meet the requirements of the application scenario, the latest remote sensing image data acquired in real time is selected as the image data source.

[0068] In high-precision update mosaic mode, use digital orthophotos as the mosaic base map and select the image data source for local updates. When selecting the image data source for local updates, if the real-time remote sensing image data meets the application scenario requirements, select the real-time remote sensing image data as the image data source for local updates; if the real-time remote sensing image data does not meet the application scenario requirements, select the most recent historical remote sensing image data as the image data source for local updates.

[0069] In the emergency update mosaic mode, digital orthophotos are used as the mosaic base map, and the image data source for local updates is selected. When selecting the image data source for local updates, time constraints are prioritized, and the orthophotos to be updated are quickly generated based on the latest remote sensing images available under emergency rescue and other time conditions, with accuracy being secondary.

[0070] In a possible implementation, when filtering the image data source, different image mosaic modes are configured with different filtering methods, including at least one of filtering based on filtering conditions and filtering based on a filtering model.

[0071] When data is screened based on the screening conditions, the screening conditions include at least one of imaging conditions, image conditions, and task constraints.

[0072] The imaging conditions may specifically include at least one of: data imaging mode, imaging season and time, imaging angle and resolution, and topography.

[0073] Data imaging mode: The data imaging mode can include regional imaging mode, off-track strip mode and off-track single-view mode. There are differences in the consistency of geometric relationships between remote sensing image data obtained under different data imaging modes, which will have a certain degree of impact on the processing algorithm and even the processing accuracy. Therefore, the data imaging mode should be considered when screening the image data source.

[0074] Imaging season and time: Data can be filtered according to the four seasons of spring, summer, autumn, and winter, or a specific time range, or by blocking a certain season or time range. Different mosaicking tasks have different time requirements for image data sources. For example, for surveying and mapping emergency repair tasks, in theory, the newer the imaging time, the better. However, if the time point is close to winter and heavy snow causes the ground to be covered, it may be necessary to avoid winter data or avoid the time period when heavy snow falls. For emergency tasks, the main focus is on the target area. As long as the target area and acquisition time meet the requirements, even if the area is covered with snow, there is no need to restrict whether it is winter imaging data.

[0075] Imaging Angle and Resolution: For regional high-precision mosaicking tasks, data acquired from surveying and mapping satellites / aerial platform observations is subject to payload design constraints, typically requiring vertical photography with an angle of less than 5 degrees. For regional emergency mosaicking tasks, however, high pitch and roll imaging is the norm for target acquisition, and constraints on data acquisition angles are appropriately relaxed. The data resolution requirements vary based on the task's requirements for data accuracy and mapping scale. For example, a 1:5000-10000 mapping scale requires an image resolution better than 1 meter; a 1:25000 mapping scale requires an image resolution better than 2.5 meters; and a 1:50000 mapping scale requires an image resolution better than 5 meters. For rapid local updates (survey revisions and supplementary surveys), resolution requirements can be appropriately relaxed. For example, 2.5-meter resolution images can be used for local survey revisions and supplementary surveys at 1:10000, while 10-meter resolution images can be used for survey revisions at 1:50000.

[0076] Topography: Different topographic conditions bring different requirements for data screening, preprocessing and mosaicking. For example, uniform field objects are prone to color cast after processing, and some loads themselves have serious color casts, so it is necessary to avoid selecting data for such loads. At the same time, uniform field objects cannot be processed based on morphological algorithms when selecting mosaic lines due to the lack of significant objects. Instead, it is necessary to use a mosaic line generation method based on synonymous points. At the same time, the effects of factors such as different solar altitude angles and mountainous terrain on radiation characteristics can also be comprehensively considered. Therefore, data screening needs to take into account topographic conditions, among which topography includes uniform fields, seas, deserts, mountains, plains, etc.

[0077] The image conditions can specifically include at least one of: radiometric quality, geometric internal and external accuracy, registration accuracy, cloud cover, and clarity. During sensor operation, factors such as payload failure, performance degradation, and changes in the space environment can degrade data quality. For example, gyroscope anomalies, camera output anomalies, camera defocus leading to reduced clarity, and noticeable streaking at certain settings can affect radiometric and geometric quality. Furthermore, data of varying accuracy has varying impacts on the accuracy of preprocessing and mosaicking. Cloud cover is also one of the most important factors influencing data usage. Generally, data with less than 10% cloud cover is selected as a filter. However, this condition can be relaxed for high mountainous areas or areas with high cloud cover due to climate-related factors.

[0078] The task constraints may specifically include: at least one of: adjacent image overlap, resolution difference within a region, and adjacent angle difference and range.

[0079] In regional high-precision mosaic mode, the screening criteria can include image conditions, imaging angle within the imaging conditions, and adjacent image overlap, as well as adjacent angle difference and range within the task constraints. The imaging angle can be less than or equal to 15°, the adjacent image overlap can be greater than or equal to 15%, and the inter-strip overlap within the adjacent angle difference and range can be greater than or equal to 5%. Cloud cover within the image conditions can be less than or equal to 10%. Radiometric consistency within the image conditions is directly screened by visual interpretation of the multispectral browsing image. Geometric registration accuracy is based on panchromatic imagery, with a threshold of distortion error within 10 pixels meeting the image requirements.

[0080] In the regional emergency mosaic mode, the screening conditions are the same as those in the regional high-precision mosaic mode. However, due to the extremely high timeliness requirements in the regional emergency mosaic mode, the constraints on imaging angle, adjacent image overlap, inter-strip overlap, and cloud cover in the image conditions can be relaxed compared to the regional high-precision mosaic mode. For example, the imaging angle can be relaxed to less than or equal to 30°, the adjacent image overlap can be relaxed to greater than or equal to 10%, the inter-strip overlap in the adjacent angle difference and range can be relaxed to greater than or equal to 10%, and the cloud cover in the image condition can be relaxed to less than or equal to 15%.

[0081] In high-precision update mosaic mode, when filtering remote sensing image data, the screening criteria can include image conditions and adjacent image overlap within task constraints. For imaging angles within 5°, the overlap can be greater than or equal to 15%. If the overlap area is small and the image positioning accuracy and image quality are good, the overlap can be relaxed to greater than or equal to 8%. Cloud cover can be less than or equal to 10%. Radiometric consistency within the image condition is directly screened by visual interpretation of the multispectral browsing image. Geometric registration accuracy is based on panchromatic imagery, with a threshold of distortion error within 10 pixels meeting the image requirements.

[0082] In the emergency update mosaic mode, when filtering remote sensing image data, the filtering conditions are the same as those in the high-precision update mosaic mode. However, due to the high timeliness requirements in the emergency update mosaic mode, the cloud cover in the image condition can be relaxed to less than or equal to 10%.

[0083] In different image mosaic modes, other screening conditions may be included to meet the needs of different application scenarios, which are not specifically limited here.

[0084] When data screening is performed based on the screening model, the screening model includes at least one of a new area task screening model and a similar task screening model.

[0085] A new region task is a mosaicking task performed on an area that has not previously undergone an image mosaicking task. For example, if an image mosaicking task has not been performed on the Haixi Autonomous Prefecture in Qinghai Province, then the mosaicking task of producing a 1-meter resolution mosaicked image of the entire prefecture in the summer of 2020 is a new region task.

[0086] For new area tasks, after filtering the image data sources based on the screening conditions, the initial image data sets that meet the screening conditions will also contain some redundant data in terms of region and imaging time, mainly including the following aspects: First, the imaging time of multi-view remote sensing image data is similar, but the resolution is different; second, the remote sensing image data of the same point has the same resolution, but the imaging time is different; third, the cloud cover and coverage rate of different remote sensing image data in the same area have their own advantages and disadvantages. For example, the cloud cover of data A is lower, but the coverage rate of the area is lower, while the cloud cover of data B is higher, and the coverage rate of the area is higher.

[0087] In order to further screen out the optimal image dataset from the initial dataset, after screening based on the screening conditions, further data screening is performed based on the new area task screening model. This can improve the retrieval efficiency of the image data source and reduce the manual participation in the screening process.

[0088] The new area task screening model is constructed based on the screening conditions. The new area task screening model can be specifically expressed as follows: γ=γ1*α1+γ2*α2+γ3*α3+……γ n *α n Among them, α1 to α n is the normalized data value of n screening conditions, γ1 to γ n is the weight coefficient preset for n screening conditions, and γ is the weight value corresponding to the image data.

[0089] Different new area task screening models are configured for different mosaicking modes. For example, in regional high-precision mosaicking mode, the new area task screening model can include four screening conditions: image condition, imaging angle within the imaging condition, adjacent image overlap, and adjacent angle difference and range within the task constraint. In regional high-precision mosaicking mode, the new area task screening model can include five screening conditions: phase, imaging angle, adjacent image overlap, inter-strip overlap, and cloud cover within the image condition. In high-precision update mode, the new area task screening model can include four screening conditions: image condition, imaging angle within the imaging condition, adjacent image overlap, and adjacent angle difference and range within the task constraint. In emergency update mode, the new area task screening model can include five screening conditions: phase, imaging angle, adjacent image overlap, inter-strip overlap, and cloud cover within the image condition. The weight coefficient for each screening condition can be set based on the application scenario. The weight coefficient value is in the range [0, 1], with values ​​closer to 0 indicating a lower weight and values ​​closer to 1 indicating a higher weight.

[0090] The steps for screening image data sources based on the new area task screening model include:

[0091] Normalize the image data values ​​corresponding to each filter condition. For example, for imaging time, the newer the imaging time, the larger the normalization value. Take the reciprocal (6.34*10-9, 1) of the difference between the current set time and the imaging time (the range is 1 second, 5 years (157680000 seconds)), normalize it to the range (0, 100), and take the value α1. For cloud cover, the percentage value (0.01, 100) within the normalized range (0, 100) is taken as α2. For resolution (0.01m, 1000m), take the reciprocal (0.001, 100), normalize it to the range (0, 100), and take the value α3.

[0092] The normalized data values ​​of each screening condition are input into the new area task screening model to obtain the weight value corresponding to the image.

[0093] Repeat the second and third steps until the weight value of each initial image data is obtained, and filter out the image data whose weight value meets the set requirements.

[0094] The results output in the third step are deduplicated to remove duplicate data, and finally an optimal data set is output to complete the screening response process.

[0095] Similar tasks refer to tasks that are similar to mosaicking tasks within the entire historical task library. These tasks are considered similar if at least one of the following is substantially consistent: task area, image resolution, imaging time, or accuracy. For example, the production of a 0.5-meter mosaic image of the entire city of Beijing in the autumn of 2021 is substantially consistent with the production of a 0.5-meter mosaic image of the entire city of Beijing in the autumn of 2017, as found in the historical task library. Therefore, these tasks are considered similar.

[0096] When acquiring a mosaic task, the similar task screening model can screen similar tasks based on at least one of the following requirements: the task area, image resolution, imaging time, and accuracy. If similar tasks exist in historical data, the task screening model used by similar tasks can be used to obtain image data sources whose weight values ​​meet the set requirements. If no mosaic task exists in historical data, the current mosaic task is determined to be a new area task, and the image data source is screened using the above screening method for new area task image data sources.

[0097] In a possible implementation, after the image data for image mosaicking is filtered out from the image data source, the process further includes: performing pre-processing on the filtered image data. Different image mosaicking modes correspond to corresponding data pre-processing methods. For example, in the regional high-precision mosaicking mode, the filtered image data can be subjected to regional network adjustment and orthorectification, or the filtered image data can be subjected to orthorectification. For another example, in the regional emergency mosaicking mode, the filtered image data can be subjected to system geometry correction, or the filtered image data can be subjected to secondary correction. For another example, in the high-precision update mosaicking model, the filtered multiple image data can be subjected to regional network adjustment and orthorectification, and the filtered single-scene data can be directly subjected to orthorectification. For another example, in the emergency update mosaicking mode, the filtered image data can be subjected to orthorectification.

[0098] S130: Based on the image data, construct mosaic lines using the mosaic line construction strategy associated with the image mosaic mode.

[0099] In a possible implementation, constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode includes: when the image mosaic mode is regional high-precision mosaic, high-precision update mosaic, or emergency update mosaic, constructing mosaic lines using the mosaic line construction strategy based on the morphological method; and when the image mosaic mode is regional emergency mosaic, constructing mosaic lines using the mosaic line construction strategy based on synonymous points.

[0100] In one possible implementation, a tessellation line construction strategy based on a morphological method is used to construct tessellation lines, including: Figure 2 The following steps are shown.

[0101] Step 1: Get the effective overlapping area of ​​the images to be mosaicked. The effective overlapping area is the overlapping area of ​​the effective areas of the images to be mosaicked. The schematic diagram of obtaining the overlapping area of ​​the effective area is as follows: Figure 3 As shown, firstly, the effective area of ​​the image to be mosaicked is obtained, and then the overlapping area between adjacent images is obtained as the effective overlapping area.

[0102] Step 2: Generate initial mosaic reference lines. Based on the overlapping area between adjacent images obtained in step 1, an automatic mosaic line extraction method based on pixel difference feathering is used to generate initial mosaic reference lines. The initial mosaic reference lines are as follows: Figure 4 As shown, the initial mosaic reference line is represented by a coordinate sequence in the overlapping area.

[0103] Step 3: Segment the overlapping area based on the primary feature library. The specific steps include:

[0104] 1) Feature library selection and classification.

[0105] The overlapping areas of acquired satellite imagery are classified for feature features within the region, creating a primary and secondary feature library. The primary library is used to determine segment boundaries, while the secondary library is used to determine mosaic line selection. The primary and secondary feature libraries are selected from the feature library data. The primary library includes dense and sparse boundary vector data, while the secondary library includes road vector data, water system vector data, and steep slope data. The feature library also primarily performs cluster analysis on the features within the overlapping areas of the images.

[0106] Overlapping areas are segmented. Sparse, dense, and mixed feature categories are determined and the features are divided into blocks. Based on feature type and established feature database data from various remote sensing geographic information resources, overlapping areas are divided into dense, sparse, and mixed segments according to the dense and sparse boundary vector data in the first-level feature database.

[0107] Determine whether the segment boundary contains road vector points (here north and south are relative to the block), such as the starting point within the block, the end point within the block, etc.

[0108] The overlapping segmentation results obtained after the above steps are as follows Figure 5 shown.

[0109] Step 4: Extract mosaic lines from the overlapping areas after segmentation based on the secondary feature library. For segmented images, there are mainly three situations: dense segments, sparse segments, and mixed segments.

[0110] 1) For dense segments: First, determine whether there is dense boundary vector data in the overlapping area. If so, segment the area according to roads and water areas, and use the automatic mosaic line selection method based on water system vectors and road vectors to obtain mosaic lines. If there is no boundary vector data, no segmentation is required, and the initial mosaic reference line in step 2 is directly selected.

[0111] 2) For sparse segments: First, determine whether the overlapping area has sparse boundary vector data. If so, construct a minimum bounding rectangle and intersect the overlapping area boundary with the known sparse segment boundary to obtain the lower boundary intersection point to obtain the mosaic line. If there is no sparse boundary vector data, then the initial mosaic reference line in step 2 is directly selected without segmentation. The remaining segments except the dense segment and the sparse segment are collectively referred to as mixed segments.

[0112] 3) For the generation of tessellation lines in mixed segments other than dense segments and sparse segments, the intersection of the initial tessellation reference line and the sparse segment in step 2, or the intersection of the initial tessellation reference line and the dense segment boundary line are selected as the starting point and end point of the tessellation line segment, and the initial tessellation line of this segment in step 2 is used as the tessellation line.

[0113] The schematic diagram of the mosaic line extraction result obtained through the above steps is as follows Figure 6 shown.

[0114] Step 5: Connect the inlay lines between each segment.

[0115] The initial mosaic reference line is used for the mixed segment; the optimal mosaic lines A2A5 and A5A3 of the characteristic vector obtained in step 4 are used for the dense segment; the optimal mosaic lines AA4 and A4A1 obtained in step 4 are used for the sparse segment. Finally, the endpoints of the segment and inter-segment mosaic lines on the segment boundary lines of the mixed segment, sparse segment, and dense segment are merged to form the complete optimal mosaic line vector in the overlapping area, that is, the complete mosaic line in this embodiment is as follows: Figure 7 In the middle, AA4A1A2A5A3B.

[0116] Step 6: After generating the complete mosaic line, store the mosaic line in the feature library. When the difference in the ground feature changes is small, use it as the mosaic line for the next mosaic of the area. When the difference in the ground feature changes is large, use the obtained complete mosaic line as the mosaic reference line and continue to perform the mosaic line optimization algorithm of steps 1 to 5 in this embodiment, thereby improving the mosaic line generation efficiency in regional mosaicking.

[0117] In a possible implementation, the mosaic line construction strategy based on the morphological method can also include the following steps: Figure 8 The following steps are shown.

[0118] Step 1: Quickly search the effective area range of the overlapping area based on image features. The effective area range of the overlapping area is as follows: Figure 9 shown.

[0119] Combining the characteristics of dynamic intelligent update mosaicking of geographic elements in image raster data, this paper studies the method of rapid retrieval of the effective area range of overlapping areas, especially the rapid retrieval of the effective range of overlapping areas of single-scene images based on the base map. On this basis, the effective range of remote sensing images that can generate mosaic lines is obtained, and useless information in the mosaic line generation process is eliminated.

[0120] Step 2: Research on geographic entity feature information extraction for image mosaicking.

[0121] Based on the characteristics of typical geographic elements (mainly roads, buildings, water systems, vegetation, etc.), we carry out research on geographic entity element extraction rules and geographic element analysis; on this basis, we study the deep learning method for efficient and rapid generation and extraction of mosaic lines in the ground feature segmentation method, construct geographic entity element units, and provide technical support for subsequent mosaic line extraction. Figure 10 shown.

[0122] Step 3: Obtain boundary information of ground object target elements.

[0123] Based on the boundary information of geographic features and the characteristics of intelligent update mosaicking, the boundaries of the segmented image are acquired, the edges are updated, the areas of interest are blocked, and the segmented mosaic lines are generated, providing the boundary information of the features for the mosaic line generation.

[0124] Update edge determination.

[0125] Since the amount of remote sensing image data is large, in order to reduce disk IO and improve efficiency, this embodiment uses the range of 10% pixels extending from the overlap range of the single-view remote sensing image and the base map image to obtain the following Figure 11 Update the edge determination result with the image shown. n To represent multiple single-view images to be updated, R is used to represent the mosaic base map, where X∈R. Determining the effective overlapping area is beneficial to reducing the time of generating updated mosaic lines while clarifying the area of ​​interest.

[0126] (2) Area of ​​interest block

[0127] For the determined image update edge, the image to be mosaicked is divided into four blocks, and after uniform color, four sub-images of the same size are determined to update the edge. Figure 12As shown. In this embodiment, the single image is set to S, and the sub-image set is set to {S1, S2, S3, S4}, where S = S1 + S2 + S3 + S4. This block strategy takes into account the ground features, mainly including: along the water system, roads, and around buildings. The updated mosaic line segments generated based on this block strategy are as follows Figure 13 shown.

[0128] (3) Generation of segmented tessellation lines

[0129] For each sub-image S i , the improved GraphCut method in this embodiment is adopted to realize the generation of segmented mosaic lines. The mosaic line search algorithm structure based on the improved GraphCut is as follows: Figure 14 As shown in the figure, the steps include: first, using Gaussian mixture model instead of histogram to describe the probability distribution of color information, thereby extending the image segmentation range from grayscale image to color image; second, using iterative method instead of estimating Gaussian mixture model parameters once, thereby improving the segmentation accuracy; third, after introducing the GMM color data model, the energy function 5 can be rewritten as:

[0130] E(α,k,θ,z)=U(α,k,θ,z)+V(α,z)

[0131] Where k=(k1,…,k n …, k N ), k n ∈{1, 2, ..., K} is the GMM label of each pixel. Its data item can be defined as:

[0132]

[0133] The fourth step is to initialize the image based on the energy function. The fifth step is to segment the image and background based on the pixels in the image. Repeat steps 1 to 3 until the conditions are met. The segmented mosaic line that takes into account the water system, road buildings is as follows: Figure 15 shown.

[0134] Step 4: Generate mosaic lines taking into account geographic entities.

[0135] Based on the boundary information of geographic features, the updated edge determination based on the integrity of the target features is carried out, and the optimal mosaic line is generated through deep learning algorithm search to obtain the optimized mosaic boundary of the features.

[0136] The data is extracted, aggregated, and segmented to simplify the geographic entity element units. When formulating extraction rules, the main focus is on analyzing the graphic and attribute characteristics of elements such as boundaries, roads, water systems, and residential areas, with a focus on the scope and boundary information of geographic entity elements. The analysis and utilization of element information such as road centerlines and road edges, water system edges and water system skeleton lines, residential area edges, and cultivated land edges are performed separately. Combined with the geographic characteristics of the mapping area, the optimal entity edge is selected as the boundary of the geographic entity element unit. The following rules are followed for extracting entity boundaries:

[0137] 1) Select linear geographic feature objects (such as roads and rivers) to construct the mosaic area skeleton, dividing the entire area into small-scale geographic entity element units with relatively regular shapes.

[0138] 2) Avoid constructing geographic entity element unit boundaries that cross overhead geographic elements (such as overpasses, elevated roads, pipelines, etc.). If avoidance is unavoidable, the mosaicked results can be re-edited to ensure the continuity of the geographic element image and prevent distortion.

[0139] 3) For large-area or close-range geographic elements (such as woodlands, roads, rivers, etc.), the boundary or obvious linear features in the image (such as vegetation, artificial structures, etc.) can be used for segmentation to refine the unit structure of the geographic entity elements.

[0140] 4) For fragmented or small geographic feature objects (such as houses, single artificial buildings, etc.), refer to the overlapping area of ​​the surrounding single-piece images, and perform aggregation processing of geographic entity feature units on the basis of ensuring the integrity of the geographic feature objects to improve the processing efficiency of image mosaicking.

[0141] Through the above rule constraints, reasonable and effective geographic information topological map units can be produced, laying the foundation for subsequent image segmentation and efficient processing based on orthophoto mosaicking.

[0142] The improved GrabCut optimization model is used to generate mosaic line segments, segment the blocked image, and intelligently bypass buildings and other landforms through topological inspection. The four generated mosaic line segments are merged to generate a complete mosaic line, realizing updated mosaicking based on the existing regional base map.

[0143] S140: mosaicking the image data according to the constructed mosaicking lines, that is, obtaining a mosaicked image according to the constructed mosaicking lines.

[0144] In a possible implementation, after mosaicking the image data according to the constructed mosaicking lines, the method further includes: performing color processing on the mosaicked image data.

[0145] In one possible implementation, when color processing is performed on the mosaicked image data, it includes: when the image mosaic mode is regional high-precision mosaic or regional emergency mosaic, color consistency processing is performed without a base map; when the image mosaic mode is high-precision update mosaic or emergency update mosaic, color consistency processing is performed based on the base map.

[0146] In this disclosure, based on the application scenario of the current mosaicking task, a corresponding image mosaicking mode is determined; image data is screened and mosaicking lines are constructed within the determined image mosaicking mode, thereby completing the mosaicking process of the image data. Since the image mosaicking method is the same under the same image mosaicking mode, the automated operation of remote sensing image mosaicking can be achieved by determining the image mosaicking mode.

[0147] <Device Example>

[0148] Figure 16 FIG. 1 is a schematic block diagram of a remote sensing image mosaicking device according to an embodiment of the present disclosure. Figure 16 As shown, the remote sensing image mosaic device 100 includes:

[0149] The mosaic mode acquisition module 110 is used to determine the corresponding image mosaic mode based on the application scenario of the current mosaic task;

[0150] The data screening module 120 is used to obtain an image data source under the determined image mosaic mode, and screen the image data source to select image data for image mosaic from the image data source;

[0151] A mosaic line construction module 130 is configured to construct mosaic lines based on the image data using a mosaic line construction strategy associated with the image mosaic mode;

[0152] The mosaic module 140 is used to perform mosaic processing on the image data according to the constructed mosaic lines.

[0153] In a possible implementation, the image mosaic mode includes at least one of regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic, and emergency update mosaic.

[0154] In one possible implementation, when filtering the image data source, different filtering methods are correspondingly set in different image mosaic modes; the filtering methods include at least one filtering method based on filtering conditions and filtering based on filtering models; wherein, when data filtering is performed based on filtering conditions, the filtering conditions include: at least one of imaging conditions, image conditions and task constraints; when data filtering is performed based on a filtering model, the filtering model includes at least one of a new area task screening model and a similar task screening model; the new area screening model is constructed according to the filtering conditions, and the similar task screening model is constructed according to the mosaic task.

[0155] In a possible implementation, constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode includes:

[0156] When the image mosaic mode is regional high-precision mosaic, high-precision update mosaic, or emergency update mosaic, mosaic lines are constructed using a mosaic line construction strategy based on morphological methods.

[0157] When the image mosaic mode is regional emergency mosaic, mosaic lines are constructed using the mosaic line construction strategy based on synonymous points.

[0158] In a possible implementation, after the image data for image mosaicking is screened out from the image data source, the method further includes: performing pre-processing on the screened image data;

[0159] Among them, different image mosaic modes correspond to corresponding data preprocessing methods.

[0160] In a possible implementation, after mosaicking the image data according to the constructed mosaicking lines, the method further includes: performing color processing on the mosaicked image data.

[0161] In a possible implementation, color processing is performed on the mosaicked image data, including:

[0162] When the image mosaic mode is regional high-precision mosaic or regional emergency mosaic, color consistency processing is performed without a base map;

[0163] When the image mosaic mode is high-precision update mosaic or emergency update mosaic, color consistency processing is performed based on the base map.

[0164] <Equipment Example>

[0165] Figure 17 FIG. 1 is a schematic block diagram of a remote sensing image mosaic device according to an embodiment of the present disclosure. Figure 17As shown, the remote sensing image mosaicking device 200 includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the above remote sensing image mosaicking methods when executing the executable instructions.

[0166] It should be noted that there may be one or more processors 210. Furthermore, the remote sensing image mosaicking device 200 according to the disclosed embodiment may further include an input device 230 and an output device 240. The processor 210, memory 220, input device 230, and output device 240 may be connected via a bus or other means, which are not specifically limited herein.

[0167] Memory 220, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and various modules, such as the programs or modules corresponding to the remote sensing image mosaicking method according to the embodiments of the present disclosure. Processor 210 executes the software programs or modules stored in memory 220 to perform various functional applications and data processing of remote sensing image mosaicking device 200.

[0168] The input device 230 may be used to receive input numbers or signals. The signals may be key signals related to user settings and function control of the device / terminal / server. The output device 240 may include a display device such as a display screen.

[0169] <Computer-readable storage medium embodiment>

[0170] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by the processor 210, any of the above remote sensing image mosaicking methods is implemented.

[0171] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technical improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A remote sensing image mosaic method, characterized in that: include: Determine the corresponding image mosaic mode based on the mapping relationship between the application scenario of the current mosaic task, image mosaic mode, task requirements, and image data source. Task requirements include product accuracy requirements, timeliness requirements, image phase requirements, and product quality requirements. In the determined image mosaic mode, obtaining an image data source, and screening the image data source to select image data for image mosaic from the image data source; Based on the image data, constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode; Performing mosaic processing on the image data according to the constructed mosaic lines; The image mosaic mode includes at least one of: regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic and emergency update mosaic; Constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode includes: When the image mosaic mode is regional high-precision mosaic, high-precision update mosaic, or emergency update mosaic, mosaic lines are constructed using a mosaic line construction strategy based on a morphological method; When the image mosaic mode is regional emergency mosaic, mosaic lines are constructed using a mosaic line construction strategy based on synonymous points.

2. The method according to claim 1, characterized in that When screening the image data source, different screening methods are set correspondingly under different image mosaic modes; The screening method includes at least one of screening based on screening conditions and screening based on screening models; Wherein, when data screening is performed based on the screening conditions, the screening conditions include: at least one of imaging conditions, image conditions and task constraints; When data screening is performed based on the screening model, the screening model includes at least one of a new area task screening model and a similar task screening model; The new regional task screening model is constructed according to the screening conditions.

3. The method according to claim 1, characterized in that After the image data for image mosaicking is screened out from the image data source, the method further includes: performing pre-processing on the screened image data; Among them, different image mosaic modes are matched with corresponding data preprocessing methods.

4. The method according to claim 1, wherein After mosaicking the image data according to the constructed mosaicking lines, the method further includes: performing color processing on the mosaicked image data.

5. The method according to claim 4, characterized in that Color processing of mosaicked image data includes: When the image mosaic mode is regional high-precision mosaic or regional emergency mosaic, color uniformity processing is performed without a base map; When the image mosaic mode is high-precision update mosaic or emergency update mosaic, color uniformity processing is performed based on the base map.

6. A remote sensing image mosaic device, characterized in that: include: The mosaic mode acquisition module is used to determine the corresponding image mosaic mode based on the mapping relationship between the application scenario of the current mosaic task, the image mosaic mode, the task requirements, and the image data source. The task requirements include product accuracy requirements, timeliness requirements, image phase requirements, and product quality requirements. A data screening model is used to obtain an image data source under the determined image mosaic mode, and to screen the image data source to select image data for image mosaicking from the image data source; A mosaic line construction module, configured to construct mosaic lines based on the image data using a mosaic line construction strategy associated with the image mosaic mode; A mosaic module, configured to perform mosaic processing on the image data according to the constructed mosaic lines; The image mosaic mode includes at least one of: regional high-precision mosaic, regional emergency mosaic, high-precision update mosaic and emergency update mosaic; Constructing mosaic lines using the mosaic line construction strategy associated with the image mosaic mode includes: When the image mosaic mode is regional high-precision mosaic, high-precision update mosaic, or emergency update mosaic, mosaic lines are constructed using a mosaic line construction strategy based on a morphological method; When the image mosaic mode is regional emergency mosaic, mosaic lines are constructed using a mosaic line construction strategy based on synonymous points.

7. A remote sensing image mosaic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 5 when executing the executable instructions.

8. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images