Remote sensing information integrated processing system and method

Abstract

The invention discloses a remote sensing information integrated processing system and method. The method comprises the following steps: fusing multi-source data; organizing and managing data; business information management software; and comprehensively judging the image. According to business demand analysis, the method comprises two processes of fixed target observation and marine ship target observation. According to the method, operation unit ground data processing adaptive to satellite resources is realized, urgent needs of satellite infrared and hyperspectral application processing are met, and target multi-dimensional feature extraction and analysis recognition capabilities are greatly improved. Through integrated construction of the system, integration of data organization and management, integration of multi-source data interpretation, integration of data comprehensive reorganization and integration of business management are realized, and accuracy and production efficiency of remote sensing data of an operation unit are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing survey, and in particular to a remote sensing information integrated processing system and method. Background Art

[0002] With the rapid development of high-altitude survey technology, the satellite resources available to operating units are becoming increasingly abundant. The massive data provided by the satellite series will greatly enhance the operating units' survey capabilities for a wide range of maritime ship targets, and will inevitably make high-altitude survey an important means to improve the operating units' informationized combat capabilities.

[0003] Based on the experience and foundation of high-altitude survey and observation research and development in the main operating environment, according to the characteristics of marine remote sensing data and the requirements of the operating environment, the processing and fusion of optical, SAR, infrared, hyperspectral and other image data and digital survey data are realized to make full use of domestic, civil and foreign commercial satellite resources, form a comprehensive guarantee capability for nearshore and offshore targets based on satellite information, and meet the operating units' requirements for high timeliness and high precision of remote sensing data in the new operating environment.

[0004] In recent years, with the gradual maturity of my country's remote sensing satellite network, remote sensing satellites have accumulated massive amounts of remote sensing data in hot spots. At the same time, other remote sensing data sources, such as commercial remote sensing satellites, have also accumulated considerable amounts of data. The existing remote sensing data processing systems are lacking in the ability to process massive amounts of remote sensing data and lack effective data organization and management models. The information mining capabilities of multi-source heterogeneous data still need to be improved, which has become a bottleneck problem currently faced by operating units in remote sensing information processing and cannot meet the needs of combat data extraction in the operating environment under the new situation. Summary of the invention

[0005] The purpose of the present invention is to provide a remote sensing information integrated processing system and method to solve the problems existing in the prior art.

[0006] To achieve the above object, a remote sensing information integrated processing system of the present invention comprises:

[0007] (1) Multi-source data fusion software, which provides support for integrated comprehensive display of multi-source high-altitude remote sensing image data and digital data for image data, digital survey data and multimedia data, and is composed of data information fusion and comprehensive display application components;

[0008] (2) Data organization and management software, which manages the visible light, SAR, infrared, and hyperspectral data connected to the system as well as digital and technical survey text data, and realizes the integrated organization and management of multi-source massive remote sensing data materials and data products. It consists of four components: basic data management, target characteristic data management, data material and data product management, and system management and maintenance;

[0009] (3) Business information management software, which realizes the information management of data production business, completes the statistical analysis of data, the monitoring and statistics of operation tasks, and the generation of statistical reports. It consists of three parts: task management component, data statistics component, and user management component;

[0010] (4) Image comprehensive judgment software, which processes hyperspectral, thermal infrared, visible light and SAR image data. It consists of three components: remote sensing image integrated processing, image data integrated interpretation, and remote sensing data integrated compilation;

[0011] System external interface: the external relationship of the remote sensing information integrated processing system includes interfaces with the data distribution system and the image and text system;

[0012] System internal interface, the remote sensing information integrated processing system internal interface is the interface between multi-source data fusion software, data organization management software, business information management software and image comprehensive judgment software;

[0013] According to the business demand analysis, the remote sensing information integrated processing system includes two major processes: fixed target observation and offshore ship target observation.

[0014] Furthermore, the data information fusion component consists of three parts: digital and imaging target association, high-altitude and heterogeneous target information association, and historical data information mining; the comprehensive display application component consists of two parts: data comprehensive display and satellite in-orbit operation status simulation.

[0015] Furthermore, the basic data management component includes six component units: digital signal data storage unit, hyperspectral product data storage unit, infrared data product storage unit, basic data product management unit, target knowledge base management, and data product retrieval and access unit; the target characteristic data management component is composed of two component units: hyperspectral spectral characteristic library management and access, and infrared characteristic library management and access; the data material and data product management component is composed of three component units: text material management and access, image material management and access, and data product management and access; the system management and maintenance component is composed of two component units: user authority log management, and database backup and monitoring.

[0016] Furthermore, the task management component is composed of two functional units: job task initiation and job task status management and prompts; the data statistics component is composed of four functional units: data acquisition statistics, data distribution statistics, data product statistics, and workload statistics; the user management component is composed of three functional units: user information management, user login and authorization, and user log information management.

[0017] Furthermore, the remote sensing image integrated processing component consists of three parts: general processing of hyperspectral images, general processing of infrared images, and general processing of visible light and SAR; the image data integrated interpretation component consists of three parts: basic interpretation function, information extraction and analysis, and target detection and auxiliary recognition; the remote sensing data integrated compilation component consists of four parts: multi-source data specification loading, multi-source data composite visualization, data style editing and decoration, and data template production and management.

[0018] Furthermore, the fixed target observation process is aimed at operating units and ports, using the second-generation distribution data packet as input to determine the number and type of ship targets in the port; the marine ship target observation process is aimed at marine ship targets, using satellite visible light wide-band data, ScanSAR data, combined with digital survey data, situation distribution data, text data, and integrated historical database and knowledge base data to determine large marine ship targets.

[0019] The present invention also provides a method using the remote sensing information integrated processing system, comprising:

[0020] Step 1: Platform data connection, based on the data information sharing service system service terminal receiving the following high-altitude main operating environment satellite resource data;

[0021] Step 2: Integrated remote sensing image processing, basic data processing for hyperspectral, thermal infrared, visible light and SAR image data;

[0022] Step 3: Integrated interpretation of image data, basic browsing, measurement and annotation; extraction of thematic information from hyperspectral and thermal infrared data; ship target detection and auxiliary recognition analysis based on visible light and SAR images;

[0023] Step 4: Multi-source data fusion, to associate digital signals with imaging targets, to associate imaging data with heterogeneous information, and to mine and visualize historical data;

[0024] Step 5: Remote sensing data integration and compilation: Based on the results of image interpretation and analysis, various data materials are analyzed, processed and integrated, and data material information is produced and standardized output using various annotation, finishing and comprehensive display functions according to standard data product specifications;

[0025] Step 6: Integrated management of business information, informatization management of data production business, initiation of operation tasks, monitoring of operation tasks, data information statistics and user management;

[0026] Step 7: Data organization and management: manage the visible light, SAR, infrared, hyperspectral data, and digital and technical survey text data connected to the system, and integrate the organization and management of multi-source massive remote sensing data materials and data products;

[0027] Step 8: Comprehensive display application, integrated display of multi-source high-altitude remote sensing image data and digital data, to achieve integrated three-dimensional display of image data, digital survey data, and multimedia data; at the same time, add a satellite on-orbit operation status simulation module to simulate and visualize the on-orbit satellite resources.

[0028] Furthermore, the integrated interpretation of image data in step 3 further includes: refined extraction of hyperspectral ground feature elements based on mixed pixel decomposition and spectral purification.

[0029] Furthermore, the integrated interpretation of image data in step 3 further includes: determining the working state of the operating target based on target thermal radiation modeling and multi-temporal analysis.

[0030] Furthermore, the integrated interpretation of image data in step 3 further includes: feature mining of massive SAR data based on a sparse deep belief network.

[0031] Furthermore, the multi-source data fusion in step 4 further includes: fusion of ship target trajectories based on multi-source information.

[0032] Furthermore, the integrated compilation of remote sensing data in step 5 further includes: integrated interpretation of software plug-in architecture and multi-source payload data of operating units.

[0033] The method of the present invention has the following advantages:

[0034] The present invention realizes the ground data processing of the operating unit that is compatible with satellite resources, meets the urgent need for satellite infrared and hyperspectral application processing, and greatly improves the multi-dimensional feature extraction and analysis and recognition capabilities of the target. Through the integrated construction of the system, the integration of data organization and management, the integration of multi-source data interpretation, the integration of data comprehensive compilation and the integration of business management are realized, which greatly improves the accuracy and production efficiency of the remote sensing data of the operating unit.

[0035] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following specifically cites a preferred embodiment and describes it in detail with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the fixed target observation process;

[0037] Figure 2 This is the internal interface diagram of the system;

[0038] Figure 3 It is a diagram of the components for integrated interpretation of image data;

[0039] Figure 4 This is a schematic diagram of the comparison of the computational efficiency of FGDA end member extraction;

[0040] Figure 5 Schematic diagram of the first 10 bands obtained by HOSVD processing;

[0041] Figure 6 Schematic diagram of feature information obtained by HOSVD processing;

[0042] Figure 7 Schematic diagram of target thermal radiation modeling analysis;

[0043] Figure 8 This is the structure diagram of the target feature learning system based on deep belief network;

[0044] Fig. 9 Schematic diagram for pre-training and model parameter initialization;

[0045] Fig.10 It is a schematic diagram of position and error ellipse;

[0046] Fig.11 Schematic diagram of trajectory data compression (compression ratio on the right);

[0047] Fig.12 This is the software architecture diagram of "microkernel + plug-in". DETAILED DESCRIPTION

[0048] The technical scheme of the present invention will be clearly and completely described below in conjunction with specific embodiments, but those skilled in the art should understand that the embodiments described below are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Based on the principle of integrated design, the present invention establishes an integrated processing platform for processing, interpreting and fusion identification of multi-source remote sensing survey images such as visible light, infrared, hyperspectral, SAR, and various types of information, realizing integrated data organization and management, integrated multi-source data interpretation, integrated data comprehensive compilation and integrated business process management, so as to enhance the comprehensive guarantee capability of multi-source remote sensing data fusion processing and data product production of operating units.

[0050] The remote sensing information integrated processing system of the operating unit is a comprehensive and highly professional system that operates in a business-oriented manner. The system involves a variety of technologies such as network and communication, data organization and management, and image processing. In order to achieve the design goals of the system, the construction of the system should follow the software engineering specifications, take "practical, reliable, efficient, and advanced" as the basic principles, establish a "standardized, secure, and open" system, and rely on advanced information technology and strict management systems to make image processing fast, efficient, safe, and stable.

[0051] A remote sensing information integrated processing system, the system consists of:

[0052] Aiming at the urgent need of satellite survey data security for operating units, solving the contradiction between the rapidly growing data volume and the bottleneck of data storage, management and information mining technology, comprehensively utilizing various existing satellite remote sensing data and digital survey data, building a "remote sensing information processing system", realizing the unified and organic management of massive remote sensing data and other operating environment data, multi-dimensional and multi-mode presentation of multi-source data, and the decoration, distribution and application services of marine operating environment situation data based on satellite survey information. The system consists of four software: multi-source data fusion software, data organization management software, business information management software, and image comprehensive judgment software.

[0053] (1) Multi-source data fusion software

[0054] The multi-source data fusion software takes into account the characteristics of image data, digital survey data and multimedia data, and provides support for the integrated comprehensive display of multi-source high-altitude remote sensing image data and digital data.

[0055] Data information fusion is based on the target information extracted from panchromatic, multispectral, SAR, infrared images, spectral images, as well as digital survey information, historical data information and other data information to achieve fusion recognition, auxiliary decision-making, and form comprehensive target recognition capabilities;

[0056] The comprehensive display application provides an integrated display of multi-source high-altitude remote sensing data, realizes the integrated three-dimensional display of image data, digital survey data, multimedia data and other auxiliary information, and has the ability to visualize the operating status of our in-orbit survey satellites. It can simulate and analyze in-orbit satellite resources according to the survey area, survey time, etc., and provide technical support for the formulation of survey needs and the selection of collaborative survey strategies.

[0057] Multi-source data fusion software consists of data information fusion and comprehensive display application components.

[0058] Data information fusion component

[0059] The data information fusion component is based on the target information extracted from full-color, multi-spectral, SAR, infrared images, spectral images, as well as digital survey information, historical data information and other data information to achieve fusion recognition, auxiliary decision-making, and form a comprehensive target recognition capability. The data information fusion component consists of three parts: digital and imaging target association, high-altitude and heterogeneous target information association, and historical data information mining.

[0060] (1) Association between numbers and imaging targets

[0061] The digital and imaging target association part is responsible for loading and displaying digital survey information data and imaging target information (mainly image slice data); associating and verifying digital survey data and imaging target data, mainly combining the position and integrating target related information (such as travel speed) to associate the position, and combining slices and digital fingerprints to perform model analysis. Specifically, the digital and imaging target association completes the access, analysis, and visualization of level 1 and level 2 data sent by the satellite; the retrieval and display of the satellite's imaging target information (mainly image slice data); and combining the position and integrating target related information (such as travel speed) to associate the digital survey data and imaging target data. Three types of component units are provided: digital information loading and display unit, imaging target data loading and display unit, and target association verification unit.

[0062] (2) High-altitude and heterogeneous target information association

[0063] The high-altitude and heterogeneous target information association provides retrieval and loading of high-altitude reconnaissance data (mainly image slice data) and heterogeneous information (mainly data), and uses an effective way to display them comprehensively, as well as associate visualization based on location description. It provides three types of component units: high-altitude survey data loading and display unit, heterogeneous information loading and display unit, and multi-source information comprehensive association visualization unit.

[0064] (3) Historical data information mining

[0065] Historical data information mining completes the comprehensive retrieval function of historical data information based on the target; based on the target information retrieval, it displays the important data information of the target, such as the trajectory, and can perform advanced display based on the time axis; it completes the statistical function of target-related information based on constraints (such as time and space), such as the frequency of occurrence in a certain area, so as to discover the movement law of the target in a period of time, etc. It provides three types of component units: target historical data information retrieval unit, target historical data information visualization unit and target data information statistical analysis unit.

[0066] Comprehensive display application components

[0067] The comprehensive display application provides integrated comprehensive display of multi-source high-altitude remote sensing image data and digital data, and realizes integrated three-dimensional display of image data, digital survey data, multimedia data, operation task status and other auxiliary information. At the same time, the simulation module of our satellite on-orbit operation status is added, which can simulate and visualize the on-orbit satellite resources according to the survey area, survey time, etc., and provide technical support for the formulation of survey requirements and the selection of collaborative survey strategies. The comprehensive display application component consists of two parts: comprehensive data display and satellite on-orbit operation status simulation.

[0068] (1) Comprehensive data display

[0069] The data comprehensive display part is responsible for parsing and loading multi-source data (image data, digital survey data, text data, image data and multimedia data), and comprehensively displaying it, realizing the integrated three-dimensional display of image data, digital survey data, multimedia data, operation task status and other auxiliary information. It consists of six component units: image data display unit, digital survey data display unit, ship track display unit, text data material display unit, image data material display unit, and multimedia data material display unit.

[0070] (2) Satellite on-orbit operation status simulation

[0071] The satellite on-orbit operation status simulation component is responsible for calculating the satellite orbit within a period of time based on the known satellite orbit elements, and using three-dimensional geographic space to load and display satellite models, satellite trajectories, and detectable ranges to simulate the satellite orbit. The time can be manually set to deduce the satellite operation status. It consists of three types of component units: satellite resource retrieval and browsing unit, satellite orbit simulation and display unit, and satellite survey and transit analysis unit.

[0072] Deployment

[0073] The multi-source data fusion software is oriented to multi-source data and realizes the display of image data, product data, etc. In terms of deployment, the multi-source data fusion software needs to be deployed on a workstation and deployed on the same machine with the image comprehensive judgment software and business information management software.

[0074] (2) Data organization and management software

[0075] The data organization and management software manages the visible light, SAR, infrared, hyperspectral data, and digital and technical survey text data connected to the system, and realizes the integrated organization and management of multi-source massive remote sensing data materials and data products.

[0076] The data organization and management software consists of four components: basic data management, target characteristic data management, data material and data product management, and system management and maintenance.

[0077] Basic data management components

[0078] The basic data management component includes the standardized storage, modification and deletion functions, and multi-condition compound retrieval functions for digital signal data, hyperspectral data, and infrared data products. The basic data management component consists of six types of component units: digital signal data storage unit, hyperspectral product data storage unit, infrared data product storage unit, basic data product management unit, target knowledge base management, and data product retrieval and access unit.

[0079] Target characteristic data management component

[0080] The target characteristic data management component provides management of the satellite hyperspectral characteristic library and infrared characteristic library, and provides manual interaction tools to realize the collection, storage and management of the hyperspectral spectral characteristic and infrared characteristic data of common objects and targets. On this basis, the unified access interface of hyperspectral characteristics and infrared spectral characteristics is encapsulated to realize the unified retrieval and access of the two characteristic data. The target characteristic data management component consists of two component units: hyperspectral spectral characteristic library management and access, and infrared characteristic library management and access.

[0081] Data material and data product management components

[0082] The data material and data product management component is mainly responsible for the standardized storage, management and unified access of text data materials, image data materials and data products. The data material and data product management component consists of three types of component units: text material management and access, image material management and access, and data product management and access.

[0083] System management and maintenance components

[0084] The system management and maintenance component provides two functions: user permission log management and database backup and monitoring. The user permission log management manages and maintains the user's permissions and access logs according to user needs; it provides a unified user interface and interactive tools to realize data backup and recovery, and monitors the storage status of the database. The system management and maintenance component is an improvement on the existing functions, and consists of two types of components: user permission log management and database backup and monitoring.

[0085] Database Design

[0086] The database system of the remote sensing information processing system of the operating unit mainly includes spatial geographic information database, basic database, target characteristic database, target knowledge base, data product library, data database, multimedia library, and operation and maintenance management library.

[0087] Deployment

[0088] Data organization and management software needs to be deployed on an independent data management server.

[0089] (3) Business information management software

[0090] Business information management software mainly realizes the information management of data production business, completes data production, data reception, statistical analysis of generated data, monitoring and statistics of job tasks, statistical report generation, etc. In terms of data task production, the software needs to provide full life cycle management functions for data production tasks; in terms of data task statistics, the software needs to provide statistical functions based on data acquisition, data product distribution, workload, etc.; in terms of data production life cycle, the software needs to provide each subsystem user with permission to access the business information management software.

[0091] Business information management software consists of three parts: task management component, data statistics component, and user management component.

[0092] Task Management Components

[0093] The task management component can build the automatic start function of daily task according to the needs of the task process, and provide interactive information entry tools to realize the manual initiation of tasks. At the same time, it can also monitor and manage the task status, and persist the key status of the task into the database, and provide information prompts for the task status. The task management component consists of two functional units: task initiation and task status management and prompts:

[0094] Data statistics component

[0095] The data statistics component can count the access of daily data, such as the load type, quantity and other indicators, for subsequent analysis; support statistics on data retrieval, download, and interpretation, etc., for analysis of data utilization efficiency; can conduct comprehensive statistics on the output of various types of data products, including data download time, satellite model, etc., and can also conduct correlation statistics on related images and data materials, so as to obtain data output efficiency; support statistical analysis of daily data interpretation work, such as statistics on image interpretation by personnel, data material output, data product output, etc. All kinds of statistical results can be displayed in charts. The data statistics component consists of four functional units: data acquisition statistics, data distribution statistics, data product statistics, and workload statistics.

[0096] User Management Widget

[0097] The user management component can provide user information management functions, including adding users, modifying user information (including passwords, roles, groups, etc.); providing a unified security authentication service access interface to facilitate user login and authorization; and logging user operations such as user login, image interpretation, data production, etc., to facilitate subsequent statistics and analysis. The user management component consists of three functional units: user information management, user login and authorization, and user log information management.

[0098] Deployment

[0099] The business information management software is mainly used to manage daily interpretation tasks, approvals, and user information. In terms of deployment, the business information management software needs to be deployed on a workstation and deployed on the same machine with the image comprehensive interpretation software and multi-source data fusion software.

[0100] (4) Image comprehensive judgment software

[0101] The image comprehensive interpretation software provides three functions: integrated remote sensing image processing, integrated image data interpretation, and integrated remote sensing data compilation, based on the characteristics of hyperspectral, thermal infrared, visible light and SAR image data. It provides integrated technical support for the transformation of images into data products, specifically including the following three aspects.

[0102] The image comprehensive judgment software consists of three components: integrated processing of remote sensing images, integrated interpretation of image data, and integrated compilation of remote sensing data.

[0103] Remote sensing image integrated processing components

[0104] For hyperspectral images, the integrated processing of remote sensing images constructs general processing tools for hyperspectral images such as spectral picking and spectral envelope removal; for thermal infrared images, it constructs general infrared processing tools such as infrared feature measurement and infrared pseudo-color display; for visible light and SAR image data, it provides common basic processing functions such as color scale adjustment, Lee filtering, and geometric correction to provide basic processing support for subsequent image data interpretation. The integrated processing component of remote sensing images consists of three parts: general processing of hyperspectral images, general processing of infrared images, and general processing of visible light and SAR.

[0105] (1) General processing of hyperspectral images

[0106] Hyperspectral image general processing supports false color display of hyperspectral images based on the band selected by the user; supports displaying the spectral curve of the target object at the specified location according to the user's mouse picking, and can dynamically adjust the display according to user settings such as band, wavelength range, etc.; can remove the envelope of the spectral curve: effectively highlight the absorption and reflection characteristics of the spectral curve, and normalize the spectral curve to a consistent spectral background for easy spectral comparison; can output a single band result image through mathematical expression-based calculations between multiple bands in the hyperspectral image, which is used to extract typical surface information such as vegetation index. Hyperspectral image general processing consists of four component units: band selection, spectrum picking, spectrum envelope removal, and spectrum spectrum calculation.

[0107] (2) General processing of infrared images

[0108] The infrared image general processing supports displaying the thermal radiation information of the target object at the specified location according to the user's mouse picking; it supports expressing the thermal radiation information of the scene objects in the form of pseudo-color, and at the same time displays the corresponding color palette to facilitate the observation of the judge. The infrared image general processing consists of two component units: infrared feature measurement and infrared data pseudo-color display.

[0109] (3) Common processing of visible light and SAR

[0110] Visible light and SAR general processing provides a series of standardized image processing tools to support users to adjust the display effects of visible light and SAR images and display them in real time, such as image brightness, contrast, color scale, color balance, grayscale curve, histogram equalization, filtering, scattering center extraction, etc. Visible light and SAR general processing consists of nine component units: brightness contrast adjustment, color scale adjustment, color balance, curve adjustment, histogram equalization, image sharpening, Lee filtering, strong scattering center extraction, and image geometry correction.

[0111] Image data integrated interpretation component

[0112] The integrated interpretation of image data provides basic browsing, measurement and annotation functions. At the same time, the software extracts thematic information from hyperspectral and thermal infrared image products, including state changes, operating environment elements, etc., and uses effective target detection algorithms to detect and extract large ship targets, and combines the knowledge base to realize auxiliary recognition and analysis of large ship targets. To provide technical support for the transformation of images to data materials, the integrated interpretation component of image data consists of three parts: basic interpretation function, information extraction and analysis, and target detection and auxiliary recognition. The composition of the integrated interpretation component of image data is shown in Figure 3 .

[0113] (1) Basic interpretation function

[0114] The basic interpretation function part provides basic browsing functions such as image display, roaming, and zooming; provides measurement functions such as length and area; and provides vector and text annotation functions. The basic interpretation function consists of three component units: basic display browsing tools, basic measurement tools, and vector annotation tools.

[0115] (2) Information extraction and analysis

[0116] Information extraction analysis can finely segment and extract targets based on the thermal radiation characteristics of targets on infrared images; support automatic statistical analysis of thermal radiation change trends of user-input areas based on historical time series images, and provide technical support for the analysis of changes in target working status; can automatically construct vegetation index algorithms based on satellite spectral parameter information to extract island and reef vegetation, and at the same time count vegetation coverage; support semi-automatic extraction of sea ice coverage information based on sea ice spectral information by user interactive selection of training samples, and at the same time count sea ice area and other information; support classification and extraction of tidal flat types by user interactive selection of training samples; support classification and mapping of working environment by user interactive selection of training samples. Information extraction analysis consists of six component units: thermal infrared image target segmentation, thermal infrared target multi-temporal comparison analysis, island and reef vegetation information extraction, hyperspectral sea ice extraction, hyperspectral nearshore working environment tidal flat classification, and hyperspectral working environment classification.

[0117] (3) Target detection and auxiliary recognition

[0118] Target detection and auxiliary identification provides automatic detection of large marine ship targets based on satellite visible light images, and the software has the function of manual confirmation and storage management of detected targets; provides automatic detection of large marine ship targets of important operating units based on satellite and satellite high-resolution visible light images, and the software has the function of manual confirmation and storage management of detected results; provides automatic detection of large marine ship targets based on satellite and satellite scanning mode SAR data, and the software has the function of manual confirmation and storage management of detected targets; provides auxiliary identification of ship target models based on manual measurement of length and width and knowledge base. Target detection and auxiliary identification consists of four types of component units: large marine ship target detection based on visible light, port ship target detection based on visible light, large marine ship target detection based on scanning mode SAR, and port ship target auxiliary identification based on knowledge base.

[0119] Remote sensing data integration component

[0120] The integrated compilation of remote sensing data is based on the results of image interpretation and analysis. According to standard data product specifications, it realizes the production and standardized output of data products through functions such as annotation, decoration, comprehensive display, and output, thereby ultimately realizing the transformation of data materials into data products. The integrated compilation of remote sensing data consists of four parts: multi-source data specification loading, multi-source data composite visualization, data style editing and decoration, and data template production and management.

[0121] (1) Standardized loading of multi-source data

[0122] Multi-source data standard loading supports direct reading of various common image data (GeoTiff, Img, Jpeg, Bmp), and also supports parsing and loading of common text formats such as Word and Txt. Multi-source data standard loading consists of two component units: image data standard loading and text data standard loading.

[0123] (2) Composite visualization of multi-source data

[0124] Composite visualization of multi-source data supports display / hide control of multiple open layers; it can support interactive position adjustment and refresh display of vector elements and texts. Composite visualization of multi-source data consists of two component units: display control and position adjustment.

[0125] (3) Data style editing and finishing

[0126] Data style editing and finishing supports modification and identification of targets according to standard data product styles and specific requirements; it can output and store data products in a unified style; it has the function of storing data products in a unified style. Data style editing and finishing consists of three component units: style editing, standardized output, and standardized storage.

[0127] Data template creation and management

[0128] Data template production and management supports the addition, editing, and production of data templates based on new data style requirements; supports the storage and management of templates in a standard format; and provides the function of quick retrieval of data templates. Data template production and management consists of three component units: data template production, data template storage, and data template management.

[0129] Deployment

[0130] Image comprehensive judgment software is mainly used to process, interpret and compile image data. In terms of deployment, image comprehensive judgment software needs to be deployed on a workstation and deployed on the same machine with business information management software and multi-source data fusion software.

[0131] System Process

[0132] Based on business needs analysis, the remote sensing information system of the operating unit includes two major processes: fixed target observation and offshore ship target observation.

[0133] (1) Fixed target observation process

[0134] The fixed target observation process is aimed at operating units and ports, using the second-generation distribution data package as input to determine the number and type of ship targets in the port.

[0135] The system first decompresses the second-generation distribution data package, obtains visible light, SAR, thermal infrared, and hyperspectral images, creates thumbnails, obtains image information, and initiates the task process based on the target information contained in the image. If it is a nearshore port interpretation based on high-resolution visible light images, the nearshore port ship target automatic extraction process based on the knowledge base is initiated. The user logs in to the task management software, seizes the interpretation task and opens the target image. If there is an automatic detection result, it is automatically loaded to assist the user in interpretation. The user uses the knowledge base, historical data materials and products to realize the interpretation and confirmation of the target, and on this basis, performs data compilation and processing, and finally closes the data storage loop. The whole process is like Figure 1 shown.

[0136] (2) Observation process of marine ship targets

[0137] The marine ship target observation process targets marine ship targets and utilizes satellite visible light wide-band data, ScanSAR data, combined with digital survey data, situation distribution data, text data, and integrates historical databases and knowledge base data to realize the determination of large marine ship targets.

[0138] The system first decompresses the second-generation distribution data packet, obtains satellite wide-band visible light data and ScanSAR data, obtains image information to initiate the operation task process, and simultaneously initiates the rapid detection task of marine targets. The system simultaneously connects satellite digital survey data, situation distribution data, and text data and stores them in the database. The user logs in to the operation task management software, seizes the interpretation task and opens the target image, uses the ship target detection result confirmation tool to confirm the ship target extracted from the image, and then pushes the slice information to the multi-source data fusion software, and combines the digital survey data, situation distribution data, and text data information to comprehensively confirm and interpret the ship target. Finally, the data is compiled and output based on the interpretation data information, and the data is finally stored in the database to realize the task closed loop.

[0139] System external interface

[0140] The external relations of this system mainly include interfaces with the "data distribution system" and the "image and text system".

[0141] (1) Interface with the data distribution system

[0142] This system accesses the image data and situation distribution package data of the data distribution system. The input information is the satellite survey data and situation distribution package data of the satellite series released by the data distribution terminal, which is sent to the directory monitored by the data access related functional module of the business information management software of this system through the network one-way transmission physical isolation device. The business information management software will automatically decompress the data packet, obtain the data information and initiate the operation task.

[0143] (2) Interface with the image text system

[0144] The interface between this system and the image text system is the input interface. The input information is the target image data and text data generated by the image text system, which are sent to the remote sensing image target information database subsystem of this system through the local area network and included in the integrated database for unified management. The data to be released is sent to the data product standardization processing station of this system through the internal local area network for standardized processing before the situation data is released.

[0145] System internal interface

[0146] The internal interfaces of the remote sensing information processing system of the operating unit are mainly the interfaces between the multi-source data fusion software, data organization and management software, business information management software and image comprehensive judgment software. Figure 2 shown.

[0147] The internal interface of the remote sensing information processing system of the operating unit adopts TCP / IP message mechanism or files to transmit instructions, messages and data.

[0148] A method using a remote sensing information processing system, comprising:

[0149] Step 1: Platform data connection

[0150] Based on the data information sharing service system, the service terminal receives the following high-altitude main operating environment satellite resource data.

[0151] Step 2: Integrated processing of remote sensing images

[0152] Remote sensing image integrated processing provides basic data processing functions for hyperspectral, thermal infrared, visible light and SAR image data. Specifically including:

[0153] (1) Basic processing functions of hyperspectral images: including band selection, spectrum picking, spectrum envelope removal, and spectrum inter-spectrum calculation;

[0154] (2) Basic processing functions of thermal infrared images: including infrared feature measurement and infrared data pseudo-color display;

[0155] (3) Basic processing functions for visible light and SAR images: including brightness and contrast adjustment, color scale adjustment, color balance, curve adjustment, histogram equalization, image sharpening, Lee filtering, strong scattering center extraction and image geometry correction.

[0156] Step 3: Integrated interpretation of image data

[0157] The integrated interpretation of image data realizes basic browsing, measurement and annotation functions; realizes the thematic information extraction function of hyperspectral and thermal infrared data; realizes the ship target detection and auxiliary recognition function based on visible light and SAR images. Specifically includes:

[0158] (1) Basic interpretation functions: including image browsing (display, roaming, zooming), measurement (length, area, etc.) and annotation (vector, text, etc.);

[0159] (2) Thematic information extraction function: including six functions: thermal infrared image target segmentation, thermal infrared target multi-time comparison analysis, island and reef vegetation information extraction, sea ice information extraction, hyperspectral image tidal flat classification, and hyperspectral image operating environment environmental element classification;

[0160] (3) Ship target detection and auxiliary recognition function: including automatic detection of large ships at sea based on satellite visible light images, automatic detection of large ships of important operating units based on high-resolution visible light images, automatic detection of large ships based on satellite scanning mode SAR data, and ship target model matching and recognition based on manual measurement and knowledge base assistance;

[0161] Step 4: Multi-source data fusion

[0162] Multi-source data fusion provides the functions of associating digital signals with imaging targets, associating imaging data with heterogeneous information, and mining and visualizing historical data. Specifically, it includes:

[0163] (1) Digital and imaged target association function: including three functions: digital information loading and display, imaged target data loading and display and retrieval, and target association verification (association and verification of digital survey data and imaged target data);

[0164] (2) High-altitude and heterogeneous target information association function: including three functions: high-altitude survey data loading and display, heterogeneous information loading and display, and multi-source information comprehensive association visualization;

[0165] (3) Historical data information mining function: including target historical data information retrieval, target historical data information (position, trajectory) visualization, and target data information statistical analysis (occurrence location, occurrence frequency, movement pattern).

[0166] Step 5: Remote sensing data integration

[0167] Remote sensing data integration and compilation is based on the results of image interpretation and analysis, and various data materials are analyzed, processed and integrated. According to standard data product specifications, various annotation, finishing, comprehensive display and other functions are used to produce and standardize data material information. Specifically include:

[0168] (1) Multi-source data standard loading function: including image data standard loading and text data standard loading;

[0169] (2) Multi-source data composite visualization function: including display control and position adjustment functions;

[0170] (3) Data style editing and finishing functions: including style editing, standardized output and standardized storage;

[0171] (4) Data template creation and management functions: including data template creation, data template storage, and data template retrieval.

[0172] Step 6: Integrated management of business information

[0173] The integrated management of business information mainly realizes the information management of data production business, and realizes the functions of initiating and monitoring operation tasks, data information statistics, and user management. Specifically, it includes:

[0174] (1) Task management function: including task initiation, task status management and prompts;

[0175] (2) Data statistics function: including data acquisition statistics, data distribution statistics, data product statistics, and workload statistics;

[0176] (3) User management function: including three functions: user information management, user login and authorization management, and user log information management.

[0177] Step 7: Data organization and management

[0178] Data organization and management realizes the management of visible light, SAR, infrared, hyperspectral data and digital and technical survey text data connected to the system, and realizes the integrated organization and management of multi-source massive remote sensing data materials and data products. Specifically including:

[0179] (1) Basic data management functions: including digital signal data storage, hyperspectral data product storage, infrared data product storage, basic data product management, data product retrieval and access, aerial high-altitude image data product storage management, and basic raster vector data storage management. Among them, data product retrieval and access is an improvement of the existing functions, while aerial high-altitude image data product storage management and basic raster vector data storage management are the inheritance of the existing functions.

[0180] (2) Target characteristic data management function: including management and access of hyperspectral image spectral characteristic library and infrared characteristic library;

[0181] (3) Data material and data product management functions: including text material management and access, image material management and access, and data product management and access;

[0182] (4) System management and maintenance functions: including user permission log management and database backup and monitoring functions.

[0183] Step 8: Comprehensive display application

[0184] The comprehensive display application completes the integrated display of multi-source high-altitude remote sensing image data and digital data, and realizes the integrated three-dimensional display of image data, digital survey data, multimedia data, etc. At the same time, we have added a simulation module for the on-orbit operation status of our satellites to simulate and visualize the on-orbit satellite resources. Specifically, it includes:

[0185] (1) Comprehensive data display function: including image data display, digital survey data display, ship track display, text data material display, image data material display, and multimedia data material display;

[0186] (2) Satellite on-orbit operation status simulation function: including satellite resource retrieval and browsing, satellite orbit simulation and display, and satellite survey and transit analysis.

[0187] The integrated interpretation of image data in step 3 further includes: refined extraction of hyperspectral ground feature elements based on mixed pixel decomposition and spectral purification.

[0188] The present invention constructs access to satellite hyperspectral payload image data, and uses hyperspectral image data to extract ground features in a refined manner, which is an important application of satellite hyperspectral data. However, due to the low spatial resolution and high spectral resolution of hyperspectral data, the phenomenon of "mixed pixels" is more significant, which directly affects the applications of battlefield feature refinement and target material determination. At the same time, there are many bands, high dimensions, and large correlation information between bands, which also affects the amount of calculation and accuracy of processing to a certain extent.

[0189] (1) Rapid endmember extraction based on FGDA algorithm

[0190] To address the problem of mixed pixels, this system constructs an endmember extraction method based on the FastGram Determinantbased Algorithm (FGDA) to quickly extract endmembers, and then uses a linear mixed model to decompose the mixed pixels into components. Finally, in the abundance composition obtained by the decomposition, the main abundance is selected and combined with the matched filter (MF) method to perform fine extraction and analysis of battlefield elements.

[0191] The key to the processing is the rapid extraction of endmembers. When performing mixed pixel decomposition, the endmembers in the actual scene are often unknown, and the amount of calculation required to directly search for endmembers is often large. Therefore, a rapid endmember extraction algorithm based on the FGDA method was developed during the implementation of the system.

[0192] First, we introduce a new high-order simplex volume (SV) V(e 1 ,...,e p ) as shown in formula (4-39):

[0193]

[0194] Among them is A p =[e 2 -e 1 ,e 3 -e 1 ,...,e p -e 1 ] end member vector, when looking for the Kth end member, let:

[0195]

[0196] Then for K≥2, the expanded relationship is as shown in formula (4-41):

[0197]

[0198] Comparison found B k+1 With B k There exists a recursive relationship as shown in formula (4-42):

[0199]

[0200] d k =[(e k+1 -e 1 ) T (e 2 -e 1),(e k+1 -e 1 ) T (e 3 -e 1 ),...,(e k+1 -e 1 ) T (e k -e 1 )] T ,

[0201] a k =(e k+1 -e 1 ) T (e k+1 -e 1 )

[0202] Symmetric matrix B k The determinant of is called the Cramer determinant, and according to the block matrix determinant calculation rule, it is easy to get formula (4-43):

[0203]

[0204] Assuming that the Kth end member has been obtained, the K+1th end member can be obtained by finding the maximum det(Bk+1). It is a constant for all pixels, so formula (4-44) is used to calculate and determine which pixel is the K+1th endmember. Therefore, the endmember score index (ESI) is defined to define this indicator.

[0205]

[0206] It can be seen that the K+1th endmember can be obtained by calculating ESI alone, which greatly reduces the amount of calculation for endmember search. In addition, the application of data has proved that the endmembers found by the above method are effective. In terms of computational complexity, let L be the number of bands and N be the total number of pixels. The computational complexity of N-FINDR is σp η+1 N, the computational complexity of SGA is The computational complexity of MVHT is The complexity of NVFS is σ(p η+1 +p 3 L-2p 2 L+pL)N, the computational complexity of OBA is 3pLN, where 2.3≤η≤2.9, 3≤σ≤5. For FGDA, extracting a new end member only requires calculating ESI, and in each cycle, It only needs to be calculated once, and the complexity of calculating ESI for each pixel is k 2+L, adding a new end member requires a cycle (k 2 + L)N times, so the computational complexity of FGDA is like Figure 4 As shown in the figure, the computational efficiency of FGDA is generally higher than that of endmember extraction algorithms such as N-FINDR, SGA, and MVHT.

[0207] After obtaining the end members, the linear mixed model is used to decompose the mixed pixels. The formula is shown in formula (4-45):

[0208]

[0209] Where r is the actual observed spectrum vector, x is the mixed spectrum, n is the noise, E = [e 1 ,e 2 ,...,e p ] is the end member vector, c i For each end member component.

[0210] In this way, the abundance of each component can be obtained by decomposing the mixed pixel. Assuming that the first abundance component of a pixel is x, then the target spectrum x 0 Construct MF to achieve matching enhancement of the extracted military targets. The relevant transformation can be described as Equation (4-46)-Equation (4-48):

[0211] y=H T x (4-46)

[0212] min(H T ΣH)stH T x 0 =1 (4-47)

[0213]

[0214] Where y is the matching result and Σ is the covariance matrix of the input image vector.

[0215] (2) Spectral feature extraction based on HOSVD

[0216] Hyperspectral data has high dimensionality, and direct calculation requires a lot of computation. In addition, there is a large correlation between the data. In order to reduce the dimensionality of the data while taking into account the efficiency of calculation, a feature extraction method based on Higher Order Singular Value Decomposition (HOSVD) was constructed.

[0217] Use tensors to describe this process. Assume that the tensor Among them I 1 ,I 2 ,...,I N∈N is the high-dimensional boundary of the tensor, then for the hyperspectral data B={r 1 ,...,r M}, then its m-order tensor can be described as formula (4-49):

[0218]

[0219] Where M is the number of pixels, r i is the image spectrum vector, then R can be regarded as the outer product of the spectrum, that is Similar to PCA operation, the data is first centralized, that is:

[0220]

[0221] In formula (4-50), is the spectral mean vector, F is the whitening matrix, so for the tensor A , define its HOSVD operation as:

[0222]

[0223] In formula (4-51) S For an I 1 ×I 2 ×…×I N The core tensor of rank, is a singular value matrix, and similar to SVD, S The subtensors of are also orthogonal, that is, Assumption A (n) (S (n) )for A ( S ) is the expanded form, then A (n) The SVD decomposition of can be expressed as:

[0224]

[0225] In formula (4-52), let U (1) =U (2) =…=U (N) =U, then A (n) Expressed as:

[0226]

[0227] And there is || S 1 ||≥|| S 2 ||≥…≥|| S L|| is the root of the eigenvalue of matrix U. So far, we have realized the HOSVD operation on the original data and the dimensionality reduction operation on the data. Figure 5 As shown, the figure gives a schematic diagram of the first 10 bands obtained by HOSVD.

[0228] In this way, by optimizing the principal components, the dimensionality reduction of the original image is achieved. Experiments have shown that for hyperspectral data, selecting the first 10 principal components can represent the main information of the original scene image. This can be seen from Figure 5 and Figure 6 Verified in.

[0229] By adopting the above algorithm, the decomposition and feature extraction of mixed pixels of hyperspectral data are realized. Further using classifiers such as support vector machines and random forests, the extraction of ground features can be realized. To prove the effectiveness of the algorithm, we selected 20 standard scenes of hyperspectral images containing typical scenes for testing, and used manual interaction to evaluate the performance of the extraction results. The experiment showed that the system's refined extraction accuracy of battlefield environmental elements is higher than 90%, which is significantly better than the extraction results of commercial software such as ENVI.

[0230] The step 3 of integrated interpretation of image data further includes: determining the working state of the operation target based on target thermal radiation modeling and multi-temporal analysis.

[0231] Extracting the target's thermal radiation information from thermal infrared images will help determine and analyze the state of the operating target. Infrared remote sensing reflects the target's thermal radiation in a quantitative form, so the target's infrared feature model must also be quantitative and must reflect the target's infrared radiation, rather than just using the features that reflect the target's geometric shape commonly used in visible light imaging. In order to reflect the role of specific heating components in the target on target interpretation, the target feature model needs to include features such as the location of the target's hot or cold components and their mutual relationship. In addition, the satellite's infrared imaging resolution is low, and the target shows a small number of pixels in the image, making it difficult to extract statistical features (such as texture, etc.) that require a certain number of pixels to be more accurate.

[0232] (1) Classification of imaging conditions

[0233] Since the infrared radiation of the target under different imaging conditions shows great differences, it is necessary to establish feature models under multiple imaging conditions. The target radiation characteristics under each imaging condition are relatively stable, which is conducive to establishing a stable feature model. The imaging conditions that need to be considered in establishing the feature model include imaging time, longitude and latitude, spatial resolution, temperature resolution, target orientation, target state, imaging band, etc. At present, the system only considers time (month and time), spatial resolution, temperature resolution and imaging band, and other conditions are not distinguished. Since the spatial resolution, temperature resolution and imaging band are given for the same sensor, the time division is mainly considered. The month is divided into 12 intervals, and each natural month is a monthly interval; the time is divided into 24 intervals, and the model of the hour is established. The scope of each moment model includes the first half hour and the second half hour of the moment. In order to enhance the scalability of the model, the following two treatments are performed: ① Although the current sensor spatial resolution, temperature resolution, and imaging band are fixed, there is still room for expansion, so that after obtaining sensor data with other spatial resolutions, temperature resolutions, and imaging bands, it can still be accurately modeled; ② Although other imaging conditions are not distinguished, these imaging conditions, if they exist, are also added to the database to provide scalability for providing more detailed models.

[0234] (2) Target infrared signature calculation

[0235] Compared with visible light images, infrared images, especially thermal infrared images, mainly reflect the thermal radiation characteristics of the target, so feature extraction needs to take into account the characteristics of infrared images. The main features used in this project include infrared radiation mean, standard deviation, infrared radiation mean difference between the target and the surrounding background, mean contrast and other statistical quantities to analyze the overall infrared radiation characteristics of the target. The calculation formula is as follows:

[0236] The target infrared radiation mean is as follows:

[0237]

[0238] Where (x, y) is the target area R O The pixels within, N O is the number of pixels in the target area;

[0239] The standard deviation of target infrared radiation is as follows:

[0240]

[0241] The mean difference of infrared radiation between the target and the surrounding background is as follows:

[0242] μ=μ O -μ B(4-56)

[0243] in R B is the background area, N B is the number of pixels in the background area.

[0244] The infrared mean contrast between the target and the surrounding background is expressed as (4-57):

[0245]

[0246] in

[0247] As for the infrared radiation characteristics inside the target, we analyze them from the aspects of absolute position and relative position relationship of special points such as the strongest point, second strongest point, weakest point, second weakest point, etc., so as to avoid statistical characteristics that require a certain number of statistical sample points to obtain.

[0248] (3) Feature model generation and update

[0249] The characteristic model of the target is established under different conditions, and the infrared characteristic model of the target under various conditions is generated. After the features of all samples under specific imaging conditions are extracted, statistical analysis is performed on them, and the mean and standard deviation are calculated. The mean and standard deviation are used to represent the characteristic distribution under this condition. The mean and standard deviation are used as the characteristic model under this imaging condition. Since the characteristic model can be used for target detection tasks, the parameters in the method of constant detection rate are given.

[0250] If there is no data in the feature model under the imaging condition corresponding to the feature to be stored, the feature can be added directly. If the feature model under the imaging condition has been established, the feature model needs to be updated. Assuming that there are N samples, their mean and variance are μ and σ respectively, and the feature to be stored is F, the updated mean and variance are respectively as equation (4-58) and equation (4-59):

[0251]

[0252]

[0253] This system has built a target-based thermal radiation modeling and multi-time comparison analysis plug-in, realizing the thermal radiation modeling tool for targets such as ships, oil tanks, and fixed target facilities. Figure 7 As shown, the modeling and display of the thermal radiation of a certain target changing with time is realized.

[0254] The step 3 of integrated interpretation of image data further includes: mining features of massive SAR data based on a sparse deep belief network.

[0255] The present invention accesses SAR payload data, and it is of great significance to interpret and obtain ship target information from SAR image data.

[0256] In view of the characteristics of massive SAR data, this project uses restricted Boltzmann machines to perform layer-by-layer unsupervised feature learning on a large amount of training data. The learning process uses the contrast divergence algorithm based on Gibbs sampling to solve the parameters. Then the parameters solved by the multi-layer restricted Boltzmann machine are used as the initial values ​​of the multi-layer neural network. The model parameters of the multi-layer neural network are then fine-tuned using the labeled training data and the multinomial logistic regression model is used as the final classification model. The fine-tuning process uses the backpropagation algorithm and the conjugate gradient descent method. Finally, the new data is forward propagated through the network to obtain the final feature representation result.

[0257] (1) Feature learning based on hybrid structure sparse constraints

[0258] In high-resolution SAR image recognition, an image contains not only the target of interest to the reader, but also a large amount of complex background. These interference information will affect the performance of the recognition method. A common method is to suppress useless or noise information in the features through function constraints or regularization to improve the generalization performance of the system. In this regard, the present invention proposes an unsupervised training method of a restricted Boltzmann machine based on a hybrid structure sparse constraint. Specifically, the defined optimization loss function is shown in formula (4-60):

[0259] L=-logP(v)+μ||P(h=1|v)|| 1 +λ||P(h=1|v)|| 1,2 +η||W|| 2 (4-60)

[0260] Among them, μ represents the hidden layer node activation probability, L 1 Norm-constrained partition coefficient, λ represents the hidden layer node activation probability L 1,2 The partition coefficient of the mixed norm constraint, η represents the L of the weight matrix W 2 Partition coefficient for norm constraint.

[0261] The second term on the right side of formula (4-60) is based on L 1 The hidden layer node sparse constraint of the norm is defined as follows:

[0262]

[0263] According to sparse coding theory, the optimization problem of adding norm constraints is equivalent to adding L 0The optimization problem with norm constraints can make most of the optimized target parameters 0, achieving the purpose of parameter sparseness. Here, adding sparse constraints can make the activation probability of most hidden layer nodes 0, thereby achieving the purpose of encoding the input data features with a small number of hidden layer nodes.

[0264] The third term on the right side of formula (4-60) is based on L 1,2 Sparse constraint of the mixed norm. Divide the hidden layer nodes of the restricted Boltzmann machine into m sets, where the L of the hidden nodes is defined 1,2 The mixed norm is as shown in formula (4-62):

[0265]

[0266] Among them G m Represents the mth hidden layer node set. By optimizing and adding L 1,2 The objective function of the mixed norm is that the sum of squares of the node activation values ​​of most hidden layer node sets is close to zero, which makes the activation values ​​of most nodes in the set also close to zero, thus achieving the same result as L 1 Therefore, when data is input to the visible layer nodes, only a small number of hidden layer nodes in the set are activated, thus achieving the purpose of sparse constraint. 1 Norm and L 1,2 Mixed norm constraints can give different types of constraints to the activation probabilities of hidden layer nodes.

[0267] The fourth term on the right side of formula (4-60) is the L of the restricted Boltzmann machine network weight matrix 2 Norm constraint, its specific form is shown in formula (4-63):

[0268]

[0269] Adding constraints on the weight matrix during optimization can make the model smoother and reduce the possibility of overfitting by suppressing excessive weights. At the same time, this constraint can suppress redundant weights in the weight matrix and enhance the generalization performance of the model.

[0270] Optimizing energy models usually relies on stochastic gradient descent, which requires the use of algorithms such as Gibbs sampling to estimate the parameters of the model and thus calculate the gradient. However, repeatedly performing Gibbs sampling on undirected graph models such as Boltzmann machines is not only computationally intensive but also very time-consuming. The present invention uses the contrast divergence algorithm to perform approximate calculations. This approximate calculation can obtain similar optimization results and greatly reduces the amount of calculation. Therefore, this method is widely used in the actual gradient solution process.

[0271] (2) Model parameter fine-tuning

[0272] By training multiple restricted Boltzmann machines layer by layer, we can obtain high-level abstract features, that is, achieve the purpose of unsupervised feature learning. The parameters of these restricted Boltzmann machines will be used as the initial values ​​of the multi-layer neural network parameters. Then we use the back propagation algorithm and conjugate gradient descent method to fine-tune the parameters of the entire network. This process is a supervised training process and will be trained with labeled data.

[0273] In the optimization problem of high-dimensional feature space, in order to avoid the possible local optimum, the model has more optimization paths to choose from, which makes it easier to find the global optimal solution for the optimization problem in high-dimensional space than in low-dimensional space. In other words, the more hidden layer nodes there are in the network, the lower the possibility of the optimization process falling into the local extreme value, and the more parameters there are, the stronger the feature expression ability of the model will be. However, the increase in model parameters will bring another more serious problem, that is, the possibility of overfitting is greatly increased, which greatly reduces the generalization performance of the model. In the relevant theories of machine learning, the most commonly used method to avoid overfitting is regularization, that is, adding regularization terms in the optimization objective function to suppress overtraining of parameters. In the present invention, we will use sparsity constraints to improve the back propagation algorithm used for multi-layer neural network optimization to improve the generalization performance of the model.

[0274] When the activation probability of a hidden layer node is close to 1, the node is considered to be activated, and when it is close to 0, the node is suppressed. When the average activation probability of a node in the hidden layer for all input data is small, it can be considered that the node is not activated most of the time, and the hidden layer node can be considered to be in a sparse state. The average activation value of the jth hidden node in the hidden layer is defined as formula (4-64):

[0275]

[0276] where a j (x (i) ) represents the activation probability of the jth hidden layer node when the i-th data is input to the visual layer, and M represents the total number of input data. When the value of is a small value, it can be considered that the hidden layer nodes have reached a sparse state. Close to a smaller value, the following sparsity constraint function is defined here as shown in formula (4-65):

[0277]

[0278] Add this constraint function to the loss function of the back propagation algorithm, and define it as formula (4-66):

[0279]

[0280] Where β represents the partition coefficient of the sparse constraint term, and J(W,b;x,y) represents the loss function of the back propagation algorithm, which is in the form of formula (4-67):

[0281]

[0282] where h w,b (x) is the predicted value of the input data x through the multi-layer neural network, and y is the manually labeled value of the data. The loss function is also optimized using the stochastic gradient descent method, and the specific method is shown in equations (4-68) and (4-69):

[0283]

[0284] The partial derivative calculation method of the model is as shown in equations (4-70) and (4-71):

[0285]

[0286] in represents the back-propagated residual of the i-th node in the k+1-th hidden layer of the neural network. In this model, the residual function can be derived:

[0287]

[0288] In formula (4-72), f'(x i ) represents the derivative of the activation function. The ρ value is set to a value close to 0. Through the above iterative optimization, a multi-layer neural network model with sparse hidden layer nodes is finally output.

[0289] (3) Unsupervised feature extraction

[0290] In the model training stage, a batch training method is used to randomly and evenly divide the training images into multiple data subsets. Each image belongs to only one of the subsets. The unsupervised pre-training method introduced above is used to train four restricted Boltzmann machines layer by layer, where the output of the hidden layer of the previous Boltzmann machine is used as the input of the visible layer of the next Boltzmann machine. After the pre-training is completed, the weights and other parameters of the four restricted Boltzmann machines are used to initialize a 5-layer neural network. The parameter initialization settings are as follows: Figure 8 and Fig. 9 shown.

[0291] After the parameters are initialized, a supervised layer is added to the top layer of the neural network. The parameters of this layer are initialized by a Gaussian distribution random number matrix with a mean of 0 and a standard deviation of 0.01. Then we use the supervised fine-tuning algorithm introduced earlier and the labeled data to perform supervised training of the neural network. The classification model of the supervised layer is defined as follows:

[0292]

[0293] After the network's unsupervised pre-training and supervised fine-tuning training are completed, a final model is obtained. When new data is obtained, the new data only needs to be simply processed and input into the network's visual layer, and the neural network is forward propagated layer by layer to finally learn the feature vector.

[0294] The multi-source data fusion in step 4 further includes: ship target trajectory fusion based on multi-source information.

[0295] The present invention introduces payload slicing and digital point information. In the satellite battlefield observation and monitoring system, digital observation satellites can currently monitor strategic targets in most parts of the world (within the range of plus or minus 60° latitude), and the number of possible targets is estimated to reach 10,000. When the target capacity is not too large and the processing speed requirement is not high, the digital target memory management is usually managed based on the digital target object data linked list. When the target is retrieved, the target linked list needs to be traversed sequentially, and the time complexity is O(n). If the traditional method is used for processing, the digital target data information cannot be processed in real time. If the data is backlogged, it will bring great risks to the stable operation of the system.

[0296] Therefore, a fast correlation retrieval method must be used to achieve fast correlation retrieval of large-capacity digital targets. This method can be used for fast target retrieval in the target correlation fusion process, and can also be used for fast filtering of important targets to further improve processing speed.

[0297] In addition, there are inconsistencies in positioning accuracy and time resolution between satellites, so effective measures need to be taken to achieve correlation and fusion analysis of different points and trajectories.

[0298] (1) Fast association based on index

[0299] With the increasing amount of radar target data, a better index structure is needed to process radar targets more effectively, so that target retrieval can be quickly realized based on the location information and characteristic parameter information of radar targets. However, the conventional one-dimensional parameter retrieval method cannot solve this problem well.

[0300] In terms of multidimensional spatial data retrieval methods, the R*-tree index theory is internationally well-known. The starting point of this theoretical research is to achieve fast retrieval of multidimensional spatial data. After more than 20 years of development, it has basically matured and has been put into practical use by well-known commercial database manufacturers.

[0301] Based on the R*-tree index theory, we proposed a new method for retrieving radar targets. We mapped the attribute features of radar targets to a point in multidimensional space, and then constructed an index - RR-Tree based on MBR. When retrieving data, we pruned the RR-tree according to the effective filtering method of the index, which can quickly approximate the query results. The RR-tree can update the index while completing the retrieval process. Experiments have shown that R*-tree is a very effective index structure. The basic definition of the index is as follows:

[0302] ①. Range query: Given a certain error range ε, a query point q is given so that the error between the query result r and q is less than ε;

[0303] ②.KNN query: given a query point q, query the N points closest to q;

[0304] ③. Radar target vector: The position information and technical characteristic parameter data of the radar target are formed into a multidimensional array, which is called the radar target vector;

[0305] ④. Radar target vector space: It is described by forming a d-dimensional space with d items of radar target vector;

[0306] ⑤. Radar target space distance: Define the distance between two points p1(m0,m1…md-1) and p2(n0,n1…nd-1) in radar target vector space as:

[0307]

[0308] ⑥.MBR: The minimum bounding rectangle containing n radar vector points.

[0309] In R*-tree, node definitions include leaf nodes and non-leaf nodes. For an RR-tree with N index nodes, if the target number of each node is at most m, the height of the tree is at most logmN–1. Leaf nodes are represented by (I, tuple-identifier), where tuple-identifier corresponds to a tuple in the target database, and I represents a rectangle or point in n-dimensional space, that is, I = (I0, I1, …, In-1), where n is the dimension of the space, Ii represents a closed interval in the i-dimensional space, and represents the expansion of a spatial data in the i-dimensional space; non-leaf nodes are represented by (I, child-pointer), where child-pointer is the pointer to the child node of the node in the index tree, and I is the large rectangle containing all child nodes.

[0310] To describe query algorithms such as range query, KNN query, and NN query, the following concepts are introduced:

[0311] (a) Expansion queue (ItemQueue): A priority queue formed by sorting the MINDIST size, storing the nodes to be expanded first;

[0312] (b) ResultQueue: It is a linked list of size K, storing the best k nodes so far. The result queue is formed by sorting according to MINMAXDIST;

[0313] (c) K distance (kthdist): MINMAXDIST corresponding to the first k items in the result queue so far, or infinity if there are fewer than k items. It is an upper bound on the K nearest neighbors.

[0314] The query algorithm can be described as:

[0315] ①. Use MINDIST to control the extended nodes;

[0316] ②. Use the nodes of ResultQueue to reduce MBR and nodes;

[0317] ③. When ResultQueue is the target node, the search ends.

[0318] (2) Target Information Fusion

[0319] The purpose of digital target information fusion is: first, to improve the accuracy of data processing, that is, to improve the accuracy of parameters or positions through fusion; second, to improve the credibility of data processing, by continuously tracking the target or accumulating parameters to improve the credibility of the target; third, to reduce the increase in target batches during data processing, by integrating the correlation characteristics of the detected targets through fusion technology, merging the same target information, and reducing the increase in target batches.

[0320] The first is position fusion. In position fusion, Fig.10 The positions of the two radars and their corresponding error ellipses are shown. The fused radar position is given by equation (4-75):

[0321]

[0322] The error ellipse of the corresponding fusion result can be calculated by the following method:

[0323] Using two position error parameters (major axis, minor axis, and major axis direction), the variance matrices of the two position measurements can be given respectively, as shown in equations (4-76) and (4-77):

[0324]

[0325] The variance matrix after fusion is shown in formula (4-78):

[0326]

[0327] Then the major axis of the corresponding error ellipse is as shown in formula (4-79):

[0328]

[0329] The short axis is shown in formula (4-80):

[0330]

[0331] The major axis direction is shown in formula (4-81):

[0332]

[0333] Then comes parameter fusion. Parameter fusion performs a similar one-dimensional variance weighted fusion. For example, for RF parameters, the RF features are first fused to merge the RF features of the two. For related RF parameters, the difference between the highest frequency and the lowest frequency is used as the variance, and the frequency center is used as the measurement mean to perform one-dimensional variance weighted fusion to obtain the fused frequency center and variance, and thus the fused highest frequency and lowest frequency. The fusion of other parameters is similar.

[0334] Targeted processing is carried out for the problem of weak satellite observation signals and large errors in some parameter strategies. For example, for small pulse width processing near the sensitivity, the pulse width grouping and large strategy is adopted, which can effectively improve the accuracy of the pulse width.

[0335] (3) Curvature-based segmented dynamic threshold trajectory compression

[0336] Aiming at the problems of low efficiency and information loss of traditional compression methods in cleaning massive high-noise data, a data compression method based on curvature segmented dynamic threshold algorithm is proposed. Fig.11 As shown in the figure, this method can avoid the crack phenomenon caused by noise and the significant loss of useful information. The compression ratio of this method can be as high as 83.3%, which is far higher than similar methods.

[0337] (4) Trajectory similarity calculation based on target prior information

[0338] Finally, an efficient and widely applicable trajectory similarity calculation algorithm is proposed. This method matches the similarity of position and trajectory based on the target prior information and calculates the confidence probability of each data source. Based on this strategy and method, the influence of noise can be significantly reduced and the solution efficiency can be optimized.

[0339] The step 5 of integrated remote sensing data compilation further includes: software plug-in architecture and integrated interpretation of multi-source payload data of operating units.

[0340] The remote sensing information processing system of the operating unit has the characteristics of multiple types of access loads, integrated and specialized data processing, etc. In addition, it faces difficulties such as large data volume, strong timeliness, and high requirements for comprehensive data processing capabilities during data processing. Therefore, it is necessary to establish an integrated and efficient data production line to realize the comprehensive application and analysis of different sensor data and give full play to the comprehensive interpretation benefits of multi-load observation data.

[0341] Establishing a unified data interpretation process can support multi-person and multi-task interpretation, realize batch production of data, and improve comprehensive interpretation efficiency.

[0342] In addition, the subsystem software architecture is required to be able to support functional expansion and updating well, have strong flexible expansion capabilities, and realize integrated access and analysis and processing of multi-source payload data, so that the system has stronger multi-source data comprehensive analysis capabilities and improves image interpretation and compilation efficiency.

[0343] (1) Integrated data production process framework

[0344] For multiple types of payloads such as visible light, SAR, thermal infrared, hyperspectral, and digital, it provides integrated interpretation process management and integrated comprehensive interpretation compilation.

[0345] The design implements a unified data production process framework, unifies data standards, operation modes and data output, standardizes processing procedures, and optimizes work modes and processing links. For time-consuming processing that can be undertaken by computers, automation is used to complete it; for links that must be manually participated, a variety of semi-automatic auxiliary tools are provided, with process-based processing as the main line to supplement the interpretation link, which significantly improves data processing efficiency.

[0346] The processing logic steps of the process framework are as follows:

[0347] ①.Task initiation: automatically access the second-generation distribution data, decompress the image compression package, and obtain the image file;

[0348] ②. Image information extraction: read the image and obtain relevant information of the image, including the image payload, imaging time, target location, etc., store the information in the database, and create a target automatic extraction interface file;

[0349] ③. Automatic target detection and recognition: automatically extract targets from images and write slices or detection result files into task files;

[0350] ④. Target manual identification, confirmation and association: The user displays the image and loads the result file of automatic detection to assist in interpretation, while using the target knowledge base to interpret and extract target information;

[0351] ⑤. Data compilation, output and storage: Based on the data information obtained, the data is compiled and output to form data products, the products are stored in the warehouse, and the next interpretation task is carried out.

[0352] (2) “Microkernel + plug-in” software system architecture

[0353] The system needs to access, process and analyze multiple payload data such as visible light, SAR, hyperspectral, thermal infrared and digital data. To ensure the integrated processing and analysis of multi-source data, the system proposes a "microkernel + application plug-in" platform design in the software architecture design, such as Fig.12 shown.

[0354] This is a flexible system design structure that realizes the sharing of information, software, computing, storage and other resources, and can provide information, software, computing and storage resources on demand, supporting the on-demand distribution of information. It standardizes the data transmission format and information code standards, provides a universal interface, and has unified information exchange and computing capabilities. It adopts virtualization design ideas and flexible and scalable software architecture design to solve integration and expansion problems.

[0355] The architectural features of this "microkernel + application plug-in" are:

[0356] Microkernel: As the core layer of the software, it implements the most basic data reading, algorithm scheduling, function hooking, etc.

[0357] Application plug-ins: It realizes functions such as dedicated interpretation and analysis, geospatial data presentation, target query retrieval and proprietary data compilation and output. The software modules can be flexibly split and customized according to usage needs, achieving scalability in data types, functional tools and equipment scale.

[0358] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A remote sensing information integrated processing system, comprising: (1) Multi-source data fusion software, which provides support for integrated comprehensive display of multi-source high-altitude remote sensing image data and digital data for image data, digital survey data and multimedia data, and is composed of data information fusion and comprehensive display application components; (2) Data organization and management software, which manages the visible light, SAR, infrared, and hyperspectral data connected to the system as well as digital and technical survey text data, and realizes the integrated organization and management of multi-source massive remote sensing data materials and data products. It consists of four components: basic data management, target characteristic data management, data material and data product management, and system management and maintenance; (3) Business information management software, which realizes the information management of data production business, completes the statistical analysis of data, the monitoring and statistics of operation tasks, and the generation of statistical reports. It consists of three parts: task management component, data statistics component, and user management component; (4) Image comprehensive judgment software, which processes hyperspectral, thermal infrared, visible light and SAR image data. It consists of three components: remote sensing image integrated processing, image data integrated interpretation, and remote sensing data integrated compilation; System external interface: the external relationship of the remote sensing information integrated processing system includes interfaces with the data distribution system and the image and text system; System internal interface, the remote sensing information integrated processing system internal interface is the interface between multi-source data fusion software, data organization management software, business information management software and image comprehensive judgment software; According to the business demand analysis, the remote sensing information integrated processing system includes two major processes: fixed target observation and offshore ship target observation.

2. A remote sensing information integrated processing system according to claim 1, characterized in that: The data information fusion component consists of three parts: digital and imaging target association, high-altitude and heterogeneous target information association, and historical data information mining; the comprehensive display application component consists of two parts: data comprehensive display and satellite on-orbit operation status simulation.

3. A remote sensing information integrated processing system according to claim 1, characterized in that: The basic data management component includes six component units: digital signal data entry unit, hyperspectral product data entry unit, infrared data product entry unit, basic data product management unit, target knowledge base management, and data product retrieval and access unit; the target characteristic data management component consists of two component units: hyperspectral spectral characteristic library management and access, and infrared characteristic library management and access; the data material and data product management component consists of three component units: text material management and access, image material management and access, and data product management and access; the system management and maintenance component consists of two component units: user authority log management, and database backup and monitoring.

4. A remote sensing information integrated processing system according to claim 1, characterized in that: The task management component consists of two functional units: job task initiation and job task status management and prompts; the data statistics component consists of four functional units: data acquisition statistics, data distribution statistics, data product statistics, and workload statistics; the user management component consists of three functional units: user information management, user login and authorization, and user log information management.

5. The remote sensing information integrated processing system according to claim 1, characterized in that: The remote sensing image integrated processing component consists of three parts: general processing of hyperspectral images, general processing of infrared images, and general processing of visible light and SAR; the image data integrated interpretation component consists of three parts: basic interpretation function, information extraction and analysis, target detection and auxiliary recognition; the remote sensing data integrated compilation component consists of four parts: multi-source data specification loading, multi-source data composite visualization, data style editing and decoration, and data template production and management.

6. A remote sensing information integrated processing system according to claim 1, characterized in that: The fixed target observation process is aimed at operating units and ports, and uses distributed data packets as input to determine the number and type of ship targets in the port; the marine ship target observation process is aimed at marine ship targets, using satellite visible light wide-band data and ScanSAR data, combined with digital survey data, situation distribution data, text data, and integrated historical databases and knowledge base data to determine large marine ship targets.

7. A method using the remote sensing information integrated processing system according to any one of claims 1 to 6, comprising: Step 1: Platform data connection, based on the data information sharing service system service terminal receiving the following high-altitude main operating environment satellite resource data; Step 2: Integrated remote sensing image processing, basic data processing for hyperspectral, thermal infrared, visible light and SAR image data; Step 3: Integrated interpretation of image data, basic browsing, measurement and annotation; extraction of thematic information from hyperspectral and thermal infrared data; ship target detection and auxiliary recognition analysis based on visible light and SAR images; Step 4: Multi-source data fusion, to associate digital signals with imaging targets, to associate imaging data with heterogeneous information, and to mine and visualize historical data; Step 5: Remote sensing data integration and compilation: Based on the results of image interpretation and analysis, various data materials are analyzed, processed and integrated, and data material information is produced and standardized output using various annotation, finishing and comprehensive display functions according to standard data product specifications; Step 6: Integrated management of business information, informatization management of data production business, initiation of operation tasks, monitoring of operation tasks, data information statistics and user management; Step 7: Data organization and management: manage the visible light, SAR, infrared, hyperspectral data, and digital and technical survey text data connected to the system, and integrate the organization and management of multi-source massive remote sensing data materials and data products; Step 8: Comprehensive display application, integrated display of multi-source high-altitude remote sensing image data and digital data, to achieve integrated three-dimensional display of image data, digital survey data, and multimedia data; at the same time, add a satellite on-orbit operation status simulation module to simulate and visualize the on-orbit satellite resources.

8. The method according to claim 7, characterized in that The integrated interpretation of image data in step 3 further includes: refined extraction of hyperspectral ground feature elements based on mixed pixel decomposition and spectral purification.

9. The method according to claim 7, characterized in that: The step 3 of integrated interpretation of image data further includes: determining the working state of the operation target based on target thermal radiation modeling and multi-temporal analysis.

10. The method according to claim 7, characterized in that The step 3 of integrated interpretation of image data further includes: mining features of massive SAR data based on a sparse deep belief network.

11. The method according to claim 7, characterized in that The multi-source data fusion in step 4 further includes: ship target trajectory fusion based on multi-source information.

12. The method according to claim 7, characterized in that The step 5 of integrated remote sensing data compilation further includes: software plug-in architecture and integrated interpretation of multi-source payload data of operating units.

Images