AIS-guided SAR image super-resolution reconstruction method
By combining AIS system and super-resolution reconstruction technology, the problem that traditional SAR imaging technology is difficult to obtain three-dimensional information of marine mobile targets is solved, and high-precision and high-resolution three-dimensional imaging is achieved, which improves imaging effects and detailed richness.
Patent Information
- Application Number
- CN202411593360.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-02-21
AI Technical Summary
Traditional SAR imaging technology is difficult to obtain three-dimensional information of moving targets at sea, and the imaging effect is poor in complex terrain or scenes requiring height information.
By combining AIS system and super-resolution reconstruction technology, AIS data is used to obtain ship dynamic information in real time and fuse it with SAR image data. It is integrated into super-resolution reconstruction algorithm as prior knowledge to reconstruct high-resolution SAR image.
High-precision and high-resolution three-dimensional imaging of marine mobile targets is achieved, which improves the clarity and detail richness of SAR images, and meets the needs of marine monitoring and target recognition.
Smart Images
Figure CN119494778A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a method for super-resolution reconstruction of SAR images guided by AIS. Background Art
[0002] SAR (Synthetic Aperture Radar) has the capability of high-resolution imaging all day and all weather, and has irreplaceable advantages in the field of earth observation. SAR images are projections of three-dimensional targets on the two-dimensional planes of slant range and azimuth, but traditional SAR only has the two-dimensional resolution imaging capability of range and azimuth, and cannot directly reflect the height information of the target.
[0003] Traditional SAR imaging technology cannot directly obtain the three-dimensional information of the target, which is particularly insufficient in dealing with complex terrain or scenes that require height information. When facing moving targets at sea, such as warships, traditional SAR imaging technology may cause imaging tasks to fail or imaging results to be inaccurate due to target movement. At sea, the demand for imaging of moving targets such as ships and warships is increasing. These targets often have characteristics such as fast movement speed and difficult to predict trajectory, and traditional SAR imaging technology is difficult to meet their imaging needs. AIS (Automatic Identification System) can provide real-time information such as the position, speed, and heading of the ship, which is of great significance for planning SAR imaging tasks and improving imaging accuracy. AIS-guided SAR three-dimensional imaging technology can achieve high-precision and high-resolution three-dimensional imaging of moving targets at sea, providing strong support for tasks such as maritime surveillance and target identification.
[0004] Although SAR technology already has a high resolution, in some application scenarios, higher resolution is still needed to meet the needs. For example, in military reconnaissance, it is necessary to accurately identify the specific details of the target; in environmental monitoring, it is necessary to accurately judge the slight changes in the ground objects, etc. Therefore, super-resolution reconstruction technology has important application value in SAR imaging. In the imaging task of moving targets at sea, due to the mobility and complexity of the target, traditional SAR imaging technology often finds it difficult to obtain high-quality three-dimensional images. By combining the AIS system and super-resolution reconstruction technology, high-precision, high-resolution three-dimensional imaging of moving targets at sea can be achieved. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention provides a method for super-resolution reconstruction of SAR images using AIS guidance. By combining the AIS system and the super-resolution reconstruction technology, the invention aims to achieve high-precision and high-resolution three-dimensional imaging of mobile targets at sea. To achieve the above purpose, the present invention provides the following technical solutions:
[0006] On the one hand, a method for super-resolution reconstruction of SAR images using AIS guidance is provided, comprising the following steps:
[0007] A method for super-resolution reconstruction of SAR images guided by AIS, characterized by comprising the following steps:
[0008] S1. Real-time acquisition of AIS data, which is the ship dynamic data sent by the AIS system;
[0009] S2, obtaining original SAR image data of the target sea area through the SAR system;
[0010] S3, using a target detection algorithm to match the AIS data with the ship target in the SAR image to determine the position of the ship in the SAR image;
[0011] S4, fusing the AIS data with the SAR image data;
[0012] S5. Incorporating AIS data as prior knowledge into the super-resolution reconstruction algorithm to reconstruct the SAR image to obtain a high-resolution SAR image.
[0013] Preferably, in the step S1, the ship dynamic data includes: the position, heading and speed of the ship.
[0014] Preferably, after step S1, the method further comprises preprocessing the ship dynamic data, wherein the preprocessing comprises: removing abnormal values or erroneous data in the AIS data, and converting the AIS data into data with a unified coordinate system and time standard.
[0015] Preferably, after step S2, the method further includes geo-locating the image using geographic information and feature points in the SAR image to ensure spatial consistency between the SAR image and the AIS data.
[0016] Preferably, in step S3, the target detection algorithm selects any one of the YOLO algorithm, Faster R-CNN, and SSD.
[0017] Preferably, the step S4 specifically includes: using the ship's speed and heading provided by the AIS data to estimate the movement trajectory and posture of the ship in the SAR image.
[0018] Preferably, in step S5, the super-resolution algorithm selects any one of an interpolation-based super-resolution reconstruction algorithm, a reconstruction-based super-resolution reconstruction algorithm, and a learning-based super-resolution reconstruction algorithm.
[0019] Preferably, step S5 specifically comprises: using the heading and speed of the ship provided by AIS to compensate for the motion trajectory of the ship in the SAR image, thereby eliminating motion blur and defocusing phenomena; and applying a super-resolution reconstruction algorithm to reconstruct the processed SAR image to obtain a high-resolution SAR image.
[0020] Preferably, the learning-based super-resolution reconstruction algorithm includes designing a super-resolution reconstruction network.
[0021] Preferably, the learning-based super-resolution reconstruction algorithm includes a specific loss term designed for the characteristics of AIS data.
[0022] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the super-resolution reconstruction method as described above when executing the computer program.
[0023] In summary, the present invention has the following beneficial effects compared with the prior art:
[0024] The present invention first obtains AIS data in real time, and AIS data is the dynamic data of the ship sent by the AIS system; uses the target detection algorithm to match the AIS data with the ship target in the SAR image to determine the position of the ship in the SAR image; performs data fusion on the AIS data and the SAR image data; and integrates the AIS data as prior knowledge into the super-resolution reconstruction algorithm to reconstruct the SAR image to obtain a high-resolution SAR image. The present invention provides the real-time position information of the target through the AIS system, helping the SAR system to accurately plan the imaging task, and the super-resolution reconstruction technology further improves the resolution of the SAR image, making the image clearer and richer in details. By combining the AIS system and the super-resolution reconstruction technology, high-precision, high-resolution three-dimensional imaging of mobile targets at sea can be achieved, providing strong support for ocean monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 An overall flow chart of a method for super-resolution reconstruction of SAR images using AIS guidance provided in this embodiment;
[0026] Figure 2 A detailed flow chart of a learning-based SAR image super-resolution reconstruction algorithm provided in this embodiment. DETAILED DESCRIPTION
[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0028] Example 1
[0029] like Figure 1 As shown, this embodiment provides a method for super-resolution reconstruction of SAR images using AIS guidance, including the following steps:
[0030] S1. Real-time acquisition of AIS data, which is the ship dynamic data sent by the AIS system;
[0031] Specifically, the ship dynamic information sent by the AIS system is received in real time, and the ship dynamic information includes the ship's latitude and longitude, heading, speed, etc. The AIS receiver of the AIS system is a device specially designed to receive and decode AIS signals. The receiver can capture information sent from ships equipped with AIS equipment in nearby waters, including static information (such as ship name, call sign, IMO number, etc.) and dynamic information (such as position, heading, altitude, etc.) of the ship.
[0032] S2, obtaining original SAR image data of the target sea area through the SAR system;
[0033] In this step, the original SAR image data of the target sea area is obtained through the "Haisi-1" satellite. This is because the coverage of the "Haisi-1" satellite is very wide, and its imaging area has covered multiple continents such as Asia, North America, South America, Europe and Antarctica. In addition, the imaging resolution of "Haisi-1" reaches 1 meter, which can capture small targets in the sea area and provide high-precision ocean observation data. The "Haisi-1" uses VV single polarization mode to help reduce the interference of ground clutter and improve the quality of SAR images. Thanks to the excellent performance of the "Haisi-1" satellite in terms of resolution, polarization mode and coverage, the original SAR data of the target sea area obtained through it provides a good foundation for the super-resolution reconstruction of SAR images.
[0034] S3, using a target detection algorithm to match the AIS data with the ship target in the SAR image to determine the position of the ship in the SAR image;
[0035] In this step, the target detection algorithm selects any one of the YOLO algorithm, Faster R-CNN, and SSD. The YOLO target retrieval algorithm is a representative of a single-stage detector, which is very fast and suitable for real-time target detection. Therefore, it is used as an optional target detection algorithm. Faster R-CNN can generate high-quality region proposals for FastR-CNN due to the introduction of the region proposal network (RPN), which greatly improves the detection speed and maintains a high detection accuracy. Therefore, it is used as an optional target detection algorithm in this step. SSD (Single Shot MultiBox Detector) combines the speed of YOLO and the accuracy of Faster R-CNN. It directly predicts the bounding box and category probability by using filters of different sizes on feature maps of different resolutions, thereby improving the speed and effect of target detection. The above three target detection algorithms are all optional algorithms in this implementation. Preferably, because the SSD target detection algorithm combines the advantages of the YOLO algorithm and Faster R-CNN, the SSD algorithm is used as the preferred target detection algorithm in this step.
[0036] The following uses the SSD target detection algorithm as an example to introduce the process of matching AIS data with ship targets in SAR images. This process can be carried out through the following steps:
[0037] S31. Convert the longitude and latitude information in the AIS data into the same coordinate system as the SAR image (such as UTM coordinates). This requires knowing the geographic positioning information when the SAR image is imaged, including the longitude and latitude of the imaging center point, image resolution, etc.
[0038] S32, traverse all detected ship targets, calculate the distance between each target and the ship position in the AIS data, and use Euclidean distance or Manhattan distance as a distance metric, set a threshold, and when the distance is less than the threshold, it is considered that the AIS data matches the ship target in the SAR image successfully.
[0039] S33. The accuracy of the matching results can be verified by comparing the characteristics of the ship targets in the AIS data and the SAR images (such as size, shape, heading, etc.), or by cross-validation using other data sources (such as optical satellite images).
[0040] S4, data fusion of AIS data and SAR image data; in this step, the fusion of AIS data and SAR image data mainly uses the ship's speed and heading provided by the AIS data to estimate the ship's motion trajectory and attitude in the SAR image, wherein the estimation of the motion trajectory includes using the speed and heading information in the AIS data, combined with the timestamp, to calculate the position of the ship at different time points, and forming the ship's motion trajectory by connecting continuous position points. Then the calculated ship motion trajectory is mapped to the SAR image to form a visual trajectory line. In the SAR image, the attitude of the ship can be inferred by the relative position of its heading and the outline of the ship in the image. Therefore, the attitude estimation can use image processing technology (such as edge detection, contour extraction, etc.) to extract the contour features of the ship from the SAR image, analyze the direction and shape of the ship's outline, to obtain clues about the ship's attitude, and combine the heading information provided by the AIS and the characteristics of the ship's outline in the SAR image to infer the attitude of the ship in the SAR image.
[0041] Since both AIS data and SAR image data may have errors, such as positioning error, measurement error, imaging error, etc., these errors will affect the accuracy of motion trajectory and attitude estimation. Therefore, it is necessary to correct and compensate the errors of AIS data and SAR image data. Multi-source data fusion technology can be used to combine other data sources (such as optical satellite images, radar altimeter data, etc.) to improve the accuracy of estimation.
[0042] S5, integrating the AIS data as prior knowledge into the super-resolution reconstruction algorithm to reconstruct the SAR image to obtain a high-resolution SAR image;
[0043] Specifically, based on the heading and speed information in the AIS data, a linear prediction model or a machine learning model can be used to predict the motion trajectory of the ship within the SAR image time window, and the ship position in the SAR image is adjusted according to the predicted trajectory to compensate for the image displacement or blur caused by the ship's motion, thereby eliminating motion blur and defocus. The processed SAR image is reconstructed using a super-resolution reconstruction algorithm to obtain a high-resolution SAR image.
[0044] In step S5, the super-resolution reconstruction algorithm is one of an interpolation-based super-resolution reconstruction algorithm, a reconstruction-based super-resolution reconstruction algorithm, and a learning-based super-resolution reconstruction algorithm.
[0045] As one implementation method, the interpolation-based super-resolution reconstruction algorithm specifically calculates the weighted average of the sixteen nearest neighboring points of a pixel point as the value of a new pixel point, thereby obtaining a high-resolution SAR image.
[0046] As one of the implementation methods, the reconstruction-based super-resolution algorithm is specifically a MAP-based super-resolution reconstruction algorithm. According to the characteristics of the low-resolution SAR image, a suitable prior knowledge model is defined, such as the statistical characteristics of the image, the edge distribution, etc., and the low-resolution SAR image and the above-mentioned prior knowledge are combined to construct a posterior probability model of the high-resolution SAR image. This usually involves modeling the image degradation process and solving the inverse process. The high-resolution SAR image parameters that maximize the posterior probability are solved through optimization algorithms (such as gradient descent, Newton's method, etc.) and the estimated high-resolution SAR image is output. In this process, the AIS provides the position, heading, speed of the ship and predicts the movement trajectory of the ship in the SAR image to timely compensate for the image displacement or blur caused by the movement of the ship, so as to assist the MAP method in constructing a more accurate posterior probability model, thereby obtaining a high-resolution SAR image.
[0047] As one of the implementation methods, Figure 2 As shown in FIG. 1 , the specific process of the learning-based super-resolution reconstruction algorithm includes:
[0048] Step S51: Based on the aforementioned acquisition of AIS data and SAR data, a deep learning model is designed, and image features are extracted from SAR images using structures such as convolutional neural networks (CNNs). The ship position, heading, and speed information in the AIS data are converted into feature vectors and introduced into the deep learning model as additional inputs or conditional constraints.
[0049] Step S52: Design an algorithm, design or select a suitable super-resolution reconstruction network, such as SRResNet, ESPCN (Efficient Sub-Pixel Convolutional Neural Network), etc. Introduce residual learning, attention mechanism and other technologies into the network to improve the quality and efficiency of the reconstructed image. Taking SRResNet as an example, the difference between image input and output is learned by constructing residual blocks, which are used to generate feature maps of each channel through convolution operations. The feature maps of the above channels are rearranged into high-resolution SAR images using pixel rearrangement technology. The residual block includes a basic residual block structure and a deep residual block structure. The basic block residual block structure mainly includes two 3x3 convolution layers. The input features are added to the original input after passing through these two convolution layers and output through the activation function. The deep residual block structure adds a 1x1 convolution layer on the basis of the basic residual block structure, which is used for dimensionality reduction and dimensionality increase, to reduce the number of parameters and improve data calculation efficiency. In order to improve the residual learning performance, the basic residual block structure can be used for preliminary training to quickly obtain a basic model. Then, on the basis of the basic residual block structure, the number of layers can be gradually increased to transition to the deep residual block structure. In this way, the complexity of the model can be gradually increased to optimize the performance of the residual block structure on training SAR images.
[0050] Step S53, using the prior information provided by the AIS data (such as the ship's position) to guide the network to perform more detailed reconstruction in a specific area (such as the location of the ship), content loss (used to measure the consistency of the image content) and perceptual loss (based on the features extracted by the network to measure the difference), as well as specific loss items for the characteristics of AIS data (such as the loss of ship position accuracy), taking SRResNet as an example, by minimizing the loss function, the SRResNet network can obtain the mapping relationship between high-resolution images and low-resolution images through learning. When training SRResNet, the average error is used as the loss function to measure the difference between the real image and the predicted image. Considering the data characteristics of AIS, the loss function is based on AIS data loss, pixel-level loss, image content loss, and perceptual loss. The loss function is:
[0051] L = a·L pixel +b·L ais +c·L content +d·L perceptual ,
[0052] Where L pixel is the pixel-level loss, L ais is the AIS data loss, L content is the image content loss, L perceptual is the perceptual loss, a, b, c, and d are the coefficients of the pixel-level loss function, AIS data loss function, image content loss function, and perceptual loss function, respectively. The coefficients are used to measure the importance of different loss items.
[0053] Step S54, training and optimization, prepare a large number of training data sets containing high- and low-resolution SAR image pairs and corresponding AIS data to enhance the training data to improve the generalization ability of the model, use the training data set to train the model, optimize the network parameters through the back propagation algorithm, monitor the loss value of the model and the quality of the reconstructed image during the training process, and adjust the learning rate, optimizer and other parameters in time. The super-resolution reconstruction algorithm can achieve more efficient and accurate SAR image super-resolution reconstruction by combining AIS data and deep learning technology.
[0054] In summary, the embodiment of the present invention first acquires AIS data in real time, and the AIS data is the ship dynamic data sent by the AIS system; uses the target detection algorithm to match the AIS data with the ship target in the SAR image to determine the position of the ship in the SAR image; performs data fusion on the AIS data and the SAR image data; and integrates the AIS data as prior knowledge into the super-resolution reconstruction algorithm to reconstruct the SAR image to obtain a high-resolution SAR image. The present invention provides the real-time location information of the target through the AIS system, helping the SAR system to accurately plan the imaging task, and the super-resolution reconstruction technology further improves the resolution of the SAR image, making the image clearer and richer in details. By combining the AIS system and the super-resolution reconstruction technology, high-precision, high-resolution three-dimensional imaging of mobile targets at sea can be achieved, providing strong support for ocean monitoring.
[0055] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the super-resolution reconstruction method described above when executing the computer program.
[0056] The processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of the electronic device. In this embodiment, the processor is used to run the program code or process data stored in the memory, such as the program code of the target recognition algorithm.
[0057] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory can also be an external storage device of the electronic device, such as a plug-in hard disk equipped on the electronic device, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory can also include both the internal storage unit of the electronic device and its external storage device. In this embodiment, the memory is generally used to store the operating method and various application software installed in the electronic device, such as the program code of the vehicle collision warning method, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are to be output.
[0058] Based on the same inventive concept, the present invention also provides a readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the infrared temperature measurement method described above are implemented.
[0059] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.
[0060] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.
[0061] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0063] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.
Claims
1. A method for super-resolution reconstruction of SAR images using AIS guidance, characterized in that: The steps include: S1. Real-time acquisition of AIS data, which is the ship dynamic data sent by the AIS system; S2, obtaining original SAR image data of the target sea area through the SAR system; S3, using a target detection algorithm to match the AIS data with the ship target in the SAR image to determine the position of the ship in the SAR image; S4, fusing the AIS data with the SAR image data; S5. Incorporating AIS data as prior knowledge into the super-resolution reconstruction algorithm to reconstruct the SAR image to obtain a high-resolution SAR image.
2. The super-resolution reconstruction method according to claim 1, characterized in that: In the step S1, the ship dynamic data includes: the position, heading and speed of the ship.
3. The super-resolution reconstruction method according to claim 1 or 2, characterized in that: After step S1, the method further includes preprocessing the ship dynamic data, wherein the preprocessing includes: removing abnormal values or erroneous data in the AIS data, and converting the AIS data into data with a unified coordinate system and time standard.
4. The super-resolution reconstruction method according to claim 1, wherein: After step S2, the method further includes geo-locating the image using the geographic information and feature points in the SAR image to ensure spatial consistency between the SAR image and the AIS data.
5. The super-resolution reconstruction method according to claim 1, characterized in that: In step S3, the target detection algorithm is selected from any one of the YOLO algorithm, Faster R-CNN, and SSD.
6. The super-resolution reconstruction method according to claim 1, characterized in that: The step S4 specifically includes: using the ship's speed and heading provided by the AIS data to estimate the ship's motion trajectory and posture in the SAR image.
7. The super-resolution reconstruction method according to claim 1, characterized in that: In step S5, the super-resolution algorithm selects any one of an interpolation-based super-resolution reconstruction algorithm, a reconstruction-based super-resolution reconstruction algorithm, and a learning-based super-resolution reconstruction algorithm.
8. The super-resolution reconstruction method according to claim 1 or 7, characterized in that: The step S5 specifically includes: using the heading and speed of the ship provided by the AIS to compensate for the motion trajectory of the ship in the SAR image, thereby eliminating motion blur and defocusing phenomena; and applying a super-resolution reconstruction algorithm to reconstruct the processed SAR image to obtain a high-resolution SAR image.
9. The method for super-resolution reconstruction of SAR images using AIS guidance as claimed in claim 7, characterized in that: The learning-based super-resolution reconstruction algorithm includes designing a super-resolution reconstruction network.
10. The method for super-resolution reconstruction of SAR images using AIS guidance as claimed in claim 7, characterized in that: The learning-based super-resolution reconstruction algorithm includes designing a specific loss term targeting the characteristics of AIS data.