Space-based and air-based target characterization method and device based on space-frequency domain combination

Abstract

The application provides a space-based and air-based target characterization method and device based on joint space-frequency domain, and relates to the technical field of image processing.The method comprises the following steps: determining a first amplitude spectrum of a first image acquired by space-based acquisition and a second amplitude spectrum of a second image acquired by air-based acquisition; determining a first spatial domain depth feature based on the first image and a second spatial domain depth feature based on the second image; determining a first frequency domain depth feature based on the first amplitude spectrum and a second frequency domain depth feature based on the second amplitude spectrum; determining target characterization of the first image based on the first spatial domain depth feature and the first frequency domain depth feature and determining target characterization of the second image based on the second spatial domain depth feature and the second frequency domain depth feature; and the target characterization of the first image and the target characterization of the second image are used for target correlation, matching or collaborative identification of space-based and air-based targets.The application can improve the accuracy of target characterization.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for characterizing space-based targets based on spatial-frequency domain integration. Background Technology

[0002] Space-based and air-based collaborative remote sensing integrates the advantages of space-based platforms' "wide-area coverage" and air-based platforms' "high-resolution close-range detailed investigation," constructing a three-dimensional observation system of "wide-area detection-close-range detailed investigation." This system plays an irreplaceable role in fields such as national defense security, disaster emergency response, and national economic services. However, space-based and air-based multi-source remote sensing images (such as optical and synthetic aperture radar images) exhibit significant differences, with a large range of target sizes and random orientations, leading to extremely high difficulty in target characterization and placing high stability requirements on subsequent interpretation algorithms.

[0003] For target representation in remote sensing images, existing technologies have transitioned from traditional manual feature representation to deep learning-based feature representation. Traditional methods rely on manually designed features, such as color histograms, Gabor filters, and directional gradient histograms, but these methods are difficult to adapt to multi-source heterogeneous data. With the development of deep learning, methods based on convolutional neural networks and visual Transformers have significantly improved target representation capabilities.

[0004] However, when remote sensing images are rotated or scaled, the accuracy of target representations, whether extracted by manually designed features or by methods based on convolutional neural networks and visual Transformers, is not high. Summary of the Invention

[0005] This invention provides a space-based and air-based target characterization method and apparatus based on spatial-frequency domain integration, which addresses the shortcomings of low accuracy in target characterization extracted in existing technologies and aims to improve the accuracy of extracted target characterization.

[0006] This invention provides a space-based and space-based target characterization method based on joint space-frequency domain analysis, comprising: Determine the first amplitude spectrum of the first image acquired by space-based system and the second amplitude spectrum of the second image acquired by air-based system; Based on the first image, a first spatial domain depth feature is determined, and based on the second image, a second spatial domain depth feature is determined. Based on the first amplitude spectrum, a first frequency domain depth feature is determined, and based on the second amplitude spectrum, a second frequency domain depth feature is determined. Based on the first spatial domain depth features and the first frequency domain depth features, the target representation of the first image is determined, and based on the second spatial domain depth features and the second frequency domain depth features, the target representation of the second image is determined. The target representation of the first image and the target representation of the second image are used to perform target association, matching or collaborative identification of the space-based and air-based systems.

[0007] According to the present invention, a space-based target characterization method based on spatial-frequency domain joint method is provided, wherein determining the first amplitude spectrum of the first image acquired by the space-based system includes: Perform a logarithmic polar coordinate transformation on the first image to obtain a first logarithmic polar coordinate representation; Perform a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation, and take the amplitude of the first spectrum obtained after the transform to obtain the first amplitude spectrum.

[0008] According to the present invention, a space-based target characterization method based on joint space-frequency domain is provided, wherein performing a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation includes: A two-dimensional discrete-time Fourier transform is performed on the first logarithmic polar coordinate representation according to the following formula (1): (1) in, This represents the first spectrum, the The first logarithmic polar coordinates represent the coordinates in the graph. Pixel value at that location, Indicates radial distance. The angle of a pixel relative to the center of the first logarithmic polar coordinate representation graph, the The horizontal frequency variable in the frequency domain, the Represents the vertical frequency variable in the frequency domain. This represents a complex exponential basis function used to implement projection from the spatial domain to the frequency domain.

[0009] According to the present invention, a space-based target characterization method based on joint spatial and frequency domains is provided, wherein determining the first spatial domain depth features based on the first image includes: The first image is input into a spatial domain feature extraction network to obtain the first spatial domain depth features output by the spatial domain feature extraction network.

[0010] According to the present invention, a space-based target characterization method based on spatial-frequency domain joint analysis is provided, wherein determining the first frequency domain depth feature based on the first amplitude spectrum includes: The first amplitude spectrum is input into the frequency domain feature extraction network to obtain the first frequency domain depth feature output by the frequency domain feature extraction network.

[0011] According to the present invention, a space-based target characterization method based on joint spatial and frequency domain features is provided, wherein determining the target characterization of the first image based on the first spatial domain depth features and the first frequency domain depth features includes: The first spatial domain depth feature and the first frequency domain depth feature are fused to obtain the target representation of the first image.

[0012] The present invention also provides a space-based target characterization device based on the joint space-frequency domain, comprising: The first determining module is used to determine the first amplitude spectrum of the first image acquired by space-based system and the second amplitude spectrum of the second image acquired by air-based system. The second determining module is used to determine a first spatial domain depth feature based on the first image, and to determine a second spatial domain depth feature based on the second image. The third determining module is used to determine a first frequency domain depth feature based on the first amplitude spectrum and to determine a second frequency domain depth feature based on the second amplitude spectrum. The fourth determining module is used to determine the target representation of the first image based on the first spatial domain depth features and the first frequency domain depth features, and to determine the target representation of the second image based on the second spatial domain depth features and the second frequency domain depth features. The target representation of the first image and the target representation of the second image are used to perform target association, matching or collaborative identification of the space-based and air-based systems.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the space-based and air-based target characterization method based on the spatial-frequency domain joint as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the space-based and air-based target characterization method based on the spatial-frequency domain joint method described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the space-based and air-based target characterization method based on the spatial-frequency domain joint method described above.

[0016] The present invention provides a space-based and space-based target characterization method and apparatus based on joint space-frequency domain. By determining a first amplitude spectrum of a first image acquired from space and a second amplitude spectrum of a second image acquired from space, a first spatial domain depth feature is determined based on the first image, and a second spatial domain depth feature is determined based on the second image. Based on the first amplitude spectrum, a first frequency domain depth feature is determined, and based on the second amplitude spectrum, a second frequency domain depth feature is determined. Based on the first spatial domain depth feature and the first frequency domain depth feature, a target characterization of the first image is determined, and based on the second spatial domain depth feature and the second frequency domain depth feature, a target characterization of the second image is determined. The target characterizations of the first and second images are used for target association, matching, or collaborative identification between space-based and space-based systems. Since image rotation and scale changes in the spatial domain can be converted into linear phase shifts in the frequency domain, the influence of this phase shift can be eliminated by determining the first and second amplitude spectrums. Therefore, even after target rotation or image scale changes, robust frequency domain rotation-scale invariant features can be extracted. Furthermore, frequency domain depth features provide robustness against geometric changes, ensuring the stability of features under different observation conditions; while spatial domain depth features preserve the detailed texture and discriminative information of the target, ensuring the ability to distinguish similar targets. Fusing frequency and spatial domain depth features results in a final feature that simultaneously possesses the strong robustness of the frequency domain to rotation and scale changes, and the high discriminative power of the spatial domain for the detailed structure of the target. This improves the accuracy of target representation in complex space-based and airborne multi-source heterogeneous scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the space-based and space-based target characterization method based on the joint space-frequency domain provided in this embodiment of the invention.

[0019] Figure 2 This is the second flowchart illustrating the space-based and air-based target characterization method based on the joint space-frequency domain provided in this embodiment of the invention.

[0020] Figure 3 This is a schematic diagram of the structure of a space-based target characterization device based on the joint space-frequency domain provided in an embodiment of the present invention.

[0021] Figure 4This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] Significant differences exist between space-based and air-based multi-source remote sensing images. The size range of targets in multi-source remote sensing images is large and the target orientation is random, making target representation extremely difficult. At the same time, the processing algorithms of the edge terminals of air-based platforms require high stability. Therefore, the output of the representation stage must have both discriminative and robust features; otherwise, it will lead to performance degradation of downstream interpretation tasks such as detection and association.

[0024] Therefore, target representation is the core foundation of remote sensing image interpretation. Traditional manual feature representation methods (such as color histograms, Gabor filters, and directional gradient histograms) have limited generalization and are difficult to adapt to multi-source heterogeneous data. Feature extraction methods based on deep learning (such as convolutional neural networks and visual Transformers) have significantly improved the target representation capability. However, when remote sensing images are rotated or their scale changes, the problem of inaccurate target representation results still exists.

[0025] To address the aforementioned issues, this invention proposes a space-based and airborne target characterization method based on joint space-frequency domain analysis. This method considers that changes in target rotation (e.g., changes in ship orientation) or image scale (e.g., differences in space-based and airborne observation distances) can lead to ineffective spatial features. In the frequency domain, such changes can be transformed into eliminable phase shifts. Therefore, by eliminating the effects of phase shifts, robust frequency domain features can be obtained even after target rotation or changes in image scale. Furthermore, by fusing spatial and frequency domain depth features, both spatial detail discrimination and frequency domain robustness can be preserved simultaneously.

[0026] The following is combined with Figure 1 and Figure 2This invention describes a space-based and airborne target characterization method based on joint space-frequency domain analysis. This invention is applicable to space-based collaborative observation systems comprised of space-based platforms (such as satellites) and airborne platforms (such as UAVs and aircraft). It can robustly extract and characterize targets in multi-source, high-difference remote sensing images (such as optical and synthetic aperture radar (SAR) images), providing high-quality features for subsequent downstream interpretation tasks such as space-based target detection, association, and collaborative identification. For example, after a space-based platform detects a target over a wide area, an airborne platform (such as a UAV or aircraft) can approach and conduct a detailed investigation to achieve high-precision identification and association of military targets, disaster-damaged facilities, or critical infrastructure.

[0027] The subject executing this method can be a terminal device, computer, server, server cluster, or specially designed space-based airborne target characterization device based on space-frequency domain integration, or a space-based airborne target characterization device based on space-frequency domain integration installed in the electronic device. The space-based airborne target characterization device based on space-frequency domain integration can be implemented by software, hardware, or a combination of both.

[0028] Figure 1 This is one of the flowcharts illustrating the space-based and space-based target characterization method based on the joint space-frequency domain provided in this embodiment of the invention, as shown below. Figure 1 As shown, the method includes: Step 101: Determine the first amplitude spectrum of the first image acquired by space-based acquisition and the second amplitude spectrum of the second image acquired by air-based acquisition.

[0029] In this step, space-based may include satellites, and air-based may include drones and aircraft. The first image can be an image slice containing a specific target, cropped from an image acquired from a space-based source, or a feature map obtained by performing feature extraction or convolution processing on the cropped image slice. Similarly, the second image can be an image slice containing a specific target, cropped from an image acquired from an air-based source, or a feature map obtained by performing feature extraction or convolution processing on the cropped image slice. The specific target may include, for example, ships or vehicles.

[0030] Based on the first amplitude spectrum determined from the first image and the second amplitude spectrum determined from the second image, all phase effects caused by rotation and scaling can be eliminated. This allows for a pure frequency domain structure representation that is unaffected by the rotation and scaling changes of the original first and second images, laying the foundation for subsequent deep networks to extract stable and robust features from the frequency domain.

[0031] Step 102: Based on the first image, determine the first spatial domain depth features, and based on the second image, determine the second spatial domain depth features.

[0032] In this step, the first spatial domain depth features extracted from the first image and the second spatial domain depth features extracted from the second image contain rich details for target discrimination, such as the structure of a ship's deck. Through the first and second spatial domain depth features, intuitive visual information about the target at the image pixel level can be captured, such as edges, textures, colors, and local shapes. This information is crucial for distinguishing different types of subtle targets.

[0033] Although the first and second spatial domain depth features are acquired from space-based and air-based images with different observation perspectives and resolutions, they still carry the inherent visual attributes of the target.

[0034] Step 103: Based on the first amplitude spectrum, determine the first frequency domain depth feature, and based on the second amplitude spectrum, determine the second frequency domain depth feature.

[0035] In this step, the first frequency domain depth feature extracted from the first amplitude spectrogram and the second frequency domain depth feature extracted from the second amplitude spectrogram inherit the robustness of the first and second amplitude spectrograms to rotation and scale changes, respectively. Although the first and second amplitude spectrograms may come from space-based and air-based platforms with large differences in viewing angle and resolution, the first and second frequency domain depth features that can point to the same target can be extracted from their inherent, stable and invariant frequency structure information.

[0036] Step 104: Based on the first spatial domain depth features and the first frequency domain depth features, determine the target representation of the first image, and based on the second spatial domain depth features and the second frequency domain depth features, determine the target representation of the second image. The target representation of the first image and the target representation of the second image are used for space-based and air-based target association, matching or collaborative recognition.

[0037] In this step, for the space-based platform, the first spatial domain depth features containing discriminative details and the first frequency domain depth features, which are robust to geometric changes, can be deeply fused to obtain a more powerful and comprehensive target representation of the first image. The target representation of the first image can be understood as a stable and unified digital representation of the targets in the first image.

[0038] Similarly, for an airborne platform, the second spatial domain depth features containing discriminative details and the second frequency domain depth features that are robust to geometric changes can be deeply fused to obtain a more powerful and comprehensive target representation of the second image.

[0039] The joint feature representation of the target in the first image and the target representation in the second image can resist the interference of changes in observation conditions and retain sufficient discriminative details, laying the foundation for subsequent cross-platform target association and recognition.

[0040] By using target representations from the first and second images, space-based and air-based target association, matching, or collaborative identification can be achieved. For example, if two target representations are very similar, it indicates that they likely correspond to the same target in the real world, thus providing a reliable and consistent feature foundation for subsequent advanced tasks such as space-based target association, cross-platform target matching, and collaborative identification and tracking.

[0041] The space-based and space-based target characterization method based on joint space-frequency domain provided in this invention determines a first amplitude spectrum of a first image acquired by space-based systems and a second amplitude spectrum of a second image acquired by space-based systems. Based on the first image, it determines a first spatial domain depth feature, and based on the second image, it determines a second spatial domain depth feature. Based on the first amplitude spectrum, it determines a first frequency domain depth feature, and based on the second amplitude spectrum, it determines a second frequency domain depth feature. Based on the first spatial domain depth feature and the first frequency domain depth feature, it determines the target characterization of the first image, and based on the second spatial domain depth feature and the second frequency domain depth feature, it determines the target characterization of the second image. The target characterizations of the first image and the second image are used for target association, matching, or collaborative recognition between space-based and space-based systems. Since image rotation and scale changes in the spatial domain can be converted into linear phase shifts in the frequency domain, the influence of this phase shift can be eliminated by determining the first and second amplitude spectrums. Therefore, even after target rotation or image scale changes, robust frequency domain rotation-scale invariant features can be extracted. Furthermore, frequency domain depth features provide robustness against geometric changes, ensuring the stability of features under different observation conditions; while spatial domain depth features preserve the detailed texture and discriminative information of the target, ensuring the ability to distinguish similar targets. Fusing frequency and spatial domain depth features results in a final feature that simultaneously possesses the strong robustness of the frequency domain to rotation and scale changes, and the high discriminative power of the spatial domain for the detailed structure of the target. This improves the accuracy of target representation in complex space-based and airborne multi-source heterogeneous scenarios.

[0042] For example, based on the above embodiments, when determining the first amplitude spectrum of the first image acquired by space-based technology, it can be done in the following manner: A logarithmic polar coordinate transformation is performed on the first image to obtain a first logarithmic polar coordinate representation. A two-dimensional discrete-time Fourier transform is then performed on the first logarithmic polar coordinate representation, and the amplitude of the first spectrum obtained after the transformation is taken to obtain a first amplitude spectrum.

[0043] Specifically, for the first image Center point of the first image As the pole, construct the first logarithmic polar coordinate representation according to formula (2): (2) in, The first logarithmic polar coordinates represent the coordinates in the graph. Pixel value at that location, Indicates radial distance. , This represents the angle of a pixel relative to the center of the first logarithmic polar coordinate system. .

[0044] Considering the discrete nature of the image, a two-dimensional discrete-time Fourier transform (DTFT) is performed on the first logarithmic polar coordinate representation to obtain the first spectrum.

[0045] For example, a two-dimensional discrete-time Fourier transform can be performed on the first logarithmic polar coordinate representation according to the following formula (1): (1) in, This represents the first spectrum. The first logarithmic polar coordinates represent the coordinates in the graph. Pixel value at that location, Indicates radial distance. This represents the angle of a pixel relative to the center of the graph in the first logarithmic polar coordinate system. Represents the horizontal frequency variable in the frequency domain. Represents the vertical frequency variable in the frequency domain. This represents a complex exponential basis function used to implement projection from the spatial domain to the frequency domain.

[0046] By performing a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation, the translation of the first image in the logarithmic polar coordinate space (caused by the rotation and scaling of the original first image) can be converted into a linear phase shift in the frequency domain, thus laying the foundation for subsequent robust feature extraction.

[0047] Furthermore, after obtaining the transformed first spectrum... Then, the frequency domain rotation-scale robust feature, i.e. the first amplitude spectrum, can be constructed according to the following formula (3). : (3) Based on the properties of a two-dimensional DTFT, it can be seen that when the first image is rotated: Or, the image size is scaled by a factor of k, i.e. At that time, the first spectrum diagram It exhibits phase-shift characteristics, as shown in formula (4): (4) Therefore, for the first spectrum The amplitude can be determined according to formula (3) to construct the first amplitude spectrum. .

[0048] The aforementioned process of performing logarithmic polar coordinate transformation, two-dimensional discrete-time Fourier transform, and amplitude-based feature transformation on the first image can be described as follows: According to formulas (3) and (4), when different images I and images If the relationship between them is one of rotation or scaling, then there exists That is, frequency domain rotation-scale robustness feature It is theoretically robust to image rotation and scaling changes.

[0049] It should be noted that, for the second image, a logarithmic polar coordinate transformation can be performed in the same manner as described above to obtain a second logarithmic polar coordinate representation. Then, a two-dimensional discrete-time Fourier transform is performed on the second logarithmic polar coordinate representation, and the amplitude of the resulting second spectrum is taken to obtain a second amplitude spectrum. The obtained second amplitude spectrum is also robust to image rotation and scaling changes.

[0050] In this embodiment, by performing a logarithmic polar coordinate transformation on the first image and a two-dimensional discrete-time Fourier transform on the resulting first logarithmic polar coordinate representation, and taking the amplitude of the transformation result, a frequency domain feature with strong robustness to image rotation and scale changes can be constructed, thereby providing a foundation for subsequent cross-modal target representation and collaborative tasks.

[0051] For example, based on the above embodiments, when determining the first spatial domain depth feature based on the first image, the first image can be input into the spatial domain feature extraction network to obtain the first spatial domain depth feature output by the spatial domain feature extraction network.

[0052] Specifically, spatial domain feature extraction network A convolutional neural network (CNN) is a deep learning model architecture. This network typically contains multiple layers, such as convolutional layers, pooling layers, and activation functions. When the first image is input into the network, the convolutional layers at the front of the network first perform local perception operations to extract basic visual elements of the image, such as edges, corners, or simple texture patterns.

[0053] As these features are passed to higher levels in the network, deeper network layers gradually combine and abstract these basic features through stacked convolutions and non-linear activation operations, thereby obtaining the final output first spatial domain depth features. ,in, This represents the first image.

[0054] Similarly, the second image can be input into the spatial domain feature extraction network to obtain the second spatial domain depth features output by the spatial domain feature extraction network.

[0055] In this embodiment, a spatial domain depth feature is automatically extracted from the first image by a spatial domain feature extraction network, which can efficiently and adaptively capture fine discriminative information of the target at the pixel level in the first image.

[0056] For example, based on the above embodiments, when determining the first frequency domain depth feature based on the first amplitude spectrum, the first amplitude spectrum can be input into the frequency domain feature extraction network to obtain the first frequency domain depth feature output by the frequency domain feature extraction network.

[0057] Specifically, frequency domain feature extraction network A convolutional neural network (CNN) is a deep learning model used to process spectral data. Its structure is typically built on convolutional neural networks. When a first amplitude spectrogram is input into the network, the network first perceives local patterns in the spectrum through its initial convolutional layers, such as the intensity distribution of specific frequency components and their neighborhood relationships. These patterns encode the structured information of the original first image that is robust to rotation and scaling changes after undergoing logarithmic polar coordinate transformation and frequency domain transformation.

[0058] As features propagate deeper into the network, higher-level convolutional layers combine and abstract these fundamental frequency domain patterns, learning to identify the most discriminative global statistical and structural features in the spectrum that are most relevant to the target category. Finally, through fully connected layers or global pooling layers at the end, this hierarchically abstracted frequency domain information is aggregated into a fixed-dimensional, high-level feature vector, i.e., the first-level frequency domain deep feature. .

[0059] Similarly, the second amplitude spectrogram can be input into the frequency domain feature extraction network to obtain the second frequency domain depth feature output by the frequency domain feature extraction network.

[0060] The frequency domain feature extraction network can extract first frequency domain deep features rich in high-level semantics from the original, theoretically robust first amplitude spectrogram, which significantly enhances the stability and reliability of the target representation under complex changes.

[0061] For example, based on the above embodiments, when determining the target representation of the first image based on the first spatial domain depth features and the first frequency domain depth features, the first spatial domain depth features and the first frequency domain depth features can be fused to obtain the target representation of the first image.

[0062] Specifically, the first frequency domain depth features It exhibits robustness to image rotation and scale changes, but loses phase spectrum information and intuitive spatial structure and detailed texture information. Considering the diversity of target orientation and scale in sky-based multi-source remote sensing data, the first frequency domain depth features are used instead. Depth features of the first spatial domain with structural and detailed information Integration and complementary advantages can improve the robustness of multi-source data representation.

[0063] In a specific implementation, a feature deep fusion network can be used. For the depth features of the first spatial domain and first frequency domain depth features By performing splicing and convolution operations, the fused spatial-frequency domain features are obtained. : Optimize the deep features of the first spatial domain through training and first frequency domain depth features The fusion weights ensure the spatio-frequency domain fusion characteristics. It retains both spatial domain discrimination details and frequency domain robustness.

[0064] Among them, feature deep fusion network Structures such as those based on convolutional neural networks can be used, through deep feature fusion networks. By combining the rich structural and detailed texture information of the spatial domain with the robustness of the frequency domain, spatial-frequency domain fusion features can be obtained. This spatial-frequency domain fusion feature This is the target representation of the first image.

[0065] It should be noted that the above-mentioned spatial-frequency domain fusion features from input image to output... The process can be viewed as a whole as a new backbone network. This is called a backbone network based on joint learning in the space and frequency domains. Complementing the advantages of spatial and frequency domain information helps improve the robustness and separability of the representation of multi-source heterogeneous data.

[0066] Similarly, for the target representation of the second image, the second spatial domain depth features and the second frequency domain depth features can be fused to obtain the target representation of the second image.

[0067] Among them, the spatial domain feature extraction network, the frequency domain feature extraction network, and the feature deep fusion network can be easily integrated into existing mainstream deep learning frameworks, adapting to multiple downstream tasks and achieving the goal of plug-and-play and wide versatility.

[0068] Figure 2 This is the second flowchart illustrating the space-based and space-based target characterization method based on the joint space-frequency domain provided in this embodiment of the invention. Figure 2 As shown, taking input image I as the first image as an example, the processing method for the second image is similar to that for the first image, and will not be repeated here. The input image I is sequentially subjected to logarithmic polar coordinate transformation, two-dimensional discrete-time Fourier transform, and frequency domain feature transformation with amplitude taking. Frequency domain rotation-scale robust features are obtained. That is, the first amplitude spectrum.

[0069] Input image I into the spatial domain feature extraction network This backbone network yields the first spatial domain depth features. The input image I is then input into the spatial domain feature extraction network. This backbone network yielded the first amplitude spectrum diagram. Input frequency domain feature extraction network This backbone network yields the first frequency domain depth features. .

[0070] Single-domain features and Input Feature Deep Fusion Network Spatial-frequency domain fusion features were obtained. This spatial-frequency domain fusion feature This refers to the target representation method in the first image.

[0071] The space-based and air-based target characterization method based on joint space-frequency domain learning provided in this invention addresses the challenge of characterizing highly variable space-based data. It leverages a joint space-frequency domain learning approach to mine robust frequency-domain depth features across multiple scales and directions, fusing them with detailed spatial-domain depth features. This improves the consistency of the fused features across diverse space-based multi-source scenarios, providing a comprehensive and robust representation of highly variable space-based target data and effectively reducing errors in subsequent target detection and association tasks. Furthermore, spatial-domain depth features preserve local target details, while frequency-domain depth features resist interference from differences in target orientation and size. The resulting fused features exhibit improved target discrimination capabilities compared to single-domain features, achieving a balance between detail and robustness.

[0072] The space-based airborne target characterization device based on the joint space-frequency domain provided by the present invention will be described below. The space-based airborne target characterization device based on the joint space-frequency domain described below can be referred to in correspondence with the space-based airborne target characterization method based on the joint space-frequency domain described above.

[0073] Figure 3 This is a schematic diagram of the structure of the space-based and airborne target characterization device based on the joint space-frequency domain provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the space-based target characterization device 300 based on the joint space-frequency domain includes: The first determining module 11 is used to determine the first amplitude spectrum of the first image acquired by space-based system and the second amplitude spectrum of the second image acquired by air-based system. The second determining module 12 is used to determine the first spatial domain depth feature based on the first image, and to determine the second spatial domain depth feature based on the second image. The third determining module 13 is used to determine the first frequency domain depth feature based on the first amplitude spectrum and to determine the second frequency domain depth feature based on the second amplitude spectrum. The fourth determining module 14 is used to determine the target representation of the first image based on the first spatial domain depth feature and the first frequency domain depth feature, and to determine the target representation of the second image based on the second spatial domain depth feature and the second frequency domain depth feature. The target representation of the first image and the target representation of the second image are used to perform target association, matching or collaborative identification of the space-based and the air-based targets.

[0074] In one example embodiment, the first determining module 11 is specifically used for: Perform a logarithmic polar coordinate transformation on the first image to obtain a first logarithmic polar coordinate representation; Perform a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation, and take the amplitude of the first spectrum obtained after the transform to obtain the first amplitude spectrum.

[0075] In one example embodiment, the first determining module 11 is specifically used for: A two-dimensional discrete-time Fourier transform is performed on the first logarithmic polar coordinate representation according to the following formula (1): (1) in, This represents the first spectrum, the The first logarithmic polar coordinates represent the coordinates in the graph. Pixel value at that location, Indicates radial distance. The angle of a pixel relative to the center of the first logarithmic polar coordinate representation graph, the The horizontal frequency variable in the frequency domain, the Represents the vertical frequency variable in the frequency domain. This represents a complex exponential basis function used to implement projection from the spatial domain to the frequency domain.

[0076] In one example embodiment, the second determining module 12 is specifically used for: The first image is input into a spatial domain feature extraction network to obtain the first spatial domain depth features output by the spatial domain feature extraction network.

[0077] In one example embodiment, the third determining module 13 is specifically used for: The first amplitude spectrum is input into the frequency domain feature extraction network to obtain the first frequency domain depth feature output by the frequency domain feature extraction network.

[0078] In one example embodiment, the fourth determining module 14 is specifically used for: The first spatial domain depth feature and the first frequency domain depth feature are fused to obtain the target representation of the first image.

[0079] The apparatus of this embodiment can be used in any embodiment of the method of the space-based airborne target characterization method based on the joint space-frequency domain. Its specific implementation process and technical effects are similar to those of the method of the space-based airborne target characterization method based on the joint space-frequency domain. For details, please refer to the detailed description in the method of the space-based airborne target characterization method based on the joint space-frequency domain, which will not be repeated here.

[0080] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a space-based and space-based target characterization method based on joint space-frequency domain. The method includes: determining a first amplitude spectrum of a first image acquired by space-based systems and a second amplitude spectrum of a second image acquired by space-based systems; determining a first spatial domain depth feature based on the first image and a second spatial domain depth feature based on the second image; determining a first frequency domain depth feature based on the first amplitude spectrum and a second frequency domain depth feature based on the second amplitude spectrum; determining a target characterization of the first image based on the first spatial domain depth feature and the first frequency domain depth feature, and determining a target characterization of the second image based on the second spatial domain depth feature and the second frequency domain depth feature. The target characterization of the first image and the target characterization of the second image are used for target association, matching, or collaborative identification between the space-based and space-based systems.

[0081] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the space-based and space-based target characterization method based on the joint space-frequency domain provided by the above methods. The method includes: determining a first amplitude spectrum of a first image acquired by space-based systems and a second amplitude spectrum of a second image acquired by space-based systems; determining a first spatial domain depth feature based on the first image and a second spatial domain depth feature based on the second image; determining a first frequency domain depth feature based on the first amplitude spectrum and a second frequency domain depth feature based on the second amplitude spectrum; determining a target characterization of the first image based on the first spatial domain depth feature and the first frequency domain depth feature, and determining a target characterization of the second image based on the second spatial domain depth feature and the second frequency domain depth feature. The target characterization of the first image and the target characterization of the second image are used for target association, matching, or collaborative identification between the space-based and space-based systems.

[0083] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the space-based and space-based target characterization method based on the above-described methods, which includes: determining a first amplitude spectrum of a first image acquired by space-based computing and a second amplitude spectrum of a second image acquired by space-based computing; determining a first spatial domain depth feature based on the first image and a second spatial domain depth feature based on the second image; determining a first frequency domain depth feature based on the first amplitude spectrum and a second frequency domain depth feature based on the second amplitude spectrum; determining a target characterization of the first image based on the first spatial domain depth feature and the first frequency domain depth feature, and determining a target characterization of the second image based on the second spatial domain depth feature and the second frequency domain depth feature, wherein the target characterization of the first image and the target characterization of the second image are used for target association, matching, or collaborative identification between the space-based and space-based computing.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A space-based target characterization method based on spatial-frequency domain integration, characterized in that, include: Determine the first amplitude spectrum of the first image acquired by space-based system and the second amplitude spectrum of the second image acquired by air-based system; Based on the first image, a first spatial domain depth feature is determined, and based on the second image, a second spatial domain depth feature is determined. Based on the first amplitude spectrum, a first frequency domain depth feature is determined, and based on the second amplitude spectrum, a second frequency domain depth feature is determined. Based on the first spatial domain depth features and the first frequency domain depth features, the target representation of the first image is determined, and based on the second spatial domain depth features and the second frequency domain depth features, the target representation of the second image is determined. The target representation of the first image and the target representation of the second image are used to perform target association, matching or collaborative identification of the space-based and air-based systems.

2. The space-based and airborne target characterization method based on spatial-frequency domain joint method according to claim 1, characterized in that, The determination of the first amplitude spectrum of the first image acquired by space-based technology includes: Perform a logarithmic polar coordinate transformation on the first image to obtain a first logarithmic polar coordinate representation; Perform a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation, and take the amplitude of the first spectrum obtained after the transform to obtain the first amplitude spectrum.

3. The space-based and airborne target characterization method based on spatial-frequency domain integration according to claim 2, characterized in that, The step of performing a two-dimensional discrete-time Fourier transform on the first logarithmic polar coordinate representation includes: A two-dimensional discrete-time Fourier transform is performed on the first logarithmic polar coordinate representation according to the following formula (1): （1） in, The first spectrum diagram is represented by the The first logarithmic polar coordinates represent the coordinates in the graph. Pixel value at that location, Indicates radial distance. The angle of a pixel relative to the center of the first logarithmic polar coordinate representation graph, the The horizontal frequency variable in the frequency domain, the This represents the vertical frequency variable in the frequency domain. This represents a complex exponential basis function used to implement projection from the spatial domain to the frequency domain.

4. The space-based and airborne target characterization method based on spatial-frequency domain joint method according to claim 1, characterized in that, Determining the first spatial domain depth features based on the first image includes: The first image is input into a spatial domain feature extraction network to obtain the first spatial domain depth features output by the spatial domain feature extraction network.

5. The space-based and airborne target characterization method based on spatial-frequency domain joint method according to claim 1, characterized in that, Determining the first frequency domain depth feature based on the first amplitude spectrum includes: The first amplitude spectrum is input into the frequency domain feature extraction network to obtain the first frequency domain depth feature output by the frequency domain feature extraction network.

6. The space-based and airborne target characterization method based on spatial-frequency domain joint analysis according to any one of claims 1-5, characterized in that, The step of determining the target representation of the first image based on the first spatial domain depth features and the first frequency domain depth features includes: The first spatial domain depth feature and the first frequency domain depth feature are fused to obtain the target representation of the first image.

7. A space-based target characterization device based on spatial-frequency domain integration, characterized in that, include: The first determining module is used to determine the first amplitude spectrum of the first image acquired by space-based system and the second amplitude spectrum of the second image acquired by air-based system. The second determining module is used to determine a first spatial domain depth feature based on the first image, and to determine a second spatial domain depth feature based on the second image. The third determining module is used to determine a first frequency domain depth feature based on the first amplitude spectrum and to determine a second frequency domain depth feature based on the second amplitude spectrum. The fourth determining module is used to determine the target representation of the first image based on the first spatial domain depth features and the first frequency domain depth features, and to determine the target representation of the second image based on the second spatial domain depth features and the second frequency domain depth features. The target representation of the first image and the target representation of the second image are used to perform target association, matching or collaborative identification of the space-based and air-based systems.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the space-based and air-based target characterization method based on the joint space-frequency domain as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the space-based and air-based target characterization method based on the joint space-frequency domain as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the space-based and air-based target characterization method based on the joint space-frequency domain as described in any one of claims 1 to 6.

Images