A PS point selection method and system based on multi-sensor information fusion

By using multi-sensor information fusion and dynamic threshold adjustment, the accuracy and reliability issues of PS point selection in complex environments were solved, and high-precision PS point selection in complex scenarios was achieved.

CN123112907APending Publication Date: 2026-11-17CHINA RAILWAY DESIGN GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611275342.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In complex environments, the accuracy and reliability of PS point selection are affected by obstructions, environmental changes, and signal interference, making accurate identification and selection difficult.

Method used

By fusing information from multiple sensors, combining optical images, radar images, humidity and temperature information, and dynamically adjusting amplitude deviation and coherence threshold, feature points are selected accurately using multi-scale depth feature maps and nonlinear diffusion scale spatial filtering.

Benefits of technology

It improves the stability and reliability of PS point selection, adapts to complex scene changes, enhances detection accuracy and reliability, and eliminates the influence of distance and environmental interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123112907A_ABST
    Figure CN123112907A_ABST
Patent Text Reader

Abstract

This invention proposes a PS point selection method and system based on multi-sensor information fusion, relating to the field of image processing technology. The method includes: acquiring multi-source detection information within the target area; performing difference comparison and feature enhancement recognition on adjacent image frames in optical image information to obtain an initial scene target, and registering the initial scene target to obtain a first candidate feature point set; adjusting a preset amplitude deviation threshold and a preset coherence threshold based on prior information corresponding to humidity and temperature information to obtain a target amplitude deviation threshold and a target coherence threshold, and adjusting the amplitude and phase of each feature point in different distance intervals to obtain a second candidate feature point set; identifying feature points in the second candidate feature point set that have an amplitude greater than the average amplitude of the radar image information as non-noise points, and identifying feature points among the non-noise points that have an amplitude deviation greater than the target amplitude deviation threshold and a coherence threshold as target PS points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a PS point selection method and system based on multi-sensor information fusion. Background Technology

[0002] PS-InSAR (Permanent Scatterer Interferometric Synthetic Aperture Radar) technology identifies stable targets (PS points, permanent scatterer points) on the Earth's surface that maintain high coherence over long periods. By utilizing the phase difference information from multi-temporal SAR images, it can achieve millimeter-level monitoring of surface deformation. PS points are typically composed of ground features with strong and stable radar scattering characteristics that do not change over time, such as building corners, bridges, and exposed rocks. They maintain a high signal-to-noise ratio in phase measurements and are less affected by temporal and baseline decoherence.

[0003] However, the structure of PS points is not static. Changes such as the stacking of construction materials or the movement of temporary support structures can alter the position or characteristics of the target scatterer, thus reducing coherence. Furthermore, obstructions such as buildings and machinery can cause multiple scatterings of the signal, leading to decreased image resolution and affecting the backscattering coefficient, making PS point identification more difficult. In addition, the complex and diverse environmental conditions—surface, soil, water, and vegetation—can all affect the reflection and propagation of microwave signals, impacting the accuracy and reliability of PS point selection. Summary of the Invention

[0004] In view of this, the present invention proposes a PS point selection method and system based on multi-sensor information fusion.

[0005] The technical solution of this invention is implemented as follows: The first aspect of this invention provides a PS point selection method based on multi-sensor information fusion, comprising: Acquire multi-source detection information within the target area to be tested; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information; Difference comparison and feature enhancement recognition are performed on adjacent image frames in the optical image information to obtain an initial scene target, and the radar image information and the optical image information are registered using the initial scene target to obtain a first candidate feature point set; Based on the prior information corresponding to the humidity information and the temperature information, the preset amplitude deviation threshold and the preset coherence threshold are adjusted to obtain the target amplitude deviation threshold and the target coherence threshold. The amplitude and phase of each feature point in the first candidate feature point set in different distance intervals are adjusted to obtain the second candidate feature point set. Feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information are determined as non-noise points, and feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold are determined as target PS points.

[0006] Based on the above technical solutions, preferably, the acquisition of multi-source detection information within the target area to be tested includes: Pulse compression technology is used to window and perform fast Fourier transform processing on radar electromagnetic wave signals to obtain scene target images at different distances; digital beamforming technology is used in conjunction with the uniform linear array characteristics of radar to obtain scene target images from different azimuths.

[0007] Based on the above technical solutions, preferably, the step of performing difference comparison and feature enhancement recognition on adjacent image frames in the optical image information to obtain the initial scene target includes: Adjacent image frames in the optical image information are input into a convolutional encoder with shared weights to obtain multi-scale depth feature maps, and the multi-scale depth feature maps are differentially processed to obtain difference feature maps. The differential feature map and the original feature map corresponding to the adjacent image frame are channel concatenated and weighted to determine the contours of non-scene targets that change in the adjacent image frames and inferior scene targets with stability below a preset threshold. The outlines of non-scene targets and inferior scene targets are removed from the optical image information to obtain the initial scene targets.

[0008] Based on the above technical solutions, preferably, the step of registering the radar image information and the optical image information using the initial scene target to obtain the first candidate feature point set includes: Nonlinear diffusion scale spaces are constructed for the radar image information and the optical image information respectively, and stable feature points are selected in the nonlinear diffusion scale space. The gradient direction and histogram description of the stable feature points are obtained, and initial matching is performed by combining the nearest neighbor distance ratio to obtain an initial candidate feature point set; The first candidate feature point set is obtained by removing erroneous matching point pairs from the initial candidate feature point set using a random sampling consensus algorithm.

[0009] Based on the above technical solutions, preferably, adjusting the preset amplitude deviation threshold and preset coherence threshold based on the prior information corresponding to the humidity information and the temperature information to obtain the target amplitude deviation threshold and target coherence threshold includes: Based on the humidity information, determine the absolute value of the humidity change rate within the preset collection period; If the absolute value of the humidity change rate is greater than the first threshold, both the target amplitude deviation threshold and the target coherence threshold are adjusted to 0. When the absolute value of the humidity change rate is greater than the second threshold but not greater than the first threshold, the product of the amplitude deviation threshold coefficient within the first preset range and the preset amplitude deviation threshold is determined as the target amplitude deviation threshold, and the product of the coherence threshold coefficient within the second preset range and the preset coherence threshold is determined as the target coherence threshold; wherein, the amplitude deviation threshold coefficient changes linearly with the humidity change rate.

[0010] Based on the above technical solutions, preferably, adjusting the preset amplitude deviation threshold and preset coherence threshold based on the prior information corresponding to the humidity information and the temperature information to obtain the target amplitude deviation threshold and target coherence threshold includes: Based on the temperature information, determine the maximum temperature difference within the preset collection time; When the maximum temperature difference is greater than the third threshold, the coherence threshold coefficient is determined based on the refractive index corresponding to the current temperature and the refractive index corresponding to the highest temperature, and the product of the coherence threshold coefficient and the preset coherence threshold is determined as the target coherence threshold.

[0011] Based on the above technical solution, preferably, the step of adjusting the amplitude and phase of each feature point in the first candidate feature point set across different distance intervals to obtain the second candidate feature point set includes: Based on the humidity information and the temperature information, the average humidity and average temperature within a preset collection period are determined, and the distinguishing distance segment is determined based on the average humidity and the average temperature. When the radar's farthest observation distance is greater than the distinguishing distance segment, the radar's observation distance is divided into multiple observation segments using the distinguishing distance segment. Within each observation segment, candidate feature points with the highest amplitude are selected at least a predetermined proportion, and the target feature point with the smallest phase standard deviation is determined from the candidate feature points. Based on the amplitude multiple change and phase change of the target feature point in adjacent images, the amplitude and phase of all feature points in each observation segment are adjusted to obtain a second candidate feature point set.

[0012] More preferably, a second aspect of the present invention provides a PS point selection system based on multi-sensor information fusion, comprising: an information acquisition module, a feature registration module, a threshold adjustment module, and a target determination module; wherein, The information acquisition module is configured to acquire multi-source detection information within the target area to be measured; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information; The feature registration module is configured to perform difference comparison and feature enhancement recognition on adjacent image frames in the optical image information to obtain an initial scene target, and to register the radar image information and the optical image information using the initial scene target to obtain a first candidate feature point set. The threshold adjustment module is configured to adjust a preset amplitude deviation threshold and a preset coherence threshold based on prior information corresponding to the humidity information and the temperature information to obtain a target amplitude deviation threshold and a target coherence threshold, and to adjust the amplitude and phase of each feature point in the first candidate feature point set in different distance intervals to obtain a second candidate feature point set. The target determination module is configured to determine feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information as non-noise points, and to determine feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold as target PS points.

[0013] More preferably, a third aspect of the present invention provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the PS point selection method based on multi-sensor information fusion as described in the first aspect.

[0014] More preferably, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the PS point selection method based on multi-sensor information fusion as described in the first aspect.

[0015] The PS point selection method and system based on multi-sensor information fusion of the present invention have the following advantages over the prior art: 1. By combining adjacent frame difference comparison with feature enhancement, the initial scene target is accurately extracted. Using the initial scene target as a control point, pixel-level alignment of optical and radar images is achieved through geometric correction, overcoming the feature loss problem caused by occlusion or environmental interference in a single sensor. Furthermore, based on humidity and temperature information, the amplitude deviation threshold and coherence threshold are dynamically adjusted. For signal attenuation differences in different distance ranges, amplitude normalization and phase correction are used to eliminate the influence of distance on feature point stability. This achieves a leap from static threshold to environmentally adaptive PS point selection, significantly improving the stability and reliability of PS point selection in complex scenes.

[0016] 2. By using dual environmental parameters of humidity change rate and maximum temperature difference, the amplitude deviation threshold and coherence threshold are adaptively and dynamically adjusted, avoiding the failure of fixed thresholds in high humidity and drastic temperature change environments. This improves the detection stability under complex working conditions, ensures the detection accuracy when the environment is stable, and can also adaptively tolerate errors when temperature and humidity fluctuate drastically, thereby improving the accuracy and reliability of subsequent PS selection.

[0017] 3. Based on humidity and temperature information, determine the distinguishing distance segments, and select the target feature points with the smallest amplitude standard deviation within each observation segment. Based on the amplitude multiple change and phase change of the target feature points in adjacent images, adjust the amplitude and phase of all feature points within each observation segment to eliminate the influence of distance on the stability of feature points and ensure the uniformity of the second candidate feature point set across the entire scene. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a PS point selection method based on multi-sensor information fusion provided in an embodiment of the present invention; Figure 2 A schematic diagram of a PS point selection system based on multi-sensor information fusion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0021] In some embodiments, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a PS point selection method based on multi-sensor information fusion provided by an embodiment of the present invention; the PS point selection method based on multi-sensor information fusion provided by the present invention includes: S110, acquire multi-source detection information within the target area to be measured; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information.

[0022] In this embodiment, an embedded camera is used to acquire optical images, ensuring that the camera and ground-based radar are always facing the same direction. Embedding reduces the angular difference between the camera and radar, preventing severe camera shake caused by adverse weather conditions. The ground-based radar acquires raw echo signals to form radar image information, from which target amplitude and phase information are extracted. Temperature and humidity sensors continuously collect temperature and humidity information. These sensors are deployed separately, away from heat sources such as radar and power sources, and from areas where moisture accumulates, and their acquisition frequency is higher than the radar image acquisition frequency.

[0023] In some embodiments, acquiring multi-source detection information within the target area to be tested includes: Pulse compression technology is used to window and perform fast Fourier transform processing on radar electromagnetic wave signals to obtain scene target images at different distances; digital beamforming technology is used in conjunction with the uniform linear array characteristics of radar to obtain scene target images from different azimuths.

[0024] Ground-based radar continuously transmits and receives electromagnetic waves to acquire the raw echo signal of a target. Information such as the target's range, azimuth, and velocity is hidden within the raw echo signal and requires a series of signal processing steps to extract this information. Pulse compression technology is used to resolve the conflict between illumination distance and range resolution; generally, the longer the illumination distance, the lower the range resolution. Ground-based radar uses a linear frequency modulated continuous wave system, and the result of its received signal after frequency mixing and beat differentiation exhibits the following relationship: ; in, For frequency, This represents the distance between a stationary target and the radar. It is a fixed constant coefficient.

[0025] After windowing and fast Fourier transform processing, the raw echo data can distinguish targets in different range cells, achieving a range resolution of [missing information]. , At the speed of light, This refers to the bandwidth of the transmitted signal.

[0026] After the above processing, targets at different range units are located at different frequency points. The magnitude of the corresponding value at each frequency point reflects the level of the target's reflected energy. However, targets at the same range but in different azimuths will be represented in the same range unit and cannot be separated solely by pulse compression. To distinguish targets in different azimuths, digital beamforming technology combined with the uniform linear array characteristics of the radar is used to acquire scene target images in different azimuths, i.e.: ; In the formula, This is a matrix composed of pulse compression results from different transmit and receive channels. The guiding matrix is ​​composed of azimuth angles. And the spacing between antenna array elements is determined. tx Characterizes the launch channel, rx Characterizes the receiving channel.

[0027] Matrix results For radar complex scattering images, to simplify the radar image expression, each pixel in the image can be represented by the following formula: ; In the formula, This represents the pixel amplitude, reflecting the target's backscattering characteristics. The pixel phase value reflects the distance between the target and the radar. The imaginary unit, row Indicates line number, col Represents the column number.

[0028] S120: Perform difference comparison and feature enhancement recognition on adjacent image frames in the optical image information to obtain the initial scene target, and use the initial scene target to register the radar image information and the optical image information to obtain the first candidate feature point set.

[0029] In some embodiments, difference comparison and feature enhancement recognition are performed on adjacent image frames in optical image information to obtain an initial scene target, including: Adjacent image frames in the optical image information are input into a convolutional encoder with shared weights to obtain multi-scale depth feature maps, and the multi-scale depth feature maps are differentially processed to obtain difference feature maps. The differential feature map and the original feature map corresponding to the adjacent image frame are channel concatenated and weighted to determine the contours of non-scene targets that change in adjacent image frames and inferior scene targets with stability below a preset threshold. The outlines of non-scene targets and inferior scene targets are removed from the optical image information to obtain the initial scene targets.

[0030] In this embodiment, adjacent image frames in the optical image information are input into a convolutional encoder with shared weights to obtain multi-scale depth feature maps. The two feature maps are then differentially processed to obtain a difference feature map. The difference feature map and the original feature map are concatenated and input into a fusion module that can weighted enhance the difference regions, suppressing misjudgments caused by environmental interference such as lighting, and highlighting the true change areas of the target. The enhanced difference features are then passed through a decoder and a pixel-level classification layer to output a change probability map. Thresholding and merging of adjacent change regions on the change probability map extracts the boundary or contour of the changed target, yielding the initial scene target. The preset threshold can be determined by the change in the center point position of the scene target in consecutive frames or the degree of overlap of the bounding boxes of the scene target in consecutive frames. The degree of overlap can be obtained by calculating the intersection-union ratio (IU). It should be noted that the convolutional encoder can be a residual backbone network, such as ResNet-18, ResNet-50, ResNet-101, etc., or a convolutional encoder equipped with a feature pyramid (FPN), or an improved convolutional encoder incorporating a deformable convolutional network (DCN). No specific limitations are imposed on this.

[0031] In some embodiments, the radar image information and optical image information are registered using the initial scene target to obtain a first candidate feature point set, including: Nonlinear diffusion scale spaces were constructed for radar image information and optical image information respectively, and stable feature points were selected in the nonlinear diffusion scale spaces. Obtain the gradient direction and histogram description of stable feature points, and perform initial matching by combining the nearest neighbor distance ratio to obtain an initial candidate feature point set; The first candidate feature point set is obtained by removing erroneous matching point pairs from the initial candidate feature point set using a random sampling consensus algorithm.

[0032] In this embodiment, nonlinear diffusion scale spaces are constructed for radar image information and optical image information respectively to suppress scattering noise in radar images and preserve structural edges, such as the basis TV total variation diffusion scale space and the structure tensor-guided anisotropic diffusion scale space. Stable feature points are monitored in the scale space to ensure that stable feature points are repeatable in heterogeneous images. After normalization processing of radar signals and optical images, the gradient direction and histogram description of stable feature points are calculated, and initial matching is performed using the nearest neighbor distance ratio. A consensus algorithm based on random sampling is used to eliminate erroneous matching point pairs, and the scene and radar image are registered based on the remaining correct matching points to obtain the first candidate feature point set. Here, stable feature points are characterized by being detectable at multiple continuous scale levels in the nonlinear diffusion scale space, having a spatial coordinate offset of key points less than a preset position deviation threshold, and maintaining consistent local neighborhood structural features.

[0033] S130, based on the prior information corresponding to humidity information and temperature information, adjust the preset amplitude deviation threshold and preset coherence threshold to obtain the target amplitude deviation threshold and target coherence threshold, and adjust the amplitude and phase of each feature point in the first candidate feature point set in different distance intervals to obtain the second candidate feature point set.

[0034] Target amplitude deviation threshold Relevance threshold to target They can be represented as follows: ; in, and These are the amplitude deviation threshold coefficient and the coherence threshold coefficient, respectively. and The preset amplitude deviation threshold and preset coherence threshold are determined empirically.

[0035] In some embodiments, adjusting a preset amplitude deviation threshold and a preset coherence threshold based on prior information corresponding to humidity and temperature information to obtain a target amplitude deviation threshold and a target coherence threshold includes: Determine the absolute value of the humidity change rate within a preset collection period based on humidity information; If the absolute value of the humidity change rate is greater than the first threshold, both the target amplitude deviation threshold and the target coherence threshold will be adjusted to 0. When the absolute value of the humidity change rate is greater than the second threshold but not greater than the first threshold, the product of the amplitude deviation threshold coefficient within the first preset range and the preset amplitude deviation threshold is determined as the target amplitude deviation threshold, and the product of the coherence threshold coefficient within the second preset range and the preset coherence threshold is determined as the target coherence threshold; wherein, the amplitude deviation threshold coefficient changes linearly with the humidity change rate.

[0036] In this embodiment, humidity information is the fundamental control factor, and temperature information is the secondary correction factor. These are not two independent, parallel, and unrelated sets of coefficients, but rather are superimposed in a sequential calculation relationship. Humidity information within the acquisition time of the most recent multiple images is obtained, and the time-humidity curve is reconstructed using an interpolation algorithm. The absolute value of the humidity change rate is calculated. If the absolute value of the humidity change rate is greater than a first threshold, such as 2% per minute, it indicates that atmospheric fluctuations are very drastic during this period, which will introduce a large residual phase error during atmospheric correction. In this case, it is best not to calculate the deformation value and set the target amplitude deviation threshold. Relevance threshold to target All values ​​are adjusted to 0. If the absolute value of the humidity change rate is greater than the second threshold but not greater than the first threshold, such as when the mean is within the range of 0.4% to 2%, the amplitude deviation threshold coefficient and the coherence threshold coefficient can be selected within the ranges of (1,2) and [0.6,1), respectively. Due to the complexity of the water vapor attenuation effect, the amplitude deviation threshold coefficient is selected linearly according to the humidity change rate; that is, the higher the humidity change rate, the larger the amplitude deviation threshold coefficient. The faster the humidity changes, the more uneven the spatial-temporal distribution of water vapor concentration, and the larger the normal fluctuation range of the signal amplitude caused by water vapor disturbance. Therefore, the amplitude deviation threshold coefficient is adjusted linearly and positively with the humidity change rate.

[0037] In some embodiments, adjusting a preset amplitude deviation threshold and a preset coherence threshold based on prior information corresponding to humidity and temperature information to obtain a target amplitude deviation threshold and a target coherence threshold includes: The maximum temperature difference within a preset collection period is determined based on temperature information; When the maximum temperature difference is greater than the third threshold, the coherence threshold coefficient is determined based on the refractive index corresponding to the current temperature and the refractive index corresponding to the highest temperature, and the product of the coherence threshold coefficient and the preset coherence threshold is determined as the target coherence threshold.

[0038] By studying the phase change law of humidity gradient through experiments, it can be found that the phase change is greater at high temperature for the same humidity value. Therefore, the higher the average temperature within a time period, the greater the change in coherence threshold coefficient. The coherence threshold coefficient is determined according to the refractive index change law.

[0039] coherence threshold coefficient It can be represented as: ; in, The rate of change of refraction at the current temperature. The rate of change of refractive index at the highest temperature. The length of the range of coherence threshold coefficient variation can be determined by water vapor microwave attenuation simulation experiment or by atmospheric refractive index temperature simulation experiment.

[0040] The magnitude of the target amplitude is affected by the backscattering coefficient. When the target structure undergoes slight changes due to temperature variations, the backscattering coefficient of common scene targets, such as bridges, concrete slopes, and buildings, will also change accordingly. Therefore, for targets susceptible to temperature effects, the amplitude deviation threshold needs to be dynamically adjusted. The threshold coefficient is generally determined based on experience, while targets less affected by temperature effects use a default amplitude deviation threshold, i.e., the amplitude deviation coefficient is 1.

[0041] In some embodiments, the amplitude and phase of each feature point in the first candidate feature point set with different distance intervals are adjusted to obtain a second candidate feature point set, including: Based on humidity and temperature information, the average humidity and average temperature within the preset collection period are determined, and the distinguishing distance segments are determined based on the average humidity and average temperature. When the radar's farthest observation range is greater than the distinguishing range segment, the radar's observation range is divided into multiple observation segments using the distinguishing range segment. Within each observation segment, candidate feature points with the highest amplitudes, at least a predetermined proportion, are selected, and the target feature point with the smallest phase standard deviation is determined from the candidate feature points. Based on the amplitude and phase changes of the target feature points in adjacent images, the amplitude and phase of all feature points in each observation segment are adjusted to obtain the second candidate feature point set.

[0042] In this embodiment, considering that when the temperature or humidity in the environment changes significantly, the phase change differences introduced by different range units are more pronounced. Near-range targets exhibit smaller amplitude and phase changes, while far-range targets show larger amplitude and phase changes. Therefore, using the same coherence threshold or amplitude deviation threshold, far more PS points are selected for near-range targets than for far-range targets. Thus, the distinguishing range segment is determined based on the average temperature and humidity within the current time period; the larger the average temperature or humidity, the smaller the distinguishing range value. Here, considering that changes in atmospheric water vapor content directly absorb and scatter microwave signals, causing significant radar phase decoherence, this is the primary factor affecting PS point stability; temperature only causes a slight phase shift by changing the atmospheric refractive index, and is a secondary interference factor. Therefore, humidity has a greater weight than temperature. The TOP algorithm can be used to select the optimal point within the range segment as the control point. When the radar's farthest observation distance is less than the distinguishing range segment, no adjustment is made to the phase and amplitude of the radar image. When the farthest observation distance is greater than the distinguishing range segment, an optimal target point needs to be extracted at each interval of the distinguishing range segment's distance value to adjust the radar pixel information.

[0043] The selection principle for the optimal point in each range segment is as follows: Sort the target amplitude within each range segment, ensuring that the operations between range segments are independent; calculate the phase stability of the top 1% of amplitude points within each range segment. For Gaussian distributed variables, i.e., the phase observations corresponding to the top 1% of amplitude points within the same range segment, their standard deviation can be used as the stability criterion; find the minimum standard deviation point, and extract the range segment, row and column number, and amplitude and phase values ​​corresponding to the minimum standard deviation point; based on the amplitude multiple change and phase change of the optimal point in different range segments across two frames of data, adjust the amplitude and phase of all points within each range segment segment by segment. For example, if the amplitude change of the optimal point in a certain range segment is 1.1 times its original value, then the amplitude of all points within that range segment is multiplied by a coefficient of 1.1; if the phase changes... If the phase changes, then the phase of all points in the current radar image within the range segment is reduced by this change value; that is, the corrected phase = the phase of all points in the current radar image - .

[0044] S140, the feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information are determined as non-noise points, and the feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold are determined as target PS points.

[0045] In this embodiment, the amplitude deviation value The calculation method is as follows: ; in, The standard deviation of the amplitude. This represents the average amplitude.

[0046] The formula for calculating coherence value is as follows: ; in, The main image signal, As auxiliary image signal, To obtain the conjugate operation, These represent the number of rows and columns in the image coherence calculation window. When calculating coherence, the radar image is divided into multiple coherence windows with overlapping regions. The number of rows and columns in each window is generally odd, indicating a center point. It should be noted that the average amplitude can be calculated as the arithmetic mean of the amplitudes of all pixels in the second candidate feature point set. The selection logic is based on the fact that radar noise amplitudes are generally lower than the global average amplitude of the image; therefore, only feature points are retained as valid non-noise points. A target PS point simultaneously satisfies both amplitude deviation and coherence thresholds, indicating that the point has both temporal scattering and phase stability, meeting the permanent scatterer selection criteria. The amplitude deviation value characterizes the long-term backscattering fluctuation of the feature point; the smaller the value, the more stable the scattering. The coherence value characterizes the phase synchronization degree between two radar frames; the closer to 1, the stronger the phase stability. Through joint selection using these dual indicators, unstable ground features with large scattering fluctuations and phase incoherence can be eliminated, accurately extracting qualified PS points.

[0047] When unstable targets such as trees and vehicles are selected as PS points, prior information is used to eliminate them, reducing the risk of misselection caused by environmental and human factors when the radar sensor detects PS points.

[0048] In some embodiments, please refer to Figure 2 , Figure 2This is a schematic diagram of a PS point selection system based on multi-sensor information fusion provided in an embodiment of the present invention. The present invention provides a PS point selection system 200 based on multi-sensor information fusion, comprising: an information acquisition module 210, a feature registration module 220, a threshold adjustment module 230, and a target determination module 240; wherein, The information acquisition module 210 is configured to acquire multi-source detection information within the target area to be measured; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information; The feature registration module 220 is configured to perform difference comparison and feature enhancement recognition on adjacent image frames in optical image information to obtain an initial scene target, and to register the radar image information and optical image information using the initial scene target to obtain a first candidate feature point set. The threshold adjustment module 230 is configured to adjust the preset amplitude deviation threshold and the preset coherence threshold based on the prior information corresponding to the humidity information and temperature information to obtain the target amplitude deviation threshold and the target coherence threshold, and to adjust the amplitude and phase of each feature point in the first candidate feature point set in different distance intervals to obtain the second candidate feature point set. The target determination module 240 is configured to determine the feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information as non-noise points, and to determine the feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold as target PS points.

[0049] In some embodiments, the information acquisition module 210 is specifically configured as follows: Pulse compression technology is used to window and perform fast Fourier transform processing on radar electromagnetic wave signals to obtain scene target images at different distances; digital beamforming technology is used in conjunction with the uniform linear array characteristics of radar to obtain scene target images from different azimuths.

[0050] In some embodiments, the feature registration module 220 is specifically configured as follows: Adjacent image frames in the optical image information are input into a convolutional encoder with shared weights to obtain multi-scale depth feature maps, and the multi-scale depth feature maps are differentially processed to obtain difference feature maps. The differential feature map and the original feature map corresponding to the adjacent image frame are channel concatenated and weighted to determine the contours of non-scene targets that change in adjacent image frames and inferior scene targets with stability below a preset threshold. The outlines of non-scene targets and inferior scene targets are removed from the optical image information to obtain the initial scene targets.

[0051] In some embodiments, the feature registration module 220 is specifically configured as follows: Nonlinear diffusion scale spaces were constructed for radar image information and optical image information respectively, and stable feature points were selected in the nonlinear diffusion scale spaces. Obtain the gradient direction and histogram description of stable feature points, and perform initial matching by combining the nearest neighbor distance ratio to obtain an initial candidate feature point set; The first candidate feature point set is obtained by removing erroneous matching point pairs from the initial candidate feature point set using a random sampling consensus algorithm.

[0052] In some embodiments, the threshold adjustment module 230 is specifically configured as follows: Determine the absolute value of the humidity change rate within a preset collection period based on humidity information; If the absolute value of the humidity change rate is greater than the first threshold, both the target amplitude deviation threshold and the target coherence threshold will be adjusted to 0. When the absolute value of the humidity change rate is greater than the second threshold but not greater than the first threshold, the product of the amplitude deviation threshold coefficient within the first preset range and the preset amplitude deviation threshold is determined as the target amplitude deviation threshold, and the product of the coherence threshold coefficient within the second preset range and the preset coherence threshold is determined as the target coherence threshold; wherein, the amplitude deviation threshold coefficient changes linearly with the humidity change rate.

[0053] In some embodiments, the threshold adjustment module 230 is specifically configured as follows: The maximum temperature difference within a preset collection period is determined based on temperature information; When the maximum temperature difference is greater than the third threshold, the coherence threshold coefficient is determined based on the refractive index corresponding to the current temperature and the refractive index corresponding to the highest temperature, and the product of the coherence threshold coefficient and the preset coherence threshold is determined as the target coherence threshold.

[0054] In some embodiments, the threshold adjustment module 230 is further configured to: Based on humidity and temperature information, the average humidity and average temperature within the preset collection period are determined, and the distinguishing distance segments are determined based on the average humidity and average temperature. When the radar's farthest observation range is greater than the distinguishing range segment, the radar's observation range is divided into multiple observation segments using the distinguishing range segment. Within each observation segment, candidate feature points with the highest amplitudes, at least a predetermined proportion, are selected, and the target feature point with the smallest phase standard deviation is determined from the candidate feature points. Based on the amplitude and phase changes of the target feature points in adjacent images, the amplitude and phase of all feature points in each observation segment are adjusted to obtain the second candidate feature point set.

[0055] It should be noted that the PS point selection system based on multi-sensor information fusion provided in this application embodiment and the PS point selection method based on multi-sensor information fusion provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned PS point selection method based on multi-sensor information fusion, and the repeated parts will not be described again.

[0056] In some embodiments, please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 300 provided in this embodiment includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program, when executed by the processor, implements the aforementioned PS point selection method based on multi-sensor information fusion.

[0057] Specifically, processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. Processor 310 may also include onboard memory for caching purposes. Processor 310 may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0058] The memory 320 may be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, the memory 320 may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, apparatuses, or propagation media. Specific examples of the memory 320 include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and may also be random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0059] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this program implements the aforementioned PS point selection method based on multi-sensor information fusion. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0060] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0061] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A PS point selection method based on multi-sensor information fusion, characterized in that, include: Acquire multi-source detection information within the target area to be tested; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information; Difference comparison and feature enhancement recognition are performed on adjacent image frames in the optical image information to obtain an initial scene target, and the radar image information and the optical image information are registered using the initial scene target to obtain a first candidate feature point set; Based on the prior information corresponding to the humidity information and the temperature information, the preset amplitude deviation threshold and the preset coherence threshold are adjusted to obtain the target amplitude deviation threshold and the target coherence threshold. The amplitude and phase of each feature point in the first candidate feature point set in different distance intervals are adjusted to obtain the second candidate feature point set. Feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information are determined as non-noise points, and feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold are determined as target PS points.

2. The PS point selection method based on multi-sensor information fusion according to claim 1, characterized in that, The acquisition of multi-source detection information within the target area includes: Pulse compression technology is used to perform windowing and fast Fourier transform processing on radar electromagnetic wave signals to obtain scene target images at different distances. Digital beamforming technology combined with the uniform linear array characteristics of radar is used to acquire scene target images from different azimuths.

3. The PS point selection method based on multi-sensor information fusion according to claim 1, characterized in that, The step of performing difference comparison and feature enhancement recognition on adjacent image frames in the optical image information to obtain the initial scene target includes: Adjacent image frames in the optical image information are input into a convolutional encoder with shared weights to obtain multi-scale depth feature maps, and the multi-scale depth feature maps are differentially processed to obtain difference feature maps. The differential feature map and the original feature map corresponding to the adjacent image frame are channel concatenated and weighted to determine the contours of non-scene targets that change in the adjacent image frames and inferior scene targets with stability below a preset threshold. The outlines of non-scene targets and inferior scene targets are removed from the optical image information to obtain the initial scene targets.

4. The PS point selection method based on multi-sensor information fusion according to claim 1, characterized in that, The process of registering the radar image information and the optical image information using the initial scene target to obtain a first candidate feature point set includes: Nonlinear diffusion scale spaces are constructed for the radar image information and the optical image information respectively, and stable feature points are selected in the nonlinear diffusion scale space. The gradient direction and histogram description of the stable feature points are obtained, and initial matching is performed by combining the nearest neighbor distance ratio to obtain an initial candidate feature point set; The first candidate feature point set is obtained by removing erroneous matching point pairs from the initial candidate feature point set using a random sampling consensus algorithm.

5. The PS point selection method based on multi-sensor information fusion according to claim 1, characterized in that, Based on prior information corresponding to the humidity and temperature information, the preset amplitude deviation threshold and preset coherence threshold are adjusted to obtain the target amplitude deviation threshold and target coherence threshold, including: Based on the humidity information, determine the absolute value of the humidity change rate within the preset collection period; If the absolute value of the humidity change rate is greater than the first threshold, both the target amplitude deviation threshold and the target coherence threshold are adjusted to 0. When the absolute value of the humidity change rate is greater than the second threshold and not greater than the first threshold, the product of the amplitude deviation threshold coefficient within the first preset range and the preset amplitude deviation threshold is determined as the target amplitude deviation threshold, and the product of the coherence threshold coefficient within the second preset range and the preset coherence threshold is determined as the target coherence threshold; wherein, the amplitude deviation threshold coefficient changes linearly with the humidity change rate.

6. The PS point selection method based on multi-sensor information fusion as described in claim 5, characterized in that, Based on prior information corresponding to the humidity and temperature information, the preset amplitude deviation threshold and preset coherence threshold are adjusted to obtain the target amplitude deviation threshold and target coherence threshold, including: Based on the temperature information, determine the maximum temperature difference within the preset collection time; When the maximum temperature difference is greater than the third threshold, the coherence threshold coefficient is determined based on the refractive index corresponding to the current temperature and the refractive index corresponding to the highest temperature, and the product of the coherence threshold coefficient and the preset coherence threshold is determined as the target coherence threshold.

7. The PS point selection method based on multi-sensor information fusion as described in claim 1, characterized in that, The step of adjusting the amplitude and phase of each feature point in the first candidate feature point set across different distance intervals to obtain the second candidate feature point set includes: Based on the humidity information and the temperature information, the average humidity and average temperature within the preset collection period are determined, and the distinguishing distance segment is determined based on the average humidity and the average temperature. When the radar's farthest observation distance is greater than the distinguishing distance segment, the radar's observation distance is divided into multiple observation segments using the distinguishing distance segment. Within each observation segment, candidate feature points with the highest amplitude are selected at least a predetermined proportion, and the target feature point with the smallest phase standard deviation is determined from the candidate feature points. Based on the amplitude multiple change and phase change of the target feature point in adjacent images, the amplitude and phase of all feature points in each observation segment are adjusted to obtain a second candidate feature point set.

8. A PS point selection system based on multi-sensor information fusion, characterized in that, include: The module comprises an information acquisition module, a feature registration module, a threshold adjustment module, and a target determination module; among which, The information acquisition module is configured to acquire multi-source detection information within the target area to be tested; the multi-source detection information includes optical image information, radar image information, humidity information, and temperature information; The feature registration module is configured to perform difference comparison and feature enhancement recognition on adjacent image frames in the optical image information to obtain an initial scene target, and to register the radar image information and the optical image information using the initial scene target to obtain a first candidate feature point set. The threshold adjustment module is configured to adjust the preset amplitude deviation threshold and the preset coherence threshold based on the prior information corresponding to the humidity information and the temperature information to obtain the target amplitude deviation threshold and the target coherence threshold, and to adjust the amplitude and phase of each feature point in the first candidate feature point set in different distance intervals to obtain the second candidate feature point set. The target determination module is configured to determine feature points in the second candidate feature point set that are greater than the average amplitude of the radar image information as non-noise points, and to determine feature points in the non-noise points that are greater than the target amplitude deviation threshold and the target coherence threshold as target PS points.

9. An electronic device comprising a processor and a memory; characterized in that, The memory stores a computer program, wherein the computer program, when executed by the processor, implements the PS point selection method based on multi-sensor information fusion as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the PS point selection method based on multi-sensor information fusion as described in any one of claims 1 to 7.