Data enhancement method for hyperspectral target detection and related device

Abstract

The invention discloses a data enhancement method for hyperspectral target detection and a related device, and relates to the field of target detection, and the method comprises the steps: precisely segmenting pixels into a target, a background and a boundary buffer region based on a hyperspectral image truth value annotation graph, calculating the average spectrum of the target and background regions, and the like. And constructing a distance gradient field by using the boundary buffer region, and calculating abundance values of the target and the background at set pixels according to the distance gradient field and a nonlinear abundance function. And determining an adaptive abundance matrix by physical constraints, and mixing the spectrums of the two regions according to abundance values by using a spectrum mixing model to generate mixed pixel simulation data. And finally performing enhancement processing on the data in a spatial domain, a spectral domain and a time phase domain. According to the invention, intelligent generation of high-quality hyperspectral training data is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and in particular to a data enhancement method for hyperspectral target detection and related devices. BACKGROUND

[0002] Hyperspectral remote sensing technology has shown great advantages in target detection field due to its rich spectral information and fine spectral resolution, and plays an important role in military reconnaissance, environmental monitoring, precision agriculture and other applications. Compared with traditional RGB images, hyperspectral images can provide hundreds of continuous spectral bands, so that target recognition is no longer dependent on spatial morphological features, but can use the unique spectral "fingerprint" information of the target for accurate identification.

[0003] However, the current development of hyperspectral target detection technology is facing a serious bottleneck of data scarcity. On the one hand, the cost of obtaining hyperspectral data is extremely high, which requires professional imaging equipment and complex data processing procedures; on the other hand, labeling high-quality hyperspectral target detection data sets requires a large amount of professional knowledge and manual cost, resulting in extremely limited labeled data available for training. This data scarcity problem is particularly prominent in small target detection scenarios, because small targets occupy a very small proportion in the image, and it is more difficult to obtain enough positive samples.

[0004] Existing data enhancement methods are mainly designed for visible light images, such as random rotation, flipping, scaling and other geometric transformations, as well as color jittering, noise addition and other pixel-level transformations. However, these methods have obvious deficiencies when directly applied to hyperspectral images: first, simple geometric transformations cannot fully utilize the spectral dimension information of hyperspectral data; second, traditional methods ignore the mixed pixel phenomenon commonly existing in hyperspectral imaging, i.e. a single pixel often contains spectral information of multiple ground objects; finally, existing methods lack consideration of the physical imaging mechanism of hyperspectral images, and the generated enhanced samples may violate the spectral physical constraints, resulting in a decrease in the quality of training data. Therefore, there is an urgent need for a data enhancement method specifically for hyperspectral target detection SUMMARY

[0005] The purpose of the present application is to provide a data enhancement method for hyperspectral target detection and related devices, which can effectively solve the problems of data scarcity in the prior art and the deficiencies of traditional data enhancement methods in the application of hyperspectral images.

[0006] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides a data enhancement method for hyperspectral target detection, comprising: Based on a ground truth label image of the hyperspectral image, an image pixel is accurately segmented into a target region, a background region, and a boundary buffer region; and the average spectrum, the spectral variance of the target region and the background region, and the spectral angle distance between them are calculated; the boundary buffer region is a region between the target region and the background region; Based on the boundary buffer region, a distance gradient field is constructed; According to the distance gradient field, based on a nonlinear abundance function, the target abundance value of the target region and the background region at a set pixel is calculated respectively; Based on the nonlinear abundance function, an adaptive abundance matrix for the physical mechanism of hyperspectral imaging is determined by taking a point spread function and an atmospheric scattering model as physical constraints; Based on the adaptive abundance matrix, the spectrum of the target region and the spectrum of the background region are mixed according to the abundance value; the mixing uses a spectral mixing model containing a nonlinear term and an interaction term to simulate the scattering phenomenon, and through a multi-scale target generation strategy and a background adaptive selection mechanism, mixed pixel simulation data with different scale targets and diverse backgrounds are generated; Based on the mixed pixel simulation data, the spatial domain, the spectral domain and the temporal domain are enhanced; the enhancement processing of the spatial domain includes simulating the elastic deformation of the geometric distortion of the hyperspectral imaging; the enhancement processing of the spectral domain includes adaptive noise addition based on the signal-to-noise ratio of the waveband and random masking of the spectral waveband; the enhancement processing of the temporal domain includes simulating seasonal changes and environmental condition disturbances.

[0007] Optionally, according to the distance gradient field, based on a nonlinear abundance function, the target abundance value of the target region and the background region at a set pixel is calculated respectively, specifically including: According to the formula , the target abundance value of the target region at pixel (i, j) is calculated; According to the formula , the target abundance value of the background region at pixel (i, j) is calculated; Wherein, The target abundance value at pixel (i, j) is represented as The amplitude coefficient of the exponential decay term is represented as The decay rate parameter is represented as The amplitude coefficient of the power function term is represented as The power function exponent is represented as The normalized distance is represented as The distance to the boundary buffer region is represented as The maximum influence radius is represented as The direction weight function is represented as The background abundance value at pixel (i, j) is represented as

[0008] Optionally, the formula expression of the point spread function is: ; wherein, is the PSF kernel function value at distance d, is the standard deviation parameter of the PSF, and d is the spatial distance, represents an exponential function, is the circular constant.

[0009] Optionally, the formula expression of the atmospheric scattering model is: ; wherein, is the scattering attenuation function at distance d, is the direct scattering coefficient, is the direct scattering attenuation rate, is the multiple scattering coefficient, is the multiple scattering attenuation exponent.

[0010] Optionally, the background adaptive selection mechanism is to calculate the matching degree of the target spectrum and each type of background spectrum in the background library; the matching degree is a weighted combination of the spectral angle distance, the spectral information divergence and the spectral correlation coefficient.

[0011] Optionally, the formula for calculating the matching degree is: ; wherein, is the matching degree index, is the spectral angle distance, is the spectral information divergence, is the spectral correlation coefficient, , , is the weight coefficient, is the average spectrum of the pth background, is the spectral reflectance; ; is the Kullback-Leibler divergence, , and are the normalized spectral probability distributions, respectively; , and are the mean values of the target spectrum and the background spectrum, respectively.

[0012] Optionally, the multi-scale target generation strategy specifically includes: According to the formula , a target scaling factor set is determined; in the formula, is the scaling factor set, and K is the number of scaling levels. scaling the target region according to a target scaling factor in the set of target scaling factors, to obtain a scaled target region ; wherein, is a target region of the kth scaling level, denotes a bilinear interpolation scaling function; the scaled abundance distribution function is calculated according to the formula ; the superscript (k) denotes parameters corresponding to the kth scaling level.

[0013] In a second aspect, the present application provides a data enhancement device for hyperspectral target detection, comprising: a region division module, configured to accurately divide image pixels into a target region, a background region and a boundary buffer region based on a ground truth label image of a hyperspectral image; and calculate the average spectrum, spectral variance and spectral angle distance between the target region and the background region; the boundary buffer region is a region between the target region and the background region; a distance gradient field construction module, configured to construct a distance gradient field based on the boundary buffer region; an abundance value calculation module, configured to calculate target abundance values of the target region and the background region at a set pixel based on a nonlinear abundance function according to the distance gradient field; a matrix construction module, configured to determine an adaptive abundance matrix for the physical mechanism of hyperspectral imaging based on the nonlinear abundance function, with a point spread function and an atmospheric scattering model as physical constraints; a mixing module, configured to mix the spectrum of the target region and the spectrum of the background region according to the abundance values based on the adaptive abundance matrix; the mixing uses a spectral mixing model containing nonlinear terms and interaction terms to simulate scattering phenomena, and generates mixed pixel simulation data with different scale targets and diverse backgrounds through a multi-scale target generation strategy and a background adaptive selection mechanism; an enhancement processing module, configured to perform enhancement processing on a spatial domain, a spectral domain and a temporal domain based on the mixed pixel simulation data; the enhancement processing of the spatial domain includes simulating elastic deformation of hyperspectral imaging geometric distortion; the enhancement processing of the spectral domain includes adaptive noise addition based on the signal-to-noise ratio of the waveband and spectral waveband random masking; the enhancement processing of the temporal domain includes simulating seasonal changes and environmental condition disturbances.

[0014] In a third aspect, the present application provides a computer device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the data enhancement method for hyperspectral target detection according to any one of the above.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the data enhancement method for hyperspectral target detection according to any one of the above aspects.

[0016] According to the embodiments provided in the present application, the following technical effects are disclosed: The present application provides a data enhancement method for hyperspectral target detection and related devices. For hyperspectral target detection, the image pixels are first accurately segmented into target, background and boundary buffer area based on the hyperspectral image ground truth label map, and the average spectrum, variance and spectral angle distance of the target and background areas are calculated. The distance gradient field is constructed based on the boundary buffer area, the target abundance value of the target area and the background area at the set pixel is calculated according to the nonlinear abundance function, the adaptive abundance matrix is determined by constraining the point spread function and the atmospheric scattering model, the target and background area spectrum are mixed according to the abundance value by using the specific spectral mixing model, the mixed pixel simulation data is generated by using the multi-scale target generation strategy and the background adaptive selection mechanism, and the spatial domain (elastic deformation simulating geometric distortion), the spectral domain (adaptive noise addition based on band signal-to-noise ratio and spectral band random shielding) and the temporal domain (simulating seasonal change and environmental condition disturbance) are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The application environment diagram of the data enhancement method for hyperspectral target detection according to an embodiment of the present application; Figure 2 The flowchart of the data enhancement method for hyperspectral target detection according to an embodiment of the present application; Figure 3 The functional module diagram of the data enhancement device for hyperspectral target detection according to an embodiment of the present application; Figure 4 The structural diagram of the computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0020] In recent years, some researchers have tried to introduce spectral mixture analysis theory into data augmentation, and generate new training samples through a linear spectral mixture model. However, these methods still have limitations: (1) the linear mixture assumption is too simplified and cannot accurately describe the complex nonlinear scattering effect; (2) the formation mechanism of target-background boundary region mixed pixels is not modeled in depth; (3) the variation law of point spread function corresponding to different scale targets is ignored.

[0021] Therefore, there is an urgent need for a data augmentation method specifically for hyperspectral target detection, which should: (1) fully consider the physical mechanism of hyperspectral imaging; (2) accurately model the formation process of target-background mixed pixels; (3) generate high-quality augmented samples that meet spectral physical constraints; (4) effectively expand small target detection training data and improve model generalization ability.

[0022] The purpose of the present application is to provide a data augmentation method for hyperspectral target detection and related devices, which realizes intelligent generation of high-quality hyperspectral training data by constructing an adaptive abundance matrix and a spectral mixture model under physical constraints, and provides an effective technical solution to solve the data scarcity problem in hyperspectral target detection.

[0023] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be described in more detail below with reference to the accompanying drawings and specific embodiments.

[0024] The data augmentation method for hyperspectral target detection provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 The terminal 102 communicates with the server 104 through a network. The data storage system can store the data required to be processed by the server 104. The data storage system can be separately arranged, or integrated on the server 104, or placed on the cloud or other servers.

[0025] The terminal 102 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by a single server or a server cluster composed of multiple servers, and can also be a cloud server.

[0026] In an exemplary embodiment, as shown in Figure 2 A data enhancement method for hyperspectral target detection is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, or both. In the embodiments of the present application, the method is applied to the server 104 in Figure 1 , which includes the following steps 201 to 206. Wherein: Step 201, based on the ground truth label map of the hyperspectral image, the image pixels are accurately segmented into target regions, background regions and boundary buffer regions; and the average spectrum, spectral variance and spectral angle distance between the target region and the background region are calculated; the boundary buffer region is a region between the target region and the background region; Step 202, based on the boundary buffer region, a distance gradient field is constructed; Step 203, according to the distance gradient field, based on a nonlinear abundance function, the target abundance value of the target region and the background region at a set pixel is calculated respectively; Step 204, based on the nonlinear abundance function, a point spread function and an atmospheric scattering model are used as physical constraints to determine an adaptive abundance matrix for the physical mechanism of hyperspectral imaging; Step 205, based on the adaptive abundance matrix, the spectrum of the target region and the spectrum of the background region are mixed according to the abundance value; the mixing uses a spectral mixing model containing nonlinear terms and interaction terms to simulate scattering phenomena, and through a multi-scale target generation strategy and a background adaptive selection mechanism, mixed pixel simulation data with different scale targets and diverse backgrounds are generated; Step 206, based on the mixed pixel simulation data, the spatial domain, the spectral domain and the temporal domain are enhanced; the enhancement of the spatial domain includes elastic deformation to simulate the geometric distortion of hyperspectral imaging; the enhancement of the spectral domain includes adaptive noise addition based on the signal-to-noise ratio of the waveband and random masking of the spectral waveband; the enhancement of the temporal domain includes simulation of seasonal changes and environmental condition disturbances.

[0027] In an exemplary embodiment, when steps 201-206 are executed, specifically as follows: Wherein, for training set data to realize multi-level target-background separation, specifically can be as follows: Based on the true value label map, the hyperspectral image is accurately separated from the target-background, and the basic parameters for subsequent abundance modeling are established. Mainly including three sub-steps: first, the target region, the background region and the boundary buffer region are accurately extracted by using the GT map; Then calculate the spectral feature statistics of each region, including average spectrum, spectral variance and target-background spectral difference measure; Finally, the boundary region is refined, and the distance gradient field and direction weight function are constructed, which provides geometric constraint conditions for modeling the spatial distribution characteristics of mixed pixels. The existing method only performs simple target-background binary classification separation, but the method in this embodiment establishes a three-level separation structure including the boundary buffer region, and introduces the direction weight function, which fully considers the spatial distribution characteristics of mixed pixels.

[0028] Specifically, based on the accurate segmentation of the GT image, it can be as follows: First, based on the ground truth (GT) of the hyperspectral image, the target region and the background region are accurately segmented. Let the hyperspectral image be Where M and N are the number of rows and columns of the image, respectively, and L is the number of spectral bands. The corresponding ground truth is Where the pixel value of 1 indicates the target region, and the pixel value of 0 indicates the background region. Through the GT map, the target region T and the pure background region B can be accurately extracted: .

[0029] In the formula, T represents the target pixel position set, B represents the background pixel position set, and (i,j) represents the spatial coordinates of the pixel.

[0030] In order to more accurately model the target-background transition region, define the target boundary buffer region M: ; Where, represents the minimum Euclidean distance of pixel to the target region T, is a preset buffer radius parameter, usually set to 2-5 pixels. The buffer region is used to model the target-background mixed pixels caused by the point spread function in the actual imaging process.

[0031] Specifically, the region spectral feature analysis can be as follows: The spectral feature statistics of each region obtained by separation are analyzed, which provides basic parameters for subsequent abundance modeling.

[0032] First, the average spectral feature vector of the target region is calculated: ; where, denotes the total number of pixels in the target region, denotes the spectral vector at position (i,j), is the average spectral feature of the target region.

[0033] Then the average spectral feature vector of the background region is calculated as: ; where, denotes the total number of pixels in the target region, is the average spectral feature of the background region.

[0034] To quantify the degree of spectral difference between the target and the background, the spectral angle distance (SAD) is defined as : ; where, denotes the inner product of vectors, denotes the norm of vectors. The larger the SAD value, the more significant the spectral difference between the target and the background.

[0035] Further, the spectral variability within the target region and the background region is calculated as: ; where, and denote the spectral variance of the target region and the background region, respectively, for subsequent noise modeling and quality control.

[0036] where, when performing adaptive abundance matrix modeling, it can be as follows: This step is in the construction of adaptive abundance matrix in line with the physical mechanism of hyperspectral imaging. Mainly includes three sub-steps: first, design a nonlinear abundance function considering distance attenuation and boundary diffusion effect; then introduce the point spread function and atmospheric scattering and other physical constraints, establish the diffusion attenuation model; finally, through regularization optimization and statistical constraints, the parameters of the abundance matrix are adaptively adjusted to ensure that the generated mixed pixels meet the spectral mixing rules under the real imaging conditions. The existing method mostly uses linear abundance allocation, the embodiment method designs an adaptive nonlinear abundance function, combined with the physical constraints of PSF and atmospheric scattering, realizes the abundance modeling in line with the imaging mechanism.

[0037] Specifically, the nonlinear abundance function design can be as follows: Based on the obtained distance gradient field, an adaptive nonlinear abundance function is designed. The abundance distribution function of the target region is defined as: ; where, denotes the target abundance value at pixel (i, j), is the amplitude coefficient of the exponential decay term , is the decay rate parameter , is the amplitude coefficient of the power function term 0.2, is the power function exponent , is the normalized distance defined in step 1.3, is the distance to the target boundary, is the maximum influence radius, is the directional weight function.

[0038] Correspondingly, the abundance distribution function of the background region is: ; wherein denotes the target abundance value at pixel (i, j), denotes the background abundance value at pixel (i, j). This constraint ensures that the sum of target and background abundance at each pixel position is 1, which meets the physical constraints of spectral mixture analysis.

[0039] To ensure the reasonableness of the abundance values, boundary constraints are imposed on the abundance function: ; wherein T is the target region set defined in step 1.

[0040] Specifically, when modeling the physical diffusion constraint, it can be as follows: To make the abundance distribution more consistent with the physical process of actual imaging, the constraints of Point Spread Function (PSF) and atmospheric scattering effect are introduced.

[0041] First, the diffusion kernel function considering the PSF effect is established: ; wherein is the PSF kernel function value at distance d, is the standard deviation parameter of PSF, usually taking a value of 0.5-1.5 pixels, d is the spatial distance, denotes the exponential function, is the circular constant.

[0042] Considering the multiple scattering effect of atmospheric scattering, a composite diffusion model is established: ; wherein is the scattering attenuation function at distance d, is the direct scattering coefficient ( ), is the direct scattering attenuation rate ( ), is the multiple scattering coefficient ( ), is the multiple scattering attenuation exponent ( ).

[0043] Combining the PSF and scattering effects, the comprehensive physical diffusion constraint function is obtained: ; where, represents the comprehensive diffusion weight at distance d.

[0044] The modified abundance function is: ; where, is the modified target abundance considering physical constraints.

[0045] where, when performing abundance matrix parameter optimization, it can be as follows: In order to ensure the statistical rationality and physical consistency of the abundance matrix, the parameter optimization objective function is established.

[0046] First, define the smoothness constraint of the abundance distribution: ; where, is the smoothness error term, and respectively represent the partial derivatives of the abundance function in i and j directions.

[0047] Define the statistical consistency constraint: ; where, is the statistical error term, |M| is the total number of pixels in the boundary region, is the expected average abundance value (usually set to 0.5).

[0048] Establish the overall optimization objective function: ; where, is the total loss function, and , are regularization weight parameters ( ), p represents the parameter set of the abundance function , is the prior value of the parameter.

[0049] Optimizing parameters by gradient descent method: where, denotes the parameter value at the t-th iteration, is the learning rate (usually takes the value 0.001-0.01), is the gradient of the loss function with respect to the parameters.

[0050] The optimized adaptive abundance matrix provides a mixed weight that meets the physical constraints for the entire data enhancement process, ensuring that the generated enhanced samples have good spectral authenticity and spatial consistency.

[0051] When performing mixing of the spectrum of the target region and the spectrum of the background region according to the abundance value based on the adaptive abundance matrix, the following can be performed: Based on the adaptive abundance matrix established in the foregoing, mixed pixel simulation that meets the physical mechanism of hyperspectral imaging is realized. It mainly includes three sub-steps: first, a spectral mixing model considering nonlinear scattering and multiple reflection effects is established, breaking through the limitations of the traditional linear mixing assumption; then a multi-scale target generation strategy is designed, and the corresponding point spread function variation law is established for different scale targets; finally, adaptive selection and intelligent matching of the background are realized to ensure reasonable combination of the target and the background in spectral characteristics, thereby generating high-quality enhanced training samples. The endmember mixing in the existing research is mainly linear model, and the method of the embodiment establishes a complex mixing model containing multiple nonlinear terms and interaction terms, which can more accurately describe the complex scattering phenomenon in actual imaging.

[0052] Specifically, the nonlinear spectral mixing modeling can be as follows: The traditional linear spectral mixing model cannot accurately describe the complex scattering phenomenon in actual imaging, and therefore a spectral mixing model considering nonlinear effects is established.

[0053] The basic nonlinear mixed spectral model is defined as: ; where, denotes the mixed spectral value of pixel (i, j) at wavelength , is the corrected target abundance, is the background abundance, is the spectral reflectance of the target at wavelength , is the spectral reflectance of the background at wavelength , is the n-th order nonlinear coefficient (usually , ), N is the maximum nonlinear order (usually 2-3), is the spectral noise term.

[0054] Considering multiple scattering effects, the interaction term is introduced: ; where, is the target-background interaction spectral term, is the interaction intensity coefficient ( ), denotes the square root function.

[0055] The complete mixed spectrum model is: ; In order to ensure the physical rationality of the spectral value, the mixed spectrum is constrained: ; where, and respectively represent the maximum and minimum value functions, which ensure that the spectral reflectance is in the range of [0, 1].

[0056] Specifically, the determination of the multi-scale target generation strategy can be as follows: For targets of different scales, the corresponding point spread function variation law and abundance distribution adjustment mechanism are established.

[0057] First, define the target scaling factor set: ; where, is the scaling factor set, and K is the number of scaling levels (K=6).

[0058] For the scaling factor , the corresponding PSF standard deviation adjustment is: ; where, is the PSF standard deviation corresponding to the kth scale, is the PSF standard deviation of the reference scale.

[0059] The scaled target region is redefined as: where, is the target region of the kth scaling level, denotes the bilinear interpolation scaling function.

[0060] The corresponding maximum influence radius adjustment is: ; where, is the maximum influence radius of the kth scale.

[0061] Recalculating the scaled abundance distribution: ; where the superscript (k) denotes the parameters corresponding to the kth scaling level, and each parameter is adaptively adjusted according to the scale: .

[0062] where, denotes the natural logarithm function.

[0063] where the background adaptive selection and intelligent matching (background adaptive selection mechanism) can be as follows: To ensure that the generated augmented samples have good diversity and authenticity, a background adaptive selection mechanism is established.

[0064] First, a background library classification system is established, and the background is divided into several categories according to spectral characteristics: ; where, is the set of background categories, P is the number of background categories, and common categories include vegetation, soil, water, artificial materials, etc.

[0065] Calculate the representative spectral features for each background category: where, is the average spectrum of the pth background, is the background pixel area belonging to the pth category, is the number of pixels in this area.

[0066] Define the target-background matching degree calculation method: ; where, is the matching degree index, is the spectral angle distance, is the spectral information divergence, is the spectral correlation coefficient, , , is the weight coefficient.

[0067] The spectral information divergence is defined as: where, is the Kullback-Leibler divergence: ; where, and are the normalized spectral probability distributions, X and Y are variables in the Kullback-Leibler divergence, for example X in S t Y is .

[0068] Specifically, the spectral correlation coefficient is defined as: ; in, and These are the mean values ​​of the target spectrum and the background spectrum, respectively.

[0069] Background selection based on matching degree: ; in, To select the optimal background category, This represents the parameter that minimizes the objective function.

[0070] Through the hybrid pixel simulation under the aforementioned physical constraints, high-quality enhanced samples were generated, laying a solid foundation for subsequent multi-dimensional data enhancement.

[0071] The data augmentation strategy based on spatial-spectral shearing, that is, when performing augmentation processing on the spatial domain, spectral domain, and temporal domain based on the hybrid pixel simulation data, can be specifically as follows: Building upon the aforementioned hybrid pixel simulation, synergistic enhancement of the spatial, spectral, and temporal domains is achieved. To simulate spectral mixing and variability in hyperspectral target detection under real-world conditions, a spatial domain enhancement method considering the geometric distortion characteristics of hyperspectral imaging is proposed. Then, a spectral domain enhancement strategy based on adaptive signal-to-noise ratio adjustment is designed, fully considering the noise distribution characteristics of hyperspectral data. Finally, the concept of temporal domain enhancement is introduced, expanding the temporal dimension diversity of the data by simulating seasonality and environmental condition changes. The core difference between this step and traditional data augmentation methods lies in fully utilizing the multi-dimensional characteristics and physical constraints of hyperspectral data.

[0072] 1) Geometric enhancement based on spatial domain: Traditional spatial domain enhancement is primarily designed for RGB images, neglecting the geometric characteristics of hyperspectral imaging. This method considers the scanning mechanism and geometric distortion features of hyperspectral imagers. Compared to traditional simple rotational transformations, this method takes into account the geometric characteristics of hyperspectral pushbroom imaging and establishes a geometric transformation model that better reflects actual imaging conditions.

[0073] First, perform basic geometric transformations, including rotation, mirroring, and translation: ; in, Rotation angle The subsequent hyperspectral image, Let be the rotation transformation matrix. , The mixed pixel image generated in step 3.

[0074] The rotation transformation matrix is defined as: ; where, and denote the cosine and sine functions, respectively.

[0075] The mirror transformation includes horizontal and vertical flips: ; where, and are the images after horizontal and vertical flips, and are the horizontal and vertical flip transformation matrices, respectively.

[0076] To simulate imaging geometric distortion, elastic deformation is introduced: ; where, is the image after elastic deformation, and are the perturbation quantities of spatial positions.

[0077] The perturbation quantities are generated by a Gaussian random field: .

[0078] where, and are independent Gaussian random fields, is a smoothing kernel, and * denotes the convolution operation.

[0079] Specifically, spectral domain enhancement can be as follows: The core purpose of spectral domain enhancement is to simulate various spectral response changes and error sources in hyperspectral imaging systems, including detector noise, atmospheric transmission effects, instrument spectral response function drift, etc. The innovation lies in the design of an adaptive spectral perturbation mechanism, which can dynamically adjust the noise intensity according to the signal-to-noise ratio characteristics of different wavebands, and process the waveband missing problem through spectral interpolation technology, better simulating actual observation conditions.

[0080] First, adaptive noise addition is implemented, and the noise intensity is dynamically adjusted according to the spectral signal-to-noise ratio: ; where, is the spectral value after adding noise to the lth waveband, is the adaptive noise standard deviation of the lth waveband, is the standard Gaussian noise.

[0081] The adaptive noise standard deviation is calculated as: ; where, is the target signal-to-noise ratio of the lth band, usually varying in the range of 20-40 dB.

[0082] Spectral band random masking is used to simulate the failure of some bands of the sensor: ; where, is the spectral value after masking processing, is the masking mask, denotes the interpolation function.

[0083] The masking mask is generated by Bernoulli distribution: ; where, is the masking probability, usually set to 0.05-0.15, denotes the Bernoulli distribution.

[0084] Spectral translation is used to simulate the instrument calibration error: ; where, is the spectral value after calibration error correction, is the calibration offset of the lth band.

[0085] The calibration offset is modeled as: ; where, is the amplitude coefficient, is the frequency parameter, is the phase offset, is the direct current offset, denotes the sine function.

[0086] where, the phase domain enhancement can be as follows: The phase domain enhancement aims to simulate the spectral characteristic changes of hyperspectral targets under different time conditions, and its physical basis comes from the time variability of the spectral characteristics of ground objects, including natural processes such as vegetation phenology change, seasonal fluctuation of soil moisture, daily change of atmospheric conditions. The innovation lies in the establishment of a seasonal change model based on cosine function and an environmental disturbance model based on first-order Markov process, which can more accurately reproduce the time evolution law of target spectrum and provide more abundant training samples for time-series hyperspectral target detection.

[0087] The seasonal spectral change is modeled as: ; in, The seasonal modulation spectrum at time t, The seasonal variation amplitude of the l-th band. It is an annual cycle. This represents the seasonal phase, where t is a time variable. \ represents the cosine function.

[0088] Environmental condition changes are modeled using random perturbations: ; in, To account for the final spectrum of environmental changes, For environmental disturbances, This represents an exponential function.

[0089] The environmental disturbance term is modeled as a first-order Markov process: ; in, The autocorrelation coefficient ( ), For environmental noise, This represents the variance of environmental noise.

[0090] Through the above multi-dimensional enhancement strategies, the hyperspectral target detection data was comprehensively expanded, significantly improving the diversity and quality of the training data and providing rich candidate samples for subsequent quality control.

[0091] Based on the same inventive concept, this application also provides a data augmentation device for hyperspectral target detection, which implements the data augmentation method for hyperspectral target detection described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the data augmentation device for hyperspectral target detection provided below can be found in the limitations of the data augmentation method for hyperspectral target detection described above, and will not be repeated here.

[0092] In one exemplary embodiment, such as Figure 3 As shown, a data enhancement device for hyperspectral target detection is provided, comprising: The region segmentation module 301 is used to accurately segment image pixels into target region, background region, and boundary buffer region based on the ground truth map of the hyperspectral image; and to calculate the average spectrum, spectral variance, and spectral angular distance between the target region and the background region; the boundary buffer region is the region located between the target region and the background region. The distance gradient field construction module 302 is used to construct a distance gradient field based on the boundary buffer region; an abundance value calculation module configured to calculate target abundance values of the target region and the background region at a set pixel based on a nonlinear abundance function according to the distance gradient field; a matrix construction module 303 configured to determine an adaptive abundance matrix for a physical mechanism of hyperspectral imaging based on the nonlinear abundance function, with a point spread function and an atmospheric scattering model as physical constraints; a mixing module 304 configured to mix the spectrum of the target region with the spectrum of the background region according to the abundance values based on the adaptive abundance matrix; the mixing uses a spectral mixing model containing nonlinear terms and interaction terms to simulate scattering phenomena, and generates mixed pixel simulation data with different scale targets and diverse backgrounds through a multi-scale target generation strategy and a background adaptive selection mechanism; an enhancement processing module 305 configured to perform enhancement processing on a spatial domain, a spectral domain and a temporal domain based on the mixed pixel simulation data; the enhancement processing on the spatial domain includes elastic deformation to simulate geometric distortion of hyperspectral imaging; the enhancement processing on the spectral domain includes adaptive noise addition based on a band signal-to-noise ratio and random masking of spectral bands; and the enhancement processing on the temporal domain includes simulation of seasonal changes and environmental condition disturbances.

[0093] In an exemplary embodiment, a computer device can be provided, which can be a server or a terminal, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a data enhancement method for hyperspectral target detection.

[0094] Those skilled in the art can understand that Figure 4 the structure shown in the above

[0095] In an example embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0096] In an example embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0097] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0098] It can be understood by those skilled in the art that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0099] The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0100] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0101] The principles and implementation modes of the present application are described by applying specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges will be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A data augmentation method for hyperspectral target detection, characterized in that, include: Based on the ground truth map of the hyperspectral image, the image pixels are accurately segmented into target region, background region and boundary buffer region; The average spectrum, spectral variance, and spectral angular distance between the target region and the background region are calculated; the boundary buffer region is the area located between the target region and the background region. Based on the aforementioned boundary buffer region, a distance gradient field is constructed; Based on the distance gradient field, the target abundance values ​​of the target region and the background region at a set pixel are calculated using a nonlinear abundance function. Based on the aforementioned nonlinear abundance function, and with the point spread function and atmospheric scattering model as physical constraints, an adaptive abundance matrix for the physical mechanism of hyperspectral imaging is determined. Based on the adaptive abundance matrix, the spectrum of the target region and the spectrum of the background region are mixed according to the abundance value; The mixture employs a spectral mixing model that includes nonlinear and interaction terms to simulate scattering phenomena, and generates mixed pixel simulation data with targets of different scales and diverse backgrounds through a multi-scale target generation strategy and a background adaptive selection mechanism. Based on the hybrid pixel simulation data, enhancement processing is performed in the spatial domain, spectral domain, and temporal domain. The enhancement processing in the spatial domain includes simulating elastic deformation of geometric distortions in hyperspectral imaging. The enhancement processing in the spectral domain includes adaptive noise addition based on band signal-to-noise ratio and random spectral band masking. The enhancement processing in the temporal domain includes simulating seasonal variations and environmental condition disturbances.

2. The data augmentation method for hyperspectral target detection according to claim 1, characterized in that, Based on the distance gradient field, and using a nonlinear abundance function, the target abundance values ​​at a set pixel are calculated for both the target region and the background region, specifically including: According to the formula Calculate the target abundance value at pixel (i,j) in the target region; According to the formula Calculate the target abundance value of the background region at pixel (i,j); in, This represents the target abundance value at pixel (i,j). This is the amplitude coefficient of the exponentially decaying term. For decay rate parameters, The magnitude coefficient of the power function term. The exponent of the power function. For normalized distance, The distance to the boundary buffer zone. To maximize the radius of influence, For the direction weight function, This represents the background abundance value at pixel (i,j).

3. The data augmentation method for hyperspectral target detection according to claim 2, characterized in that, The formula for the point spread function is: ； in, The value of the PSF kernel function at a distance d. Here, d represents the standard deviation parameter of the PSF, and d represents the spatial distance. Represents an exponential function. Pi is the mathematical constant of a circle.

4. The data augmentation method for hyperspectral target detection according to claim 3, characterized in that, The formula for the atmospheric scattering model is: ； in, Let be the scattering attenuation function at a distance d. The direct scattering coefficient, For direct scattering attenuation rate, The multiple scattering coefficient, This is the multiple scattering attenuation index.

5. The data augmentation method for hyperspectral target detection according to claim 4, characterized in that, The background adaptive selection mechanism calculates the matching degree between the target spectrum and the background spectra of each category in the background library; The matching degree is a weighted combination of spectral angular distance, spectral information divergence, and spectral correlation coefficient.

6. The data augmentation method for hyperspectral target detection according to claim 5, characterized in that, The formula for calculating the matching degree is: ； in, As a matching index, Spectral angular distance, For spectral information divergence, The spectral correlation coefficient is... , , These are the weighting coefficients. The average spectrum of the p-th background. Spectral reflectance; ; For Kullback-Leibler divergence, , and These are the normalized spectral probability distributions; ; and These are the mean values ​​of the target spectrum and the background spectrum, respectively.

7. The data augmentation method for hyperspectral target detection according to claim 6, characterized in that, Multi-scale target generation strategies, specifically including: According to the formula Determine the set of target scaling factors; where, Let K be the set of scaling factors, where K is the number of scaling levels; The target region is scaled according to the target scaling factors in the target scaling factor set to obtain the scaled target region. ;in, For the target region at the k-th zoom level, This represents the bilinear interpolation scaling function; According to the formula Calculate the scaled abundance distribution function; the superscript (k) indicates the parameter corresponding to the k-th scaling level.

8. A data enhancement device for hyperspectral target detection, characterized in that, include: The region segmentation module is used to accurately segment image pixels into target regions, background regions, and boundary buffer regions based on the ground truth map of hyperspectral images. The average spectrum, spectral variance, and spectral angular distance between the target region and the background region are calculated; the boundary buffer region is the area located between the target region and the background region. A distance gradient field construction module is used to construct a distance gradient field based on the boundary buffer region; The abundance value calculation module is used to calculate the target abundance value of the target region and the background region at a set pixel based on the distance gradient field and a nonlinear abundance function. The matrix construction module is used to determine an adaptive abundance matrix for the physical mechanism of hyperspectral imaging based on the nonlinear abundance function, with the point spread function and atmospheric scattering model as physical constraints. The mixing module is used to mix the spectra of the target region and the background region according to their abundance values ​​based on an adaptive abundance matrix. The mixture employs a spectral mixing model that includes nonlinear and interaction terms to simulate scattering phenomena, and generates mixed pixel simulation data with targets of different scales and diverse backgrounds through a multi-scale target generation strategy and a background adaptive selection mechanism. The enhancement processing module is used to perform enhancement processing on the spatial domain, spectral domain, and temporal domain based on the hybrid pixel simulation data; the enhancement processing in the spatial domain includes simulating elastic deformation of geometric distortion in hyperspectral imaging; the enhancement processing in the spectral domain includes adaptive noise addition and random spectral band masking based on the band signal-to-noise ratio; and the enhancement processing in the temporal domain includes simulating seasonal variations and environmental condition disturbances.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement a data augmentation method for hyperspectral target detection according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements a data augmentation method for hyperspectral target detection as described in any one of claims 1-7.

Images