A method and system for predicting crop yield by integrating multi-source remote sensing data

Abstract

The application discloses a crop yield prediction method and system integrating multi-source remote sensing data, optical images and radar images of a target crop plot are acquired, a crop canopy vertical height profile is constructed based on the backscattering characteristics of the radar images, a canopy geometric projection relationship is established in combination with a solar incident azimuth angle and a sensor angle, a canopy shielding area is identified through a three-dimensional light transmission simulation algorithm, and canopy transmission light distribution characteristics are derived according to the spatial distribution difference between vegetation and shadows, the transmission light distribution is used to correct optical image reflectivity, and the height profile is combined to generate comprehensive features of a vegetation canopy structure, a deep feature fusion network based on an attention mechanism is used to fuse the canopy space and height features, and thus high-precision crop yield prediction is realized. The method effectively overcomes the influence of illumination conditions and image geometric distortion on estimation accuracy, and improves the accuracy and stability of multi-source remote sensing data in crop yield prediction.

Description

Technical Field

[0001] This invention relates to the field of agricultural remote sensing application technology, specifically to a method and system for predicting crop yield by integrating multi-source remote sensing data. Background Technology

[0002] With the development of high-resolution remote sensing imagery technology, the use of multi-source remote sensing data for refined monitoring and prediction of crop growth status and yield has become an important research direction in the field of agricultural remote sensing. Among existing technologies, optical remote sensing images, due to their rich spectral information, are widely used for vegetation index calculation and biomass estimation; while synthetic aperture radar (SAR) remote sensing can penetrate clouds and vegetation surfaces to obtain structural information of the crop canopy, especially in polarization modes, where it can extract the backscattering difference between the canopy and the ground for structural estimation.

[0003] However, in practical applications, optical imagery is limited by the observation angle, the direction of solar incidence, and the complexity of the canopy structure, making it prone to shading and transmitted light interference, leading to errors in reflectivity calculation and consequently affecting the accuracy of crop yield prediction. Existing studies have attempted to jointly model radar and optical remote sensing, but these efforts have mostly focused on feature-level fusion, failing to fully utilize radar polarimetric scattering information to construct a high-precision canopy vertical height structure, and also failing to systematically simulate the impact of the three-dimensional light transmission characteristics of canopy-shaded areas on reflectivity.

[0004] Furthermore, most existing deep learning-based crop yield prediction models use two-dimensional image features as input, ignoring the spatial distribution differences and vertical layer information of the canopy structure. They lack a deep fusion mechanism between height dimension and spatial image features, making it difficult to meet the generalization requirements of different crop varieties, plot environments, and remote sensing conditions.

[0005] In view of this, the present invention provides a method and system for predicting crop yield by integrating multi-source remote sensing data, thereby solving the above-mentioned problems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for predicting crop yield by integrating multi-source remote sensing data. This method integrates multi-source remote sensing data and considers the influence of canopy transmitted light distribution on spectral reflectance correction, thereby improving the accuracy and robustness of crop three-dimensional canopy structure modeling and yield prediction.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a method for predicting crop yield by integrating multi-source remote sensing data, comprising the following steps:

[0009] S101: Acquire optical and radar remote sensing images of the target crop plot, construct the vertical height profile of the canopy based on the backscattering characteristics of the crop canopy in the radar remote sensing image, and construct the geometric projection relationship of the canopy based on the solar incident azimuth angle and the observation sensor angle.

[0010] S102: Based on the geometric projection relationship of the canopy, the shading area of ​​the canopy structure is determined by the three-dimensional canopy light transmission simulation algorithm, and the spatial distribution difference between vegetation and shadow in the shading area is used to deduce the distribution characteristics of canopy transmitted light.

[0011] S103: Correct the canopy reflectance in optical remote sensing images based on the distribution characteristics of canopy transmitted light, and use the corrected canopy reflectance and canopy vertical height profile to generate comprehensive features of vegetation canopy structure.

[0012] S104: Predicting crop yield based on a deep feature fusion network driven by comprehensive features of vegetation canopy structure. The deep feature fusion network uses an attention mechanism to fuse spatial map features and height features.

[0013] As a preferred embodiment of the first aspect of the present invention, constructing a canopy vertical height profile based on the backscattering characteristics of crop canopies in radar remote sensing images includes:

[0014] Based on the polarization scattering characteristics of radar remote sensing images, the polarization phase difference between the scattered signals from the top of the canopy and the ground is obtained;

[0015] The effective propagation path of radar signals penetrating the canopy is determined by the polarization phase difference.

[0016] The echo locations at different depths in the canopy are determined based on the effective propagation path.

[0017] A height profile that can represent the true vertical height of the crop canopy is generated based on the echo positions of each layer.

[0018] As a preferred embodiment of the first aspect of the present invention, the polarization phase difference between the top of the canopy and the ground scattering signal is obtained based on the polarization scattering characteristics of radar remote sensing images, including:

[0019] HH polarization and VV polarization echo signals of stable scattering targets on the ground within the same imaging area are extracted, and the average complex scattering coefficient is calculated to obtain the ground reference target signal.

[0020] Based on the HH polarization and VV polarization echoes acquired simultaneously by the polarization radar sensor, the canopy polarization phase of the scattered signal at the top of the canopy is calculated.

[0021] The polarization phase difference between the canopy top scattering signal and the ground reference target signal is calculated by utilizing the complex difference in the complex scattering coefficients of the canopy top scattering signal and the ground scattering signal in the HH and VV channels.

[0022] As a preferred embodiment of the first aspect of the present invention, the distribution characteristics of canopy transmitted light are derived by using the spatial distribution differences between vegetation and shadow within the shaded area, including:

[0023] Based on the spectral reflectance of vegetation and shade in different bands of the shaded area, the spectral difference pattern between the shaded area and the vegetated area is obtained;

[0024] Based on the spectral difference pattern, the specific spatial locations of the multiple scattering regions within the shadow area and the single scattering regions within the vegetation area are identified.

[0025] By analyzing the spectral attenuation characteristics of the multiple scattering region, the spatial location and extent of transmitted light attenuation within the blocked area can be determined.

[0026] Based on the spatial location and degree of transmitted light attenuation, the transmitted light distribution characteristics that characterize the true light distribution characteristics inside the canopy are obtained.

[0027] As a preferred embodiment of the first aspect of the present invention, the specific spatial locations of multiple scattering regions within the shadow area and single scattering regions within the vegetation area are identified based on the spectral difference pattern, including:

[0028] Extract the spectral reflectance distribution of multi-band images of shaded and vegetated areas;

[0029] Analyze the reflectivity variation patterns of each band to identify stable single-scattering characteristic bands in vegetated areas and significant multiple-scattering characteristic bands in shaded areas.

[0030] By using the identified single scattering characteristic band and multiple scattering characteristic band, the specific spatial locations of single scattering and multiple scattering are determined.

[0031] As a preferred embodiment of the first aspect of the present invention, the specific spatial locations of single scattering and multiple scattering are determined using defined single scattering characteristic bands and multiple scattering characteristic bands, including:

[0032] Based on the characteristic bands of the shaded and vegetated areas, a spatial spectral difference map is constructed.

[0033] The spatial autocorrelation algorithm is used to analyze the spatial spectral difference map to determine the single scattering or multiple scattering properties at each spatial location;

[0034] Based on the scattering properties of each spatial location, the specific spatial distribution results of single scattering regions and multiple scattering regions in the canopy-shaded area are obtained.

[0035] As a preferred embodiment of the first aspect of the present invention, a comprehensive feature of vegetation canopy structure is generated by combining the corrected canopy reflectivity and the canopy vertical height profile, including:

[0036] The corrected canopy reflectance was decomposed into different spectral bands to obtain detailed reflectance characteristics of vegetation leaves and branches.

[0037] The vertical height profile of the canopy is divided into several vertical layers, and the canopy thickness information corresponding to each layer is extracted.

[0038] Based on the detailed reflective features of vegetation leaves and branches, the main vegetation composition types in each vertical layer are identified.

[0039] Based on vertical layer thickness information and vegetation composition type information, comprehensive characteristics of vegetation canopy structure that characterize the differences in three-dimensional canopy structure are obtained.

[0040] As a preferred embodiment of the first aspect of the present invention, a training method for a comprehensive feature-driven deep feature fusion network includes:

[0041] A dual-input network structure is constructed, where the first input branch receives canopy height features and the second input branch receives canopy spatial map features.

[0042] Convolutional operations are used to extract local pattern features of canopy height and spatial map features, respectively;

[0043] An attention mechanism module was designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features.

[0044] By fusing key regions of canopy spatial map features and local patterns of canopy height features, a trained deep feature fusion network is obtained.

[0045] As a preferred embodiment of the first aspect of the present invention, an attention mechanism module is designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features, including:

[0046] A height-guided weight map is generated by utilizing local patterns of canopy height features;

[0047] The height-guided weight map and the canopy spatial map features are multiplied point by point to form a weighted distribution of spatial map features.

[0048] Based on the weighted distribution of spatial map features, the key regions for yield prediction in the canopy spatial map features are selected.

[0049] In a second aspect, the present invention provides a crop yield prediction system integrating multi-source remote sensing data, based on the implementation of the first aspect, including an image acquisition module, a canopy illumination simulation module, a canopy feature correction module and a yield prediction module, wherein the modules transmit data to each other via wired and / or wireless means.

[0050] The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of the target crop plot, and to construct the vertical height profile of the canopy based on the backscattering characteristics of the crop canopy in the radar remote sensing image; the image acquisition module is also used to construct the geometric projection relationship of the canopy based on the solar incidence azimuth angle and the observation sensor angle.

[0051] The canopy illumination simulation module is used to determine the canopy structure shading area based on the canopy projection relationship using a three-dimensional canopy light transmission simulation algorithm, and to derive the canopy transmitted light distribution characteristics by using the spatial distribution differences between vegetation and shadow within the shading area.

[0052] The canopy feature correction module is used to correct the canopy reflectance in the optical remote sensing image based on the canopy transmitted light distribution characteristics, and combine the corrected canopy reflectance with the canopy vertical height profile to generate a comprehensive feature of the vegetation canopy structure.

[0053] The yield prediction module is used to predict crop yield based on the comprehensive features of the vegetation canopy structure driven by a deep feature fusion network; wherein, the deep feature fusion network uses an attention mechanism to fuse spatial map features and height features.

[0054] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0055] This invention utilizes the combined use of optical and radar remote sensing data, leveraging radar backscattering characteristics to construct a canopy vertical height profile, and integrates this profile with optical reflectivity. This overcomes the limitations of single-source remote sensing in terms of spatial resolution, imaging conditions, and data stability, achieving complementarity and enhancement of multi-source information. By introducing canopy geometric projection relationships and a three-dimensional transmittance simulation algorithm, it effectively identifies and corrects canopy shading effects caused by solar incidence and observation angles, eliminating interference from shadow distribution and geometric distortion on image reflectivity correction, thereby improving the realism of canopy structural features.

[0056] This invention utilizes transmitted light distribution to correct optical reflectivity and combines it with canopy vertical height profiles to generate comprehensive canopy structure features that simultaneously characterize spatial distribution and vertical stratification. This ensures accurate characterization of crop three-dimensional structural differences. By combining a deep feature fusion network with a dual-branch input structure and attention mechanism, an effective correlation is established between spatial and height features, focusing on key areas related to yield prediction. It does not rely on excessive computing resources and can be processed on conventional remote sensing data and computing platforms, ensuring the application value of the technical solution in practical scenarios such as agricultural production management, crop growth monitoring, and yield assessment. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0058] Figure 1 This is a flowchart of the crop yield prediction method integrating multi-source remote sensing data according to the present invention.

[0059] Figure 2 This is a framework diagram of the crop yield prediction system integrating multi-source remote sensing data according to the present invention. Detailed Implementation

[0060] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0061] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure may be practiced with one or more specific details omitted, or methods, components, steps, etc. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0062] Example 1

[0063] like Figure 1As shown, this invention provides a method for predicting crop yield by integrating multi-source remote sensing data, comprising the following steps:

[0064] S101: Acquire optical and radar remote sensing images of the target crop plot, construct the vertical height profile of the canopy based on the backscattering characteristics of the crop canopy in the radar remote sensing image, and construct the geometric projection relationship of the canopy based on the solar incident azimuth angle and the observation sensor angle.

[0065] It should be noted that wheat was the target crop, and the target plot area was approximately 500m × 500m. Utilizing the complementary characteristics of optical and radar remote sensing imagery, image data was acquired simultaneously during the crop's key growth stages (such as from heading to grain filling). The image acquisition time coincided with the key growth stages of the crop, thereby obtaining detailed three-dimensional canopy structure information of the target crop plot, reflecting the representativeness of the canopy structure. Specifically, the optical remote sensing imagery was preferably from high-resolution multispectral satellite or UAV imagery, used to extract the spectral reflectance characteristics of the canopy; the radar remote sensing imagery was preferably Polarimetric Synthetic Aperture Radar (PolSAR) imagery, used to extract the vertical structural features of the vegetation canopy.

[0066] The specific implementation process of constructing the vertical height profile of the crop canopy based on the backscattering characteristics of the crop canopy in radar remote sensing images includes:

[0067] Based on the polarization scattering characteristics of radar remote sensing images, the polarization phase difference between the scattered signals from the top of the canopy and the ground is obtained;

[0068] It should be noted that polarization scattering characteristics refer to the complex amplitude and phase combination relationship of the target scattering response observed by a synthetic aperture radar system under different transmission and reception polarization states. In microwave radar, the polarization direction of electromagnetic waves can be divided into two orthogonal states: horizontal (H) and vertical (V). When the radar transmits H or V polarized waves and simultaneously receives H or V polarized echoes, four basic echo signals can be obtained:

[0069] HH polarization: horizontal transmission, horizontal reception;

[0070] HV polarization: horizontal transmission, vertical reception;

[0071] VH polarization: vertical transmission, horizontal reception;

[0072] VV polarization: vertical transmission, vertical reception.

[0073] For any pixel The above echo signals are all in complex form:

[0074] ;

[0075] in: The complex scattering coefficients; It reflects the target's ability to reflect incident waves, and represents the scattering intensity (amplitude), which is related to the reflectivity of the scattering surface; Phase indicates the relationship between the electromagnetic wave propagation path difference and the target dielectric properties.

[0076] The four complex scattering coefficients constitute the polarization scattering matrix of the target: ;

[0077] The four complex scattering coefficients are expressed as follows: ;

[0078] The polarization scattering matrix reflects the scattering mechanism of the target: that is, the physical process of electromagnetic waves being reflected, refracted and re-radiated in vegetation, ground surface and artificial structures. Based on the scattering differences reflected by the polarization scattering matrix, the different polarization responses of radar waves in the canopy and the ground can be used to accurately identify and distinguish the position of the top of the canopy and the ground, and to determine the position reference in the vertical direction of the canopy.

[0079] The process of obtaining the polarization phase difference between the scattered signals from the top of the canopy and the ground includes the following steps:

[0080] HH polarization and VV polarization echo signals of stable scattering targets on the ground within the same imaging area are extracted, and the average complex scattering coefficient is calculated to obtain the ground reference target signal.

[0081] It should be noted that the surface reference target signal refers to a non-vegetation-covered area (such as farmland, roads, bare land, or metal targets) selected as the ground reference point within the same radar imaging area. Its scattering characteristics are stable, with surface scattering dominating. The polarization response of this signal typically exhibits the following characteristics: strong HH polarization channel, followed by VV polarization channel, high polarization coherence (|ρ|>0.9), and stable phase (standard deviation <0.1 rad). The complex scattering coefficients of the HH horizontal polarization channel and the VV vertical polarization channel are extracted separately as follows:

[0082] , ;

[0083] To eliminate local angle and noise differences, a high coherence window is used at the surface. The surface reference values ​​are obtained by averaging complex amplitudes. ;

[0084] Subsequently, to obtain the principal definition values ​​of the surface specular reflection component, the Freeman–Durden three-component polarization decomposition algorithm was used to construct the polarization covariance matrix based on the polarization scattering matrix: , where the scattering vector for: ;

[0085] The formula for the polarization covariance matrix is: ;

[0086] Among them, angle brackets This indicates a local averaging operation within a spatial sliding window (typically 3×3 or 5×5 pixels), indicated by the superscript. The polarization covariance matrix represents the conjugation; each element in the polarization covariance matrix represents the complex correlation between different polarization channels, with diagonal elements reflecting the scattering energy of each channel and off-diagonal elements reflecting the coherence between channels.

[0087] According to electromagnetic scattering theory, the polarization response of different scattering mechanisms can be described by an idealized geometric model. The Freeman–Durden ternary model characterizes the three types of scattering template matrices—surface scattering, volume scattering, and dual reflection—by establishing three typical template matrices. , and Specifically:

[0088] Surface scattering template matrix Simulate a single reflection of radar waves onto a smooth surface or a mirror-like object:

[0089] ;

[0090] in: It is the dip angle of the ground or the equivalent angle of incidence.

[0091] Volume scattering template matrix Characterizing isotropic scattering from randomly oriented leaves and branches in the vegetation canopy:

[0092] ;

[0093] Dual reflection template matrix Describe the two-way reflection caused by a two-layer structure (such as between the canopy and the ground, or between branches and leaves):

[0094] ;

[0095] in: The geometric tilt angle of the dual reflection path.

[0096] All three can be obtained through incident angle, canopy geometry parameters, or experimental calibration, thus reflecting the geometric and electromagnetic properties of different scattering mechanisms in the model.

[0097] After obtaining the observed polarization covariance matrix and the three types of scattering template matrices, the overall scattering model is transformed into a solvable linear system; a scattering mechanism model is established, specifically as follows:

[0098] ;

[0099] in, , and Let represent the power coefficients of the surface, volume, and double reflection, respectively, constituting the total scattered energy distribution of the observed polarization signal; the polarization covariance matrix contains 9 matrix elements but only 3 unknown coefficients, belonging to an overdetermined system of equations. To make this system solvable numerically, each matrix is ​​expanded into a vector form, and the matrix elements are expanded row-wise into one-dimensional vectors. Let:

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] The linear system can then be written as: ;

[0105] To reduce the impact of observation noise, the least squares method is used to solve the problem:

[0106] ;

[0107] Right now: ;

[0108] The surface scattering power is obtained by least squares solution. And obtain the surface reference polarization phase within the surface window: ;

[0109] Therefore, the surface reference target signal is defined as: ;

[0110] It should be noted that the surface reference target signal refers to the complex surface scattering component obtained from this decomposition; the surface reference quantity obtained by averaging the complex amplitude is used as a reference check quantity and is not included in the subsequent main calculation. The surface reference target signal... It serves as a phase reference for subsequent canopy signal calculations, providing a stable and reliable phase reference for canopy scattering signals.

[0111] Based on the HH polarization and VV polarization echoes acquired simultaneously by the polarization radar sensor, the canopy polarization phase of the scattered signal at the top of the canopy is calculated.

[0112] It should be noted that the scattered signal at the top of the canopy In polarimetric radar imagery, the phase center of the complex signal dominated by volume scattering from the upper leaves and branches of the crop canopy corresponds to the upper surface of the canopy. Typically, the scattering energy in the HV and VH channels is relatively strong, reflecting the presence of numerous randomly oriented volume scattering units such as leaves and branches within the crop canopy, which are a significant source of canopy volume scattering. However, the phase stability and spatial coherence of the HV and VH channels are poor, making them unsuitable for use as a basis for interferometric phase analysis. In contrast, the scattering in the HH and VV channels is mainly influenced by both the upper canopy structure and surface reflection, exhibiting continuous phase variation with space and high coherence, thus providing a more stable reflection of the average propagation path difference from the upper canopy surface to the ground. Therefore, the phase difference is calculated using the complex scattering coefficients of the HH and VV channels to ensure the coherence and physical interpretability of the calculation results.

[0113] Specifically, the scattered signal at the top of the canopy The calculation formula is:

[0114] ;

[0115] in, and These are the complex scattering coefficients for the corresponding horizontal and vertical polarization channels, respectively. That is, the scattering signal at the top of the canopy is the average composite component of the combined echo from above the canopy via the HH and VV channels minus the surface reference target signal. It eliminates surface reflection interference and has stable spatial continuity.

[0116] The complex representation of the scattered signal at the top of the canopy is as follows: ;

[0117] in: This represents the amplitude component of the canopy scattering; This represents the canopy polarization phase of the volume scattering at the top of the canopy, used for subsequent phase difference calculation.

[0118] By utilizing the difference between the HH polarization and VV polarization signals of the scattering target at the top of the canopy and the ground, the polarization phase difference between the scattering signal at the top of the canopy and the scattering signal at the ground is calculated.

[0119] Specifically, HH and VV polarization signals are the original observations of the two main polarization channels of the radar and are the basic components of all scattering features; the canopy top scattering signal is a complex signal component formed by scattering from the upper layer of the vegetation canopy under the same polarization channel, representing the main reflection center of the canopy surface; the ground reference target signal is a complex signal component formed by scattering from the surface of bare land or stable artificial structures under the same polarization channel, representing the ground reflection center; that is, both the canopy top scattering signal and the ground scattering signal are decomposed from the HH / VV polarization channel components. By comparing the complex phase difference between the two under the same polarization framework, the equivalent path difference of radar waves propagating from the canopy surface to the ground reflection surface can be quantified.

[0120] To further explain, when calculating the polarization phase difference, the scattered signal from the top of the canopy is used. With surface reference target signal Given the complex difference, calculate the polarization phase difference between the two:

[0121] ;

[0122] in: Indicates complex conjugation. Representing the phase angle as a complex number, the polarization phase difference quantifies the equivalent path difference of radar waves propagating from the upper surface of the canopy to the ground reflector, reflecting the comprehensive differences between the canopy and the ground surface in terms of propagation medium, electromagnetic properties, and geometric height.

[0123] To further clarify, the signal from the top of the canopy and the reference signal from the ground both originate from the same radar scene and the same set of HH / VV polarization channels, simply representing the canopy volume scattering center and the ground surface specular scattering center, respectively. By calculating the complex phase difference within the same polarization framework, the relative delay of the canopy's vertical propagation path can be obtained; the polarization phase difference distribution... It can be directly used as input parameters for canopy height inversion and polarization interferometry calculation, providing a reliable phase reference for crop canopy vertical structure modeling.

[0124] The effective propagation path of radar signals penetrating the canopy is determined by the polarization phase difference.

[0125] In practice, based on the aforementioned polarization phase difference, polarization interferometry processing technology is used to determine the actual propagation trajectory of radar waves after entering the canopy. This propagation trajectory can be determined by establishing a polarization interferometry scattering model and combining it with prior structural information about the vegetation canopy to deduce the specific penetration path of the radar signal within the canopy, thereby enabling the identification of the detection path in the vertical direction of the canopy.

[0126] The echo locations at different depths in the canopy are determined based on the effective propagation path.

[0127] It is understandable that radar waves, after entering the canopy, will interact with vegetation structures at different heights, thus generating echo signals at different locations within the canopy. By utilizing the aforementioned clearly defined and effective propagation paths and performing joint analysis in the time and spatial domains, the specific locations of the echoes at different depths within the vegetation canopy can be determined layer by layer, thereby obtaining echo location information that can clearly characterize the vertical structure within the vegetation canopy.

[0128] A height profile that can represent the true vertical height of the crop canopy is generated based on the echo positions of each layer.

[0129] In practice, the echo locations of each canopy depth layer, as determined above, are progressively connected using hierarchical clustering based on the echo signal intensity distribution curves. Combined with ground surface reference points, this ultimately generates a refined, realistic, and stable profile of the crop canopy's vertical height. This profile directly reflects the actual vertical structural changes within the canopy, providing core data for subsequent canopy structural feature analysis.

[0130] Furthermore, the specific implementation method for constructing the geometric projection relationship of the canopy based on the solar incidence azimuth angle and the observation sensor angle is as follows:

[0131] First, based on the position parameters of the sun at the time of remote sensing image acquisition and the orbit and attitude information of the sensor, the incident azimuth and incident elevation angles of sunlight illuminating the Earth's surface, as well as the viewing angle and viewing tilt angle of the remote sensing sensor during imaging, are determined.

[0132] Secondly, based on the aforementioned canopy height contour information, the three-dimensional structure of the canopy is projected onto a two-dimensional plane. Using ray tracing algorithms or the ray method, the projection area and shading relationship of each spatial location within the canopy under illumination conditions can be obtained, so as to accurately establish the spatial positional relationship between vegetation and shaded areas within the canopy.

[0133] It should be noted that the vertical height profile of the canopy characterizes the structural distribution features of the crop canopy in the vertical direction, while the geometric projection relationship of the canopy defines the spatial shading and projection relationship of each structural unit of the canopy under the direction of solar incidence and the observation view. Based on the geometric projection relationship of the canopy, the spatial distribution differences between vegetated and shaded areas are identified, and the spatial locations of the canopy leaf layer, gaps, and shaded areas are clearly defined in three-dimensional space, providing accurate geometric constraints for the calculation of light distribution and simulation of light transmission characteristics in optical images. In step S102, a three-dimensional light transmission simulation algorithm and spectral scattering analysis are used to establish the physical correspondence between the canopy structure and light propagation, realizing the collaborative modeling of canopy geometric structure information and optical radiation information.

[0134] S102: Based on the geometric projection relationship of the canopy, the shading area of ​​the canopy structure is determined by the three-dimensional canopy light transmission simulation algorithm, and the spatial distribution difference between vegetation and shadow in the shading area is used to deduce the distribution characteristics of canopy transmitted light.

[0135] It should be noted that: based on the established canopy geometric projection relationship, the spatial distribution of the canopy structure's shading area is simulated, and the distribution characteristics of canopy transmitted light are derived by utilizing the differences in radiation characteristics between vegetation and shadow within the shading area. This directly serves the structural correction of reflectance in subsequent optical remote sensing images and is an important preprocessing step for image physical property correction.

[0136] Understandably, the non-uniformity of the canopy structure in three-dimensional space can lead to illumination occlusion in some areas of remote sensing images. These occluded areas exhibit localized brightness reduction and distorted spectral curves in optical images. To accurately model the light propagation patterns within the canopy, it is necessary to construct a canopy projection model and extract the occluded areas based on geometric projection relationships combined with measured solar incidence angles and remote sensing sensor imaging angles. In this embodiment, a three-dimensional transmission simulation algorithm (such as Monte Carlo photon tracing or the Zhang model) is used to estimate the penetration path of the incident beam for each pixel and identify the spatial locations affected by occlusion.

[0137] Specifically, the spatial distribution differences between vegetation and shadow within the shaded area are used to deduce the canopy transmittance characteristics, including:

[0138] Based on the spectral reflectance of vegetation and shade in different bands of the shaded area, the spectral difference pattern between the shaded area and the vegetated area is obtained;

[0139] It should be understood that the optical remote sensing reflectance of the shaded area is affected by physical mechanisms such as multiple scattering and shadow attenuation, exhibiting spectral response characteristics that are significantly different from those of the directly illuminated area. By comparing and analyzing typical vegetation bands (such as near-infrared, red edge, and visible green bands) in multi-band optical remote sensing images, and by calculating the normalized difference index (such as NDVI change rate and RVI attenuation rate) between the vegetation area and the shaded area in the same region under each band, a pattern matrix describing the spectral differences is established.

[0140] Based on the spectral difference pattern, the specific spatial locations of the multiple scattering regions within the shadow area and the single scattering regions within the vegetation area are identified.

[0141] For example, a spatial sliding window is used to traverse the spectral reflectance values ​​of each pixel and its neighborhood within the occluded area, and a feature vector set is constructed by combining the difference pattern matrix. Subsequently, an unsupervised classification algorithm based on principal component analysis (PCA) + K-means clustering is introduced to divide the region into single scattering-dominant and multiple scattering-dominant regions, and the spatial boundaries of each type of region are further corrected by comparing the canopy contour projection mapping position.

[0142] To improve spatial positioning accuracy, the above results can be cross-corrected with the backscattering intensity variation areas in radar images to improve the accuracy of scattering type identification. Furthermore, based on the spectral difference pattern recognition, the specific spatial locations of the multiple scattering regions within the shadow area and the single scattering regions within the vegetation area include:

[0143] Extract the spectral reflectance distribution of multi-band images of shaded and vegetated areas;

[0144] Specifically, reflectance data for the direct sunlight and shadow areas of the canopy are extracted from optical remote sensing images. The average reflectance difference between the two types of areas under different wavelengths is calculated to characterize the influence of illumination conditions on the wavelength response. For each wavelength... The rate of change of its reflectivity is defined as:

[0145] ;

[0146] in, The average reflectance of the area directly exposed to sunlight by vegetation. The average reflectance of the shaded area. To prevent small quantities with a denominator of zero.

[0147] Analyze the reflectivity variation patterns of each band to identify stable single-scattering characteristic bands in vegetated areas and significant multiple-scattering characteristic bands in shaded areas.

[0148] It should be noted that the intensity of light irradiation on a wavelength band is determined based on a preset threshold for the rate of change of reflectivity, thereby calibrating single scattering and multiple scattering; specifically:

[0149] like A smaller value (e.g., below 0.1) indicates that the band is less affected by light, and its signal mainly originates from single reflections from the canopy leaves, which can be classified as a single-scattering characteristic band.

[0150] like A larger value (e.g., above 0.2) indicates that the band is significantly affected by multiple scattering and shadowing, reflecting the multi-layer reflection effect inside the canopy, and can be classified as a multiple scattering characteristic band.

[0151] Based on the above determination, the set of single-scattering characteristic bands with stable reflectivity changes can be denoted as: The set of characteristic bands of multiple scattering that change significantly is denoted as .

[0152] Using the identified single scattering and multiple scattering characteristic bands, the specific spatial locations of single and multiple scattering are determined; this includes the following steps:

[0153] Based on the characteristic bands of the shaded and vegetated areas, a spatial spectral difference map is constructed; the spatial autocorrelation algorithm is used to analyze the spatial spectral difference map to determine the single or multiple scattering properties of each spatial location.

[0154] This can be understood as: for each pixel The formula for calculating the difference in average reflectance between the two characteristic bands is as follows:

[0155] ;

[0156] in, This represents the average reflectivity of the set of multiple scattering bands. This represents the average reflectance of a set of single-scattering bands. The difference in average reflectance per pixel. The larger the value, the more significant the attenuation of reflectivity in the multiple scattering band, and the more reflection and absorption processes the light undergoes from the leaves.

[0157] Based on the scattering properties of each spatial location, the specific spatial distribution results of single scattering regions and multiple scattering regions in the canopy-shaded area are obtained.

[0158] Specifically, the system classifies and labels each spatial location based on the intensity of reflectance differences and spatial clustering characteristics in the spatial spectral difference map. For each pixel, the deviation between its reflectance difference value and the average value of the entire region is calculated. A spatial autocorrelation analysis algorithm is used to perform spatial clustering detection on the spectral difference map to determine whether there is regional consistency in the reflectance differences of adjacent pixels.

[0159] If a certain area Regions consistently above the global average and showing a significant positive correlation (p<0.05) are labeled as multiple scattering regions; regions consistently below the average and showing a significant negative correlation are labeled as single scattering regions. By introducing spatial autocorrelation analysis, local anomalous pixels caused by noise are eliminated, ensuring the spatial distribution of scattering types remains consistent with the canopy's geometric occlusion characteristics. This yields a spatial distribution label map containing single and multiple scattering classification results, providing regional constraints for subsequent transmitted light attenuation calculations.

[0160] To improve spatial continuity, smoothing calculations can be performed within local neighborhoods (such as 3×3 or 5×5 pixels) to ensure that the region division is consistent with the canopy geometry. Simultaneously, the calibration results within geometrically shaded areas are corrected based on the canopy projection relationship: if a pixel is located in the canopy shadow region and the average reflectance difference is greater than the regional average, it is maintained as a multiple scattering region; if it is located in the direct sunlight region and the average reflectance difference is less than or equal to the regional average, it is corrected to a single scattering region. This yields a continuous spatial distribution map of single and multiple scattering on the canopy projection plane, providing input conditions for subsequent transmitted light attenuation estimation.

[0161] To further explain, the logical relationship of the above methods is as follows:

[0162] Through calculation The sensitivity of each band to changes in illumination was analyzed, and the bands dominated by single and multiple scattering were identified.

[0163] Through calculation By projecting spectral differences onto spatial locations, regions can be defined based on scattering types.

[0164] Finally, an interpretable mapping relationship of canopy scattering type was established in the spectral-spatial dual domain, so that the canopy shading and light attenuation characteristics correspond physically.

[0165] By analyzing the spectral attenuation characteristics of the multiple scattering region, the spatial location and extent of transmitted light attenuation within the blocked area can be determined.

[0166] Understandably, in the multiple scattering region, due to reflection and absorption of light among multiple leaves and branches, the effective transmitted light intensity reaching the sensor will decrease significantly. To quantitatively describe this attenuation process, a spatial labeling map is used to calculate the slope of the decrease in multi-band reflectivity with wavelength in each region, denoted as:

[0167] ;

[0168] in, Reflects the attenuation trend of the local spectrum; when A large value indicates that the light has undergone stronger multiple scattering attenuation at that location. This should be considered in conjunction with the average brightness value of the area. The spectral attenuation characteristics of the transmitted light can be further calculated:

[0169] ;

[0170] in, The absorption coefficient is an empirical value, which can be corrected for in-situ radiation or calibrated using existing literature (typically taken as 0.4–0.6). This calculation process is based on the Beer–Lambert extended model and can estimate the attenuation of transmitted light within the canopy at the pixel level.

[0171] Based on the spatial location and degree of transmitted light attenuation, the transmitted light distribution characteristics that characterize the true light distribution characteristics inside the canopy are obtained.

[0172] In practice, the spatial distribution of the multiple scattering region and the single scattering region, as well as the spectral attenuation characteristics of each region, are considered. A transmitted light field distribution map is constructed on a three-dimensional canopy structure. The transmitted light distribution map describes the relative illumination intensity received by the lower layers of the canopy in pixels, and serves as a spatial weight reference for reflectivity correction of optical remote sensing images. The output is a two-dimensional layer or multiple raster layers, representing the changes in transmitted light intensity at different depth intervals.

[0173] S103: Correct the canopy reflectance in optical remote sensing images based on the distribution characteristics of canopy transmitted light, and use the corrected canopy reflectance and canopy vertical height profile to generate comprehensive features of vegetation canopy structure.

[0174] It should be noted that the core of step S103 lies in correcting the canopy reflectivity in the original optical remote sensing image through the distribution characteristics of canopy transmitted light, thereby eliminating reflectivity anomalies caused by light shading, differences in incident angle, etc. This corrected reflectivity is then combined with the canopy vertical height profile extracted by radar remote sensing to construct a comprehensive feature reflecting the three-dimensional structure of the crop canopy. This comprehensive feature includes both the vertical distribution structure of vegetation and reflects the physical properties of different vertical layers, serving as a crucial foundation for achieving deep feature fusion prediction.

[0175] Specifically, due to the significant spatial non-uniformity of light distribution in vegetated areas under natural conditions, the same type of vegetation in the original remote sensing image will exhibit obvious differences in spectral reflectance at different locations. The reflectance values ​​of each pixel are structurally corrected using a canopy transmittance distribution feature map. The corrected canopy reflectance is then combined with the canopy vertical height profile to generate comprehensive features of the vegetation canopy structure, including:

[0176] The corrected canopy reflectance was decomposed into different spectral bands to obtain detailed reflectance characteristics of vegetation leaves and branches.

[0177] In this embodiment, the reflectance data of each pixel in the optical remote sensing image after S102 correction is spectrally decomposed in the main vegetation-sensitive bands (such as visible green, red, red-edge, and near-infrared bands). By comparing the difference between the standard vegetation reflectance curve model and the measured reflectance, spectral characteristic indicators such as leaf reflectance intensity and branch scattering rate are extracted. For example, the first derivative of reflectance can be used to calculate the reflectance edge displacement to characterize the nitrogen content and activity state of leaves. Finally, a refined reflectance feature vector for each pixel in different bands is output as input for subsequent structure recognition.

[0178] For example, each pixel has a set of reflection feature values ​​after structural decomposition as follows: ,in Indicates different frequency bands, This indicates the corresponding band number, which is a positive integer.

[0179] The vertical height profile of the canopy is divided into several vertical layers, and the canopy thickness information corresponding to each layer is extracted.

[0180] This can be understood as follows: After completing the reflectance spectral feature decomposition, the entire remote sensing image is divided into several vertical layers (e.g., each layer is spaced 0.3m apart) using the canopy vertical height profile constructed in step S101. For each pixel, the vertical thickness information of its layer is statistically analyzed based on its spatial location within the canopy height profile; that is, the distance from the ground to that layer and the canopy thickness information of that layer are recorded. The canopy thickness information is used to establish the influence weights of different vertical layer reflectances on the overall vegetation canopy spectral response.

[0181] For example, each cell carries a layer number and corresponding canopy thickness information, such as {layer 1: 0.4m, layer 2: 0.5m, layer 3: 0.3m}.

[0182] Based on the detailed reflective features of vegetation leaves and branches, the main vegetation composition types in each vertical layer are identified.

[0183] It should be noted that the vegetation composition type within each layer is identified by combining detailed reflectance features with the corresponding canopy thickness information. In this embodiment, a support vector machine (SVM) model is used to classify the reflectance features of different layers, labeling them as leaf-dominant, branch-dominant, or mixed types. Radar backscattering features are also introduced as an auxiliary dimension. By training the model with joint features, vegetation layers with similar reflectance but different structural compositions can be distinguished.

[0184] For example, each vertical layer corresponds to a vegetation type label, such as {layer 1: leaves, layer 2: branches, layer 3: mixed}.

[0185] Based on vertical layer thickness information and vegetation composition type information, comprehensive characteristics of vegetation canopy structure that characterize the differences in three-dimensional canopy structure are obtained.

[0186] In other words, by combining the thickness information of each vertical layer with the identified vegetation composition type, a spatial map feature vector of the canopy structure is constructed. The comprehensive structural features of each cell include: the thickness ratio of each vertical layer, the vegetation type in each layer, and the height gradient change between different layers.

[0187] For example, the integrated features can be constructed as a vector of the following form:

[0188]

[0189] in: Indicates the first Vegetation type coding for each layer (e.g., leaf = 1, branch = 2, mixed = 3). For the first Layered vegetation corresponds to thickness. When the comprehensive features of vegetation canopy structure are used as input to deep learning models, they can enhance the model's sensitivity to changes in three-dimensional spatial distribution, effectively improving the spatial resolution of crop yield prediction.

[0190] It should be noted that by combining the corrected canopy reflectance with the canopy vertical height profile, a comprehensive feature of the vegetation canopy structure that can simultaneously characterize both the vertical dimensional features and spectral reflectance features of the crop is obtained. This comprehensive feature of the vegetation canopy structure not only includes the differences in spectral reflectance of leaves and branches at different wavelengths, but also integrates the three-dimensional structural information formed by the vertical layer thickness of the canopy and the vegetation composition type, thus forming a complete representation of the overall canopy state of the vegetation.

[0191] Understandably, the comprehensive characteristics of vegetation canopy structure are not features from a single source, but rather combine the complementary advantages of optical and radar imagery: the former provides spectral sensitivity, while the latter provides vertical height information. Therefore, the comprehensive characteristics of vegetation canopy structure can not only reflect the optical differences on the canopy surface, but also reveal the spatial distribution patterns of its internal structure.

[0192] It should be understood that, in order for the synthesized features to play a further role, they need to be used as input to drive a deep feature fusion network. Because the deep feature fusion network has a multi-branch input structure, it can receive both the height features and spatial features decomposed from the synthesized features. The deep feature fusion network can effectively resolve feature dimensions from different sources at the structural level and, guided by an attention mechanism, fuse them into key features that contribute most to yield prediction.

[0193] For example, the comprehensive feature matrix can be divided into a "height component" and a "spatial component": the height component is input into the network as the first input branch, and the spatial component is input into the network as the second input branch. Through this input mapping method, the deep feature fusion network can automatically learn the complementary relationship between height and space during training, thereby achieving a seamless connection from comprehensive features to yield prediction.

[0194] S104: Predicting crop yield based on a deep feature fusion network driven by comprehensive features of vegetation canopy structure. The deep feature fusion network uses an attention mechanism to fuse spatial map features and height features.

[0195] It should be noted that by inputting the comprehensive features of vegetation canopy structure into a specially designed deep feature fusion network model, high-precision prediction of crop yield is achieved through the structural fusion of spatial map features and height features. The deep feature fusion network adopts a dual-input structure, incorporating two-dimensional optical data reflecting the spatial distribution characteristics of the canopy and vertical height information reflecting the canopy hierarchy. An attention mechanism is then used to accurately locate the yield prediction-related areas. After training, the network can adaptively capture spatial-structural correspondences, achieving effective collaborative modeling of local and global features.

[0196] This embodiment provides an exemplary network structure and training method, which has good interpretability and implementability. Specifically, the training method for a comprehensive feature-driven deep feature fusion network includes:

[0197] A dual-input network structure is constructed, where the first input branch receives canopy height features and the second input branch receives canopy spatial map features.

[0198] In this embodiment, the deep feature fusion network adopts a dual-branch input structure: the first input branch receives the canopy height feature, i.e., the vertical structure vector output by step S103; the second input branch receives the canopy spatial map feature, i.e., the spectral reflectance distribution of each pixel in the corrected remote sensing image.

[0199] in:

[0200] Height input: Receives vertical height sequence using a multilayer sensing structure;

[0201] Spatial input: 2D spatial spectral images are received using a convolutional neural network (CNN) structure.

[0202] The network initial weights are initialized using Kaiming, and the loss function is mean squared error (MSE).

[0203] Convolutional operations are used to extract local pattern features of canopy height and spatial map features, respectively;

[0204] To further explain, in order to uncover potential local patterns in structural and spatial information, feature extraction operations are performed on both input channels:

[0205] Canopy height features are processed through a one-dimensional convolution operation (Conv1D) to extract local canopy height features corresponding to pattern variations between different vertical layers. ;

[0206] Canopy spatial map features are processed by a two-dimensional convolution module (Conv2D+ReLU+Pooling) to extract canopy spatial map features corresponding to discriminative spectral patterns in local spatial patches. .

[0207] An attention mechanism module was designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features.

[0208] It should be understood that not all regions in the spatial map features are correlated with crop yield. Therefore, an attention mechanism based on height features is introduced to perform weighted filtering of spatial map features, retaining only information from regions that contribute highly to the prediction.

[0209] Specifically, an attention mechanism module is designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns in canopy height features. These regions include:

[0210] A height-guided weight map is generated by utilizing local patterns of canopy height features;

[0211] Utilizing local features of canopy height As a guiding signal, it is input into a weight mapping network (such as an MLP layer + Sigmoid activation), which outputs a height-guided weight map with the same size as the feature size of the spatial map. (Value range is [0,1]), representing the importance distribution of various locations in the canopy spatial features. The height-guided weighting map... ,in: Indicates the height of the canopy spatial map features. The width of the canopy spatial map features is represented by a matrix where each element is a real value, indicating the importance of the corresponding spatial location to yield prediction.

[0212] It should be understood that, Indicates the first line, number The weight value of a column position indicates its contribution to yield prediction; a larger value indicates a higher contribution, while a smaller value indicates a lower contribution. By establishing a two-dimensional weight matrix, key vegetation areas can be highlighted in the spatial distribution dimension, thereby reducing the interference of redundant areas on the feature fusion process.

[0213] In the specific implementation process, a height-guided weight map is generated based on the local pattern of canopy height features; the height-guided weight map is multiplied point by point with the canopy spatial features to form a weighted spatial distribution feature map; then, based on the weighted distribution map, the regions with larger weight values ​​are extracted and retained, and finally the key spatial regions used for yield prediction are obtained.

[0214] The height-guided weight map and the canopy spatial map features are multiplied point by point to form a weighted distribution of spatial map features.

[0215] High-guided weight map Features of canopy spatial map Perform a product operation pixel by pixel to form a spatially weighted feature map. :

[0216] ;

[0217] Among them: features of the canopy space map Indicates spatial location Place, No. Original spatial feature values ​​on each channel; Indicates spatial location The attention weight value reflects the importance of that position in the yield prediction task; This represents the feature value after attention weighting.

[0218] This can be understood as: taking a certain spatial location The original eigenvalues ​​on Then use the attention weight at that position. We then weight them to obtain new weighted eigenvalues. This involves scaling up or down each pixel in the spatial feature map based on the value in the attention map; that is: if If the value is large, the location is important, the corresponding feature value is amplified, and the network will pay more attention to this location; if... If the value is smaller, the importance of that location is lower, and the corresponding eigenvalue is suppressed, reducing its influence in the final prediction. This automatically focuses on areas useful for yield prediction (such as vegetation-covered areas) while weakening the interference of irrelevant areas (such as bare soil or shaded areas).

[0219] Based on the weighted distribution of spatial map features, the key regions for yield prediction in the canopy spatial map features are selected.

[0220] Through the Perform global attention-weighted pooling to select several key regions as the spatial basis for yield judgment. The locations of the key regions are output in the form of spatial coordinates.

[0221] By fusing key regions of canopy spatial map features and local patterns of canopy height features, a trained deep feature fusion network is obtained.

[0222] The local canopy height feature H_feat obtained in step S104.2 is concatenated with the key region spatial map feature vector extracted in step S104.3 and then fed into the fusion prediction module (such as the FC layer of a fully connected network) for regression prediction.

[0223] The network training adopts an end-to-end strategy, with the input being a set of structural integrated feature images and corresponding measured yield labels, and the output being continuous yield predictions.

[0224] Once trained, the model can estimate yield for any crop region, and the prediction results have high spatial resolution and hierarchical interpretation capabilities.

[0225] Example 2

[0226] like Figure 2 As shown in the figure, the parts not described in detail in this embodiment are as shown in Embodiment 1. This embodiment provides a crop yield prediction system integrating multi-source remote sensing data, including an image acquisition module, a canopy illumination simulation module, a canopy feature correction module and a yield prediction module. The modules transmit data to each other through wired and / or wireless means.

[0227] The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of the target crop plot, and to construct the vertical height profile of the canopy based on the backscattering characteristics of the crop canopy in the radar remote sensing image; the image acquisition module is also used to construct the geometric projection relationship of the canopy based on the solar incidence azimuth angle and the observation sensor angle.

[0228] To further explain, the vertical height profile of the crop canopy is constructed based on the backscattering characteristics of the crop canopy in radar remote sensing images, including:

[0229] Based on the polarization scattering characteristics of radar remote sensing images, the polarization phase difference between the scattered signals from the top of the canopy and the ground is obtained; including the following steps:

[0230] HH polarization and VV polarization echo signals of stable scattering targets on the ground within the same imaging area are extracted, and the average complex scattering coefficient is calculated to obtain the ground reference target signal.

[0231] Based on the HH polarization and VV polarization echoes acquired simultaneously by the polarization radar sensor, the canopy polarization phase of the scattered signal at the top of the canopy is calculated.

[0232] The polarization phase difference between the canopy top scattering signal and the ground reference target signal is calculated by utilizing the complex difference in the complex scattering coefficients of the canopy top scattering signal and the ground scattering signal in the HH and VV channels.

[0233] The effective propagation path of radar signals penetrating the canopy is determined by the polarization phase difference.

[0234] The echo locations at different depths in the canopy are determined based on the effective propagation path.

[0235] A height profile that can represent the true vertical height of the crop canopy is generated based on the echo positions of each layer.

[0236] The canopy illumination simulation module is used to determine the canopy structure shading area based on the canopy projection relationship using a three-dimensional canopy light transmission simulation algorithm, and to derive the canopy transmitted light distribution characteristics by using the spatial distribution differences between vegetation and shadow within the shading area.

[0237] Specifically, the spatial distribution differences between vegetation and shadow within the shaded area are used to deduce the canopy transmittance characteristics, including:

[0238] Based on the spectral reflectance of vegetation and shade in different bands of the shaded area, the spectral difference pattern between the shaded area and the vegetated area is obtained;

[0239] The spatial locations of multiple scattering regions within the shaded area and single scattering regions within the vegetated area are identified based on the spectral difference pattern recognition; wherein, the spatial locations of multiple scattering regions within the shaded area and single scattering regions within the vegetated area are identified based on the spectral difference pattern recognition, including:

[0240] Extract the spectral reflectance distribution of multi-band images of shaded and vegetated areas;

[0241] Analyze the reflectivity variation patterns of each band to identify stable single-scattering characteristic bands in vegetated areas and significant multiple-scattering characteristic bands in shaded areas.

[0242] Using the identified single scattering characteristic band and multiple scattering characteristic band, the specific spatial locations of single scattering and multiple scattering are determined;

[0243] To further explain, using the defined characteristic bands of single scattering and multiple scattering, the specific spatial locations of single and multiple scattering are determined, including:

[0244] Based on the characteristic bands of the shaded and vegetated areas, a spatial spectral difference map is constructed.

[0245] The spatial autocorrelation algorithm is used to analyze the spatial spectral difference map to determine the single scattering or multiple scattering properties at each spatial location;

[0246] Based on the scattering properties of each spatial location, the specific spatial distribution results of the single scattering area and multiple scattering area in the canopy-shaded region are obtained;

[0247] By analyzing the spectral attenuation characteristics of the multiple scattering region, the spatial location and extent of transmitted light attenuation within the blocked area can be determined.

[0248] Based on the spatial location and degree of transmitted light attenuation, the transmitted light distribution characteristics that characterize the true light distribution characteristics inside the canopy are obtained.

[0249] The canopy feature correction module is used to correct the canopy reflectance in the optical remote sensing image based on the canopy transmitted light distribution characteristics, and combine the corrected canopy reflectance with the canopy vertical height profile to generate a comprehensive feature of the vegetation canopy structure.

[0250] Specifically, the comprehensive characteristics of vegetation canopy structure are generated by combining the corrected canopy reflectivity with the canopy vertical height profile, including:

[0251] The corrected canopy reflectance was decomposed into different spectral bands to obtain detailed reflectance characteristics of vegetation leaves and branches.

[0252] The vertical height profile of the canopy is divided into several vertical layers, and the canopy thickness information corresponding to each layer is extracted.

[0253] Based on the detailed reflective features of vegetation leaves and branches, the main vegetation composition types in each vertical layer are identified.

[0254] Based on vertical layer thickness information and vegetation composition type information, comprehensive characteristics of vegetation canopy structure that characterize the differences in three-dimensional canopy structure are obtained.

[0255] The yield prediction module is used to predict crop yield based on the comprehensive features of the vegetation canopy structure driven by a deep feature fusion network; wherein, the deep feature fusion network uses an attention mechanism to fuse spatial map features and height features.

[0256] Specifically, training methods for deep feature fusion networks driven by comprehensive features include:

[0257] A dual-input network structure is constructed, where the first input branch receives canopy height features and the second input branch receives canopy spatial map features.

[0258] Convolutional operations are used to extract local pattern features of canopy height and spatial map features, respectively;

[0259] An attention mechanism module was designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features.

[0260] By fusing key regions of canopy spatial map features and local patterns of canopy height features, a trained deep feature fusion network is obtained.

[0261] To further explain, an attention mechanism module is designed, guided by local patterns in canopy height features, to identify key regions in the canopy spatial map features that are relevant to yield prediction, including:

[0262] A height-guided weight map is generated by utilizing local patterns of canopy height features;

[0263] The height-guided weight map and the canopy spatial map features are multiplied point by point to form a weighted distribution of spatial map features.

[0264] Based on the weighted distribution of spatial map features, the key regions for yield prediction in the canopy spatial map features are selected.

[0265] This embodiment provides a crop yield prediction system integrating multi-source remote sensing data, used to execute a crop yield prediction method integrating multi-source remote sensing data provided in the above embodiments of the present invention. The specific methods and processes for implementing the corresponding functions of each structure in the crop yield prediction system based on integrated multi-source remote sensing data are detailed in the above embodiments of the crop yield prediction method integrating multi-source remote sensing data, and will not be repeated here.

[0266] Example 3

[0267] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0268] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0269] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0270] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0271] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0272] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0273] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0274] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for integrated multi-source remote sensing data crop yield prediction, characterized in that, The method comprises the following steps: S101: Obtain optical remote sensing images and radar remote sensing images of a target crop plot, construct a crown vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images, and construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; S102: According to the canopy geometric projection relationship, determine the canopy structure shadow area by a three-dimensional canopy light transmission simulation algorithm, and derive the canopy transmission light distribution characteristics by using the spatial distribution difference between the vegetation and the shadow in the shadow area; S103: Correct the canopy reflectivity in the optical remote sensing images according to the canopy transmission light distribution characteristics, and generate comprehensive features of the vegetation canopy structure by combining the corrected canopy reflectivity and the canopy vertical height profile; S104: Based on the comprehensive features of the vegetation canopy structure, drive a deep feature fusion network to predict the crop yield, and the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

2. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, Based on the backscattering characteristics of the crop canopy in the radar remote sensing images, the crown vertical height profile is constructed, comprising: Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the top of the canopy and the ground scattering signal is obtained; The effective propagation path of the radar signal penetrating the canopy is determined through the polarization phase difference; Based on the effective propagation path, the echo positions of different depth layers in the canopy are determined; According to the echo positions of each layer, a height profile capable of representing the true vertical height of the crop canopy is generated.

3. The crop yield prediction method integrating multi-source remote sensing data according to claim 2, characterized in that, Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the top of the canopy and the ground scattering signal is obtained, comprising: Extract the HH polarization and VV polarization echo signals of the ground stable scattering target in the same imaging area, and calculate the average complex scattering coefficient to obtain the ground reference target signal; Based on the HH polarization and VV polarization echoes obtained by the polarization radar sensor simultaneously, the canopy polarization phase of the top of the canopy scattering signal is calculated; The complex difference of the complex scattering coefficients of the top of the canopy scattering signal and the ground reference target signal under HH and VV channels is used to calculate the polarization phase difference between the top of the canopy scattering signal and the ground scattering signal. 4.The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, The spatial distribution difference between the vegetation and the shadow in the shadow area is used to derive the canopy transmission light distribution characteristics, comprising: According to the spectral reflectance of the vegetation and the shadow in the shadow area in different bands, the spectral difference pattern of the shadow area and the vegetation area is obtained; Based on the spectral difference pattern, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified; Through the spectral attenuation characteristics of the multiple scattering area, the spatial position and degree of transmission light attenuation in the shadow area are determined; Based on the spatial position and degree of transmission light attenuation, the transmission light distribution characteristics representing the real light distribution characteristics in the canopy are obtained.

5. The crop yield prediction method integrating multi-source remote sensing data according to claim 4, characterized in that, Based on the spectral difference pattern, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified, comprising: Extract the spectral reflectance distribution of the multi-band images of the shadow area and the vegetation area; Analyze the change pattern of the reflectance of each band to determine the stable single scattering characteristic band in the vegetation area and the significant multiple scattering characteristic band in the shadow area; The specific spatial positions of single scattering and multiple scattering are calibrated by using the determined single scattering characteristic band and multiple scattering characteristic band.

6. The crop yield prediction method integrating multi-source remote sensing data according to claim 5, wherein, The specific spatial positions of single scattering and multiple scattering are calibrated by using the determined single scattering characteristic band and multiple scattering characteristic band, comprising: A spatial spectrum difference map is constructed according to the characteristic bands of the shadow area and the vegetation area; A spatial autocorrelation algorithm is used to analyze the spatial spectrum difference map to determine the single scattering or multiple scattering attribute of each spatial position; According to the scattering attribute of each spatial position, classification calibration is performed to obtain the specific spatial position distribution results of the single scattering area and the multiple scattering area of the canopy sheltered area.

7. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, A vegetation canopy structure comprehensive feature is generated by combining the corrected canopy reflectance and the canopy vertical height profile, comprising: The corrected canopy reflectance is spectrally decomposed at different bands to obtain the detailed reflection characteristics of vegetation leaves and branches; The canopy vertical height profile is divided into several vertical layers, and the canopy thickness information of each layer is extracted; According to the detailed reflection characteristics of vegetation leaves and branches, the main composition types of vegetation in each vertical layer are identified; According to the vertical layer thickness information and the vegetation composition type information, the vegetation canopy structure comprehensive feature representing the three-dimensional structure difference of the canopy is obtained.

8. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, The training method of the deep feature fusion network driven by the comprehensive feature comprises: A double-input network structure is constructed, wherein a first input branch receives the canopy height feature, and a second input branch receives the canopy spatial map feature; Convolution operations are used to extract local pattern features of the canopy height feature and the spatial map feature, respectively; An attention mechanism module is designed to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local pattern of the canopy height feature; Feature fusion is performed using the key regions of the canopy spatial map feature and the local pattern of the canopy height feature to obtain the trained deep feature fusion network.

9. The integrated multi-source remote sensing data crop yield prediction method of claim 8, wherein, The attention mechanism module is designed to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local pattern of the canopy height feature, comprising: An altitude guide weight map is generated using the local pattern of the canopy height feature; A point-by-point multiplication operation is performed between the altitude guide weight map and the canopy spatial map feature to form a spatial map feature weighted distribution; According to the spatial map feature weighted distribution, the key region positions in the canopy spatial map feature used for yield prediction are screened.

10. A system for integrated multi-source remote sensing data-based crop yield prediction, configured to perform the method of integrated multi-source remote sensing data-based crop yield prediction according to any one of claims 1-9, characterized in that, An image acquisition module, a canopy illumination simulation module, a canopy feature correction module, and a yield prediction module are included. The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of a target crop plot, and to construct a canopy vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images; the image acquisition module is also used to construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; The canopy illumination simulation module is used to determine a canopy structure sheltered area by a three-dimensional canopy light transmission simulation algorithm according to the canopy projection relationship, and to derive a canopy transmission light distribution feature using the spatial distribution difference between vegetation and shadows in the sheltered area; a canopy feature correction module configured to correct the canopy reflectivity in the optical remote sensing image according to the canopy transmission light distribution feature, and combine the corrected canopy reflectivity with the canopy vertical height profile to generate a comprehensive feature of a vegetation canopy structure; a yield prediction module configured to predict crop yield based on the comprehensive feature of the vegetation canopy structure driving a deep feature fusion network; wherein the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

Images