Crop yield prediction method and system integrating multi-source remote sensing data

Abstract

The invention discloses a crop yield prediction method and system integrated with multi-source remote sensing data, and the method comprises the steps: obtaining an optical image and a radar image of a target crop plot, and constructing a vertical height contour of a crop canopy based on the back scattering characteristics of the radar image; meanwhile, a canopy geometric projection relation is established by combining a sun incidence azimuth angle and a sensor angle, a canopy shielding area is identified through a three-dimensional light transmission simulation algorithm, canopy transmission light distribution characteristics are deduced according to the spatial distribution difference of vegetation and shadow, optical image reflectivity is corrected by using transmission light distribution, and the canopy shielding area is obtained. And combining with the height contour to generate comprehensive features of the vegetation canopy structure, and fusing the canopy space and the height features by adopting a depth feature fusion network based on an attention mechanism, thereby realizing high-precision prediction of the crop yield. According to the method, the influence of illumination conditions and image geometric distortion on the estimation precision is effectively overcome, and the accuracy and stability of the multi-source remote sensing data in crop yield prediction are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural remote sensing application, and in particular to a crop yield prediction method and system integrated with multi-source remote sensing data. BACKGROUND

[0002] With the development of high-resolution remote sensing image technology, fine monitoring and prediction of crop growth status and yield using multi-source remote sensing data has become an important research direction in the field of agricultural remote sensing. In the prior art, optical remote sensing images are widely used in vegetation index calculation and biomass estimation due to their rich spectral information; and synthetic aperture radar (SAR) remote sensing can penetrate clouds and vegetation surface to obtain structural information of crop canopy, especially in polarization mode, which can extract the difference in backscattering between canopy and ground for structure estimation.

[0003] However, in practical applications, optical images are limited by observation angle, solar incident direction and canopy structure complexity, which are prone to shadow blocking and transmitted light interference, resulting in errors in reflectance calculation and affecting the accuracy of crop yield prediction. Existing research attempts to combine radar remote sensing and optical remote sensing for joint modeling, but most of them only fuse features at the feature level, do not fully utilize radar polarization scattering information to construct high-precision canopy vertical height structure, and do not systematically simulate the influence of three-dimensional light transmission characteristics of canopy shadow area on reflectivity.

[0004] In addition, most of the existing crop yield prediction models based on deep learning use two-dimensional image features as input, ignoring the spatial distribution differences and vertical level information of the canopy structure, lacking a deep fusion mechanism between height dimension and spatial image features, and being difficult to meet the generalization requirements under different crop varieties, land environment and remote sensing conditions.

[0005] In view of the above problems, the present application provides a crop yield prediction method and system integrated with multi-source remote sensing data. SUMMARY

[0006] The purpose of the present application is to provide a crop yield prediction method and system integrated with multi-source remote sensing data, a multi-source remote sensing data integration method considering the influence of canopy transmitted light distribution on spectral reflectance correction, to improve the accuracy and robustness of crop three-dimensional canopy structure modeling and yield prediction.

[0007] In order to achieve the above purpose, the present application provides the following technical solutions: In a first aspect, the present application provides a crop yield prediction method integrated with multi-source remote sensing data, comprising the following steps: S101: Obtain optical remote sensing image and radar remote sensing image of a target crop plot, construct a crown vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing image, and construct a crown geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; S102: According to the crown geometric projection relationship, determine the crown structure shielding area by a three-dimensional crown light transmission simulation algorithm, and derive the crown transmission light distribution characteristics by using the spatial distribution difference between the vegetation and the shadow in the shielding area; S103: Correct the crown reflectivity in the optical remote sensing image according to the crown transmission light distribution characteristics, and generate the comprehensive features of the vegetation canopy structure by combining the corrected crown reflectivity and the crown vertical height profile; S104: Based on the comprehensive features of the vegetation canopy structure, drive the deep feature fusion network to predict the crop yield, and the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

[0008] As a preferred technical solution of the first aspect of the application, the crown vertical height profile is constructed based on the backscattering characteristics of the crop canopy in the radar remote sensing image, comprising: Based on the polarization scattering characteristics of the radar remote sensing image, the polarization phase difference between the crown top scattering signal and the ground scattering signal is obtained; The effective propagation path of the radar signal penetrating the crown layer is determined through the polarization phase difference; Based on the effective propagation path, the echo positions of different depth layers in the crown layer are determined; According to the echo positions of each layer, a height profile capable of representing the true vertical height of the crop canopy is generated.

[0009] As a preferred technical solution of the first aspect of the application, based on the polarization scattering characteristics of the radar remote sensing image, the polarization phase difference between the crown top scattering signal and the ground scattering signal is obtained, comprising: Extract the HH polarization and VV polarization echo signals of the ground stable scattering target in the same imaging area, and calculate the average complex scattering coefficient to obtain the ground reference target signal; Based on the HH polarization and VV polarization echoes obtained by the polarization radar sensor simultaneously, the crown polarization phase of the crown top scattering signal is calculated; The complex difference of the complex scattering coefficients of the crown top scattering signal and the ground reference target signal under HH and VV channels is used to calculate the polarization phase difference between the crown top scattering signal and the ground scattering signal.

[0010] As a preferred technical solution of the first aspect of the application, the crown transmission light distribution characteristics are derived by using the spatial distribution difference between the vegetation and the shadow in the shielding area, comprising: According to the spectral reflectivity of the different bands of the shadow area and the vegetation area, a spectral difference pattern of the shadow area and the vegetation area is obtained; Based on the spectral difference pattern, specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified; Through the spectral attenuation characteristics of the multiple scattering area, spatial positions and degrees of the transmission light attenuation in the shadow area are determined; Based on the spatial positions and degrees of the transmission light attenuation, transmission light distribution characteristics representing the real light distribution characteristics in the canopy are obtained.

[0011] As a preferred technical solution of the first aspect of the present application, based on the spectral difference pattern, specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified, comprising: The spectral reflectivity distribution of the multi-band image of the shadow area and the vegetation area is extracted; The change pattern of the reflectivity of each band is analyzed, and the stable single scattering characteristic band in the vegetation area and the significant multiple scattering characteristic band in the shadow area are determined; The specific spatial positions of the single scattering and the multiple scattering are calibrated by using the determined single scattering characteristic band and the multiple scattering characteristic band.

[0012] As a preferred technical solution of the first aspect of the present application, the specific spatial positions of the single scattering and the multiple scattering are calibrated by using the determined single scattering characteristic band and the multiple scattering characteristic band, comprising: According to the characteristic bands of the shadow area and the vegetation area, a spatial spectral difference map is constructed; The spatial autocorrelation algorithm is used to analyze the spatial spectral difference map, and the single scattering or multiple scattering attribute of each spatial position is determined; According to the scattering attribute of each spatial position, classification calibration is performed, and the specific spatial position distribution result of the single scattering area and the multiple scattering area in the canopy shadow area is obtained.

[0013] As a preferred technical solution of the first aspect of the present application, the corrected canopy reflectivity and the canopy vertical height profile are combined to generate the vegetation canopy structure comprehensive feature, comprising: The corrected canopy reflectivity is subjected to spectral decomposition of different bands, and the detailed reflection characteristics of the vegetation leaves and branches are obtained; The canopy vertical height profile is divided into a plurality of vertical layers, and the canopy thickness information corresponding to each layer is extracted; According to the detailed reflection characteristics of the vegetation leaves and branches, the main component types of the vegetation in each vertical layer are identified; According to the vertical layer thickness information and the vegetation component type information, the vegetation canopy structure comprehensive feature representing the three-dimensional structure difference of the canopy is obtained.

[0014] As a preferred embodiment of the first aspect of the present invention, a training method for a comprehensive feature-driven deep feature fusion network includes: A dual-input network structure is constructed, where the first input branch receives canopy height features and the second input branch receives canopy spatial map features. Convolutional operations are used to extract local pattern features of canopy height and spatial map features, respectively; An attention mechanism module was designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features. By fusing key regions of canopy spatial map features and local patterns of canopy height features, a trained deep feature fusion network is obtained.

[0015] As a preferred embodiment of the first aspect of the present invention, an attention mechanism module is designed to identify key regions in the canopy spatial map features that are relevant to yield prediction, guided by local patterns of canopy height features, including: A height-guided weight map is generated by utilizing local patterns of canopy height features; The height-guided weight map and the canopy spatial map features are multiplied point by point to form a weighted distribution of spatial map features. Based on the weighted distribution of spatial map features, the key regions for yield prediction in the canopy spatial map features are selected.

[0016] In a second aspect, the present invention provides a crop yield prediction system integrating multi-source remote sensing data, based on the implementation of the first aspect, including an image acquisition module, a canopy illumination simulation module, a canopy feature correction module and a yield prediction module, wherein the modules transmit data to each other via wired and / or wireless means. The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of the target crop plot, and to construct the vertical height profile of the canopy based on the backscattering characteristics of the crop canopy in the radar remote sensing image; the image acquisition module is also used to construct the geometric projection relationship of the canopy based on the solar incidence azimuth angle and the observation sensor angle. The canopy illumination simulation module is used to determine the canopy structure shading area based on the canopy projection relationship using a three-dimensional canopy light transmission simulation algorithm, and to derive the canopy transmitted light distribution characteristics by using the spatial distribution differences between vegetation and shadow within the shading area. The canopy feature correction module is used to correct the canopy reflectance in the optical remote sensing image based on the canopy transmitted light distribution characteristics, and combine the corrected canopy reflectance with the canopy vertical height profile to generate a comprehensive feature of the vegetation canopy structure. The yield prediction module is configured to predict crop yield based on the integrated feature of the vegetation canopy structure driving a deep feature fusion network; wherein the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

[0017] In the above technical solution, the present application provides technical effects and advantages: The present application uses optical remote sensing and radar remote sensing data in combination, uses radar backscattering characteristics to construct a canopy vertical height profile, and combines with optical reflectivity, breaks through the limitations of single remote sensing source in spatial resolution, imaging conditions and data stability, realizes the complement and enhancement of multi-source information. By introducing the canopy geometric projection relationship and three-dimensional light transmission simulation algorithm, the canopy shielding effect caused by the sun incident angle and the observation angle is effectively identified and corrected, the interference of shadow distribution and geometric distortion on image reflectivity correction is eliminated, and the authenticity of the canopy structure feature is improved.

[0018] The present application corrects the optical reflectivity by using the transmission light distribution, and generates a canopy structure integrated feature which can represent the spatial distribution and vertical stratification at the same time by combining with the canopy vertical height profile, so as to ensure the accurate description of the differences of crop three-dimensional structure. The deep feature fusion network with double-branch input structure and attention mechanism establishes effective association between spatial features and height features, and focuses on the key area related to yield prediction, without relying on high computing resources, and can complete processing on the conventional remote sensing data and computing platform, ensuring the application value of the technical solution in the actual scenes such as agricultural production management, crop growth monitoring and yield evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0020] Figure 1 The flow chart of the crop yield prediction method integrated with multi-source remote sensing data of the present application; Figure 2 The system framework diagram of the crop yield prediction system integrated with multi-source remote sensing data of the present application. DETAILED DESCRIPTION

[0021] Example implementations are now described with reference to the drawings. Example implementations can, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example implementations to those skilled in the art. The accompanying drawings are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification. The drawings are not intended to be to scale. Like reference numerals refer to like or similar elements throughout the several views.

[0022] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more example implementations. In the following description, numerous specific details are provided to give a thorough understanding of example implementations. One skilled in the relevant art will recognize, however, that the

[0023] Example 1 As Figure 1 shown, the present application provides a crop yield prediction method integrating multi-source remote sensing data, comprising the following steps: S101: Obtain optical remote sensing images and radar remote sensing images of a target crop plot, construct a canopy vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images, and construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle.

[0024] It should be noted that wheat is taken as the target crop, and the target plot area is about 500m x 500m. By utilizing the complementary characteristics of optical remote sensing images and radar remote sensing images, image data is synchronously obtained in the critical growth period of crops (such as the heading to the filling period), the image acquisition time is consistent with the critical growth period of crops, and thus the fine canopy three-dimensional structure information of the target crop plot is obtained to reflect the representativeness of the canopy structure. Among them, the optical remote sensing images are preferably from high-resolution multispectral satellite or unmanned aerial vehicle images, which are used to extract the spectral reflection characteristics of the canopy; the radar remote sensing images are preferably polarimetric synthetic aperture radar (PolSAR) images, which are used to extract the vertical structure characteristics of the vegetation canopy.

[0025] The specific implementation process of constructing the canopy vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images includes: Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the scattering signals of the canopy top and the ground is obtained; It should be noted that the polarization scattering feature refers to the combination relationship of the complex amplitude and phase of the target scattering response observed by the synthetic aperture radar system under different transmitting and receiving polarization states. In a microwave radar, the polarization direction of the electromagnetic wave can be divided into two orthogonal states of horizontal (H) and vertical (V). When the radar transmits H or V polarized waves and receives H or V polarized echoes, four basic echo signals can be obtained: HH polarization: horizontal transmission, horizontal reception; HV polarization: horizontal transmission, vertical reception; VH polarization: vertical transmission, horizontal reception; VV polarization: vertical transmission, vertical reception.

[0026] For any pixel , the above echo signals are complex numbers: Among them: is a complex scattering coefficient; reflects the reflection ability of the target to the incident wave, represents the scattering intensity (amplitude), and is related to the reflection ability of the scattering surface; represents the phase, which reflects the electromagnetic wave propagation path difference and target dielectric characteristics, The four complex scattering coefficients constitute the polarization scattering matrix of the target: That is, the four complex scattering coefficients are respectively: Based on the polarization scattering matrix, the scattering mechanism of the target is reflected: that is, the physical process of electromagnetic wave reflection, refraction and re-radiation in vegetation, ground and artificial structure. Based on the scattering difference embodied by the polarization scattering matrix, the different polarization responses of radar waves in the crown layer and the ground can be used, and the scattering characteristic difference of the polarization radar signal between the vegetation crown layer and the ground can be used to accurately identify and distinguish the position of the crown layer top and the ground. The position reference in the vertical direction of the crown layer is determined.

[0027] Among them, the polarization phase difference between the crown layer top and the ground scattering signals is obtained, which specifically includes the following steps: Extract the HH polarization and VV polarization echo signals of the ground stable scattering target in the same imaging area, and calculate the average complex scattering coefficient to obtain the ground reference target signal; ​​​It should be noted that the ground reference target signal refers to selecting a non-vegetation covered area (such as a farmland road, bare land or a metal target) as a ground reference point in the same radar imaging area, and the scattering characteristics of the ground reference target signal are stable, the surface scattering is dominant, and the polarization response of the signal is usually as follows: the HH polarization channel is strong, the VV polarization channel is second, the polarization coherence is high (|ρ|>0.9), and the phase is stable (standard deviation<0.1 rad). The complex scattering coefficients of the HH horizontal polarization channel and the VV vertical polarization channel are extracted, specifically as follows: , ; In order to eliminate local angle and noise differences, the complex amplitude average is used in the ground high coherence window to obtain the ground reference quantity: ; Subsequently, in order to obtain the main definition value of the ground mirror reflection component, the Freeman-Durden three-component polarization decomposition algorithm is used, and the polarization covariance matrix is constructed according to the polarization scattering matrix: , wherein the scattering vector is: ; The formula of the polarization covariance matrix is: ; , wherein the angle bracket represents a local average operation in a spatial sliding window (usually 3*3 or 5*5 pixels), and the superscript represents conjugation; each element in the polarization covariance matrix represents the complex correlation relationship between different polarization channels, and the diagonal elements reflect the scattering energy of each channel, and the non-diagonal elements reflect the coherence between channels.

[0028] According to the electromagnetic scattering theory, the polarization response of different scattering mechanisms can be described by an idealized geometric model, and the Freeman-Durden three-component model represents three kinds of scattering template matrices of surface scattering, volume scattering and double reflection by establishing three kinds of typical template matrices , and , specifically as follows: The surface scattering template matrix simulates single reflection of radar waves and smooth ground or mirror bodies: ; , wherein: is the ground inclination angle or the equivalent incidence angle.

[0029] The volume scattering template matrix represents the isotropic volume scattering generated by randomly oriented leaves and branches in the vegetation canopy: ; Dual-reflection template matrix The dual-reflection path is described as: ; Where: is the geometric dip of the dual-reflection path.

[0030] All of them can be obtained by the incident angle, crown layer geometric parameters or experimental calibration, so as to reflect the geometric and electromagnetic characteristics of different scattering mechanisms in the model.

[0031] After obtaining the observed polarization covariance matrix and three kinds of scattering template matrix, the total scattering model is converted into a solvable linear system; the scattering mechanism model is established, specifically: ; Where, , and respectively represent the power coefficients of surface, volume and dual-reflection, which constitute the total scattering energy distribution of the observed polarization signal; there are 9 matrix elements in the polarization covariance matrix but only 3 unknown coefficients, which belong to the overdetermined equation group, in order to make the equation can be solved numerically, each matrix is expanded into a vector form, and the matrix elements are expanded into a one-dimensional vector according to the row, and let: ; ; ; ; Then the linear system can be written as: ; In order to reduce the influence of observation noise, the least square method is used to solve: ; That is: ; The surface scattering power is obtained by least square solution, and the ground reference polarization phase is obtained in the ground window: ; The ground reference target signal is defined as: ; It should be noted that the ground reference target signal refers to the surface scattering complex component obtained by the decomposition; the complex amplitude average ground reference quantity is used as a reference check quantity and does not participate in the subsequent main calculation. The ground reference target signal serves as the phase reference for subsequent crown layer signal calculation, providing a stable and reliable phase reference for crown layer scattering signal.

[0032] The canopy top scattering signal is calculated based on HH polarization and VV polarization echoes obtained by the polarization radar sensor simultaneously. It is to be noted that the canopy top scattering signal is a complex signal dominated by volume scattering caused by the upper leaves and twigs of the crop canopy in the polarization radar image, and the phase center of which corresponds to the upper surface of the canopy. In general, the scattering energy in the HV and VH channels is strong, and this feature reflects that there are a large number of randomly oriented leaf blades, branches and other volume scattering units in the crop canopy, which is an important source of volume scattering components of the canopy. However, the phase stability of the HV and VH channels is poor, and the spatial coherence is low, which makes it difficult to be used as the basis for interferometric phase analysis. In contrast, the scattering of the HH and VV channels is mainly affected by the upper structure of the canopy and the ground reflection, and the phase changes continuously with space and has high coherence, which can stably reflect the average propagation path difference from the upper surface of the canopy to the ground. That is, the phase difference is calculated by using the complex scattering coefficients of the HH and VV channels to ensure the coherence and physical interpretability of the calculation results.

[0033] Specifically, the calculation formula of the canopy top scattering signal is as follows: ; wherein, and are the complex scattering coefficients corresponding to the horizontal polarization and vertical polarization channels, respectively, that is, the canopy top scattering signal is the average complex component of the HH and VV channels to the comprehensive echo above the canopy minus the ground reference target signal , which eliminates the interference of the ground reflection and has stable spatial continuity.

[0034] The complex representation of the canopy top scattering signal is as follows: ; wherein: is the amplitude component of the canopy volume scattering; is the canopy polarization phase of the canopy top volume scattering, which is used for subsequent phase difference calculation.

[0035] The polarization phase difference between the canopy top scattering signal and the ground scattering signal is calculated by using the difference between the HH polarization and VV polarization signals of the canopy top and ground scattering targets.

[0036] Specifically, the HH polarization and the VV polarization signal are the original observation of the two main polarization channels of the radar, which are the basic components of all scattering characteristics; the canopy top scattering signal is a complex signal component formed by the scattering of the upper layer of the vegetation canopy under the same polarization channel, which represents the main reflection center of the upper surface of the canopy; the ground reference target signal is a complex signal component formed by the scattering of the surface of bare land or stable artificial structure under the same polarization channel, which represents the ground reflection center; that is, the canopy top scattering signal and the ground scattering signal are both decomposed from the HH / VV polarization channel component, and by comparing the complex phase difference of the two under the same polarization framework, the equivalent path difference of the radar wave propagating from the upper surface of the canopy to the ground reflection surface can be quantified.

[0037] Further, in the calculation of the polarization phase difference, the complex difference between the canopy top scattering signal and the ground reference target signal is used to calculate the polarization phase difference between them: ; Wherein: represents the complex conjugate, represents the phase angle of the complex number, and the polarization phase difference quantifies the equivalent path difference of the radar wave propagating from the upper surface of the canopy to the ground reflection surface, reflecting the comprehensive difference of the canopy and the ground in the propagation medium, electromagnetic properties and geometric height.

[0038] Further, the canopy top signal and the ground reference signal both come from the same radar scene and the same set of HH / VV polarization channels, but only represent the canopy scattering center and the ground mirror scattering center respectively. By calculating the complex phase difference under the same polarization framework, the relative delay of the vertical propagation path of the canopy can be obtained; the polarization phase difference distribution can be directly used as an input parameter for canopy height inversion and polarization interference calculation, providing a reliable phase reference for modeling the vertical structure of the crop canopy.

[0039] The effective propagation path of the radar signal penetrating the canopy is determined by the polarization phase difference; In specific implementation, based on the polarization phase difference obtained as described above, the actual propagation trajectory of the radar wave into the canopy is determined by using polarization interference processing technology. The propagation trajectory can be determined by establishing a polarization interference scattering model and combining the prior information of the structure of the vegetation canopy to deduce the specific penetration path of the radar signal in the canopy, so as to realize the identification of the detection path in the vertical direction of the canopy.

[0040] The echo positions of different depth layers in the canopy are determined based on the effective propagation path. It can be understood that the radar wave will interact with the vegetation structure of different height layers after entering the canopy, thereby forming echo signals at different positions at different height layers of the canopy. By using the aforementioned explicit effective propagation path and performing joint analysis in the time domain and the spatial domain, the specific positions of the echoes at different depths of the vegetation canopy can be determined layer by layer, so that the echo position information capable of clearly characterizing the vertical structure inside the vegetation canopy is obtained.

[0041] According to the echo positions of each layer, a height profile capable of characterizing the true vertical height of the crop canopy is generated.

[0042] In specific implementation, the echo positions of each canopy depth layer determined in the foregoing are connected step by step based on the echo signal intensity distribution curve by using a hierarchical clustering method, and in combination with a ground position reference point, a fine, true and stable vertical height profile of the crop canopy can be finally generated. The profile directly reflects the true vertical structure change inside the canopy, and provides core data basis for subsequent analysis of the canopy structure characteristics.

[0043] Further, the specific implementation method of constructing the canopy geometric projection relationship according to the solar incident azimuth and the observation sensor angle is as follows: First, according to the position parameters of the sun at the time of remote sensing image acquisition and the orbit and attitude information of the sensor, the incident azimuth and incident elevation angle of the sunlight irradiating the ground, and the viewing azimuth and viewing tilt angle of the remote sensing sensor imaging are determined.

[0044] Secondly, based on the foregoing canopy height profile information, the three-dimensional structure of the canopy is projected onto a two-dimensional plane. Through a ray tracing algorithm or a ray method, the projection area and the occlusion relationship of each spatial position in the canopy under the light condition can be obtained, so as to accurately establish the spatial position relationship between the vegetation and the shadow area inside the canopy.

[0045] It should be noted that the canopy vertical height profile characterizes the structural distribution characteristics of the crop canopy in the vertical direction, and the canopy geometric projection relationship defines the spatial occlusion and projection relationship of each structural unit of the canopy under the incident direction of the sun and the observation viewing angle. Based on the canopy geometric projection relationship, the spatial distribution difference between the vegetation area and the shadow area is identified, and the spatial positions of the leaf layer, the gap and the shadow area in the canopy are clearly defined in the three-dimensional space, so as to provide accurate geometric constraints for the illumination distribution calculation and the light transmission characteristic simulation of the optical image. For step S102, the physical correspondence between the canopy structure and the illumination propagation is established by the three-dimensional light transmission simulation algorithm and the spectral scattering analysis, and the collaborative modeling of the canopy geometric structure information and the optical radiation information is realized.

[0046] S102: According to the canopy geometric projection relationship, the canopy structure occlusion area is determined by a three-dimensional canopy light transmission simulation algorithm, and the spatial distribution difference between the vegetation and the shadow in the occlusion area is used to derive the transmission light distribution characteristics of the canopy; It should be noted that: according to the established crown layer geometric projection relationship, the spatial distribution of the crown layer structure shielding area is simulated, and the radiation characteristics difference between the vegetation and the shadow in the shielding area is used to derive the crown layer transmission light distribution characteristics. Directly serve the subsequent reflectivity correction of optical remote sensing image structure, which is an important pre-processing process of image physical property correction.

[0047] It can be understood that the non-uniformity of the crown layer structure in the three-dimensional space will cause the light shielding phenomenon in some areas of the remote sensing image. Such shielding area presents the characteristics of local brightness reduction and spectral curve distortion on the optical image. In order to accurately model the light propagation mode in the crown layer, it is necessary to construct the crown layer projection model based on the geometric projection relationship combined with the measured solar incident angle and the imaging angle of the remote sensing sensor, and to extract the shielding area. In this embodiment, the three-dimensional light transmission simulation algorithm (such as Monte Carlo photon tracking or Zhang model) is used to estimate the penetration path of the incident light beam of each pixel, and to identify the spatial position affected by the shielding.

[0048] Specifically, the spatial distribution difference between the vegetation and the shadow in the shielding area is used to derive the crown layer transmission light distribution characteristics, including: According to the spectral reflectivity of the vegetation and the shadow in the shielding area in different wave bands, the spectral difference mode of the shadow area and the vegetation area is obtained; It should be noted that the optical remote sensing reflectivity of the shielding area is affected by multiple scattering, shadow attenuation and other physical mechanisms, and shows a spectral response characteristic significantly different from that of the direct illumination area. The typical vegetation wave bands (such as near-infrared, red edge, visible green wave band) in the multi-band optical remote sensing image are compared and analyzed, the normalized difference index (such as NDVI change rate, RVI attenuation rate) of the vegetation area and the shadow area in the same area in each wave band is calculated, and the mode matrix describing the spectral difference is established.

[0049] Based on the spectral difference mode, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified; Exemplarily, the spectral reflectivity values of each pixel and its neighborhood in the shielding area are traversed using a spatial sliding window, and a feature vector set is constructed in combination with the difference mode matrix. Subsequently, an unsupervised classification algorithm based on principal component analysis (PCA) + K-means clustering is introduced, the area is divided into single scattering dominant area and multiple scattering dominant area, and the spatial boundaries of each area are further corrected by comparing the projection mapping position of the crown layer contour.

[0050] To improve the spatial positioning accuracy, the above results can be cross corrected with the backscattering intensity change area in the radar image to improve the accuracy of scattering type recognition. Further, based on the spectral difference mode, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified, including: extracting the spectral reflectance distribution of multi-band images in shadow and vegetation regions; Specifically, the reflectance data of the direct region and the shadow region of the canopy layer in the optical remote sensing image are extracted, and the average reflectance difference of the two types of regions under different bands is calculated to represent the influence degree of the light condition on the band response. For each band , the reflectance variation rate is defined as: ; Among them, is the average reflectance of the vegetation direct region, is the average reflectance of the shadow region, is a small amount to prevent the denominator from being zero.

[0051] The reflectance variation mode of each band is analyzed to determine the stable single scattering characteristic band in the vegetation region and the significant multiple scattering characteristic band in the shadow region; It should be noted that the influence intensity of the band on the light is judged based on the preset reflectance variation rate threshold, so as to calibrate the single scattering and the multiple scattering; Specifically: If is small (such as less than 0.1), it indicates that the band is less affected by the light, and its signal mainly comes from the single reflection of the canopy leaf, which can be classified as a single scattering characteristic band; the single scattering characteristic band If is large (such as greater than 0.2), it indicates that the band is obviously affected by multiple scattering and shadow, reflecting the multi-layer reflection effect inside the canopy layer, which can be classified as a multiple scattering characteristic band.

[0052] Through the above determination, the single scattering characteristic band set with stable reflectance variation is recorded as , and the multiple scattering characteristic band set with significant variation is recorded as .

[0053] Using the determined single scattering characteristic band and multiple scattering characteristic band, the specific spatial position of the single scattering and the multiple scattering is calibrated; including the following steps: According to the characteristic bands of the shadow region and the vegetation region, a spatial spectral difference map is constructed; the spatial self-correlation algorithm is used to analyze the spatial spectral difference map to determine the single scattering or multiple scattering attribute of each spatial position; It can be understood that: for each pixel , the average reflectance difference value is calculated in the two types of characteristic bands, and the formula is: ; Among them, represents the average reflectance of the multiple scattering band set, This represents the average reflectance of a set of single-scattering bands. The difference in average reflectance per pixel. The larger the value, the more significant the attenuation of reflectivity in the multiple scattering band, and the more reflection and absorption processes the light undergoes from the leaves.

[0054] Based on the scattering properties of each spatial location, the specific spatial distribution results of single scattering regions and multiple scattering regions in the canopy-shaded area are obtained.

[0055] Specifically, the system classifies and labels each spatial location based on the intensity of reflectance differences and spatial clustering characteristics in the spatial spectral difference map. For each pixel, the deviation between its reflectance difference value and the average value of the entire region is calculated. A spatial autocorrelation analysis algorithm is used to perform spatial clustering detection on the spectral difference map to determine whether there is regional consistency in the reflectance differences of adjacent pixels.

[0056] If a certain area Regions consistently above the global average and showing a significant positive correlation (p<0.05) are labeled as multiple scattering regions; regions consistently below the average and showing a significant negative correlation are labeled as single scattering regions. By introducing spatial autocorrelation analysis, local anomalous pixels caused by noise are eliminated, ensuring the spatial distribution of scattering types remains consistent with the canopy's geometric occlusion characteristics. This yields a spatial distribution label map containing single and multiple scattering classification results, providing regional constraints for subsequent transmitted light attenuation calculations.

[0057] To improve spatial continuity, smoothing calculations can be performed within local neighborhoods (such as 3×3 or 5×5 pixels) to ensure that the region division is consistent with the canopy geometry. Simultaneously, the calibration results within geometrically shaded areas are corrected based on the canopy projection relationship: if a pixel is located in the canopy shadow region and the average reflectance difference is greater than the regional average, it is maintained as a multiple scattering region; if it is located in the direct sunlight region and the average reflectance difference is less than or equal to the regional average, it is corrected to a single scattering region. This yields a continuous spatial distribution map of single and multiple scattering on the canopy projection plane, providing input conditions for subsequent transmitted light attenuation estimation.

[0058] To further explain, the logical relationship of the above methods is as follows: Through calculation The sensitivity of each band to changes in illumination was analyzed, and the bands dominated by single and multiple scattering were identified. Through calculation By projecting spectral differences onto spatial locations, regions can be defined based on scattering types. Finally, an interpretable mapping relationship of canopy scattering type was established in the spectral-spatial dual domain, so that the canopy shading and light attenuation characteristics correspond physically.

[0059] The spectral attenuation characteristics of the multiple scattering region are determined by the spectral attenuation characteristics of the light passing through the occluded region. It can be understood that the effective transmitted light intensity reaching the sensor will be significantly reduced due to the reflection and absorption of light between multiple leaves and branches in the multiple scattering region. In order to quantitatively describe the attenuation process, the spatial label map is used to calculate the decline slope of the multi-band reflectance with wavelength in each region, denoted as: ; wherein, reflects the local spectral attenuation trend; when the value is large, it indicates that the light has experienced stronger multiple scattering attenuation at this position. In combination with the average brightness value in the region, ; wherein, is an empirical absorption coefficient, which can be calibrated by field radiation correction or existing literature (generally 0.4-0.6). The calculation process is based on the Beer-Lambert extended model, which can estimate the attenuation degree of the transmitted light inside the canopy at the pixel level.

[0060] Based on the spatial position and degree of attenuation of the transmitted light, the transmitted light distribution characteristics representing the real light distribution characteristics inside the canopy are obtained; In specific implementation, in combination with the spatial position distribution of the multiple scattering region and the single scattering region, and the spectral attenuation characteristics of each region , a transmitted light field distribution map is constructed on the three-dimensional canopy structure. The transmitted light distribution map describes the relative light intensity that can be received by the lower layer of the canopy in units of pixels, and serves as a spatial weight reference for reflectance correction of optical remote sensing images. The output result is a two-dimensional layer or multi-layer grid, representing the change of transmitted light intensity in different depth intervals.

[0061] S103: correcting the canopy reflectance in the optical remote sensing image according to the canopy transmitted light distribution characteristics, and combining the corrected canopy reflectance with the canopy vertical height profile to generate a comprehensive feature of the vegetation canopy structure; It should be noted that the core of step S103 is to correct the canopy reflectance in the original optical remote sensing image by the canopy transmitted light distribution characteristics, so as to eliminate the abnormal reflectance caused by light occlusion, angle of incidence difference, etc., and combine the corrected reflectance with the canopy vertical height profile extracted by radar remote sensing to construct a comprehensive feature reflecting the three-dimensional structure of the crop canopy. The comprehensive feature contains not only the vertical distribution structure of the vegetation, but also reflects the physical properties of different vertical layers, which is the key basis for realizing the depth feature fusion prediction.

[0062] Specifically, due to the significant spatial non-uniformity of light distribution in vegetated areas under natural conditions, the same type of vegetation in the original remote sensing image will exhibit obvious differences in spectral reflectance at different locations. The reflectance values ​​of each pixel are structurally corrected using a canopy transmittance distribution feature map. The corrected canopy reflectance is then combined with the canopy vertical height profile to generate comprehensive features of the vegetation canopy structure, including: The corrected canopy reflectance was decomposed into different spectral bands to obtain detailed reflectance characteristics of vegetation leaves and branches. In this embodiment, the reflectance data of each pixel in the optical remote sensing image after S102 correction is spectrally decomposed in the main vegetation-sensitive bands (such as visible green, red, red-edge, and near-infrared bands). By comparing the difference between the standard vegetation reflectance curve model and the measured reflectance, spectral characteristic indicators such as leaf reflectance intensity and branch scattering rate are extracted. For example, the first derivative of reflectance can be used to calculate the reflectance edge displacement to characterize the nitrogen content and activity state of leaves. Finally, a refined reflectance feature vector for each pixel in different bands is output as input for subsequent structure recognition.

[0063] For example, each pixel has a set of reflection feature values ​​after structural decomposition as follows: ,in Indicates different frequency bands, This indicates the corresponding band number, which is a positive integer.

[0064] The vertical height profile of the canopy is divided into several vertical layers, and the canopy thickness information corresponding to each layer is extracted. This can be understood as follows: After completing the reflectance spectral feature decomposition, the entire remote sensing image is divided into several vertical layers (e.g., each layer is spaced 0.3m apart) using the canopy vertical height profile constructed in step S101. For each pixel, the vertical thickness information of its layer is statistically analyzed based on its spatial location within the canopy height profile; that is, the distance from the ground to that layer and the canopy thickness information of that layer are recorded. The canopy thickness information is used to establish the influence weights of different vertical layer reflectances on the overall vegetation canopy spectral response.

[0065] For example, each cell carries a layer number and corresponding canopy thickness information, such as {layer 1: 0.4m, layer 2: 0.5m, layer 3: 0.3m}.

[0066] Based on the detailed reflective features of vegetation leaves and branches, the main vegetation composition types in each vertical layer are identified. It should be noted that the vegetation composition type within each layer is identified by combining detailed reflectance features with the corresponding canopy thickness information. In this embodiment, a support vector machine (SVM) model is used to classify the reflectance features of different layers, labeling them as leaf-dominant, branch-dominant, or mixed types. Radar backscattering features are also introduced as an auxiliary dimension. By training the model with joint features, vegetation layers with similar reflectance but different structural compositions can be distinguished.

[0067] For example, each vertical layer corresponds to a vegetation type label, such as {layer 1: leaves, layer 2: branches, layer 3: mixed}.

[0068] Based on vertical layer thickness information and vegetation composition type information, comprehensive characteristics of vegetation canopy structure that characterize the differences in three-dimensional canopy structure are obtained.

[0069] In other words, by combining the thickness information of each vertical layer with the identified vegetation composition type, a spatial map feature vector of the canopy structure is constructed. The comprehensive structural features of each cell include: the thickness ratio of each vertical layer, the vegetation type in each layer, and the height gradient change between different layers. For example, the integrated features can be constructed as a vector of the following form:

[0070] in: Indicates the first Vegetation type coding for each layer (e.g., leaf = 1, branch = 2, mixed = 3). For the first Layered vegetation corresponds to thickness. When the comprehensive features of vegetation canopy structure are used as input to deep learning models, they can enhance the model's sensitivity to changes in three-dimensional spatial distribution, effectively improving the spatial resolution of crop yield prediction.

[0071] It should be noted that by combining the corrected canopy reflectance with the canopy vertical height profile, a comprehensive feature of the vegetation canopy structure that can simultaneously characterize both the vertical dimensional features and spectral reflectance features of the crop is obtained. This comprehensive feature of the vegetation canopy structure not only includes the differences in spectral reflectance of leaves and branches at different wavelengths, but also integrates the three-dimensional structural information formed by the vertical layer thickness of the canopy and the vegetation composition type, thus forming a complete representation of the overall canopy state of the vegetation.

[0072] Understandably, the comprehensive characteristics of vegetation canopy structure are not features from a single source, but rather combine the complementary advantages of optical and radar imagery: the former provides spectral sensitivity, while the latter provides vertical height information. Therefore, the comprehensive characteristics of vegetation canopy structure can not only reflect the optical differences on the canopy surface, but also reveal the spatial distribution patterns of its internal structure.

[0073] It should be appreciated that in order to enable the comprehensive features to further play a role, it is necessary to drive them as inputs into the deep feature fusion network. Since the deep feature fusion network has a multi-branch input structure, it can respectively receive the height features and spatial features decomposed in the comprehensive features. The deep feature fusion network can effectively analyze the feature dimensions of different sources at the structural level and fuse them into key features that contribute most to yield prediction under the guidance of the attention mechanism.

[0074] Exemplarily, the comprehensive feature matrix can be divided into a "height component" and a "spatial component": the height component is input into the network as a first input branch, and the spatial component is input into the network as a second input branch. Through such an input mapping mode, the deep feature fusion network can automatically learn the complementary relationship between height and space during the training process, thereby realizing seamless connection from the comprehensive features to yield prediction.

[0075] S104: driving the deep feature fusion network based on the vegetation canopy structure comprehensive features to predict crop yield, wherein the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

[0076] It should be noted that the vegetation canopy structure comprehensive features are input into a specially designed deep feature fusion network model, and through structural fusion of spatial graph features and height features, high-precision prediction of crop yield is realized. The deep feature fusion network adopts a double-input structure, on the one hand, two-dimensional optical data reflecting the spatial distribution characteristics of the canopy are introduced, and on the other hand, vertical height information reflecting the hierarchical structure of the canopy is introduced, and the attention mechanism is used to accurately locate the yield prediction related area. After training, the network can adaptively capture the spatial-structural correspondence relationship, and realize effective collaborative modeling of local and global features.

[0077] The embodiment provides an exemplary network structure and training method, which has good interpretability and implementability. Specifically, the training method of the deep feature fusion network driven by comprehensive features comprises the following steps: constructing a double-input network structure, wherein a first input branch receives canopy height features, and a second input branch receives canopy spatial graph features; In the embodiment, the deep feature fusion network as a whole adopts a double-branch input structure: a first input branch receives canopy height features, i.e., the vertical structure vector output in the step S103; and a second input branch receives canopy spatial graph features, i.e., the spectral reflectance value distribution of each pixel in the corrected remote sensing image.

[0078] Among them: height input: receiving the vertical height sequence in a multilayer perception structure; spatial input: receiving a 2D spatial spectrum image in a convolutional neural network (CNN) structure.

[0079] The network initial weights are initialized by Kaiming, and the loss function is mean square error (MSE).

[0080] The local pattern features of the canopy height features and the spatial map features are extracted by convolution operations respectively; It should be further explained that, in order to mine the potential local patterns in the structure and spatial information, feature extraction operations are respectively performed on the two input channels: The canopy height features are subjected to one-dimensional convolution operation (Conv1D), and the canopy height local features corresponding to the pattern changes between different vertical layers are extracted . The canopy spatial map features are subjected to two-dimensional convolution module (Conv2D+ReLU+Pooling), and the canopy spatial map features corresponding to the discriminative spectral patterns in the local spatial patches are extracted .

[0081] An attention mechanism module is designed to determine the key regions in the canopy spatial map features related to yield prediction, guided by the local patterns of the canopy height features; It should be understood that not all regions in the spatial map features have relevance to crop yield, therefore, the height feature guided attention mechanism is introduced to weight and screen the spatial map features, and only the region information with high contribution to prediction is reserved.

[0082] Specifically, the attention mechanism module is designed to determine the key regions in the canopy spatial map features related to yield prediction, guided by the local patterns of the canopy height features, including: The height guided weight map is generated by using the local patterns of the canopy height features; The canopy height local features are input into a weight mapping network (such as MLP layer+Sigmoid activation) as a guide signal, and a height guided weight map (value range [0, 1]) consistent with the size of the spatial map features is output , which represents the importance distribution of each position of the canopy spatial features. The height guided weight map , wherein: represents the height of the canopy spatial map features, represents the width of the canopy spatial map features, and each element in the matrix is a real value, which is used to represent the importance of the corresponding spatial position to yield prediction.

[0083] It should be understood that, represents the element in the th row and the The weight value of a column position indicates its contribution to yield prediction; a larger value indicates a higher contribution, while a smaller value indicates a lower contribution. By establishing a two-dimensional weight matrix, key vegetation areas can be highlighted in the spatial distribution dimension, thereby reducing the interference of redundant areas on the feature fusion process.

[0084] In the specific implementation process, a height-guided weight map is generated based on the local pattern of canopy height features; the height-guided weight map is multiplied point by point with the canopy spatial features to form a weighted spatial distribution feature map; then, based on the weighted distribution map, the regions with larger weight values ​​are extracted and retained, and finally the key spatial regions used for yield prediction are obtained.

[0085] The height-guided weight map and the canopy spatial map features are multiplied point by point to form a weighted distribution of spatial map features. High-guided weight map Features of canopy spatial map Perform a product operation pixel by pixel to form a spatially weighted feature map. : ; Among them: features of the canopy space map Indicates spatial location Place, No. Original spatial feature values ​​on each channel; Indicates spatial location The attention weight value reflects the importance of that position in the yield prediction task; This represents the feature value after attention weighting.

[0086] This can be understood as: taking a certain spatial location The original eigenvalues ​​on Then use the attention weight at that position. We then weight them to obtain new weighted eigenvalues. This involves scaling up or down each pixel in the spatial feature map based on the value in the attention map; that is: if If the value is large, the location is important, the corresponding feature value is amplified, and the network will pay more attention to this location; if... If the value is smaller, the importance of that location is lower, and the corresponding eigenvalue is suppressed, reducing its influence in the final prediction. This automatically focuses on areas useful for yield prediction (such as vegetation-covered areas) while weakening the interference of irrelevant areas (such as bare soil or shaded areas).

[0087] Based on the weighted distribution of spatial map features, the key regions for yield prediction in the canopy spatial map features are selected.

[0088] Through the The global attention weighted pooling is performed to screen a plurality of key regions as the spatial basis for yield determination. The key region positions are output in the form of spatial position coordinates.

[0089] The key region and the local mode of the canopy height feature are fused to obtain a trained deep feature fusion network.

[0090] The canopy height local feature H_feat obtained in step S104.2 is spliced with the key region spatial graph feature vector extracted in step S104.3, and is sent to a fusion prediction module (such as a fully connected network FC layer) for regression prediction.

[0091] The overall network training adopts an end-to-end strategy, the input is a set of structural comprehensive feature images and corresponding measured yield labels, and the output is a continuous yield prediction value.

[0092] The trained model can estimate the yield of any crop area, and the prediction result has high spatial resolution and hierarchical interpretation ability.

[0093] Embodiment 2 As shown in Figure 2 , parts not detailed in this embodiment are as shown in embodiment 1, this embodiment provides a crop yield prediction system integrating multi-source remote sensing data, including an image acquisition module, a canopy illumination simulation module, a canopy feature correction module and a yield prediction module, each module transmits data through wired and / or wireless; The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of a target crop plot, and to construct a canopy vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images; the image acquisition module is also used to construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; Further explanation, based on the backscattering characteristics of the crop canopy in the radar remote sensing images, the canopy vertical height profile is constructed, including: Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the canopy top scattering signal and the ground scattering signal is obtained; including the following steps: Extract the HH polarization and VV polarization echo signals of the stable scattering targets on the ground surface in the same imaging area, and calculate the average complex scattering coefficient to obtain the ground reference target signal; Based on the HH polarization and VV polarization echoes simultaneously acquired by the polarization radar sensor, the canopy polarization phase of the canopy top scattering signal is calculated; The complex difference of the complex scattering coefficients of the canopy top scattering signal and the ground reference target signal under HH and VV channels is used to calculate the polarization phase difference between the canopy top scattering signal and the ground scattering signal.

[0094] determine an effective propagation path of the radar signal penetrating the canopy layer through the polarization phase difference; determine echo positions of different depth layers in the canopy layer based on the effective propagation path; generate a height profile capable of representing the real vertical height of the crop canopy according to the echo positions of each layer.

[0095] A canopy illumination simulation module is configured to determine a shielding area of the canopy structure by a three-dimensional canopy light transmission simulation algorithm according to the canopy projection relationship, and derive canopy transmission light distribution characteristics using the spatial distribution difference between vegetation and shadows in the shielding area. Specifically, the derivation of the canopy transmission light distribution characteristics using the spatial distribution difference between vegetation and shadows in the shielding area includes: obtain a spectral difference pattern of the shadow area and the vegetation area according to the spectral reflectivity of the vegetation and the shadows in the shielding area at different wave bands; identify the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area based on the spectral difference pattern; wherein the identification of the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area based on the spectral difference pattern includes: extract the spectral reflectivity distribution of the multi-band image of the shadow area and the vegetation area; analyze the reflectivity change pattern of each wave band to determine the stable single scattering characteristic wave band in the vegetation area and the significant multiple scattering characteristic wave band in the shadow area; use the determined single scattering characteristic wave band and multiple scattering characteristic wave band to calibrate the specific spatial positions of the single scattering and the multiple scattering; Further, the use of the determined single scattering characteristic wave band and multiple scattering characteristic wave band to calibrate the specific spatial positions of the single scattering and the multiple scattering includes: construct a spatial spectral difference map according to the characteristic wave bands of the shadow area and the vegetation area; use a spatial autocorrelation algorithm to analyze the spatial spectral difference map to determine the single scattering or multiple scattering attribute of each spatial position; classify and calibrate according to the scattering attribute of each spatial position to obtain the specific spatial position distribution result of the single scattering area and the multiple scattering area in the shielding area of the canopy; determine the spatial position and degree of transmission light attenuation in the shielding area through the spectral attenuation characteristics of the multiple scattering area; obtain transmission light distribution characteristics representing the real light distribution characteristics inside the canopy based on the spatial position and degree of transmission light attenuation.

[0096] a canopy feature correction module configured to correct the canopy reflectivity in the optical remote sensing image according to the canopy transmission light distribution feature, and combine the corrected canopy reflectivity with the canopy vertical height profile to generate a comprehensive canopy structure feature of the vegetation canopy; Specifically, the comprehensive canopy structure feature of the vegetation canopy is generated by combining the corrected canopy reflectivity with the canopy vertical height profile, including: spectrum decomposition of the corrected canopy reflectivity in different wavebands to obtain detailed reflectance features of the vegetation leaves and branches; dividing the canopy vertical height profile into a plurality of vertical layers to extract the canopy thickness information corresponding to each layer; identifying the main types of vegetation composition in each vertical layer according to the detailed reflectance features of the vegetation leaves and branches; obtaining the comprehensive canopy structure feature representing the three-dimensional structure difference of the vegetation canopy according to the vertical layer thickness information and the vegetation composition type information.

[0097] a yield prediction module configured to drive a deep feature fusion network to predict the crop yield based on the comprehensive canopy structure feature of the vegetation canopy; wherein the deep feature fusion network adopts an attention mechanism to fuse the spatial map feature and the height feature.

[0098] Specifically, the training method of the comprehensive feature driven deep feature fusion network includes: constructing a double-input network structure, wherein a first input branch receives the canopy height feature and a second input branch receives the canopy spatial map feature; extracting the local pattern features of the canopy height feature and the spatial map feature respectively by convolution operation; designing an attention mechanism module to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local patterns of the canopy height feature; performing feature fusion using the key regions of the canopy spatial map feature and the local patterns of the canopy height feature to obtain the trained deep feature fusion network.

[0099] Further, the attention mechanism module is designed to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local patterns of the canopy height feature, including: generating a height-guided weight map using the local patterns of the canopy height feature; performing point-by-point multiplication operation between the height-guided weight map and the canopy spatial map feature to form a spatial map feature weighted distribution; according to the spatial map feature weighted distribution, screening to obtain the key region positions in the canopy spatial map feature for yield prediction.

[0100] The embodiment provides a crop yield prediction system integrating multi-source remote sensing data, which is used for executing the crop yield prediction method integrating multi-source remote sensing data provided in the above-mentioned embodiments of the application. The specific method and process of realizing the corresponding function of each structure included in the crop yield prediction system based on the integration of multi-source remote sensing data are described in the above-mentioned embodiment of the crop yield prediction method integrating multi-source remote sensing data, and will not be described here again.

[0101] Embodiment 3 The embodiment also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize the steps of the method in the embodiment 1.

[0102] The above-mentioned embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the above-mentioned embodiments can be realized in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired network or a wireless network. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center and the like containing one or more available medium collections. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD) or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0103] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-mentioned system, device and unit can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here again.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the embodiments of the device described above are merely schematic, and the division of the units is merely logical function division. There can be other division manners in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0106] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0107] In addition, each function unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit.

[0108] The above only describes some exemplary embodiments of the present application in a descriptive manner, and it is self-evident that those skilled in the art can modify the described embodiments in various manners without departing from the spirit and scope of the present application. Therefore, the above drawings and descriptions are illustrative in nature, and should not be understood as limiting the scope of protection of the claims of the present application.

Claims

1. A method for integrated multi-source remote sensing data crop yield prediction, characterized in that, The method comprises the following steps: S101: Obtain optical remote sensing images and radar remote sensing images of a target crop plot, construct a crown vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images, and construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; S102: According to the canopy geometric projection relationship, determine the canopy structure shadow area by a three-dimensional canopy light transmission simulation algorithm, and derive the canopy transmission light distribution characteristics by using the spatial distribution difference between the vegetation and the shadow in the shadow area; S103: Correct the canopy reflectivity in the optical remote sensing images according to the canopy transmission light distribution characteristics, and generate comprehensive features of the vegetation canopy structure by combining the corrected canopy reflectivity and the canopy vertical height profile; S104: Based on the comprehensive features of the vegetation canopy structure, drive a deep feature fusion network to predict the crop yield, and the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

2. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, characterized in that, Based on the backscattering characteristics of the crop canopy in the radar remote sensing images, the crown vertical height profile is constructed, comprising: Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the top of the canopy and the ground scattering signal is obtained; The effective propagation path of the radar signal penetrating the canopy is determined through the polarization phase difference; Based on the effective propagation path, the echo positions of different depth layers in the canopy are determined; According to the echo positions of each layer, a height profile capable of representing the true vertical height of the crop canopy is generated.

3. The crop yield prediction method integrating multi-source remote sensing data according to claim 2, characterized in that, Based on the polarization scattering characteristics of the radar remote sensing images, the polarization phase difference between the top of the canopy and the ground scattering signal is obtained, comprising: Extract the HH polarization and VV polarization echo signals of the ground stable scattering target in the same imaging area, and calculate the average complex scattering coefficient to obtain the ground reference target signal; Based on the HH polarization and VV polarization echoes obtained by the polarization radar sensor simultaneously, the canopy polarization phase of the top of the canopy scattering signal is calculated; The complex difference of the complex scattering coefficients of the top of the canopy scattering signal and the ground reference target signal under HH and VV channels is used to calculate the polarization phase difference between the top of the canopy scattering signal and the ground scattering signal. 4.The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, The spatial distribution difference between the vegetation and the shadow in the shadow area is used to derive the canopy transmission light distribution characteristics, comprising: According to the spectral reflectance of the vegetation and the shadow in the shadow area in different bands, the spectral difference pattern of the shadow area and the vegetation area is obtained; Based on the spectral difference pattern, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified; Through the spectral attenuation characteristics of the multiple scattering area, the spatial position and degree of transmission light attenuation in the shadow area are determined; Based on the spatial position and degree of transmission light attenuation, the transmission light distribution characteristics representing the real light distribution characteristics in the canopy are obtained.

5. The crop yield prediction method integrating multi-source remote sensing data according to claim 4, characterized in that, Based on the spectral difference pattern, the specific spatial positions of the multiple scattering area in the shadow area and the single scattering area in the vegetation area are identified, comprising: Extract the spectral reflectance distribution of the multi-band images of the shadow area and the vegetation area; Analyze the change pattern of the reflectance of each band to determine the stable single scattering characteristic band in the vegetation area and the significant multiple scattering characteristic band in the shadow area; The specific spatial positions of single scattering and multiple scattering are calibrated by using the determined single scattering characteristic band and multiple scattering characteristic band.

6. The crop yield prediction method integrating multi-source remote sensing data according to claim 5, wherein, The specific spatial positions of single scattering and multiple scattering are calibrated by using the determined single scattering characteristic band and multiple scattering characteristic band, comprising: A spatial spectrum difference map is constructed according to the characteristic bands of the shadow area and the vegetation area; A spatial autocorrelation algorithm is used to analyze the spatial spectrum difference map to determine the single scattering or multiple scattering attribute of each spatial position; According to the scattering attribute of each spatial position, classification calibration is performed to obtain the specific spatial position distribution results of the single scattering area and the multiple scattering area of the canopy sheltered area.

7. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, A vegetation canopy structure comprehensive feature is generated by combining the corrected canopy reflectance and the canopy vertical height profile, comprising: The corrected canopy reflectance is spectrally decomposed at different bands to obtain the detailed reflection characteristics of vegetation leaves and branches; The canopy vertical height profile is divided into several vertical layers, and the canopy thickness information of each layer is extracted; According to the detailed reflection characteristics of vegetation leaves and branches, the main composition types of vegetation in each vertical layer are identified; According to the vertical layer thickness information and the vegetation composition type information, the vegetation canopy structure comprehensive feature representing the three-dimensional structure difference of the canopy is obtained.

8. The crop yield prediction method integrating multi-source remote sensing data according to claim 1, wherein, The training method of the deep feature fusion network driven by the comprehensive feature comprises: A double-input network structure is constructed, wherein a first input branch receives the canopy height feature, and a second input branch receives the canopy spatial map feature; Convolution operations are used to extract local pattern features of the canopy height feature and the spatial map feature, respectively; An attention mechanism module is designed to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local pattern of the canopy height feature; Feature fusion is performed using the key regions of the canopy spatial map feature and the local pattern of the canopy height feature to obtain the trained deep feature fusion network.

9. The integrated multi-source remote sensing data crop yield prediction method of claim 8, wherein, The attention mechanism module is designed to determine the key regions in the canopy spatial map feature related to yield prediction guided by the local pattern of the canopy height feature, comprising: An altitude guide weight map is generated using the local pattern of the canopy height feature; The altitude guide weight map and the canopy spatial map feature are multiplied point by point to form a spatial map feature weighted distribution; According to the spatial map feature weighted distribution, the key region positions in the canopy spatial map feature used for yield prediction are screened.

10. A system for integrated multi-source remote sensing data-based crop yield prediction, configured to perform the method of integrated multi-source remote sensing data-based crop yield prediction according to any one of claims 1-9, characterized in that, An image acquisition module, a canopy illumination simulation module, a canopy feature correction module, and a yield prediction module are included. The image acquisition module is used to acquire optical remote sensing images and radar remote sensing images of a target crop plot, and to construct a canopy vertical height profile based on the backscattering characteristics of the crop canopy in the radar remote sensing images; the image acquisition module is also used to construct a canopy geometric projection relationship according to the solar incident azimuth angle and the observation sensor angle; The canopy illumination simulation module is used to determine a canopy structure sheltered area by a three-dimensional canopy light transmission simulation algorithm according to the canopy projection relationship, and to derive a canopy transmission light distribution feature using the spatial distribution difference between vegetation and shadows in the sheltered area; a canopy feature correction module configured to correct the canopy reflectivity in the optical remote sensing image according to the canopy transmission light distribution feature, and combine the corrected canopy reflectivity with the canopy vertical height profile to generate a comprehensive feature of a vegetation canopy structure; a yield prediction module configured to predict crop yield based on the comprehensive feature of the vegetation canopy structure driving a deep feature fusion network; wherein the deep feature fusion network adopts an attention mechanism to fuse spatial graph features and height features.

Images