A modeling method for multi-scale spatial structure spectrum of fruits

Abstract

The application discloses a fruit multi-scale space structure spectrum modeling method. The technical scheme is as follows: a fruit multi-scale space structure spectrum modeling method comprises the following steps: a to-be-measured space is established according to the shape characteristics of a fruit tree and a spectrum scale law; a multi-angle and multi-polarization database with different angle resolutions of the fruit is established through space grid quantization marking and temperature and illumination grid calibration; a space distribution map with different azimuths and angles is obtained through multi-angle iterative fusion; near-infrared spectra of the fruit at different time periods and multi-angles are collected to establish a fruit quality spectrum model; and simulation analysis is conducted on the multi-angle and multi-polarization database and the fruit spectrum to perform inversion and verify and optimize the model. The fruit multi-scale space structure spectrum modeling method can improve the detection precision and stability of a quantitative remote sensing model through multi-angle space background segmentation and classification matching recognition, and can be used for rapid and accurate detection of fruits.

Description

Technical Field

[0001] This invention belongs to the field of rapid detection of fruit quality, specifically involving a modeling method for multi-scale spatial structure spectrum of fruit. Background Technology

[0002] Southern Xinjiang possesses unique geographical conditions such as large diurnal temperature variations and abundant sunshine, resulting in the high sugar content and unique advantage of its specialty fruits. Temperature and sunshine conditions vary across different regions, and even the quality of fruit can differ significantly depending on the part of the fruit growing on the same tree. In other words, fruit quality is influenced by environmental factors such as outdoor sunlight and temperature, exhibiting certain regional distribution characteristics and serving as an indicator of the fruit's environmental adaptability. However, this also makes it difficult to quickly and extensively assess fruit quality.

[0003] While UAV multispectral cameras can quickly acquire multispectral images of orchards, their low spectral resolution hinders rapid quantitative detection of fruit quality. Furthermore, sunlight exhibits polarization during atmospheric scattering and transmission; additionally, fruit spectral reflectance characteristics differ at different angles and scales, with multi-angle reflection and polarization naturally intertwined. To refine the study of the impact of environmental temperature and light on fruit quality at different scales and to rapidly evaluate fruit quality in batches across different regions, this invention proposes a multi-scale spatial structure spectral modeling method for fruits. This method improves spectral resolution through iterative fusion of multi-angle spectra, enabling rapid and accurate fruit detection. Summary of the Invention

[0004] The purpose of this invention is to provide a modeling method for the multi-scale spatial structure spectrum of fruits. This method involves simulating the structure of fruit trees to perform gridded calibration of fruits in different spatial orientations, collecting multi-angle spectra to improve spectral resolution, and enhancing the detection accuracy and stability of the quantitative remote sensing model through multi-angle spatial background segmentation and classification matching. By quantitatively calibrating and modeling the scattering spectra of fruits at different angles and scales, this method can be used for rapid and accurate fruit detection.

[0005] To achieve the above objectives, the technical solution adopted is as follows:

[0006] A method for modeling the multi-scale spatial structure spectrum of fruit products, the modeling comprising the following steps:

[0007] Based on the shape characteristics of fruit trees and the law of spectral scaling, a simulated test space is established. Through quantitative marking of spatial grids and grid calibration of temperature and light conditions, a multi-angle and multi-polarization database of fruit products with different angular resolutions is established. Through multi-angle iterative fusion, spatial distribution maps of different orientations and angles are obtained.

[0008] Near-infrared spectra and physicochemical indicators of fruits were collected from different time periods and from multiple angles to establish a multi-angle characteristic spectral model of fruit quality.

[0009] The multi-angle, multi-polarization database and fruit spectral model were incorporated into the LESS software simulation model for simulation analysis, inversion, and model verification and optimization.

[0010] Furthermore, the method for quantitative marking and grid calibration of the spatial grid is as follows:

[0011] First, the spectral density of the light source at different angles and distances is collected to obtain the coherence and polarization of the light source; then, according to the scaling law, the spatial scale and orientation to be measured are subdivided into grids and the system is calibrated.

[0012] Then, spectral data were collected for different row and column spacings;

[0013] Then, calibration and standardization are performed.

[0014] Furthermore, the calibration and standardization method is as follows: collect and create spectral distribution maps of fruits from different orientations, correspond to spatial gridded locations, stitch together the single-point spectra of single fruits from different locations, and form multiple spatial quality distribution images of fruits through calibration and standardization.

[0015] Furthermore, the simulation analysis includes: using ENVI software to perform simulation analysis of the variance and spectral mean of different regional ROIs; performing regional segmentation, classification, and spectral feature matching and recognition through ROIs at different distances and angles to realize the model transfer of multispectral and near-infrared spectrometers; and using MATLAB to perform distance and angle fitting simulations to create distance and orientation segmentation, classification, matching, and recognition models.

[0016] Furthermore, the modeling method described above involves first modeling indoors and then modeling outdoors.

[0017] Furthermore, in the outdoor modeling process, a BP neural network is used for training, inversion, and model verification.

[0018] Furthermore, in the outdoor modeling process, temperature spatial distribution calibration and verification are also required during the verification process.

[0019] Furthermore, in the outdoor modeling process, the spectra of fruits and leaves from different angles are used as input parameters for BP network reconstruction training, and a multi-scale fruit quality detection model is reconstructed through multi-angle spectral weight allocation.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0021] This invention examines the changes in spectral coherence and polarization during spectral propagation, and the scale characteristics of spectral spatial distribution. It establishes a multi-angle, multi-polarization standard database through quantitative spatial grid labeling and temperature-illuminance grid calibration, uses a semi-variogram to describe spatial statistical distribution characteristics, and obtains calibrated spatial distribution maps at different orientations and angles. Spatial statistical characteristic models at different scales are established, and spatial inversion and verification are performed at different calibration scales.

[0022] It can also achieve high-precision near-ground inversion from horizontal detection to vertical retrieval using mobile multi-scale ground-based spectral detection devices, covering different angles and distances. It can acquire multi-source data on fruit quality through multi-scale data acquisition platforms such as drones and airborne systems. Through multi-scale fruit spectral calibration and fusion, it can transfer characteristic scale models, achieving consistent expression of data and fruit traits across multiple platforms. This serves precision agriculture and smart agriculture. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the technical solution of the present invention;

[0024] Figure 2 A schematic diagram of a linear reciprocating indoor calibration and calibration device;

[0025] Figure 3 This is a schematic diagram of the BRDF data acquisition method. Detailed Implementation

[0026] To further illustrate the modeling method for multi-scale spatial structure spectra of fruits according to the present invention and to achieve the intended purpose of the invention, the following detailed description, in conjunction with preferred embodiments, details the specific implementation, structure, features, and effects of the modeling method for multi-scale spatial structure spectra of fruits proposed according to the present invention. In the following description, different "an embodiment" or "an embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0027] The following will provide a more detailed description of the modeling method for multi-scale spatial structure spectra of fruits according to the present invention, with reference to specific embodiments:

[0028] The technical solution of this invention is as follows:

[0029] A method for modeling the multi-scale spatial structure spectrum of fruit products, the modeling comprising the following steps:

[0030] Based on the shape characteristics of fruit trees and the law of spectral scaling, a simulated test space is established. Through quantitative marking of spatial grids and grid calibration of temperature and light, a multi-angle and multi-polarization database of fruit products with different angular resolutions is established. Through multi-angle iterative fusion, spatial distribution maps of different orientations and angles are obtained.

[0031] Near-infrared spectra and physicochemical indicators of fruits were collected from different time periods and from multiple angles to establish a multi-angle characteristic spectral model of fruit quality.

[0032] The multi-angle, multi-polarization database and fruit spectral model were incorporated into the LESS software simulation model for simulation analysis, inversion, and model verification and optimization.

[0033] Preferably, the method for spatial grid quantitative marking and grid calibration is as follows:

[0034] First, the spectral density of the light source at different angles and distances is collected to obtain the coherence and polarization of the light source; then, according to the scaling law, the spatial scale and orientation to be measured are subdivided into grids and the system is calibrated.

[0035] Then, spectral data were collected for different row and column spacings;

[0036] Then, calibration and standardization are performed.

[0037] More preferably, the calibration and standardization method is as follows: collect and create spectral distribution maps of fruits from different orientations, correspond to spatial gridded locations, stitch together the single-point spectra of single fruits from different locations, and form multiple spatial quality distribution images of fruits through calibration and standardization.

[0038] Preferably, the simulation analysis includes: using ENVI software to perform simulation analysis of the variance and spectral mean of different spatial ROIs, segmenting different background regions through ROIs at different distances and angles, and then performing classification spectral feature matching and recognition to realize the transfer of spectral models for multispectral imaging and near-infrared spectrometers; and using MATLAB to perform distance and angle fitting simulations to create distance and orientation segmentation and classification matching recognition models.

[0039] Preferably, the modeling method involves first modeling indoors and then modeling outdoors.

[0040] More preferably, in the outdoor modeling process, a BP neural network is used for training, inversion, and model verification.

[0041] More preferably, during the outdoor modeling process, the spatial distribution of temperature is also verified during the verification process.

[0042] More preferably, in the outdoor modeling process, the spectra of fruits and leaves from different angles are used as input parameters of the BP network for reconstruction training, and a multi-scale fruit quality detection model is reconstructed through multi-angle spectral weight allocation.

[0043] Example 1.

[0044] The specific steps are as follows:

[0045] It mainly consists of the following three parts: spatial gridding calibration of the test area, multi-angle characteristic spectral model of fruit quality, and outdoor calibration model verification and optimization (specific multi-angle spatial spectral modeling methods are as follows). Figure 1 As shown):

[0046] A Interior Modeling

[0047] (1) Spatial gridded quantitative calibration method: By simulating the quantitative marking of the spatial grid to be measured and temperature-illuminated gridded calibration, a multi-angle, multi-polarization standard database is established, and spatial distribution maps of calibration at different orientations and angles are obtained. Specifically:

[0048] a: Based on the spectral scaling law of fruit tree shape characteristics (e.g., Y-shaped, open-center, round-headed), a simulated test space is established. Typical angles and dimensions of fruit tree branches are measured to abstract the structural characteristics of the fruit tree for orientation simulation. The spectral diffusion factor in the xy direction at these characteristic angles and distances is calculated using the spectral scaling law. Spectral densities of light sources at different angles and distances are collected using a spectrometer to obtain the coherence and polarization of the light source. The scale and orientation of the test space are then subdivided into grids and calibrated systematically according to the scaling law.

[0049] Secondly, fruits (red dates, apricots, fragrant pears, etc.) were placed in a calibrated space to conduct multi-angle experiments with horizontal azimuth angles of 0-180 degrees and different angular resolutions (5 degrees, 10 degrees, 20 degrees).

[0050] Finally, through multi-angle iterative fusion, the spectral coherence, polarization, and characteristic angles of the fruit from different orientations are obtained. Effective calibration data can only be obtained after system calibration. Data collected under different experimental environments or at different stages are compared to obtain physical parameters of structural characteristic scales at different angles and distances, thus optimizing the experimental data and theoretical model data.

[0051] b: Linear reciprocating ground calibration of fruit spectra:

[0052] The linear reciprocating motion of the grid-like fruit racks based on the spacing between fruit trees and fruit varieties simulates the reciprocating flight of a drone, collecting spectral data at different row and column spacings, and then stitching the acquired data into images.

[0053] c: Circular BRDF multi-angle data acquisition method:

[0054] Indoor calibration and verification were performed using BRDF data acquisition devices at different elevation and azimuth angles. Background segmentation and matching recognition of multi-angle spectral images under different lighting conditions were used to simulate high-precision spectral detection of UAV outdoor BRDF at multiple angles near the ground.

[0055] The indoor calibration method in step c is as follows: Figure 2A linear reciprocating indoor calibration and adjustment device was constructed. The linear reciprocating flight of a drone was simulated. Sixteen grid-point fruit supports, driven by stepper motors, moved sequentially along the horizontal and vertical directions. Simultaneously, a fixed-location PSR-100 spectrometer collected spectra of the fruit on the 16 supports at different angles and distances (adjustable angle and distance): (1, 1; 1, 2; 1, 3; 1, 4); (2, 1; 2, 2; 2, 3; 2, 4); (3, 1; 3, 2; 3, 3; 3, 4); (4, 1; 4, 2; 4, 3; 4, 4). The 16 spectral files at the marked locations were merged into a single spectral image file using ENVI software, creating 16 spatial distribution maps of the fruit spectra from different orientations. The above process mainly involved spectral manipulation, adding the marked spatial coordinates, and stitching each band into a single image. Spatial resolution was increased by increasing the number of acquisition points at each spatial location. This process is similar to the progressive scanning of a hyperspectral camera to achieve spectral imaging of fruit quality. Then, by changing the distance, the average spectrum of the fruit in the area was collected using a PSR-100 ground-based spectrometer, and multi-band images were collected using a multispectral camera for spatial registration and fusion.

[0056] Specifically: First, individual fruits are marked and grouped using the Region of Interest (ROI) in ENVI, corresponding to spatial grid positions. Background spectra at different spatial angles are established by comparing and judging the mean and variance of the regional spectra. Spatial registration and fusion are then performed using high-resolution and low-resolution ENVI spectra to obtain quality distribution maps of fruits in different locations.

[0057] Then, a multispectral camera was used to capture multi-band images of the fruit from the same position and angle. These images were then compared with the single-point scanned fruit quality spectra of the spectrometer for calibration and verification. The single-point spectra of individual jujubes from different positions were saved sequentially by row and column in ENVI BIP format and stitched together to form a hyperspectral image showing the spatial distribution of multiple jujube quality locations. This is equivalent to labeling the spatial location information with spectral information. Through calibration and standardization, multiple spatial quality distribution images of the fruit were generated, thus obtaining calibrated spatial distribution maps at different orientations and angles.

[0058] When a drone's multispectral camera performs long-distance imaging, the spectral resolution is low. By establishing a multi-angle polarization database for training simulation, the spectral resolution of fruit quality can be improved.

[0059] (2) Dynamic spectrum: The quality spectrum model of fruit is collected by a near-infrared spectrometer, and the near-infrared spectrum of fruit is collected at different drying times to realize real-time dynamic spectrum of drying.

[0060] (3) Spatial gridding and simulation at the fruit tree canopy scale: Multi-angle, multi-polarization databases and fruit spectral models are input into Less software for simulation analysis, inversion, and model verification. Specifically:

[0061] A simulation model of jujube canopy quality was established using Less software, incorporating jujube spectra with different angles and polarization characteristics. Spectra of fruit samples were collected at different distances and angles, and Regions of Interest (ROIs) of different scales were selected using ENVI software for variance and spectral mean simulation analysis. Regional segmentation and spectral feature matching and identification were performed using ROIs at different distances and angles to achieve model transfer from multispectral and near-infrared spectrometers. MATLAB was used for distance and angle fitting simulations. Variance was used as an important evaluation index. The Rourjean model was used to obtain K-parameters for different angles and structures from the experimental spectral data of fruit samples at different angles. Based on the tree structure characteristics, the spectral response function of the fruit tree canopy was calculated. Multiple iterations of simulation and experimentation were conducted for prediction inversion and experimental verification to improve the model inversion accuracy and stability.

[0062] B Outdoor Modeling

[0063] The steps for outdoor modeling are the same as those for indoor modeling, with the following differences: meshing of the spatial features under typical ambient temperature and lighting conditions, as detailed below:

[0064] (1) Grid generation of the space to be tested:

[0065] Based on the shape and scale characteristics of the fruit trees under test, a fine grid was divided to achieve spatial calibration of the images from the spectral camera at different distances and angles, establishing a one-to-one correspondence between image pixel reflectance and scale space. The outdoor fruit growing environment has a certain impact on fruit quality distribution; the higher the angular resolution of fruit detection, the higher the accuracy of fruit quality detection, but the corresponding data volume increases significantly. Near-infrared spectrometers and multispectral cameras were used to collect in-situ, gridded fruit spectra with typical angular characteristics from the tree canopy. Standard plates were placed at different locations to calibrate and standardize the spatial positions of spectral reflectance. Multi-angle standard plate calibration experiments were conducted to establish a multi-angle, multi-polarization standard plate database, obtaining distribution maps of different orientations and angles in the calibration space. A semi-variogram was used to describe the spatial statistical distribution characteristics, establishing spatial statistical models at different scales for spatial inversion and verification at different calibration scales.

[0066] (2) Thermal imager to create temperature spatial distribution model

[0067] A temperature difference distribution field was created using outdoor ambient temperature and temperature-controlled infrared heating lamps. Fruits were placed on supports at different angles, and the spatial temperature distribution was calibrated and standardized using a thermal imager and thermometer to establish a multi-angle temperature characteristic orientation database. Then, multispectral images of the fruits were captured, and different spatial angles were segmented and temperature spectrum matching calibrations were performed. Spectral calibrations of fruits with different temperature distribution characteristics were conducted under different lighting conditions.

[0068] Through routine experiments on the spectral and physicochemical indicators of fruits at different times, temperature and humidity, spectral, thermal imager, multi-source data acquisition and quality testing devices were used. A multispectral, multi-angle fruit quality imaging database was classified and multi-scale spectral calibration modeling was performed.

[0069] (3) Multi-angle quantitative detection method for fruits

[0070] Multi-angle quantitative detection methods for fruits mainly involve collecting and processing spectral data, finding the optimal band based on the band index method, and selecting the optimal band using the correlation coefficient method and standard deviation to obtain the band index. Through the optimal combination of scale and angle, characteristic quality parameters such as reflectance, azimuth, and elevation angle are determined.

[0071] Band indices, standard deviations of leaf grayscale values, and correlation coefficients were collected for jujubes and leaves under different light angles. Multi-angle spectral quantitative description, band index method, and correlation coefficient method were used to identify characteristic angles corresponding to quality. Using physical quality indicators such as moisture, sugar content, and acidity, along with multi-angle spectral quantitative description methods, a characteristic angle database was established. Spectra of single leaves or canopy samples at different angles and distances (jujubes, apricots, pears, apples) were obtained, and the optimal decomposition scale and optimal principal component factor (characteristic wavelength) were selected. An angle detection method was obtained by training and calculating using CNN or BP neural networks.

[0072] (4) Near-ground multi-angle orientation model of UAV

[0073] Set the GPS coordinates and angles for multi-angle flight of the drone, and conduct BRDF simulation of fruit tree spectral quality detection, including different lighting conditions and angles (BRDF performs spectral acquisition at different angles, such as...). Figure 3 (As shown).

[0074] Through multispectral quantitative imaging experiments using a BRDF UAV at different angles, the peak at the optimal reflection angle and azimuth angle can be reproduced. This method is used to explore other quality characteristics of the jujube canopy. The focus is on collecting data from the relatively concentrated jujube canopy area, where the jujube canopy spectrum is a mixture of jujube and leaf spectra. The spectra of jujubes and leaves at different angles are used as input parameters for a backpropagation (BP) network for reconstruction training. The jujube canopy spectrum is reconstructed through multi-angle spectral weight allocation. The BP network is then used to train and reconstruct the jujube canopy spectrum. Less simulations of the jujube canopy spectrum provide an important reference for multi-angle inversion of fruit canopy quality using UAVs at low altitudes.

[0075] Device C

[0076] (1) Indoor multi-angle BRDF calibration device

[0077] Different light sources exhibit varying coherence during transmission, resulting in angularly distributed scattering characteristics after interacting with fruit. These characteristics can be detected at specific azimuth angles. Multi-angle iterative fusion can improve spectral resolution and the accuracy of fruit quality detection. This study focuses on establishing a multi-angle characteristic spectral library of fruit and developing a quantitative detection model for fruit quality detection accuracy at different angular resolutions by establishing the correspondence between characteristic angles and characteristic qualities.

[0078] (2) Multi-angle spectral response function and angle model transfer

[0079] A multi-angle, multi-band polarization spectral database of fruits, based on a calibration device, is used for the inversion and verification of fruit experimental quality at different angles and distances, and spectral response functions with different angular resolutions and bands are generated. Multi-angle spectral response functions are used for multi-angle spectral model transfer, enabling quantitative inversion of multi-angle, multi-spectral imaging from UAVs.

[0080] (3) Outdoor fruit tree canopy BRDF calibration field

[0081] The outdoor hyperspectral BRDF measuring instrument consists of a spatial multi-angle rotating scanning mechanism, a spectral radiance meter, and a spectral irradiance meter. The spectral radiance meter is fixed on the spatial scanning mechanism to observe surface reflection in any direction, while the spectral irradiance meter measures the incident irradiance. Under natural outdoor conditions, it automatically measures the hyperspectral bidirectional reflectance distribution function (BRDF) of ground targets (samples). It is mainly used for site radiometric calibration of UAV near-ground optical remote sensors, measurement of target (sample) directional reflectance characteristics, and verification and inversion of multi-angle remote sensing data.

[0082] The main purpose is to calibrate and verify temperature and illumination orientation by placing thermometers, illuminometers, and laser rangefinders at multiple grid points in the space to be measured. The thermometers and illuminometers (GPS) are used for spatial illumination and temperature field calibration and alignment. A horizontally and vertically movable track support is constructed to house the camera and light source. Horizontal detection and vertical inversion are performed (the movable base can secure the UAV's multispectral camera for wireless communication and image transmission).

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method of modeling a fruit multiscale spatial structure spectrum, characterized in that, The modeling comprises the following steps: The modeling comprises the following steps: The modeling comprises the following steps: The modeling comprises the following steps: The modeling method comprises indoor modeling and outdoor modeling in sequence; The indoor modeling comprises: spectrum collection and image splicing are performed through linear reciprocating movement of a net-shaped fruit carrier to simulate unmanned aerial vehicle flight; data collection and calibration are performed at different elevation angles and azimuth angles by using a disc-type BRDF device; near-infrared dynamic spectrum is collected during fruit drying; and crown scale simulation is performed on the multi-angle and multi-polarization database and the fruit spectrum model, and model inversion, verification and transmission are realized through variance and spectral mean analysis; The outdoor modeling comprises: a fine grid is divided according to the shape of the fruit tree, a corresponding relationship between pixel reflectivity and space is established, and a spatial statistical model is established by using a semi-variogram function; multi-angle temperature distribution calibration and spectrum matching are performed in an artificial temperature field by using a thermal imager; a characteristic angle database is established by using a waveband index method, a correlation coefficient method and physical and chemical indexes, model inversion and verification are performed by using a BP neural network, and temperature field matching calibration and verification are performed by using the multi-angle temperature spatial distribution model established by the thermal imager; and unmanned aerial vehicle GPS and angle are set to perform BRDF simulation and crown spectrum reconstruction.

2. The modeling method of claim 1, wherein, The method for grid quantization marking and grid calibration comprises: Spectrum density of light sources at different angles and distances is collected to obtain coherence and polarization of the light sources; grid subdivision processing and system calibration are performed on the scale and azimuth of the space to be measured according to the scaling law; Spectrum collection is performed on different row spacings and column spacings; Calibration and calibration are performed.

3. The modeling method of claim 2, wherein, The calibration and calibration method comprises: fruit spectrum distribution maps at different azimuths are collected and prepared, corresponding to the grid positions in space, single-point spectra of single fruits at different positions are spliced, and multiple fruit spatial quality distribution images are formed through calibration and calibration.

4. The modeling method of claim 1, wherein, The method for performing crown spectrum simulation of fruits on different structure fruit trees comprises: ROI variance and spectral mean simulation analysis are performed on different regional spaces, regional segmentation, classification spectrum feature matching and identification are realized through ROI at different distances and angles, and model transmission of multispectral and near-infrared spectrometers is realized; distance and angle fitting simulation is performed to realize distance and azimuth segmentation and classification matching and identification. The method for performing crown spectrum simulation of fruits on different structure fruit trees comprises: ROI variance and spectral mean simulation analysis are performed on different regional spaces, regional segmentation, classification spectrum feature matching and identification are realized through ROI at different distances and angles, and model transmission of multispectral and near-infrared spectrometers is realized; distance and angle fitting simulation is performed to realize distance and azimuth segmentation and classification matching and identification.

Images