Method for discriminating water stress of facility tomato by fusing multi-modal information and neural network

Abstract

The present application relates to the technical field of agricultural intelligent monitoring, and particularly relates to a facility tomato water stress discrimination method fusing multi-modal information and neural networks, comprising: collecting a plurality of physiological parameters and image information of a single crop at different growth stages; determining whether a sub-region is a single leaf region based on a gray level fluctuation value; determining a leaf range under the condition that the sub-region is a single leaf region; determining whether the leaf range is qualified according to a texture characteristic parameter, and adjusting the preset fluctuation value based on the unqualified leaf range condition; under the condition that the leaf range is qualified, determining whether the leaf contour is qualified based on a contour characteristic value, and adjusting the preset contrast based on the unqualified leaf contour condition; and constructing a neural network discrimination model based on the qualified leaf contour, leaf color skewness distribution parameters and corresponding physiological parameters. The present application improves the intelligent discrimination accuracy of the water stress degree of crops.

Description

Technical Field

[0001] This invention relates to the field of crop water stress discrimination technology, and in particular to a method for judging water stress in greenhouse tomatoes that integrates multimodal information and neural networks. Background Technology

[0002] Water scarcity and frequent extreme weather events have become key factors restricting the sustainable development of global agriculture. As a typical high-value fruit and vegetable crop, tomatoes are highly sensitive to abnormal soil moisture conditions. Water balance not only directly affects physiological processes such as photosynthesis and transpiration in plants, but also significantly reduces fruit quality and final yield.

[0003] Traditional methods for assessing water stress in tomatoes mainly rely on manual experience or point-based index measurements, which suffer from drawbacks such as high sampling costs, poor spatial representativeness, and response lag, making it difficult to meet the real-time, high-throughput, and intelligent monitoring needs of modern precision agriculture. In recent years, high-throughput crop phenotypic acquisition technologies have developed rapidly. Image acquisition, due to its non-contact and high-frequency advantages, has become a research hotspot for water stress monitoring. However, image parameters (such as RGB and texture) are easily affected by lighting, angle, and background interference, and cannot reliably reflect the true physiological state of plants when used alone. While field-measured parameters such as SPAD value and leaf water content have clear physiological significance, they suffer from low acquisition efficiency and limited information dimensions.

[0004] Furthermore, the physiological response of tomatoes to water stress exhibits significant nonlinear characteristics. For example, mild drought may promote root growth, while severe drought may inhibit photosynthesis. Traditional linear statistical models cannot fully capture the complex coupling relationships among multiple factors, resulting in insufficient accuracy in water stress level discrimination (typically below 40%). Therefore, how to integrate the advantages of multi-source data to improve the accuracy and practicality of tomato water stress discrimination has become a pressing technical challenge in the field of precision agriculture.

[0005] Chinese Patent Application Publication No. CN106097372A discloses a method for detecting water stress phenotypes in crop plants based on image processing. The method includes: S1: segmenting an image of the crop into a plant pattern and a background pattern; S2: segmenting the plant pattern to obtain a stem pattern and a leaf pattern; S3: refining the stem pattern into straight lines and the leaf pattern into leaf curves, calculating the angle SOP1 formed by straight lines OS and OP1, and the angle SOP2 formed by straight lines OS and OP2; S4: calculating the ratio of the angle SOP1 to the angle SOP2, and evaluating the changes in the crop leaves under water stress based on this ratio. This invention provides a method for detecting water stress phenotypes in crop plants based on image processing, which can accurately perceive and monitor subtle changes in crops and accurately reflect the interaction between crops and the environment.

[0006] Existing technologies suffer from the following problems: In intelligent crop water stress discrimination systems based on multimodal information, image information serves as a crucial perceptual input, and its quality directly determines the reliability of subsequent analysis and discrimination. However, current methods generally lack effective monitoring mechanisms for the suitability of plant patterns. This means that during image acquisition or preprocessing, a large number of low-quality or invalid plant images may be mixed in, such as those with severe occlusion, incomplete leaves, blurred images, uneven lighting, or excessive background interference. When these substandard patterns enter the discrimination model, they cause severe distortion or bias in the extracted visual features, such as leaf texture, color distribution, and canopy structure, failing to accurately reflect the true physiological state of the plant. When the model infers the degree of water stress based on these unreliable visual features, it not only struggles to correctly correlate corresponding physiological parameters such as leaf water potential and stomatal conductance but also introduces significant noise and misjudgment risks. This leads to error accumulation and feature confusion in multimodal fusion decision-making, ultimately resulting in a significant reduction in the overall discrimination accuracy of the system for crop water stress. Summary of the Invention

[0007] To address this, the present invention provides a method for determining water stress in greenhouse tomatoes that integrates multimodal information and neural networks. This method overcomes the shortcomings of existing technologies that, when determining water stress based on crop image information and physiological parameters, lack monitoring of the quality of plant patterns in the image information. This leads to the inclusion of low-quality or invalid plant images, resulting in severe distortion or deviation of extracted visual features such as leaf texture, color distribution, and canopy structure. Consequently, these features fail to accurately reflect the true physiological state of the plant, leading to low accuracy in intelligently determining the degree of water stress in crops based on plant pattern determination methods.

[0008] To achieve the above objectives, this invention provides a method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks, comprising:

[0009] Several crops were planted under water stress conditions with preset irrigation levels, and several physiological parameters and image information of individual crops at different growth stages were collected.

[0010] The image information after grayscale processing is divided into several sub-regions, and the grayscale fluctuation value of several pixel blocks in a single sub-region is compared with a preset fluctuation value to determine whether the sub-region is a single leaf region.

[0011] Under the condition that the sub-region is determined to be a single leaf region, the grayscale contrast is determined with the sub-region as the center region and the preset pixel distance as the interval, and the leaf range is determined according to the comparison result of the grayscale contrast and the preset contrast.

[0012] Obtain the texture representation parameters of the blade range, determine whether the blade range is qualified based on the texture representation parameters, and adjust the preset fluctuation value accordingly;

[0013] Under the condition that the blade range is qualified, the contour characterization value of the blade range is determined, so as to determine whether the blade contour is qualified based on the contour characterization value, and the preset contrast is adjusted based on the condition that the blade contour is unqualified.

[0014] Based on the qualified leaf profile, several leaf color skewness distribution parameters of the leaf are calculated. The leaf color skewness distribution parameters and the corresponding physiological parameters are preprocessed to obtain several sets of data to be simulated. A neural network discrimination model is constructed based on the data to be simulated.

[0015] The physiological parameters and leaf color skewness distribution parameters of the crop are input into the neural network discrimination model to output the degree of water stress of the crop.

[0016] Furthermore, the process of determining whether the sub-region is a single leaf region based on the grayscale fluctuation values ​​of several pixel blocks includes:

[0017] The grayscale fluctuation value is compared with a preset fluctuation value;

[0018] Based on the comparison result that the grayscale fluctuation value is less than or equal to the preset fluctuation value, the sub-region is determined to be a single leaf region.

[0019] Furthermore, the process of determining the blade range based on the grayscale contrast includes:

[0020] The grayscale contrast is determined by taking a single leaf area as the center area and using a preset pixel distance as the spacing.

[0021] The grayscale contrast is compared with the preset contrast.

[0022] The preset pixel distance range is determined based on the comparison result that the grayscale contrast is less than or equal to the preset contrast, which is the leaf range;

[0023] Based on the comparison result that the grayscale contrast is greater than the preset contrast, it is determined that the preset pixel distance is reduced by a preset amount to obtain the second pixel distance;

[0024] The second grayscale contrast is determined by taking a single leaf area as the central area and the second pixel distance;

[0025] The second grayscale contrast is compared with the preset contrast until the second grayscale contrast is less than or equal to the preset contrast, and the second pixel distance range is determined as the leaf range.

[0026] Furthermore, the blade area is divided into sub-regions at different scales, and the gray-level entropy of any sub-region is determined.

[0027] The sum of the products of the variation coefficients of the gray-level entropy of several sub-regions at a single scale and the corresponding weight coefficients is determined as the texture representation parameter.

[0028] Furthermore, the process of determining whether the blade range is qualified based on the texture characterization parameters includes:

[0029] The texture representation parameters are compared with preset texture representation parameters;

[0030] The blade range is determined to be unqualified based on the comparison result that the texture characterization parameter is greater than the preset texture characterization parameter.

[0031] Furthermore, under the condition that the blade range is determined to be unqualified, the process of adjusting the preset fluctuation value includes:

[0032] The difference between the texture representation parameter and the preset texture representation parameter is compared with the preset parameter difference.

[0033] Based on the comparison result between the parameter difference and the preset parameter difference, several fluctuation value adjustment coefficients are set to reduce the preset fluctuation value.

[0034] Furthermore, the process for determining the contour characterization value of the blade range is as follows:

[0035] The spatial position signal of the blade range contour is converted into a frequency domain signal based on Fourier transform to establish a power spectral density map.

[0036] Determine the intensity and width of the dominant frequency peak in the power spectral density plot;

[0037] The ratio of the main frequency peak intensity to the peak width is determined as the profile characterization value of the blade range.

[0038] Furthermore, the process of determining whether the blade profile is qualified based on the profile characterization value includes:

[0039] The contour representation value is compared with a preset representation value;

[0040] The blade profile is determined to be unqualified based on the comparison result that the profile characterization value is less than the preset characterization value.

[0041] Furthermore, under the condition that the blade profile is determined to be substandard, the process of adjusting the preset contrast includes:

[0042] Determine the length of continuous lines in the blade profile;

[0043] Compare the length of the continuous line with a preset length;

[0044] Determine the number of lines whose continuous line length is less than the preset length;

[0045] Based on a comparison of the percentage of the number of lines to the total number of lines in the blade outline with a preset percentage, several contrast adjustment coefficients are set to adjust the contrast based on these coefficients.

[0046] Furthermore, under the condition that the leaf outline is qualified, several leaf color skewness distribution parameters within the leaf outline are extracted using a higher-order equation. Based on the leaf color skewness distribution parameters and the corresponding physiological parameters, a neural network discrimination model is constructed to determine the water stress level of the crop based on the tomato image information and physiological parameters. The convergence error of the neural network discrimination model is 0.0001, the learning rate is 0.05, and the maximum number of training iterations is 1000.

[0047] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention determines whether a sub-region is a single leaf region by using the grayscale fluctuation value of the sub-region of image information. In its natural state, the mesophyll tissue of a leaf is composed of uniform cells, with stable chlorophyll and water distribution and no significant color / brightness differences. Reflected in the image, the grayscale values ​​of pixels within the same leaf will be concentrated in a narrow range. However, in non-leaf regions or mixed areas of leaves and non-leaf regions, due to the diverse pixel sources, the grayscale differences will increase sharply, and the grayscale fluctuation value will rise significantly. Taking a single leaf region as the central region and using a preset pixel distance as the spacing to determine the grayscale contrast, the leaf range is determined based on the grayscale contrast. The grayscale contrast of different regions within the leaf is low and stable, while the grayscale contrast outside the leaf boundary will increase and change abruptly. By iteratively reducing the pixel distance, the critical position from the low-contrast interior of the leaf to the high-contrast leaf boundary is found, and finally, the area within the critical position is locked as the leaf range. After determining the leaf area, the model uses texture representation parameters to determine if the leaf area is acceptable. Determining the leaf area only solves the problem of physically selecting the leaf region from the image; it cannot eliminate hidden defects within that region, such as insect holes formed by bitten leaves. Only pure leaf regions with uniform texture can extract leaf color skewness distribution parameters that are representative and reflect the true physiological state of the leaf. Under the condition that the leaf area is acceptable, the model determines the contour representation value of the leaf area to determine if the leaf contour is acceptable. Determining the leaf area is acceptable only ensures that the leaf region is free of impurities, damage, and has uniform tissue, but it does not address the effectiveness of the overall leaf contour morphology. Even if the interior of the leaf area is pure, its contour may still have physical damage or acquisition errors. Using images of leaves with unacceptable contours for model training can cause the model to mistakenly equate parameters caused by damage with parameters caused by stress, leading to non-stressed samples being classified as stressed, thus reducing the model's discrimination accuracy. Therefore, this invention uses a step-by-step screening process of several acquired image information to ensure that the association between the acquired samples and water stress is unique and conflict-free, thereby improving the model's intelligent discrimination accuracy for the degree of crop water stress.

[0048] Furthermore, under the condition that the blade outline is unqualified, the present invention determines the adjustment range of the preset contrast based on the length of the continuous lines of the blade outline. The value of the preset contrast determines the boundary accuracy of the blade range. If the preset contrast is too high, the area with slightly fluctuating grayscale at the edge of the blade will be mistakenly identified as the background, resulting in a smaller extracted blade range, broken or missing outlines, and discontinuous lines that should be continuous at the edge of the blade. If the preset contrast is too low, the background area around the blade will be mistakenly identified as the blade, resulting in a larger extracted blade range, redundant and blurred outlines, and the outline containing irrelevant background lines. When the preset contrast is too high, the blade boundary will be judged too strictly. Even if a small segment of the blade edge still belongs to the area of ​​the blade itself and the grayscale fluctuates only slightly, it will be mistakenly identified as the background because the grayscale contrast is slightly higher than the preset value, resulting in broken outline lines and forming a large number of short lines with a continuous line length less than the preset length. Therefore, the adjustment range of the preset contrast is determined based on the length of the continuous lines to further improve the intelligent discrimination accuracy of the neural network discrimination model built based on image information. Attached Figure Description

[0049] Figure 1 This is a flowchart of a method for determining water stress in greenhouse tomatoes that integrates multimodal information and neural networks, according to an embodiment of the present invention.

[0050] Figure 2 This is a flowchart illustrating how to determine whether a sub-region is a single-blade region in an embodiment of the present invention.

[0051] Figure 3 This is a flowchart for determining whether the blade range is qualified according to an embodiment of the present invention;

[0052] Figure 4 This is a flowchart for determining whether the blade profile is qualified in an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0054] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0055] Please see Figure 1 The flowchart shown is a process for determining water stress in facility-grown tomatoes, which integrates multimodal information and neural networks, according to an embodiment of the present invention.

[0056] The present invention provides a method for determining water stress in greenhouse tomatoes that integrates multimodal information and neural networks, comprising:

[0057] Step S1: Plant several crops under water stress environment with preset irrigation level, and collect several physiological parameters and image information of each crop at different growth stages. The physiological parameters include single leaf fresh weight, single leaf dry weight, chlorophyll value, leaf area, leaf perimeter, leaf thickness, leaf water content and plant height.

[0058] Step S2: Divide the image information after grayscale processing into several sub-regions, and determine whether the sub-region is a single leaf region by comparing the grayscale fluctuation value of several pixel blocks in a single sub-region with a preset fluctuation value.

[0059] Step S3: Under the condition that the sub-region is a single leaf region, the grayscale contrast is determined with the sub-region as the center region and the preset pixel distance as the interval, and the leaf range is determined according to the comparison result of the grayscale contrast and the preset contrast.

[0060] Step S4: Under the condition that the blade range is qualified, determine the contour characterization value of the blade range, so as to determine whether the blade contour is qualified based on the contour characterization value, and set several contrast adjustment coefficients based on the condition that the blade contour is unqualified to adjust the preset contrast.

[0061] Step S5: Calculate several leaf color skewness distribution parameters of the leaf based on the qualified leaf outline, preprocess the leaf color skewness distribution parameters and the corresponding physiological parameters to obtain several sets of data to be simulated, and construct a neural network discrimination model based on the data to be simulated.

[0062] Step S6: Input the crop's physiological parameters and leaf color skewness distribution parameters into the neural network discrimination model to output the crop's water stress level.

[0063] Specifically, the tomato planting plan was as follows: tomato seedlings were transplanted into plastic pots with a diameter of 35cm, a height of 45cm, and a volume of 43.3L. The planting substrate was a pure commercial substrate without soil mixing. The entire experiment was conducted under natural light conditions, and the temperature and humidity range suitable for tomato growth was maintained in the greenhouse. All plants were irrigated once after transplanting until the soil was saturated. The initial irrigation amount was denoted as Q (the measured value was 2500mL), which served as the standard value for setting subsequent water gradients.

[0064] The experiment consisted of five irrigation gradient treatment groups, representing different levels of flooding and drought: flood (Level I), moderately flooded (Level II), normal (Level III), moderately droughty (Level IV), and droughty (Level V). The irrigation program was conducted in stages, with a treatment and observation interval of 5 days. Irrigation was carried out every 5 days, and relevant measurements were completed on the day of irrigation. Specific settings are as follows:

[0065] Phase 1:

[0066] ,

[0067] ,

[0068] Phase Two:

[0069] ,

[0070] Phase Three:

[0071] ,

[0072] Basic fertilization management was carried out during the experiment in conjunction with the tomato growth process:

[0073] During the vegetative growth period (before flowering): Use a balanced compound fertilizer (N:P:K≈1:1:1) at a concentration of 5g / 5L. -1 It is diluted before irrigation.

[0074] During the reproductive growth period (after flowering): switch to a high-potassium compound fertilizer to promote fruit enlargement and improve quality, keeping the ratio and concentration consistent.

[0075] Physiological parameters collected during the tomato growth stages included: chlorophyll content was measured using a SPAD-502 Plus handheld chlorophyll meter. Three to five unfolded leaves were selected from the central region of each plant, and measurements were taken at three points on each leaf, with the average value calculated. Leaf area was measured using a CI-202 leaf area meter. Complete single leaves were selected to measure their area, perimeter, and morphological characteristics to calculate unit leaf weight, thickness, and specific leaf area. Leaf thickness was measured three times at the midrib using vernier calipers, and the average value was calculated. Fresh and dry weight of single leaves were weighed using an electronic balance on the day of sampling. Dry weight was obtained after drying at a constant temperature of 80℃ to constant weight, and the leaf moisture content was then calculated. Plant height was measured using a ruler, referring to the length from the ground to the top of the main stem. All measurements were performed with three replicates, and the average values ​​were calculated for subsequent statistical analysis and modeling. The models of the electronic balance and drying device were not limited.

[0076] Specifically, when collecting physiological parameters of tomatoes at different stages, an image sensor is used simultaneously to collect image information of the tomato canopy. The specific model and parameters of the image sensor are not limited.

[0077] Specifically, the grayscale processed image information is divided into several sub-regions, which can be done by dividing the area equally, without any specific limitation.

[0078] Please see Figure 2 As shown, it is a flowchart of an embodiment of the present invention for determining whether a sub-region is a single blade region.

[0079] Specifically, the process of determining whether the sub-region is a single leaf region based on the grayscale fluctuation values ​​of several pixel blocks includes:

[0080] The grayscale fluctuation value is compared with a preset fluctuation value;

[0081] Based on the comparison result that the grayscale fluctuation value is less than or equal to the preset fluctuation value, the sub-region is determined to be a single-leaf region;

[0082] Based on the comparison result that the grayscale fluctuation value is greater than the preset fluctuation value, it is determined that the sub-region is not a single leaf region.

[0083] Specifically, the grayscale fluctuation value refers to the maximum absolute difference in grayscale between adjacent pixel blocks in the sub-region, and the range of the grayscale fluctuation value is set to [10, 30]. Embodiment 15 of the present invention is preferred.

[0084] Specifically, the process of determining the blade range based on the grayscale contrast includes:

[0085] The grayscale contrast is determined by taking a single leaf area as the center area and using a preset pixel distance as the spacing.

[0086] The grayscale contrast is compared with the preset contrast.

[0087] The preset pixel distance range is determined based on the comparison result that the grayscale contrast is less than or equal to the preset contrast, which is the leaf range;

[0088] Based on the comparison result that the grayscale contrast is greater than the preset contrast, it is determined that the preset pixel distance is reduced by a preset amount to obtain the second pixel distance;

[0089] The second grayscale contrast is determined by taking a single leaf area as the central area and the second pixel distance;

[0090] The second grayscale contrast is compared with the preset contrast until the second grayscale contrast is less than or equal to the preset contrast, and the second pixel distance range is determined as the leaf range.

[0091] Specifically, the preset pixel distance is set to a value range of [10 pixels, 20 pixels], and is preferably 13 pixels in this embodiment of the invention; the grayscale contrast refers to the difference between the average grayscale value of the pixel block at the edge of a single leaf region and the average grayscale value of the pixel block within the preset pixel distance outside the edge, and the preset contrast is set to a value range of [8, 15], and is preferably 10 in this embodiment of the invention; the preset amplitude is 3 pixels.

[0092] Specifically, the process of determining the texture characterization parameters of the blade range includes:

[0093] The blade area is divided into sub-regions at different scales, and the gray entropy of any sub-region is determined.

[0094] The coefficient of variation of gray-level entropy of several sub-regions at a single scale is determined as a single-scale texture representation parameter;

[0095] The sum of the products of several single-scale texture representation parameters and their corresponding weight coefficients is determined as the texture representation parameter.

[0096] Specifically, for the determined leaf range, the leaf range is evenly divided into several independent sub-regions according to three different scales: coarse, medium, and fine. The process of determining the grayscale entropy is to extract the grayscale value of each pixel block in a single region and substitute the grayscale value into the grayscale entropy calculation formula to obtain the grayscale entropy. This is existing technology and will not be elaborated further. The scale size is, for example, 32×32 pixels for the coarse scale, 16×16 pixels for the medium scale, and 8×8 pixels for the fine scale, and the specific size is not limited.

[0097] Specifically, the weighting coefficient for the single-scale texture representation parameter determined by dividing a single sub-region at a coarse scale is 0.2, the weighting coefficient for the single-scale texture representation parameter determined by dividing a single sub-region at a medium scale is 0.5, and the weighting coefficient for the single-scale texture representation parameter determined by dividing a single sub-region at a fine scale is 0.3.

[0098] Please see Figure 3 As shown, it is a flowchart for determining whether the blade range is qualified according to an embodiment of the present invention.

[0099] Specifically, the process of determining whether the blade range is qualified based on the texture characterization parameters includes:

[0100] The texture representation parameters are compared with preset texture representation parameters;

[0101] The blade range is determined to be unqualified based on the comparison result that the texture characterization parameter is greater than the preset texture characterization parameter;

[0102] The blade range is determined to be qualified based on the comparison result of the texture characterization parameter being less than or equal to the preset texture characterization parameter.

[0103] Specifically, the preset texture representation parameter is set to a value range of [0.1, 0.3], and preferably 0.2 in this embodiment of the invention.

[0104] Understandably, a smaller texture representation parameter indicates a more uniform texture across regions, while a larger texture representation parameter indicates the presence of regions with significant texture differences. In crop water stress discrimination models, a qualified leaf range must meet the following conditions: it must contain only a single, complete leaf region, without background soil / weeds, leaf damage / insect holes, or overlapping leaves. The texture of such pure leaf regions is consistent. Whether it is the transition between leaf veins and leaf tissue or the grayscale distribution of different parts of the leaf, it follows the physiological structural laws of the leaf itself and there will be no sudden texture jumps. Therefore, its grayscale entropy variation coefficient will remain at a low level.

[0105] Specifically, when the blade range is determined to be unqualified, the process of adjusting the preset fluctuation value includes:

[0106] The difference between the texture representation parameter and the preset texture representation parameter is compared with the preset parameter difference.

[0107] Based on the comparison result between the parameter difference and the preset parameter difference, several fluctuation value adjustment coefficients are set to reduce the preset fluctuation value.

[0108] Specifically, based on the comparison result that the parameter difference is greater than the preset parameter difference, a first fluctuation value adjustment coefficient is used to reduce the preset fluctuation value;

[0109] Based on the comparison result that the parameter difference is less than or equal to the preset parameter difference, a second fluctuation value adjustment coefficient is used to reduce the preset fluctuation value.

[0110] Specifically, the preset fluctuation value is a key threshold for determining whether a sub-region is a single leaf region. However, if the initial preset fluctuation value is too large, the judgment standard for grayscale uniformity of the sub-region will be too lenient. This may lead to the misjudgment of sub-regions containing background noise (such as soil or light spots) or multiple overlapping leaves as single leaf regions. If the leaf range is determined with the incorrect single leaf region as the center, the subsequent leaf range will be mixed with interference regions such as background and overlapping leaves. The grayscale entropy of the interference regions is very different from that of the normal leaf region, which will cause the texture representation parameters to exceed the standard. In the end, the leaf range will be judged as unqualified. Reducing the preset fluctuation value is to tighten the judgment standard for single leaf regions. A smaller preset fluctuation value requires the grayscale fluctuation of the pixel blocks in the sub-region to be smoother, so as to accurately exclude interference sub-regions containing background and overlapping leaves, and retain only the truly pure single leaf regions. This will reduce the coefficient of variation of grayscale entropy to a qualified level, thereby correcting the judgment result of the leaf range.

[0111] Specifically, the range of the preset parameter difference is set to [0.05, 0.15], and preferably 0.1 in this embodiment of the invention; the range of the first fluctuation value adjustment coefficient is set to [0.6, 0.8], and preferably 0.7 in this embodiment of the invention; the range of the second fluctuation value adjustment coefficient is set to [0.85, 0.95], and preferably 0.9 in this embodiment of the invention.

[0112] Specifically, the process for determining the contour characterization value of the blade range is as follows:

[0113] The spatial position signal of the blade range contour is converted into a frequency domain signal based on Fourier transform to establish a power spectral density map.

[0114] Determine the intensity and width of the dominant frequency peak in the power spectral density plot;

[0115] The ratio of the main frequency peak intensity to the peak width is determined as the profile characterization value of the blade range.

[0116] Specifically, the essence of a qualified blade profile is a continuous and regular edge shape. The role of Fourier transform is to convert the profile shape in the spatial domain into frequency components in the frequency domain. Different profile shapes correspond to different frequency distributions. The profile of a healthy blade is composed of a single dominant shape, which will form a high-intensity main frequency peak in the frequency domain, indicating that the frequency component of the dominant shape is absolutely dominant. If the blade profile is unqualified, such as if there are damages, gaps, or deformities at the edges, the profile will be mixed with local irregular shapes. These irregular shapes will generate a large number of noise peaks in the frequency domain, resulting in low energy of the main frequency peak and ultimately a significant reduction in the intensity of the main frequency peak. The profile edges of a healthy blade are smooth and continuous, and the frequency components of the dominant shape are highly concentrated. Therefore, the peak width of the main frequency peak is relatively narrow (peak width refers to the full width at half maximum of the main frequency peak). The narrower the peak width, the more concentrated the frequency components and the more stable the profile shape. If the blade profile is unqualified, the frequency of the dominant shape will shift slightly, causing the peak width of the main frequency peak to widen. Therefore, the larger the ratio of the intensity of the main frequency peak to the peak width, the better the continuity and regularity of the blade profile.

[0117] It is understandable that converting the spatial position signal of the blade range contour into a frequency domain signal based on Fourier transform and establishing a power spectral density map, and determining the main frequency peak intensity and peak width based on the power spectral density map, can be achieved using Python's OpenCV, NumPy, and SciPy libraries. This is existing technology and will not be elaborated further.

[0118] Please see Figure 4 As shown, it is a flowchart for determining whether the blade profile is qualified according to an embodiment of the present invention.

[0119] Specifically, the process of determining whether the blade profile is qualified based on the profile characterization value includes:

[0120] The contour representation value is compared with a preset representation value;

[0121] Based on the comparison result that the profile characterization value is less than the preset characterization value, the blade profile is determined to be unqualified.

[0122] The blade profile is deemed qualified based on the comparison result that the profile characterization value is greater than or equal to the preset characterization value.

[0123] Specifically, the preset representation value is set to a range of [20, 30], and the preferred embodiment of the present invention is 25.

[0124] Specifically, when the blade profile is determined to be substandard, the process of adjusting the preset contrast includes:

[0125] Determine the length of continuous lines in the blade profile;

[0126] Compare the length of the continuous line with a preset length;

[0127] Determine the number of lines whose continuous line length is less than the preset length;

[0128] Based on a comparison of the percentage of the number of lines to the total number of lines in the blade outline with a preset percentage, several contrast adjustment coefficients are set to adjust the preset contrast based on these coefficients.

[0129] Specifically, based on the comparison result that the percentage is greater than or equal to the preset percentage, it is determined to increase the preset contrast by a first contrast adjustment coefficient;

[0130] Based on the comparison result that the percentage is less than the preset percentage, the preset contrast is increased by a second contrast adjustment coefficient.

[0131] Specifically, the continuous line length of the blade outline refers to the length of the pixel line segment between two adjacent inflection points on the outline. The preset length is set to a value range of [13 pixels, 18 pixels], and preferably 14 pixels in this embodiment of the invention; the preset percentage is set to a value range of [20%, 30%], and preferably 23% in this embodiment of the invention; the first contrast adjustment coefficient is set to a value range of [1.3, 1.6], and preferably 1.4 in this embodiment of the invention; the second contrast adjustment coefficient is set to a value range of [1.1, 1.29], and preferably 1.15 in this embodiment of the invention.

[0132] Specifically, under the condition that the leaf outline is qualified, several leaf color skewness distribution parameters within the leaf outline are extracted using a higher-order equation. Based on the leaf color skewness distribution parameters and the corresponding physiological parameters, a neural network discrimination model is constructed to determine the water stress level of the crop based on the tomato image information and physiological parameters. The convergence error of the neural network discrimination model is 0.0001, the learning rate is 0.05, and the maximum number of training iterations is 1000.

[0133] Specifically, higher-order equations are used to extract several leaf color skewness distribution parameters within the leaf outline. These parameters include mean-type parameters and distribution-type parameters. The mean-type parameters include the mean (RM1), median (RM2), and mode (RM3) of the red (R) channel color levels, the mean (GM1), median (GM2), and mode (GM3) of the green (G) channel color levels, the mean (BM1), median (BM2), and mode (BM3) of the blue (B) channel color levels, and the mean (YM1), median (YM2), and mode (YM3) of the grayscale (Y) image color levels. The distribution-type parameters include the skewness (RS) and kurtosis (RK) of the red (R) channel color levels, the skewness (GS) and kurtosis (GK) of the green (G) channel color levels, the skewness (BS) and kurtosis (BK) of the blue (B) channel color levels, and the skewness (YS) and kurtosis (YK) of the grayscale (Y) image color levels.

[0134] Specifically, a neural network discrimination model is constructed based on several of the aforementioned leaf color skewness distribution parameters and several of the aforementioned physiological parameters, firstly using empirical formulas. Different numbers of neurons were determined, and training comparisons were performed to obtain the optimal number of neurons in the hidden layer. Here, p, o, and l represent the number of neurons in the hidden layer, input layer, and output layer, respectively, and α is a constant between [1, 10]. Then, a BP neural network discriminant model of tomato water stress was constructed using the Neural Network Toolbox in MATLAB 2016R. The transfer function of the intermediate layer neurons was the Logsig function, the transfer function of the output layer neurons was the Purelin linear function, and the training function was Trainlm. 75% of the data was used for model training, 15% for prediction validation, and 10% for model testing. The convergence error of the model was set to 0.0001, the learning rate was set to 0.05, and the maximum number of training iterations was set to 1000. It is understood that the process of constructing the BP neural network model is existing technology and will not be elaborated further.

[0135] In the implementation of this invention, firstly, linear regression models are established based on the physiological parameters, mean parameters, and distribution parameters of greenhouse tomatoes during their growth stages. Using SPSS software, irrigation level is used as the dependent variable, and mean parameters, tomato physiological parameters, and distribution parameters are used as independent variables. A stepwise regression method using least squares is employed to establish the linear models. When the significance test of the regression equation is P≤0.05, the independent variables are added to the regression equation; when the significance test of the regression equation is P≥0.1, the independent variables are removed from the regression equation, resulting in several alternative regression models. The alternative regression models are then subjected to selection of the coefficient of determination, significance testing of the regression models and regression coefficients, and diagnosis of collinearity of the independent variables. Finally, the optimal regression model expression is determined, as shown in Table 1.

[0136] Table 1 Linear Regression Model

[0137] ;

[0138] Among them, model M1 is determined based on the physiological parameters of tomatoes, model M2 is determined based on the mean class parameters of tomatoes, model M3 is determined based on the distribution class parameters of tomatoes, model M4 is determined based on the mean class parameters and physiological parameters of tomatoes, and model M5 is determined based on the distribution class parameters and physiological parameters of tomatoes. The accuracy rates of each model are shown in Table 2. The accuracy rate of the model is the percentage of the absolute difference between the simulated value and the measured value to the measured value. The simulated value and the measured value are both the volumetric water content of the soil. The volumetric water content of the soil is greater than 35% and indicates waterlogging, greater than or equal to 20% and less than or equal to 35% and indicates moderate waterlogging, greater than or equal to 15% and less than 20% and indicates normal, greater than or equal to 10% and less than 15% and indicates moderate drought, and less than 10% and indicates drought.

[0139] Table 2 shows the accuracy of the model's judgments.

[0140] ;

[0141] As shown in the figure, the maximum accuracy of the M1-M5 model is 63.64%.

[0142] Then, based on the skewed distribution parameters of tomato leaf color and physiological parameters, a neural network discrimination model is constructed, first using empirical formulas. Different numbers of neurons were determined, and training comparisons were performed to obtain the optimal number of neurons in the hidden layer. Here, p, o, and l represent the number of neurons in the hidden layer, input layer, and output layer, respectively, and α is a constant between [1, 10]. The input layer has 4 neurons, including two leaf color skewness distribution parameters: skewness of the red channel color gradation and kurtosis of the green channel color gradation, as well as two physiological parameters: chlorophyll value and leaf water content. The output layer has 1 neuron, representing the tomato's water content level. Then, a BP neural network discriminant model of tomato water stress was constructed using the Neural Network Toolbox in MATLAB 2016R. The transfer function for the intermediate layer neurons was the Logsig function, the transfer function for the output layer neurons was the Purelin linear function, and the training function was Trainlm. 75% of the data was used for model training, 15% for prediction validation, and 10% for model testing. The model convergence error was set to 0.0001, the learning rate was set to 0.05, and the maximum number of training iterations was set to 1000, as shown in Table 3.

[0143] Table 3 Neural Network Discriminant Model

[0144] ;

[0145] As can be seen from the table, the BP neural network discriminant model constructed in this embodiment of the invention has a discrimination accuracy of 86.36%, which is much higher than the maximum discrimination accuracy of 63.64% of the M1-M5 linear regression model. Therefore, this embodiment of the invention selects the BP neural network discriminant model for water stress discrimination model of greenhouse tomatoes.

[0146] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks, characterized in that, include: Several crops were planted under water stress conditions with preset irrigation levels, and several physiological parameters and image information of individual crops at different growth stages were collected. The image information after grayscale processing is divided into several sub-regions. The grayscale fluctuation value of several pixel blocks in a single sub-region is compared with a preset fluctuation value to determine whether the sub-region is a single leaf region. The grayscale fluctuation value refers to the maximum absolute difference in grayscale between adjacent pixel blocks in the sub-region. Under the condition that the sub-region is a single leaf region, the grayscale contrast is determined with the sub-region as the center region and the preset pixel distance as the interval. The leaf range is determined according to the comparison result of the grayscale contrast and the preset contrast. The grayscale contrast refers to the difference between the average grayscale value of the pixel block at the edge of the single leaf region and the average grayscale value of the pixel block within the preset pixel distance outside the edge. The texture representation parameters of the blade range are obtained, and the blade range is used to determine whether it is qualified. Based on the unqualified condition of the blade range, the preset fluctuation value is adjusted. The process of determining the texture characterization parameters of the blade range includes: The blade area is divided into sub-regions at different scales, and the gray entropy of any sub-region is determined. The sum of the products of the coefficients of variation of the gray-level entropy of several sub-regions at a single scale and the corresponding weight coefficients is determined as the texture representation parameter; If the blade range is determined to be acceptable, a contour characterization value for the blade range is determined. Based on this value, it is determined whether the blade range contour is acceptable. The preset contrast is then adjusted based on the condition that the blade range contour is unacceptable. The process for determining the contour characterization value of the blade range is as follows: The spatial position signal of the blade range contour is converted into a frequency domain signal based on Fourier transform to establish a power spectral density map. Determine the intensity and width of the dominant frequency peak in the power spectral density plot; The ratio of the main frequency peak intensity to the peak width is determined as the profile characterization value of the blade range; Based on the qualified leaf profile, several leaf color skewness distribution parameters of the leaf are calculated. The leaf color skewness distribution parameters and the corresponding physiological parameters are preprocessed to obtain several sets of data to be simulated. A neural network discrimination model is constructed based on the data to be simulated. The leaf color skewness distribution parameters include mean parameters and distribution parameters. The process of adjusting the preset fluctuation value when the blade range is determined to be unqualified includes: The difference between the texture representation parameters and the preset texture representation parameters is compared with the preset parameter difference. Based on the comparison result between the parameter difference and the preset parameter difference, several fluctuation value adjustment coefficients are set to reduce the preset fluctuation value.

2. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 1, characterized in that, The process of determining whether a sub-region is a single leaf region based on the grayscale fluctuation values ​​of several pixel blocks includes: The grayscale fluctuation value is compared with a preset fluctuation value; Based on the comparison result that the grayscale fluctuation value is less than or equal to the preset fluctuation value, the sub-region is determined to be a single-blade region.

3. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 2, characterized in that, The process of determining the blade range based on the grayscale contrast includes: The grayscale contrast is determined by taking a single leaf area as the center area and using a preset pixel distance as the spacing. The grayscale contrast is compared with the preset contrast. The preset pixel distance range is determined based on the comparison result that the grayscale contrast is less than or equal to the preset contrast, which is the leaf range; Based on the comparison result that the grayscale contrast is greater than the preset contrast, a second pixel distance is obtained by reducing the preset pixel distance by a preset amount. The second grayscale contrast is determined by taking a single leaf area as the central area and the second pixel distance. The second grayscale contrast is compared with the preset contrast until the second grayscale contrast is less than or equal to the preset contrast, and the second pixel distance range is determined as the leaf range.

4. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 3, characterized in that, The process of determining whether the blade range is qualified based on the texture characterization parameters includes: The texture representation parameters are compared with preset texture representation parameters; The blade range is determined to be unqualified based on the comparison result that the texture characterization parameter is greater than the preset texture characterization parameter.

5. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 4, characterized in that, The process of determining whether the blade profile is qualified based on the profile characterization value includes: The contour representation value is compared with a preset representation value; The blade profile is determined to be unqualified based on the comparison result that the profile characterization value is less than the preset characterization value.

6. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 5, characterized in that, The process of adjusting the preset contrast when the blade profile is determined to be substandard includes: Determine the length of continuous lines in the blade profile; Compare the length of the continuous line with a preset length; Determine the number of lines whose continuous line length is less than the preset length; Based on a comparison of the percentage of the number of lines to the total number of lines in the blade outline with a preset percentage, several contrast adjustment coefficients are set to adjust the contrast based on these coefficients.

7. The method for determining water stress in greenhouse tomatoes by integrating multimodal information and neural networks according to claim 6, characterized in that, Under the condition that the leaf outline is qualified, several leaf color skewness distribution parameters within the leaf outline are extracted using higher-order equations. Based on the leaf color skewness distribution parameters and the corresponding physiological parameters, a neural network discrimination model is constructed to determine the water stress level of the crop based on the tomato image information and physiological parameters. The convergence error of the neural network discrimination model is 0.0001, the learning rate is 0.05, and the maximum number of training iterations is 1000.

Images