Illumination condition adaptive soil humidity estimation method based on multi-modal data fusion

Abstract

The invention relates to the field of robot terrain perception, and provides an illumination condition adaptive soil humidity estimation method based on multi-modal data fusion. The method comprises the following steps: firstly, providing an optimal exposure time calculation method for constructing a multi-modal soil humidity data set; secondly, soil image shadows are removed through an SS-GAN algorithm, a channel attention mechanism ECA and a standard convolutional layer are improved in combination with environmental information, ResNet-50 is embedded as a network backbone, and soil humidity estimation sub-models adapted to different illumination conditions are designed; then, training each sub-model by using an improved loss function to enable the sub-model to adapt to a specific illumination condition; finally, a sub-model fusion and switching method is provided, and a soil humidity estimation map is generated through gray value mapping. According to the invention, multi-modal data fusion is carried out through the improved neural network model, and a fusion and switching algorithm of the sub-models is designed, so that the soil humidity estimation precision of the robot under a complex illumination condition can be improved, and thus the terrain perception capability is improved.

Description

Technical Field

[0001] The present invention relates to the field of robot terrain perception, and in particular to a method for estimating soil moisture based on illumination conditions and adaptively based on multimodal data fusion. Background Art

[0002] In the field of robot terrain perception and autonomous navigation, accurate estimation of soil moisture is crucial to ensure that robots can travel safely and efficiently on slippery roads. Changes in soil moisture directly affect the interaction characteristics between the robot and the ground, including traction, grip, and friction, which are key factors affecting the safety and efficiency of robot driving. Especially in application scenarios such as agriculture, rescue, and exploration, robots need to adapt to changing soil conditions, and accurate soil moisture perception helps optimize driving strategies and path planning.

[0003] Traditional soil moisture measurement methods, such as the use of inserted soil moisture sensors or drying methods, can provide relatively accurate moisture data in local areas, but have significant limitations in large areas or complex terrain. These methods require direct contact with the soil, which is complex and time-consuming to operate, and cannot achieve real-time monitoring, making it difficult to reflect dynamic changes in the environment in a timely manner. In addition, the equipment cost is high, which is not conducive to large-scale deployment on robotic platforms. On the other hand, although satellite and aerial images have been used for soil moisture monitoring in large-scale agriculture, their temporal and spatial resolution is low and the update frequency is insufficient, which makes it difficult to meet the robot's needs for accurate, real-time, and in-situ moisture measurement.

[0004] In recent years, vision-based soil moisture estimation methods have gradually attracted attention. These methods mainly use single-modal data captured by RGB cameras perpendicular to the soil to estimate soil moisture through image processing and machine learning techniques. However, existing methods usually assume fixed lighting conditions and do not fully consider the various lighting changes that robots may encounter during actual driving, such as changes in light intensity and color temperature, as well as complex environmental factors such as shadows and reflections. In practical applications, the variability of lighting conditions will lead to degraded image quality and difficulty in feature extraction, which will significantly reduce the accuracy of moisture estimation. In addition, existing methods lack effective fusion of multimodal data, such as infrared images, converted images in different color spaces, and environmental sensor data. This limits the robustness and applicability of the model in complex environments and cannot meet the robot's needs for accurate soil moisture estimation in a changing environment.

[0005] Given the existing challenges, it is necessary to develop a new soil moisture estimation method. This method will integrate multimodal data collected under different lighting and viewing conditions, and use a conditional adaptation network to dynamically adapt to lighting changes to improve the accuracy and stability of soil moisture estimation. This will enable the robot to achieve safer and more efficient autonomous navigation in slippery soil environments. Summary of the invention

[0006] The present invention provides a method for estimating soil moisture under illumination conditions based on multimodal data fusion, which can accurately estimate soil moisture under various illumination conditions and is particularly suitable for autonomous navigation and environmental monitoring of robots on soil roads. Specifically, the method includes the following steps:

[0007] Calculation of optimal exposure time: A weighted fusion texture evaluation index is designed, which comprehensively considers the image GLCM energy (GLCM Energy), contrast, and information entropy. This index dynamically adjusts the weight coefficient according to the light intensity and soil moisture. On this basis, the calculation formula for the optimal exposure time is designed. The D435i camera is used to take soil images under different lighting conditions and set different exposure times. The GLCM Energy, contrast, and information entropy of the image at each exposure time are calculated, and the optimal exposure time is determined by the calculation formula to ensure that the acquired image has the best texture features, thereby providing information-rich input data for subsequent soil moisture estimation.

[0008] Data collection and construction: A soil container placement scheme was designed to ensure that the layout covers the camera field of view, so that the trained model can be generalized to the entire field of view of the camera. Soil images and related environmental data were collected according to the optimal exposure time determined under various lighting conditions to construct a multimodal dataset. Data augmentation and annotation were performed on the dataset to enrich the training samples and improve the generalization ability of the model.

[0009] Network model design: We designed an improved conditional channel attention (C-ECA) that considers environmental information and a convolutional generative network (C-Conv) that considers lighting conditions, and embedded these two improved networks into ResNet-50 as the network backbone. Based on the network backbone, we embedded targeted network modules into the backbone network according to the characteristics of soil images under different lighting conditions, and designed four soil moisture estimation sub-models: inverse photon model, backlight sub-model, sidelight sub-model, and cloudy sub-model. Each sub-model estimates soil moisture for images under specific lighting conditions.

[0010] Sub-model fusion / switching algorithm design: The algorithm includes a cloudy recognition module and a sub-model weight assignment module based on fuzzy rules. First, a cloudy recognition module implemented by a convolutional neural network (CNN) is introduced to determine whether it is cloudy to decide whether the cloudy sub-model needs to be called. In the weight assignment module, the angle θ between the driving direction and the direct direction of the sun, the illumination intensity I, and the weight α of the side photon sub-model are used as fuzzy linguistic variables, and the membership function and fuzzy rules are designed considering the relationship between the driving heading and the adjacent sub-models. Through fuzzy reasoning and defuzzification, the weight coefficient α of the side photon sub-model is obtained, and then the weight 1-α of the backlight sub-model or the reverse photon sub-model fused with the side photon sub-model is determined.

[0011] Improved loss function: The cross entropy loss function is optimized by introducing illumination weights to reflect the impact of different illumination conditions on soil moisture estimation. This illumination weight is dynamically adjusted according to the current illumination intensity, making the model more sensitive to errors under high illumination conditions, thereby improving estimation accuracy. In addition, in order to enable the fully connected neural networks in the conditional channel attention mechanism (C-ECA) and conditional convolution (C-Conv) to generate better channel weights and convolution kernels, a regularization term is introduced into the loss function. By imposing constraints on network parameters, the regularization term can prevent overfitting and improve the stability and generalization ability of the model in complex lighting environments.

[0012] In practical applications, the trained network model is deployed on a wheeled robot to estimate soil moisture from real-time soil images. Based on the estimation results, a soil moisture estimation map is generated through grayscale value mapping to visualize the spatial distribution of soil moisture. This method can improve the robot's perception of the soil road environment, thereby enhancing its autonomous decision-making and navigation capabilities in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The schematic diagram of the method for estimating soil moisture adaptively to light conditions;

[0014] Figure 2 It is a flowchart of the light condition adaptive soil moisture estimation method;

[0015] Figure 3 This is a diagram of the moisture soil placement plan;

[0016] Figure 4 To simulate the driving direction map;

[0017] Figure 5 This is the SS-GAN shadow removal network model diagram;

[0018] Figure 6 It is the network backbone diagram;

[0019] Figure 7 Inverse photon model diagram;

[0020] Figure 8 Side photon model diagram;

[0021] Fig. 9 Backlight sub-model diagram;

[0022] Fig.10 This is the model diagram of Yin Tianzi.

[0023] Fig.11 It is a fuzzy rule table diagram. DETAILED DESCRIPTION

[0024] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0025] The embodiment of the present invention provides a soil moisture estimation method based on multimodal data and conditional adaptive network, such as Figure 1 As shown, Figure 2 The schematic diagram of the process according to the present invention shows that the method comprises the following steps:

[0026] The determination of the optimal exposure time is based on the variable lighting conditions that the robot may encounter during actual driving. First, the light intensity is divided into three levels from 10,000 lux to 100,000 lux, and the color temperature is adjusted according to the actual environment, and three color temperature levels of 2800K, 4500K, and 5500K are set. Specifically, the color temperature is set to 2800K at a light intensity of 10,000 lux, which is suitable for natural light conditions in the morning or evening, simulating the soft light of sunrise or sunset; the color temperature is set to 4500K at a light intensity of 20,000 lux and 30,000 lux, which is suitable for a mild light environment in the morning or afternoon; from 40,000 lux to 100,000 lux, the color temperature is unified to 5500K to adapt to the high-intensity light at noon or summer noon. These settings ensure realistic simulation of lighting conditions in different time periods. In addition, in order to simulate the changes in light intensity under cloudy conditions, the cloudy light intensity is divided into three levels, namely 10,000 lux, 20,000 lux and 30,000 lux, and the color temperature is uniformly set to 6500K to reflect the higher color temperature and cool light on cloudy days. 10,000 lux simulates cloudy mornings or evenings, 20,000 lux is suitable for cloudy mornings or afternoons, and 30,000 lux corresponds to cloudy noon. These lighting settings under cloudy conditions are in sharp contrast to the high light intensity and lower color temperature under sunny conditions, making the lighting effects of sunny and cloudy days clearly distinguishable. Through such settings of light intensity and color temperature, the various lighting conditions encountered by the robot in sunny and cloudy environments can be effectively simulated, thereby ensuring that the selection of the optimal exposure time can adapt to various natural lighting conditions. In addition, the position of the sun is divided into five azimuths: east, southeast, south, southwest, and west, which is closely related to the changes in geographical location and time, providing a full range of lighting simulation for the environmental perception system. Combined with five soil moisture levels (0%, 10%, 20%, 30%, 40%) and eight soil container placements, such as Figure 3 As shown in Figure 1, there are eight potential directions of travel (east, west, south, north, southeast, southwest, northeast, northwest), such as Figure 4 As shown. For each combination, a preliminary exposure time interval is estimated first as the basis for subsequent adjustments. Then, the exposure time is adjusted step by step through the automatic exposure time program, reducing it by 1 microsecond (us) each time, and 20 photos are recorded. To ensure data consistency, only the texture indicators of the last 10 photos in each group of settings are averaged. The texture richness of the image is evaluated by calculating the GLCM Energy, contrast, and information entropy of these images. The weighted fusion texture evaluation index formula is as follows: V=w C (I, M)·C(T)+w E (I, M)·E(T)+w H (I, M)·H(T), Where V represents texture richness, C(T), E(T), and H(T) represent the contrast, GLCM Energy, and information entropy calculated from the soil image at exposure time T, respectively; w C , w E , w H , which are the weight coefficients corresponding to contrast, GLCM Energy, and information entropy. Under high humidity conditions, the soil surface is smoother, the change of GLCM Energy is more significant, and the weight should increase with the increase of humidity M. Under high light intensity, the information entropy H has richer image details and more obvious changes in information entropy, and the weight should increase with the increase of light intensity L. Contrast C is a basic feature, and the weight can be set as a constant or fine-tuned according to needs. The calculation formula is as follows: where k C ,k E ,k H is a constant coefficient used to adjust the baseline influence of each feature, I threshold ,M threshold is the light intensity and soil moisture threshold. Based on this, the optimal exposure time is calculated as follows: Where T opt is the calculated optimal exposure time.

[0027] Data collection plan, first mix water and soil to saturation, use soil moisture sensor to measure actual moisture, then divide soil sample into 26 parts: 25 parts of soil are placed in the test site, the specific placement plan is as follows Figure 3 As shown in the figure, the main field of view of the camera is divided into 8 equal parts, and then the soil containers are placed in the positions of one of the parts. The soil containers are composed of 25 rectangular soil containers to facilitate the change of soil moisture. Another part of the soil is placed on an electronic scale to obtain real-time soil moisture on the site. Next, set different lighting combinations (light intensity, color temperature, sun position, driving direction), soil moisture levels (0%, 10%, 20%, 30%, 40%) and the driving direction of the car (east, west, south, north, southeast, southwest, northeast, northwest), as shown in the figure. Figure 4As shown. Under each lighting condition, the optimal exposure time is preset. Then, the D435i camera is used to collect RGB and IR images of wet soil under the corresponding conditions, as well as the environmental variable configuration to construct a multimodal dataset. OpenCV is used to convert RGB images into CIEXYZ and CIELCH color spaces to enhance the representation channels of the input images. The final modal dataset is soil image data in multiple color spaces such as RGB, infrared (IR), CIEXYZ, and CIELCH, as well as sun azimuth data, light intensity, color temperature, robot driving heading, and camera exposure time. These soil image data from different color spaces and spectral ranges increase the representation dimension of soil characteristics. In addition, the dataset combines changes in light intensity and color temperature, as well as sun azimuth and robot driving heading to simulate the imaging environment of the camera under different lighting conditions.

[0028] Data labeling and data enhancement. First, the collected images in the soil trough are labeled to accurately mark the actual moisture value of the soil in the soil trough, and other areas in the image are marked as 0. Next, the robustness and generalization ability of the model are improved through data enhancement methods: Gaussian noise is added to simulate the tiny noise caused by illumination changes, salt and pepper noise is added to simulate the sudden noise points that may be generated by the sensor, and mild Gaussian blur is applied to enhance the model's ability to extract features at different focal lengths. Motion blur is used to simulate the image blur of the car at different driving speeds to improve the model's adaptability to dynamic scenes. In addition, slight color jittering is performed in the CIEXYZ and CIELCH color spaces to enhance the model's robustness to color changes, especially to improve the detection accuracy of tiny color differences caused by different soil moisture under the same illumination conditions. After completing data enhancement, the soil moisture estimation submodel is trained using the processed diversified data set, and the model performance is evaluated on the validation set and test set. Necessary optimization is performed based on the evaluation results to ensure that the model has high stability and accuracy in practical applications. Throughout the entire process, it is necessary to reasonably select noise and blur parameters and control the amplitude of color jitter to ensure that data enhancement can effectively improve model performance without causing loss of image information or introducing unnecessary deviations, thereby achieving accurate detection and robust application of soil moisture.

[0029] The network model for soil shadow removal is based on the generative adversarial network (GAN) architecture, called soil shadow generative adversarial network (SS GAN). The SS GAN structure adopts a recurrent network form, including two generators (GXY and GYX) and two discriminators (DX and DY), where the generator and the discriminator are as follows: Figure 5As shown. A mutual mapping relationship is formed between two image domains: shadow image (X domain) and non-shadow image (Y domain). The generator GXY is responsible for converting the shadow soil surface image into a non-shadow image, and the discriminator DY judges the authenticity of the generated non-shadow image. GYX performs the opposite operation and generates the corresponding shadow image from the non-shadow image. The model uses the cycle consistency loss function and the content loss function to ensure the consistency of the generated image with the original image content, and uses the identity loss function to ensure that when the non-shadow image is input, the generator does not change its content. The network architecture is an unsupervised learning method and does not require paired shadow and non-shadow images for training;

[0030] The network model design and specific implementation method are as follows: Improved conditional channel attention C-ECA adds a fully connected layer based on the traditional ECA attention mechanism. The fully connected layer takes environmental information (light intensity I, light color temperature K, driving direction O, exposure time T, solar altitude H, solar azimuth W) as input, and outputs coefficients for adjusting image channel weights. The coefficients are element-wise multiplied with the channel weights output by the ECA mechanism to obtain the final channel weights, thereby achieving dynamic adaptation to environmental conditions and enhancing the model's ability to extract soil moisture features under different lighting conditions. Convolution kernel generation network: Through a fully connected convolution kernel generation network, the illumination condition information is encoded as a feature vector input, and the parameters of the convolution kernel are automatically adjusted through a fully connected neural network to achieve dynamic generation of the convolution kernel. This method can optimize the convolution operation in a targeted manner according to real-time environmental information, improve the feature extraction effect under various illumination conditions, and enhance the robustness and generalization ability of the model. Network backbone: It is characterized by replacing a standard convolution layer in the residual block of ResNet-50 with improved conditional convolution, and C-ECA is embedded after 4 residual blocks, such as Figure 6 shown. Sub-model design: The inverse photon model integrates the SKNet (Selective Kernel Networks) module, such as Figure 7 As shown in Figure 1. SKNet dynamically selects convolution kernels of different sizes to adapt to multi-scale feature extraction, thereby capturing image details at different scales and further reducing the impact of strong reflections on key features. The side photon model integrates the HiLo Attention (High-Low Attention) module, as shown in Figure 1. Figure 8 The backlight sub-model integrates CFP (Centralized Feature Pyramid), as shown in Fig. 9As shown in the figure. CFP can effectively capture global long-distance dependencies through a centralized feature pyramid structure and enhance feature differences through the adjustment mechanism of intra-layer features. HiLo Attention enhances the feature representation of high-contrast texture areas through the interaction of high-frequency and low-frequency features, ensuring that soil moisture-related features can be accurately extracted in areas with large light changes. The cloudy sub-model integrates a dilated convolution module for cloudy conditions, such as Fig.10 As shown in the figure, images under cloudy conditions have the characteristics of low contrast, uniform illumination, and low illumination intensity. The dilated convolution effectively handles the problems of weak texture and uneven brightness by expanding the receptive field, ensuring efficient extraction and enhancement of soil features, thereby improving the accuracy of soil moisture estimation.

[0031] The model classification training process includes inputting the preprocessed multimodal data (RGB images, CIEXYZ images, CIELCH images, IR images, light intensity, color temperature, solar azimuth, solar altitude, camera exposure time, driving heading) into the soil moisture estimation network. During the training process, in order to better adapt to changes in lighting conditions, a weighting factor related to light intensity is introduced on the basis of the standard cross entropy loss function. The weighting factor is defined as: Among them, α, β, γ, δ are adjustment factors, I is the current light intensity, I_max is the maximum light intensity, I threshold is the light intensity threshold. The loss function after introducing the light weight is as follows: L C = -∑yC(I)log(p), The weighting factor C(I) ensures that the model is more sensitive to errors under special lighting conditions. On this basis, in order to make the fully connected neural network in C-ECA better generate channel weights, we can add a special regularization term to the loss function to encourage the channel weight w C-ECA The specific formula is as follows: L C-ECA =λ C-ECA Regularization(w C-ECA ), Regularization(w C-ECA )=||w C-ECA ||1 is a sparsity constraint, and L1 regularization is used to make important channel weights prominent and unimportant weights approach zero. In order to make the fully connected neural network in C-Conv generate a more suitable convolution kernel, the convolution kernel parameter K can be added to the loss function. C-Conv The regularization term of is as follows: L C-Conv=λ C-Conv Regularization(K C-Conv ), in For smoothness constraints, L2 regularization is used to prevent the convolution kernel parameters from being too large and to promote the stability of the model. Combining the above loss terms, the final loss function is as follows: L total =L C +L C-ECA +L C-Conv , When dividing the data set, the training data is selected according to the angle between the robot's driving direction and the direct direction of the sun: when the angle between the driving direction and the direct direction of the sun is between 0° and 45°, this part of the data is used to train the inverse photon model; when the angle is between 45° and 135°, the data is used to train the side photon model; when the angle is between 135° and 180°, the data is used to train the backlight sub-model; data for cloudy conditions (low light intensity and high color temperature) are used to train the cloudy sub-model. The optimizer can choose Adam or SGD, the initial learning rate is set to 0.001, and the learning rate decay strategy is combined to adapt to the training process at different stages. In order to improve the generalization ability of the model, the cross-validation method is used to divide the data set into multiple parts, and use one part as the validation set in turn, and the rest is used for training. This method can effectively reduce the risk of overfitting and improve the adaptability and robustness of the model in practical applications;

[0032] Fusion / switching algorithm, cloudy recognition module, uses ResNet network to identify whether the weather is cloudy. If it is detected as cloudy, it switches to the cloudy sub-model. The weight distribution module designs a set of fuzzy logic rules based on the angle (azimuth) between the direction of the car's travel and the direction of direct sunlight and the dynamic changes in light intensity. It dynamically selects or weightedly fuses three sub-models (backlight sub-model, sidelight sub-model, and backlight sub-model) to estimate soil moisture. The cloudy sub-model is switched by other means, and this algorithm does not include the calculation of the cloudy sub-model. In any case, the angle θ must be between the angle ranges of the two sub-models, and the weights of the two sub-models are jointly determined by the degree of angle deviation and the light intensity. The membership function of the angle is designed to be based on the degree of angle deviation: Among them, the backlight angle membership function is as follows: The membership function of the side light angle (left) is as follows: The membership function of the side light angle (right) is as follows: The backlight angle membership function is as follows: Fuzzification of light intensity I: Light intensity I is divided into three fuzzy sets, and the membership function is as follows: The membership function of the output weight α is as follows: Fuzzy rules such as Fig.11 As shown; Based on minimum-maximum fuzzy reasoning, the centroid method is used to defuzzify the final weight coefficient α model1 On this basis, weighted fusion is performed to calculate the final estimated value, and the formula is as follows: Y final =α model1 ·Y model1 +α model2 ·Y model2 , where α model1 is the side photon model weight, α model2 =1-α model1 is another weight involved in the fusion model, Y model1 and Y model2 is the output result of the corresponding model;

[0033] In the inference stage, the system deploys the trained soil moisture estimation model to a mobile robot equipped with an Intel D435i camera to achieve real-time estimation of soil moisture. First, the robot uses the Intel D435i camera to collect RGB and infrared (IR) images of the soil surface at a frame rate of 30FPS and a resolution of 640x480. At the same time, the ambient light sensor monitors the light intensity and color temperature in real time, and adjusts the camera exposure time T according to the lighting conditions to ensure image quality. In addition, the robot records the driving direction every second through the electronic compass module, and the heading error is controlled within ±2°. All data are processed on NVIDIA Jetson AGX Xavier or equivalent edge computing devices. The collected RGB images are converted into CIE XYZ and CIE LCH images through color space conversion, which together with the standardized infrared images constitute a multimodal input data matrix. In order to eliminate the interference of ambient light and shadows, the system applies the SS-GAN (Shadow-Removal Generative Adversarial Network) algorithm to remove shadows in the image through the generator network, and the discriminator ensures the image quality. The hyperparameters of SS-GAN are set to a learning rate of 0.0002, and the Adam optimizer is used. The training time is 100 epochs. Subsequently, the system dynamically adjusts the sub-model weights according to the real-time ambient lighting conditions using a fusion switching algorithm. Finally, the grayscale value is used to map different levels of soil moisture, and the soil moisture value S from 0% to 40% is mapped to the grayscale value G from 0 to 255. The mapping relationship is a linear mapping, and the calculation formula is as follows: Among them, when the soil moisture S is 0%, the gray value G is 0 (completely black); when the soil moisture S is 10%, the gray value G is 63.75 (dark gray); when the soil moisture S is 20%, the gray value G is 127.5 (medium gray); when the soil moisture S is 30%, the gray value G is 191.25 (light gray); when the soil moisture S is 40%, the gray value G is 255 (completely white). Through this linear mapping method, different moisture soil levels can be intuitively displayed.

Claims

1. A method for estimating soil moisture based on illumination conditions and adaptive to multimodal data fusion, characterized in that: The following steps are involved: Step 1: Design a weighted fusion soil image texture evaluation index that comprehensively considers the energy of the gray-level co-occurrence matrix (GLCM Energy), contrast, and information entropy, and dynamically adjusts the weight coefficient according to the light intensity and soil moisture. Based on this evaluation index, a calculation formula is formulated to determine the optimal exposure time of the camera under different lighting conditions. Step 2: Propose a method to construct a multimodal soil moisture dataset. Under different lighting conditions, adjust the camera parameters based on the optimal exposure time determined in step 1. Combine the robot's driving path with the actual lighting environment to design a representative combination of lighting conditions and driving directions. Place soil containers in batches to ensure that the entire camera field of view is covered. Simultaneously collect soil images and related environmental data at different moisture levels, integrate multiple data modalities, and construct a multimodal dataset. Subsequently, perform data enhancement and annotation on the dataset. Step 3: Combine environmental information to improve the ECA attention mechanism and standard convolutional layer, design conditional channel attention (Condition ECA, C-ECA) and conditional convolution (Condition Conv, C-Conv), and embed ResNet-50 as the network backbone. According to the characteristics of soil images under different lighting conditions, four sub-models (reverse photon model, backlight sub-model, side photon model and cloudy sub-model) are designed to achieve multimodal data fusion through sub-models to estimate soil moisture under various lighting conditions. Step 4: Design an improved cross entropy loss function. The cross entropy loss function is optimized by introducing illumination weights, which are dynamically adjusted according to the current illumination intensity. In addition, in order to enable the fully connected neural network in the conditional channel attention mechanism (C-ECA) and conditional convolution (C-Conv) to generate better channel weights and convolution kernels, a regularization term is introduced in the loss function. Step 5: Design a sub-model fusion / switching algorithm, which includes a cloudy recognition module and a sub-model weight assignment module based on fuzzy rules. First, a cloudy recognition module implemented by a convolutional neural network (CNN) is introduced to determine whether it is cloudy and whether the cloudy sub-model needs to be called. In the weight assignment module, the angle θ between the driving direction and the direct direction of the sun, the illumination intensity I, and the weight α of the side photon sub-model are used as fuzzy linguistic variables, and the membership function and fuzzy rules are designed considering the relationship between the driving direction and the adjacent sub-models. Through fuzzy reasoning and defuzzification, the weight coefficient α of the side photon model is obtained, and then the weight 1-α of the backlight sub-model or the inverse photon model fused with the side photon model is determined. Step 6: In practical applications, the trained network model is deployed on a wheeled robot to estimate soil moisture for real-time soil images. Based on the estimation results, a soil moisture estimation map is generated through grayscale value mapping.

2. The weighted fusion texture evaluation index according to step 1 of claim 1, characterized in that: The formula is as follows: V=w C (I,M)·C(T)+w E (I,M)·E(T)+w H (I,M)·H(T), Where V represents texture richness, C(T), E(T), and H(T) represent the contrast, GLCM Energy, and information entropy calculated from the soil image at exposure time T, respectively; w C , w E , w H , is the weight coefficient corresponding to contrast, GLCM Energy, and information entropy, and its calculation formula is as follows: where k C ,k E ,k H is a constant coefficient used to adjust the baseline influence of each feature, I threshold ,M threshold is the light intensity and soil moisture threshold. Based on this, the optimal exposure time is calculated as follows: Where T opt is the calculated optimal exposure time.

3. The multimodal dataset according to step 2 of claim 1, characterized in that Contains multiple modalities and their corresponding data. Specifically, the dataset includes: image modality, i.e. soil image data acquired in multiple color spaces and spectral ranges such as RGB, infrared (IR), CIEXYZ, CIELCH, etc., which enriches the representation dimension of soil characteristics; illumination modality, including data on light intensity and color temperature, recording changes in ambient lighting conditions; position information modality, including sun position data and robot driving heading, simulating the imaging environment of the camera at different times and spaces; and camera parameter modality, including camera exposure time, recording the camera setting parameters during the imaging process.

4. The C-ECA according to step 3 of claim 1, characterized in that: On the basis of the traditional ECA attention mechanism, a fully connected layer is added, which takes environmental information (light intensity I, light color temperature K, driving direction O, exposure time T, solar altitude angle H, solar azimuth W) as input to generate coefficients for adjusting image channel weights. This coefficient is multiplied element by element with the channel weight output by the ECA mechanism to obtain the final channel weight.

5. The conditional convolution according to step 3 of claim 1, characterized in that: Through a fully connected convolution kernel generation network, the lighting condition information (light intensity I, light color temperature K, driving direction O, exposure time T, sun altitude angle H, sun azimuth W) is used as input to dynamically adjust the parameters of the convolution kernel. Specifically, the lighting condition information is first processed by a multi-layer perceptron to generate the corresponding convolution kernel weights, and the convolution kernel parameters are automatically adjusted according to the current environmental conditions.

6. The network backbone according to step 3 of claim 1, characterized in that: The improved conditional convolution replaces the 3×3 standard convolution layer in the residual block in ResNet-50, and C-ECA is embedded after each residual block.

7. The targeted network module according to step 3 of claim 1, characterized in that: According to the characteristics of soil images under different lighting conditions, targeted network modules are embedded in each sub-model to improve the efficiency and accuracy of feature extraction: Inverse photon model: The selective convolution kernel network module (SKNet) is embedded, which enhances the ability to capture multi-scale features under backlighting conditions by adaptively selecting convolution kernels of different scales. Backlight sub-model: Embedded in the centralized feature pyramid module (CFP), it enhances the ability to extract detail features under backlight conditions and improves sensitivity to low-contrast areas through the fusion and transmission of multi-level features. Side-light model: Embeds a high-low frequency attention module (HiLo Attention) to focus on both high-frequency and low-frequency information, enhances feature expression and discrimination capabilities under side-light conditions, and effectively addresses image feature differences caused by uneven lighting. Cloudy model: Embedded dilated convolution module, by expanding the receptive field, obtains more contextual information without significantly increasing parameters, enhances the global feature extraction effect under cloudy and low-light conditions, and improves the overall perception of the image.

8. The improved cross entropy loss function according to step 4 of claim 1, characterized in that: A weight factor related to light intensity is introduced based on the standard cross entropy loss function. The weight factor is defined as: Among them, α, β, γ, δ are adjustment factors, I is the current light intensity, I_max is the maximum light intensity, I threshold is the light intensity threshold. The loss function after introducing the light weight is as follows: L C =-∑yC(I)log(p), The weight factor C(I) ensures that the model is more sensitive to errors under special lighting conditions. On this basis, in order to make the fully connected neural network in C-ECA better generate channel weights, a special regularization term is added to the loss function to encourage the channel weight w C-ECA The specific formula is as follows: L C-ECA =λ C-ECA ·Regularization(w C-ECA ), Regularization(w C-ECA )=||w C-ECA ||1 is a sparsity constraint, and L1 regularization is used to make important channel weights prominent and unimportant weights approach zero. In order to make the fully connected neural network in C-Conv generate a more suitable convolution kernel, the convolution kernel parameter K is added to the loss function C-Conv The regularization term is as follows: L C-Conv =λ C-Conv ·Regularization(K C-Conv ), in For smoothness constraints, L2 regularization is used to prevent the convolution kernel parameters from being too large and to promote the stability of the model. Combining the above loss terms, the final loss function is as follows: L total =L C +L C-ECA +L C-Conv .

9. The fuzzy linguistic variable according to step 5 of claim 1, characterized in that: The angle θ represents the angle between the direction of the car and the direction of direct sunlight, ranging from [0,180], and is divided into four levels: backlight angle, sidelight angle (left) (sidelight-left), sidelight angle (right) (sidelight-right), and backlight angle (frontlight); the range of light intensity I is [0,100000 lux], and is divided into three levels: low light (Low), medium light (Medium), and high light (High). The output is the weight α of the side photon model, ranging from [0,1], and is divided into four levels: low (Low), medium (Medium), high (High), and very high (Very High).

Images