Water and soil conservation index determination method and device based on multi-modal perception

Abstract

The invention provides a water and soil conservation index determination method and device based on multi-modal perception, and the method comprises the steps: obtaining target perception data which comprises target image data, target laser point cloud data and target multi-spectral data; inputting the target image data into an image segmentation model for image segmentation to obtain vegetation coverage; obtaining a normalized vegetation index according to the target multispectral data; according to the target laser point cloud data, a terrain gradient is extracted; obtaining a soil erosion modulus according to the vegetation coverage and the terrain gradient; obtaining a water and soil conservation index report according to the vegetation coverage, the normalized vegetation index and the soil erosion modulus; wherein the image segmentation model is obtained by training according to the target image data and the ground calibration data. According to the invention, the efficiency, accuracy and standardization degree of water and soil conservation index determination can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for determining soil and water conservation indicators based on multimodal perception. Background Technology Currently, after the completion of construction projects with water and soil conservation plans, acceptance inspections are required in accordance with the approved plans. Inspection personnel must verify the specific implementation of engineering and vegetation measures on-site, according to the requirements of the approved water and soil conservation plans. Traditional inspection methods generally have the following limitations: First, the efficiency of obtaining inspection indicators (such as forest and grassland coverage, soil erosion modulus, soil erosion control degree, spoil protection rate, and drainage system integrity) is low. For example, manual surveys of 10km... 2 First, the area requires 3-5 days to fully cover, and due to terrain limitations (such as steep slopes and gullies), it is difficult to achieve complete coverage. The acceptance area is often complex and vast, making manual surveys time-consuming, labor-intensive, and lengthy. Second, the accuracy of indicator measurements is poor. Traditional vegetation coverage relies on visual estimation (error > 15%), and soil erosion relies on manually constructed runoff plots. However, the number of plots is limited, resulting in insufficient representativeness. Judgments on indicators such as vegetation coverage and structural integrity depend on the experience of the acceptance personnel, lacking unified and objective quantitative standards, which easily leads to disputes. Third, the acceptance data is fragmented, making it impossible to form a quantitative report that links "space and time," resulting in highly subjective acceptance conclusions and poor traceability. Fourth, some acceptance areas (such as high slopes, steep cliffs, and deep gullies) pose safety hazards, making manual surveys risky and potentially leading to personal safety accidents, thus making it impossible to grasp the actual situation of the area. Fifth, manual records are mostly text and simple photos, making it difficult to form quantifiable, multi-dimensional digital archives, which is not conducive to later traceability and big data analysis.

[0002] Currently, drone technology is mostly used solely for image capture, and no dedicated data processing algorithms have been developed for soil and water conservation acceptance indicators. Existing technology has the following drawbacks: (1) Limitations of a single data source: Most existing solutions only use UAV orthophotos (RGB cameras) to analyze vegetation cover through two-dimensional images. This method cannot accurately calculate earthwork volume (such as the siltation volume of dam retainers and the volume of cut and fill slopes), it is difficult to assess slope stability, and it cannot penetrate the vegetation canopy to assess the true surface condition.

[0003] (2) Low level of automation: Many solutions still require a lot of manual intervention, such as manually selecting feature points, manually annotating images, and semi-automatic calculation, failing to achieve full automation from data collection to report generation.

[0004] (3) Incomplete indicator evaluation: Existing methods mostly focus on single indicators (such as vegetation coverage) and lack the systematic and comprehensive key indicators of the "three major measures" of soil and water conservation (engineering measures, vegetation measures, and temporary measures) (such as slope gradient, engineering structure integrity, drainage system smoothness, soil erosion intensity, etc.) for collaborative automatic interpretation and evaluation.

[0005] (4) Accuracy and reliability issues: Visible light images alone are easily affected by light, shadow and seasonal changes, and have limited ability to identify vegetation species, detect minor soil erosion and micro-topographic changes. The accuracy and reliability of the assessment results need to be improved. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide a method and apparatus for determining soil and water conservation indicators based on multimodal sensing. This can improve the efficiency, accuracy, and standardization of soil and water conservation indicator determination.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A method for determining soil and water conservation indicators based on multimodal sensing includes: Acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data; The target image data is input into an image segmentation model for image segmentation to obtain the vegetation coverage. Based on the target multispectral data, the normalized vegetation index is obtained; Based on the target laser point cloud data, the terrain slope is extracted; Based on the vegetation cover and terrain slope, the soil erosion modulus is obtained; Based on the vegetation cover, normalized vegetation index and soil erosion modulus, a report on soil and water conservation indicators is obtained; The image segmentation model is trained based on the target image data and ground calibration data.

[0008] Optionally, acquire target perception data, including: Raw image data is obtained by using a color camera, and the raw image data is preprocessed to obtain target image data. Raw laser point cloud data is obtained through a laser sensor, and the raw laser point cloud data is then denoised to obtain target laser point cloud data. Raw multispectral data is obtained by using a multispectral camera, and radiometric calibration is performed on the raw multispectral data to obtain target multispectral data.

[0009] Optionally, the original image data is preprocessed to obtain target image data, including: Based on the dark channel map of the original image data, atmospheric light value and transmittance are obtained; Based on the atmospheric light value and transmittance, target image data is obtained.

[0010] Optionally, the training process of the image segmentation model includes: The feature image is input into the encoder of the preset convolutional model for feature extraction and multi-scale fusion to obtain the first output; The first output is input into the decoder of the preset convolutional model and focused to obtain the second output; Based on the second output and the loss function, the hyperparameters of the preset convolutional model are adjusted to obtain the image segmentation model.

[0011] Optionally, based on the target multispectral data, a normalized vegetation index is obtained, including: The normalized vegetation index is obtained based on the near-infrared and red reflectance of the target multispectral data.

[0012] Optionally, the terrain slope is extracted based on the target laser point cloud data, including: The terrain slope is extracted based on the elevation value and planar coordinates of the target laser point cloud data.

[0013] Optionally, the soil erosion modulus is obtained based on the vegetation cover and topographic slope, including: The terrain factor is calculated based on the terrain slope; Vegetation management factors are calculated based on vegetation cover. Soil erosion modulus is obtained based on the topographic factors and vegetation management factors.

[0014] Optionally, based on the vegetation cover, normalized difference vegetation index, and soil erosion modulus, a soil and water conservation index report is obtained, including: The vegetation coverage, normalized vegetation index, and soil erosion modulus are compared with their respective preset thresholds to calculate the compliance rate of soil and water conservation indicators, and a soil and water conservation indicator report is obtained.

[0015] Embodiments of the present invention also provide a device for determining soil and water conservation indicators based on multimodal sensing, comprising: The acquisition module is used to acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data. The processing module is used to input the target image data into an image segmentation model for image segmentation to obtain vegetation coverage; obtain the normalized vegetation index based on the target multispectral data; extract the terrain slope based on the target laser point cloud data; obtain the soil erosion modulus based on the vegetation coverage and terrain slope; and obtain a soil and water conservation index report based on the vegetation coverage, normalized vegetation index, and soil erosion modulus; wherein the image segmentation model is trained based on the target image data and ground calibration data.

[0016] Embodiments of the present invention also provide a computing device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining soil and water conservation indicators based on multimodal perception as described in the present invention.

[0017] The above-described technical solution of the present invention has at least the following technical effects: The above-described method for determining soil and water conservation indicators based on multimodal perception of the present invention acquires target perception data, including target image data, target laser point cloud data, and target multispectral data; inputs the target image data into an image segmentation model for image segmentation to obtain vegetation cover; obtains the normalized vegetation index (NVI) based on the target multispectral data; extracts the terrain slope based on the target laser point cloud data; obtains the soil erosion modulus based on the vegetation cover and terrain slope; and obtains a soil and water conservation indicator report based on the vegetation cover, NVI, and soil erosion modulus. The image segmentation model is trained based on the target image data and ground calibration data. This method can improve the efficiency, accuracy, and standardization of soil and water conservation indicator determination. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the method for determining soil and water conservation indicators based on multimodal sensing according to the present invention; Figure 2 This is a schematic diagram of the water and soil conservation index determination device based on multimodal perception of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0020] like Figure 1As shown, an embodiment of the present invention proposes a method for determining soil and water conservation indicators based on multimodal sensing, comprising: Step S1: Acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data; Step S2: Input the target image data into the image segmentation model to perform image segmentation and obtain the vegetation coverage. Step S3: Obtain the normalized vegetation index based on the target multispectral data; Step S4: Extract the terrain slope based on the target laser point cloud data; Step S5: Obtain the soil erosion modulus based on the vegetation coverage and terrain slope; Step S6: Based on the vegetation coverage, normalized vegetation index and soil erosion modulus, obtain a soil and water conservation index report; The image segmentation model is trained based on the target image data and ground calibration data.

[0021] In this embodiment, as Figure 1 As shown, in the method for determining soil and water conservation indicators based on multimodal perception, firstly, an industrial-grade drone equipped with an RGB camera, a LiDAR sensor, and a multispectral camera is used to acquire target perception data of the target area. When using the RGB camera and multispectral camera for data acquisition, the drone's flight altitude is set to 100-150m to ensure an image resolution ≥5cm / pixel; when using the LiDAR sensor for data acquisition, the drone's flight altitude is set to 150-200m to ensure a point cloud density ≥50 points / m²; the target perception data includes target image data, target... The system first obtains laser point cloud data and target multispectral data. Then, using a pre-trained image segmentation model, it segments the target image data into vegetation areas to obtain vegetation coverage. Next, based on the acquired target multispectral data, it calculates the normalized vegetation index. Then, based on the acquired target laser point cloud data, it extracts the terrain slope. Next, based on the vegetation coverage and terrain slope, combined with the influencing factors of the target area, it obtains the soil erosion modulus. Finally, based on the vegetation coverage, normalized vegetation index, and soil erosion modulus, it automatically generates a soil and water conservation index report.

[0022] This invention patent is primarily applied in the field of soil and water conservation acceptance technology for construction projects. Specifically, it employs an automated system and method integrating multimodal data acquisition from unmanned aerial vehicles (UAVs), key indicator inversion, and quantitative evaluation of acceptance results. Specifically, it relates to a system and method that utilizes UAVs equipped with multimodal sensors for data acquisition, combined with computer vision, photogrammetry, and deep learning technologies, to achieve automated, high-precision quantitative evaluation of the effectiveness of soil and water conservation facilities and vegetation restoration status. This is applicable to soil and water conservation monitoring and acceptance work in projects such as new energy projects, power transmission and transformation projects, and thermal power projects.

[0023] In an optional embodiment of the present invention, step S1, acquiring target perception data, includes: Step S11: Obtain raw image data through a color camera, and preprocess the raw image data to obtain target image data; Step S12: Obtain raw laser point cloud data through a laser sensor, and perform noise reduction processing on the raw laser point cloud data to obtain target laser point cloud data; Step S13: Obtain raw multispectral data through a multispectral camera, and perform radiometric calibration on the raw multispectral data to obtain target multispectral data.

[0024] In this embodiment, raw image data is obtained through an RGB color camera, and target image data is obtained after preprocessing the raw image data; raw laser point cloud data is obtained through a LiDAR laser sensor, and statistical filtering is used to denoise the raw laser point cloud data, preferably with a neighboring point count of 15 and a distance threshold of 1.5 times the standard deviation, to obtain target laser point cloud data; raw multispectral data is obtained through a multispectral camera, and radiometric calibration is performed on the raw multispectral data to obtain target multispectral data; the radiometric calibration formula is:

[0025] in, DN represents the surface reflectance, and DN represents the image grayscale value. This represents the grayscale value of the dark current. The grayscale value is the standard plate value. The reflectance is that of the standard plate.

[0026] Data preprocessing can eliminate error sources such as data noise and atmospheric interference, so that the preprocessed data can meet the accuracy requirements of index inversion.

[0027] In an optional embodiment of the present invention, step S21, preprocessing the original image data to obtain target image data, includes: Step S211: Obtain atmospheric light value and transmittance based on the dark channel map of the original image data; Step S212: Obtain target image data based on the atmospheric light value and transmittance.

[0028] In this embodiment, when preprocessing the original image data, the original image data is first converted from the RGB color space to the HSV or Lab color space to separate each channel. Then, at each pixel, the minimum value among the three channels is calculated to obtain the dark channel image. The formula for calculating the dark channel image is:

[0029] in, This represents the dark channel plot. This represents the intensity value of pixel y in channel c. This represents a local window centered at pixel x.

[0030] From the dark channel diagram Select the top 0.1% of the brightest pixels, find the corresponding pixels in the original foggy image I(x), and take the maximum value of these pixels as the atmospheric light value A. The formula for calculating transmittance is:

[0031] Where t(x) represents transmittance. It is an adjustable parameter, generally with a value between 0.9 and 0.99, used to control the defogging intensity.

[0032] Target image data is obtained based on atmospheric light value and transmittance; the formula for calculating target image data is:

[0033] in, Represents target image data; It is a very small positive number to prevent the denominator from being 0.

[0034] In an optional embodiment of the present invention, the training process of the image segmentation model includes: Step S71: Input the feature image into the encoder of the preset convolutional model for feature extraction and multi-scale fusion to obtain the first output; Step S72: Focus attention on the decoder of the preset convolutional model input to the first output to obtain the second output; Step S73: Adjust the hyperparameters of the preset convolutional model according to the second output and loss function to obtain the image segmentation model.

[0035] In this embodiment, the preset convolutional model achieves feature fusion through a symmetrical structure of encoder downsampling and decoder upsampling; firstly, feature extraction is performed using the encoder, and residual connections are introduced into each convolutional layer, specifically expressed as:

[0036] in, Represents the characteristic map of the cavity. denoted by , where W and b are convolution parameters, and p is the input feature map; This feature extraction method can avoid gradient vanishing and enhance the ability to extract deep features.

[0037] Then, dilated convolution is used for multi-scale fusion, and the fusion formula is as follows:

[0038] Where q represents the fused feature map. Represents the characteristic map of the cavity. Represents the convolution kernel. Indicates the convolution operation; Multiscale fusion can expand the receptive field and capture multiscale information without increasing parameters.

[0039] Next, the decoder is used to focus attention, resulting in a spatial attention map, which is the second output. The formula for calculating the spatial attention map is:

[0040] in, The output spatial attention map is used to emphasize important spatial regions in the fused feature map q; q represents the fused feature map; AP(q) represents the average pooling operation on the fused feature map q, which reduces the spatial dimension of the feature map by calculating the average value of the elements within the pooling window, while retaining the average feature information; MP(q) represents the max pooling operation on the fused feature map, which selects the maximum value within the pooling window, thus compressing the spatial dimension while highlighting the most significant features.

[0041] Next, ground calibration data collection was conducted. Three to five calibration points were set up according to the representative topography of the acceptance area (covering typical landforms such as slopes, excavated faces, and vegetated areas). A three-dimensional observation area for "soil-vegetation-runoff" was established at each calibration point. In the soil sampling area, five sampling points were arranged in a quincunx pattern within a 10m×10m area to collect 0-20cm soil samples and measure soil bulk density. For vegetation quadrats, three 1m×1m quadrats were set up, and images of the quadrats were taken with a digital camera, consistent with the viewpoint of the UAV's RGB camera. Vegetation height and cover were measured manually. A tipping bucket runoff meter was installed at the downhill section of each calibration point to record the total runoff volume and runoff intensity during rainfall periods. The GPS coordinates of the calibration points (accuracy ±0.05m) were collected to establish the spatial correspondence between the ground calibration data and the UAV data.

[0042] Next, based on the loss function, the second output is compared with the ground calibration data, and the hyperparameters of the preset convolutional model are adjusted to obtain the image segmentation model; the specific loss function is:

[0043] Here, LF represents the loss function, which measures the difference between the model's prediction and the actual result. The model parameters are optimized by minimizing this function. It is a hyperparameter, usually ranging from [0,1], used to balance the weights of the cross-entropy loss term and the Dice coefficient loss term, controlling their relative importance in the total loss; Cross-entropy is used to measure the difference between two probability distributions, where y represents the probability distribution of the true label. The smaller the cross-entropy, the closer the predicted result is to the actual result. The Dice coefficient is a metric used to calculate the similarity between two samples. Its value ranges from [0,1]. The closer the value is to 1, the higher the overlap between the predicted result and the actual result, i.e., the higher the similarity. This is then transformed into a loss term, which, together with the cross-entropy loss, constitutes the total loss function, balancing the class imbalance problem and strengthening the learning of boundary features.

[0044] In an optional embodiment of the present invention, step S3, obtaining the normalized vegetation index based on the target multispectral data, includes: Step S31: Obtain the normalized vegetation index based on the near-infrared band reflectance and red band reflectance of the target multispectral data.

[0045] In this embodiment, the normalized vegetation index (NDI) is obtained by combining the target multispectral data; the formula for calculating the NDI is:

[0046] NVDI represents the Normalized Difference Vegetation Index. Indicates near-infrared reflectivity. Indicates the reflectivity in the red light band; The formula for calculating vegetation cover is:

[0047] VFC represents vegetation cover. This represents the pixel area of ​​the vegetation region. Indicates the total pixel area of ​​the region; By combining normalized vegetation index (VFC) and vegetation coverage, the problem of low accuracy of existing methods can be solved. By combining multispectral data, a multidimensional assessment of vegetation indicators can be achieved, providing accurate data for the acceptance of vegetation measures. For example, when VFC ≥ 85% and NVDI ≥ 0.7, the vegetation measures are judged to be excellent.

[0048] In an optional embodiment of the present invention, step S4, extracting the terrain slope based on the target laser point cloud data, includes: Step S41: Extract the terrain slope based on the elevation value and plane coordinates of the target laser point cloud data.

[0049] In this embodiment, the terrain slope is extracted based on the elevation value and planar coordinates of the target laser point cloud data; the formula for calculating the terrain slope is:

[0050] Where SL represents the slope, z is the elevation value, m is the horizontal coordinate of the plane, and n is the vertical coordinate of the plane.

[0051] In an optional embodiment of the present invention, step S5, obtaining the soil erosion modulus based on the vegetation cover and terrain slope, includes: Step S51: Calculate the terrain factor based on the terrain slope; Step S52: Calculate the vegetation management factor based on the vegetation coverage. Step S53: Obtain the soil erosion modulus based on the topographic factors and vegetation management factors.

[0052] In this embodiment, the terrain factor is calculated based on the terrain slope. The formula for calculating the terrain factor is as follows:

[0053] Where LS represents the topographic factor, L represents the slope length, and SL represents the slope.

[0054] Based on vegetation cover, the vegetation management factor is calculated. The formula for calculating the vegetation management factor is as follows: C = 0.6508 - 0.3436log(VFC) Where C represents vegetation management factor and VFC represents vegetation coverage.

[0055] Then, combining the existing rainfall erosivity factors, soil erodibility factors, and soil and water conservation measures factors in the target area, the soil erosion modulus is obtained; the formula for calculating the soil erosion modulus is: M = R × K × LS × C × D Where M represents the soil erosion modulus, R represents the rainfall erosivity factor, K represents the soil erodibility factor, LS represents the topographic factor, C represents the vegetation management factor, and D represents the soil and water conservation measures factor.

[0056] In an optional embodiment of the present invention, step S6, based on the vegetation cover, normalized difference vegetation index, and soil erosion modulus, generates a soil and water conservation index report, including: Step S61: Compare the vegetation coverage, normalized vegetation index and soil erosion modulus with the corresponding preset thresholds to calculate the compliance rate of soil and water conservation indicators and obtain a soil and water conservation indicator report.

[0057] In this embodiment, the vegetation coverage, normalized vegetation index, and soil erosion modulus are compared with their respective preset thresholds to mark non-compliant indicators and calculate the compliance rate. The compliance rate statistics table for acceptance indicators is output in Excel format, including indicator name, calculated value, threshold, compliance status, and compliance rate. In the as-built acceptance map, non-compliant areas are marked in red in the GIS platform, and the location of engineering entities and geomorphic unit types are overlaid to generate a vector map. Finally, a soil and water conservation indicator report is automatically generated, including data sources (UAV model, sensor parameters, ground calibration point location), inversion process (model parameters, calculation steps), conclusions (compliant indicators, non-compliant areas and reasons), and suggestions (such as replanting vegetation in non-compliant areas and repairing damaged retaining dams). This achieves automated and standardized output of acceptance evaluation results, reducing the time for manual report compilation from 3 days to 2 hours. The visualized map facilitates intuitive location of non-compliant areas, providing clear guidance for subsequent rectification.

[0058] The advantages of the present invention mainly include: 1. Efficiency improvement: 10km 2 The time for data collection and analysis in the acceptance area has been shortened from 3-5 days to 4-6 hours, improving efficiency by more than 80%. This reduces months of manual work to hours, and the accuracy of indicator calculations reaches the centimeter level, far exceeding manual estimation.

[0059] 2. Improved accuracy: The measurement errors of vegetation coverage and soil erosion modulus have been reduced to within 8% and 10% respectively, meeting the accuracy requirements of the acceptance specifications; 3. Standardization: Construct an end-to-end automated pipeline from intelligent planning, data collection, cloud processing to report generation, realize the digitalization and intelligentization of acceptance work, output reports directly match industry acceptance procedures, reduce subjective human judgment, and have strong traceability.

[0060] 4. By integrating multiple sensors (optical, laser, multispectral), high-precision, multi-dimensional geospatial data is acquired, overcoming the limitations of single data sources. For the first time, integrated automatic assessment of multi-dimensional indicators such as engineering, vegetation, and landforms is achieved.

[0061] 5. By utilizing artificial intelligence and 3D modeling technology, multiple key acceptance indicators for soil and water conservation are automatically interpreted and extracted, greatly reducing manual intervention and improving the efficiency and objectivity of the assessment.

[0062] 6. It avoids personnel entering dangerous areas, and the provided realistic 3D model makes the acceptance results clear at a glance, providing strong decision support capabilities.

[0063] like Figure 2 As shown, embodiments of the present invention also provide a soil and water conservation index determination device 20 based on multimodal sensing, comprising: The acquisition module 21 is used to acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data. The processing module 22 is used to input the target image data into an image segmentation model for image segmentation to obtain vegetation coverage; obtain the normalized vegetation index based on the target multispectral data; extract the terrain slope based on the target laser point cloud data; obtain the soil erosion modulus based on the vegetation coverage and terrain slope; and obtain a soil and water conservation index report based on the vegetation coverage, normalized vegetation index, and soil erosion modulus; wherein the image segmentation model is trained based on the target image data and ground calibration data.

[0064] Optionally, acquire target perception data, including: Raw image data is obtained by using a color camera, and the raw image data is preprocessed to obtain target image data. Raw laser point cloud data is obtained through a laser sensor, and the raw laser point cloud data is then denoised to obtain target laser point cloud data. Raw multispectral data is obtained by using a multispectral camera, and radiometric calibration is performed on the raw multispectral data to obtain target multispectral data.

[0065] Optionally, the original image data is preprocessed to obtain target image data, including: Based on the dark channel map of the original image data, atmospheric light value and transmittance are obtained; Based on the atmospheric light value and transmittance, target image data is obtained.

[0066] Optionally, the training process of the image segmentation model includes: The feature image is input into the encoder of the preset convolutional model for feature extraction and multi-scale fusion to obtain the first output; The first output is input into the decoder of the preset convolutional model and focused to obtain the second output; Based on the second output and the loss function, the hyperparameters of the preset convolutional model are adjusted to obtain the image segmentation model.

[0067] Optionally, based on the target multispectral data, a normalized vegetation index is obtained, including: The normalized vegetation index is obtained based on the near-infrared and red reflectance of the target multispectral data.

[0068] Optionally, the terrain slope is extracted based on the target laser point cloud data, including: The terrain slope is extracted based on the elevation value and planar coordinates of the target laser point cloud data.

[0069] Optionally, the soil erosion modulus is obtained based on the vegetation cover and topographic slope, including: The terrain factor is calculated based on the terrain slope; Vegetation management factors are calculated based on vegetation cover. Soil erosion modulus is obtained based on the topographic factors and vegetation management factors.

[0070] Optionally, based on the vegetation cover, normalized difference vegetation index, and soil erosion modulus, a soil and water conservation index report is obtained, including: The vegetation coverage, normalized vegetation index, and soil erosion modulus are compared with their respective preset thresholds to calculate the compliance rate of soil and water conservation indicators, and a soil and water conservation indicator report is obtained.

[0071] It should be noted that all implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0072] Embodiments of the present invention also provide a computing device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining soil and water conservation indicators based on multimodal perception described in the present invention. All implementations in the above method embodiments are applicable to the embodiments of this computing device and can achieve the same technical effects.

[0073] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0074] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0075] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0076] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0078] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0079] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve by using their basic programming skills after reading the description of the present invention.

[0080] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code for implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps for performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0081] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for determining soil and water conservation indicators based on multimodal sensing, characterized in that, include: Acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data; The target image data is input into an image segmentation model for image segmentation to obtain the vegetation coverage. Based on the target multispectral data, the normalized vegetation index is obtained; Based on the target laser point cloud data, the terrain slope is extracted; Based on the vegetation cover and terrain slope, the soil erosion modulus is obtained; Based on the vegetation cover, normalized vegetation index and soil erosion modulus, a report on soil and water conservation indicators is obtained; The image segmentation model is trained based on the target image data and ground calibration data.

2. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, Acquire target perception data, including: Raw image data is obtained by using a color camera, and the raw image data is preprocessed to obtain target image data. Raw laser point cloud data is obtained through a laser sensor, and the raw laser point cloud data is then denoised to obtain target laser point cloud data. Raw multispectral data is obtained by using a multispectral camera, and radiometric calibration is performed on the raw multispectral data to obtain target multispectral data.

3. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 2, characterized in that, The original image data is preprocessed to obtain target image data, including: Based on the dark channel map of the original image data, atmospheric light value and transmittance are obtained; Based on the atmospheric light value and transmittance, target image data is obtained.

4. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, The training process of the image segmentation model includes: The feature image is input into the encoder of the preset convolutional model for feature extraction and multi-scale fusion to obtain the first output; The first output is input into the decoder of the preset convolutional model and focused to obtain the second output; Based on the second output and the loss function, the hyperparameters of the preset convolutional model are adjusted to obtain the image segmentation model.

5. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, Based on the target multispectral data, the normalized vegetation index is obtained, including: The normalized vegetation index is obtained based on the near-infrared and red reflectance of the target multispectral data.

6. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, Based on the target laser point cloud data, the terrain slope is extracted, including: The terrain slope is extracted based on the elevation value and planar coordinates of the target laser point cloud data.

7. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, Based on the vegetation cover and terrain slope, the soil erosion modulus is obtained, including: The terrain factor is calculated based on the terrain slope; Vegetation management factors are calculated based on vegetation cover. Soil erosion modulus is obtained based on the topographic factors and vegetation management factors.

8. The method for determining soil and water conservation indicators based on multimodal sensing according to claim 1, characterized in that, Based on the aforementioned vegetation cover, normalized difference vegetation index, and soil erosion modulus, a report of soil and water conservation indicators is obtained, including: The vegetation coverage, normalized vegetation index, and soil erosion modulus are compared with their respective preset thresholds to calculate the compliance rate of soil and water conservation indicators, and a soil and water conservation indicator report is obtained.

9. A device for determining soil and water conservation indicators based on multimodal sensing, characterized in that, include: The acquisition module is used to acquire target perception data, which includes target image data, target laser point cloud data, and target multispectral data. The processing module is used to input the target image data into an image segmentation model for image segmentation to obtain vegetation coverage; obtain the normalized vegetation index based on the target multispectral data; extract the terrain slope based on the target laser point cloud data; obtain the soil erosion modulus based on the vegetation coverage and terrain slope; and obtain a soil and water conservation index report based on the vegetation coverage, normalized vegetation index, and soil erosion modulus; wherein the image segmentation model is trained based on the target image data and ground calibration data.

10. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 8.

Images