Lidar-based crop density detection method, system, and storage medium

Abstract

The disclosure provides a laser radar-based crop density detection method, system and storage medium, relating to the technical field of intelligent agricultural equipment and artificial intelligence, wherein the method comprises: acquiring three-dimensional point cloud data of a crop area currently collected by a laser radar; determining a predicted area of a harvester in the three-dimensional point cloud data, and dividing the predicted area into multiple sub-areas along the advancing direction of the harvester; extracting height distribution features of the three-dimensional point cloud data in each sub-area, and constructing a multi-dimensional feature vector of each sub-area; adaptively fusing different features of the multi-dimensional feature vector of each sub-area based on an attention mechanism to generate a region code of each sub-area; acquiring a sample-level global feature of the predicted area based on the region code of each sub-area and a corresponding position code; and synchronously predicting a discrete density level and a continuous density estimation value of the crop based on the sample-level global feature, and outputting the crop density detection result.

Description

Technical Field

[0001] This disclosure belongs to the field of interdisciplinary technology of agricultural intelligent equipment and artificial intelligence, specifically involving a crop density detection method, system and storage medium based on lidar. Background Technology

[0002] In the mechanized harvesting of crops such as wheat, accurately obtaining crop density is a core prerequisite for combine harvesters to optimize operating parameters and improve harvesting quality and efficiency. When crop density is high, the harvester's feeding speed should be reduced and the threshing device speed adjusted to avoid grain breakage or machine blockage. When crop density is low, the harvester's feeding speed can be appropriately increased to improve operating efficiency and reduce ineffective energy consumption.

[0003] In related technologies, traditional crop density detection methods mainly rely on manual sampling, requiring pre-sampling and testing before harvesting. Wheat density is obtained through manual observation and measurement or contact sensors. Manual observation and measurement requires stopping the machine for sampling, which cannot meet the real-time requirements of harvesting operations. Contact sensors are easily entangled and worn by wheat straw or grains, resulting in a very high failure rate in high-intensity harvesting scenarios, making it difficult to operate continuously and stably. These methods are inefficient, time-consuming, and labor-intensive, unable to cover large areas, and mostly involve sampling measurements at fixed time points, failing to provide real-time data updates and struggling to support dynamic management needs.

[0004] Among related technologies, crop density detection methods based on remote sensing technology, as a non-contact detection method observing from the top of crops, can achieve large-scale non-destructive monitoring of wheat in the field. However, most mainstream methods acquire data through satellite platforms, combine ground sampling data and vegetation indices, and use empirical or statistical models for spectral inversion, which has the following obvious limitations: First, since the spatial resolution of conventional satellite imagery is mostly 10-30 meters, the surface area corresponding to a single pixel is relatively large, making it difficult for schemes based on conventional satellite imagery to accurately characterize wheat density. Second, due to the high cost and long data acquisition cycle of high-resolution satellite imagery, schemes based on high-resolution satellite imagery cannot meet the real-time requirements of agricultural monitoring. Third, due to the spatial resolution limitation, this type of method generally suffers from mixed pixel phenomenon, that is, a single pixel usually contains multiple land cover types such as crop canopy, weeds, and soil background, which limits the accuracy of wheat density inversion. Fourth, this type of method is susceptible to interference from environmental factors such as lighting conditions, cloud cover, and soil background, resulting in insufficient stability of time series data. Clouds, haze, and atmospheric interference can significantly reduce image clarity and usability, leading to unstable detection results. Fifth, this type of method relies on empirical relationships or statistical models between spectral reflectance, vegetation index and crop growth status. Although effective in specific regions and specific crop types, its empirical relationships and models have poor transferability to crop types and regions, and cannot adapt to complex field environments and variable wheat growth status. Summary of the Invention

[0005] This disclosure provides a method, system, and storage medium for crop density detection based on lidar, aiming to at least partially solve the technical problem that related technologies cannot provide accurate and reliable crop density detection results for harvester operations in real time and efficiently.

[0006] At least one embodiment of this disclosure provides a crop density detection method based on lidar, applied to a harvester, including:

[0007] Acquire the current 3D point cloud data of the crop area in front of the harvester, collected by lidar; Based on the geometry of the harvester header, the predicted harvesting area in the three-dimensional point cloud data is determined, and the predicted area is divided into multiple sub-regions along the direction of the harvester's movement. Height distribution features are extracted from the 3D point cloud data of each sub-region to construct a multi-dimensional feature vector for each sub-region; The different features of the multidimensional feature vector of each sub-region are adaptively fused based on an attention mechanism to generate the region code of each sub-region; Based on the region encoding and corresponding location encoding of each sub-region, global features at the sample level of the predicted region are obtained; and... Based on the global features at the sample level of the predicted region, the discrete density level and continuous density estimate of the crop are predicted synchronously and output as the crop density detection result.

[0008] The above-mentioned solution has the following technical advantages: it provides an automatic crop density detection method based on lidar, which significantly improves crop density detection efficiency, real-time performance, spatial resolution, and applicability compared to related technologies. The improvements include at least four key points: First, by determining the predicted area of ​​the harvester's movement, sub-regions are divided along the forward direction, and a density representation approach is established based on the height distribution differences of the 3D point cloud data of each sub-region. Second, for each sub-region, multi-dimensional feature vectors are extracted, and an attention mechanism is used to achieve adaptive fusion processing to improve the accuracy and robustness of feature representation. Third, for each sub-region, its unique region encoding is effectively fused with its corresponding location encoding, thereby integrating and constructing sample-level global features to enhance the overall representation and capture richer spatial context information. Fourth, through a multi-task synchronous prediction method using discrete density levels and continuous density estimates, classification and regression problems can be handled simultaneously, integrating discrete and continuous features to improve the overall performance and generalization ability of the model, thus achieving more accurate and efficient predictions in complex data environments. The above key points together form a complete closed loop from point cloud acquisition, feature construction, cross-feature fusion to secure output control, enabling this method to achieve real-time and high-precision density detection, which distinguishes it from related technologies. The advantages of this method include at least the following four aspects: First, compared with related technologies that require manual sampling during downtime or rely on prior remote sensing inversion, this method significantly improves the timeliness and operational continuity of crop density information acquisition. It does not rely on illumination and spectral indices, broadens the applicable scenarios, and reduces model migration costs, providing a feedforward basis for real-time adjustment of parameters such as combine harvester feed rate and travel speed. Second, by utilizing the height distribution characteristics of radar 3D point cloud data, and constructing a prediction region and dividing it into multiple sub-regions, the method models the 3D spatial density of crops at a scale close to the actual cutting width and sub-cutting width. Third, by using the multidimensional features of each sub-region as multi-source inputs, and through adaptive fusion processing based on an attention mechanism combined with position encoding, the method can fully explore the internal dependencies of features and the complementary relationships across features, greatly enhancing the modeling ability for complex point cloud distribution patterns. Fourth, by performing a series of processing steps on the multidimensional features of each sub-region to achieve fusion and adopting a multi-task structure of synchronous prediction, the method enhances the overall perception of continuous spatial changes in crop density while ensuring on-board real-time performance, enabling more reliable feed control decisions and operational safety assurance.

[0009] The method provided in at least one embodiment of this disclosure further includes: Obtain the confidence level of the crop density detection results; In response to the confidence level exceeding a preset first threshold, a first instruction is generated to use the discrete density level and the continuous density estimate as harvester operation control variables; and... In response to the confidence level being lower than a preset second threshold, a second instruction is generated to trigger manual review or lidar rescanning, wherein the first threshold is greater than the second threshold.

[0010] The above scheme has the following technical effects: improving the effectiveness of decision-making based on crop density detection results.

[0011] In at least one embodiment of the method provided in this disclosure, before extracting height distribution features from the 3D point cloud data in each sub-region, the method further includes: The 3D point cloud data in each sub-region is preprocessed to remove abnormal data, and the preprocessed 3D point cloud data is used for feature extraction.

[0012] The above scheme has the following technical effect: ensuring the effectiveness of feature extraction.

[0013] In at least one embodiment of the method provided in this disclosure, the lidar is positioned at a fixed location in front of the harvester, and the step of acquiring the current three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar includes: The lidar is controlled to perform a vertical scan of the crop area in front of the harvester; Acquire the initial three-dimensional point cloud data of the crop area in front of the harvester, collected by the lidar through the vertical scan; The initial 3D point cloud data is transformed from the lidar coordinate system to the harvester's vehicle coordinate system; and, After coordinate system transformation, the laser ranging value of each position coordinate in the initial three-dimensional point cloud data is converted into the crop height value relative to the ground to generate the final three-dimensional point cloud data. The three-dimensional point cloud data contains multiple position coordinates and the crop height value relative to the ground for each position coordinate.

[0014] The above solution has the following technical advantages: acquiring efficient 3D point cloud data.

[0015] In at least one embodiment of the method provided in this disclosure, determining the predicted harvesting area of ​​the harvester in the three-dimensional point cloud data based on the geometry of the harvester header includes: Obtain a straight line parallel to the harvester's header and at a predetermined distance from the front of the harvester as the basic boundary; and, Based on the basic boundary and the installation position and scanning angle of the lidar, a rectangular area is determined in the vehicle coordinate system as the prediction area.

[0016] The above scheme has the following technical effects: it ensures the operability of the detection process based on the prediction area and ensures that accurate real-time crop density detection results are obtained.

[0017] In at least one embodiment of the method provided in this disclosure, the preprocessing of the 3D point cloud data in each sub-region includes: For the 3D point cloud data within each sub-region, a first-level screening is performed on the 3D point cloud data of the sub-region based on a preset reasonable height range for the crop canopy, to filter out first-type abnormal data that does not conform to the reasonable height range for the crop canopy; and, A second-level filtering is performed on the point cloud data after the first-level filtering based on the quartiles and interquartile ranges of the point cloud height values, in order to filter out the second type of outlier data that are outliers.

[0018] The above solution has the following technical advantages: to ensure the accuracy of crop density detection results.

[0019] In at least one embodiment of the method provided in this disclosure, the step of extracting height distribution features from the three-dimensional point cloud data within each sub-region and constructing a multi-dimensional feature vector for each sub-region includes: For each sub-region, the 3D point cloud data is statistically analyzed to obtain multiple statistical features of the sub-region, and a statistical feature vector of the sub-region is constructed. The multiple statistical features include: Robust height range characteristics, which characterize the penetration of laser light into the crop canopy in the sub-region. The height statistical discrete feature is used to characterize the degree of dispersion of crop height values ​​relative to the ground in the sub-region, and Height distribution shape features, which are used to characterize the clustering pattern of crop height values ​​relative to the ground in the sub-region; Based on the point cloud height value of each sub-region, the sub-region is divided into multiple sub-intervals. The number of point clouds in each sub-interval is counted and normalized. A height histogram feature vector is constructed for each sub-region, containing the proportion of point clouds in each sub-interval. The point cloud height value is the maximum value of the crop's height relative to the ground in the corresponding sub-region. Construct a multidimensional feature vector for each sub-region, wherein the multidimensional feature vector includes the statistical feature vector and the height histogram feature vector.

[0020] The above scheme has the following technical effect: extracting effective multidimensional features.

[0021] In the method provided in at least one embodiment of this disclosure, the robust height interval feature is related to the difference between the 95th percentile and the 5th percentile of the point cloud height values ​​of the corresponding sub-region; the height statistical dispersion feature includes at least one of the point cloud height mean, point cloud height standard deviation, and point cloud height coefficient of variation of the corresponding sub-region; and the height distribution shape feature includes at least one of the point cloud height skewness and point cloud height kurtosis of the corresponding sub-region. The step of extracting height distribution features from the 3D point cloud data within each sub-region and constructing a multi-dimensional feature vector for each sub-region also includes: Obtain multiple operating condition parameters for the current operating condition, and construct an operating condition parameter vector containing the multiple operating condition parameters; and, The operating condition parameter vector is incorporated into the multidimensional feature vector of each sub-region, so that the multidimensional feature vector of each sub-region includes the statistical feature vector, the height histogram feature vector, and the operating condition parameter vector. The adaptive fusion process of different features of the multidimensional feature vector of each sub-region based on an attention mechanism includes: For each sub-region, the statistical feature vector and the height histogram feature vector in its multidimensional feature vector are encoded respectively to generate the first encoded feature corresponding to the statistical feature vector and the second encoded feature corresponding to the height histogram feature vector; Adaptive weighting is applied to the first coding feature and the second coding feature to generate enhanced first coding feature and enhanced second coding feature; Construct a bidirectional cross-attention interaction network related to the enhanced first coding feature and the enhanced second coding feature, such that one of the enhanced first coding feature and the enhanced second coding feature can extract complementary information from the other; and, The enhanced first coding feature, the enhanced second coding feature, and the complementary information in the bidirectional cross-attention interaction network are fused to generate the region code of the sub-region; The process of obtaining global features at the sample level for the predicted region based on the region encoding and corresponding position encoding of each sub-region includes: Arrange the area codes of each sub-region into the first sequence according to the direction of the harvester's movement; Obtain the location code of each sub-region and arrange them into a second sequence; The first sequence and the second sequence are superimposed to obtain a feature sequence with added positional information; The feature sequence with added location information is input into the Transformer encoder to obtain the context-enhanced feature sequence. The context-enhanced feature sequence is weighted based on a channel attention mechanism; and... The different elements of the weighted context-enhanced feature sequence are subjected to average pooling to obtain sample-level global features. The method of synchronously predicting the discrete density level and continuous density estimate of crops based on the global features at the sample level of the prediction region includes: The global features at the sample level of the prediction region are input into a preset multi-task model to simultaneously predict the discrete density level and continuous density estimate of the crop. The multi-task model is configured to convert the input global features at the sample level of the prediction region into the output crop density level through its first fully connected layer and Softmax function, and to convert the input global features at the sample level of the prediction region into the output continuous density estimate through its second fully connected layer.

[0022] The above solution has the following technical advantages: to ensure the accuracy of crop density detection results.

[0023] At least one embodiment of this disclosure also provides a crop density detection system based on lidar, applied to a harvester, comprising: The acquisition unit is configured to acquire the current operating status parameters of the motor; The first-level processing unit is configured to generate enhanced modulation waves for each phase to improve the utilization rate of motor voltage based on the operating state parameters. The second-level processing unit is configured to acquire the duty cycle of each phase generated by the motor controller based on the enhanced modulation wave, acquire the target carrier signal corresponding to the phase to which the median value of each phase duty cycle belongs, and perform phase-shifting processing on the target carrier signal, wherein the phase-shifting processing is used to eliminate the zero-voltage vector generated by the motor controller; and, The third-level processing unit is configured to generate phase PWM signals for driving the inverter in the motor controller based on the phase carrier signals after phase shifting, so as to suppress the common-mode voltage of the motor.

[0024] At least one embodiment of this disclosure also provides a storage medium storing a program or instructions, wherein the program or instructions, when executed by a processor, implement the steps of the method provided in any embodiment of this disclosure.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram of lidar scanning a sparsely planted wheat field; Figure 2 A schematic diagram of a lidar scan of a densely planted wheat field; Figure 3 A flowchart of a crop density detection method based on lidar provided for at least one embodiment of this disclosure; Figure 4 Flowchart of a three-dimensional point cloud data acquisition scheme provided for at least one embodiment of this disclosure; Figure 5 A flowchart illustrating a prediction region determination and partitioning scheme provided for at least one embodiment of this disclosure; Figure 6 A schematic diagram of a sub-region of a sparsely planted wheat field provided for at least one embodiment of this disclosure; Figure 7 A schematic diagram of a sub-region of a densely planted wheat field provided for at least one embodiment of this disclosure; Figure 8 Flowchart of a sub-region multidimensional feature vector construction scheme provided in at least one embodiment of this disclosure; Figure 9 A flowchart of a sub-region multidimensional feature adaptive fusion scheme provided in at least one embodiment of this disclosure; Figure 10 A schematic diagram of a hierarchical attention mechanism provided for at least one embodiment of this disclosure; Figure 11 Flowchart of a sample-level global feature generation scheme provided for at least one embodiment of this disclosure; Figure 12 A schematic diagram of the basic gating fusion mechanism provided for at least one embodiment of this disclosure; Figure 13 A schematic diagram illustrating the process of sequence modeling and sample-level global feature aggregation provided for at least one embodiment of this disclosure; Figure 14 A flowchart of a crop density prediction scheme provided in at least one embodiment of this disclosure; Figure 15 Example flowchart of a crop density detection method based on lidar provided for at least one embodiment of this disclosure; Figure 16A structural block diagram of a LiDAR-based crop density detection system provided for at least one embodiment of this disclosure; Figure 17 A schematic diagram illustrating the composition of a program product provided for at least one embodiment of this disclosure.

[0028] Figure label: 1- LiDAR; 2- Wheat plant; 10- LiDAR-based crop density detection system; 11- Acquisition unit; 12- First-level processing unit; 13- Second-level processing unit; 14- Third-level processing unit; 21- Processor; 22- Memory; 23- Input device; 24- Output device; ⊕- Element-by-element addition; ⊙- Hadamard product; ©- Element-by-element multiplication; - Matrix multiplication; - First coding feature; - Second coding feature; - Enhanced first encoded feature; - Enhanced second coding features; - First comprehensive feature representation; - Second comprehensive feature representation; - First interactive information; - Second interactive information; MLP - Multilayer Perceptron; - First-channel attention; - Second channel attention. Detailed Implementation

[0029] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the disclosure. Similarly, the following embodiments are only some, not all, embodiments of the present disclosure, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0030] The terms "first," "second," and "third" used in the embodiments of this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," and "third" may explicitly or implicitly include at least one of that feature.

[0031] In the description of this disclosure, "multiple" means at least two, such as two or three, unless otherwise expressly and specifically limited.

[0032] In this disclosure, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0033] The terms “comprising” and “having”, and any variations thereof, used in this disclosure are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or components inherent to such processes, methods, products, or devices.

[0034] The term "prediction region" in the embodiments of this disclosure is also called the ROI region.

[0035] The design concept of this disclosure will be explained below.

[0036] LiDAR technology, with its advantages of high precision, non-contact operation, and rapid response, has become one of the optional detection solutions suitable for wheat harvesting scenarios. Although there are LiDAR-based crop density detection methods, the point cloud data processing algorithms they use are too simple, and the extracted features are not effective enough, resulting in a large error in the final crop density estimation.

[0037] Research has shown that crop density can be detected by analyzing the height distribution characteristics of point clouds returned by lidar, based on the morphological characteristics of wheat plants before harvest. For example... Figure 1 As shown, when the laser beam emitted by lidar 1 scans a sparsely planted wheat field, the gaps between the wheat plants 2 are relatively large, allowing the laser to penetrate the canopy and reach the underlying straw area. The return point exhibits a large vertical span, with a large height range ΔH. For example... Figure 2 As shown, when the laser beam emitted by lidar 1 scans a densely planted wheat field, the canopy of wheat plants 2 is tightly intertwined, and the laser mainly reflects off the canopy on the surface of the wheat. The vertical height range of the return point is significantly narrowed, and the height interval ΔH is small. By combining this feature with deep learning, the wheat density in the field can be quickly inverted, providing data support for the combine harvester to adjust its operating parameters in real time.

[0038] To improve the efficiency, real-time performance, spatial resolution, and applicability of crop density detection, this disclosure focuses on extracting the height distribution features (also known as vertical distribution features) of three-dimensional point cloud data. By combining multi-dimensional point cloud height distribution features, optimizing region division, and introducing adaptive fusion processing based on attention mechanisms, the three-dimensional spatial density distribution of crops is finely characterized at the scale of sub-regions. This significantly improves the accuracy and stability of density estimation, enhances adaptability to different varieties, soil types, and climatic conditions, and provides reliable real-time density information support for optimizing operational parameters and improving operational quality.

[0039] Figure 3 This is a flowchart illustrating a crop density detection method based on lidar, provided for at least one embodiment of this disclosure. The method can be applied to harvesters, and the crops include, but are not limited to, wheat. Figure 3 As shown, the method may include the following steps S10-S60.

[0040] Step S10: Obtain the current three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar.

[0041] Step S20: Based on the geometry of the harvester header, determine the predicted area (also known as the ROI area of ​​vehicle travel) of the harvester in the three-dimensional point cloud data, and divide the predicted area into multiple sub-regions along the direction of the harvester's movement.

[0042] Step S30: Extract height distribution features from the 3D point cloud data in each sub-region and construct a multi-dimensional feature vector for each sub-region.

[0043] Step S40: Perform adaptive fusion processing based on attention mechanism on the different features of the multidimensional feature vector of each sub-region to generate the region code of each sub-region.

[0044] Step S50: Based on the region encoding and corresponding position encoding of each sub-region, obtain the global features at the sample level of the prediction region.

[0045] Step S60: Simultaneously predict the discrete density level and continuous density estimate of the crop based on the global features at the sample level of the prediction area, and output them as the crop density detection result.

[0046] It should be noted that step S10 is used to acquire three-dimensional point cloud data, step S20 is used to determine the prediction area and divide it into partitions, step 30 is used to construct multi-dimensional feature vectors, steps S40 and S50 are used to fuse multiple height distribution features of each sub-region to obtain sample-level global features of the prediction area, and step S60 is used to predict crop density based on sample-level global features.

[0047] In the above scheme, this disclosure does not specifically limit the scheme for acquiring 3D point cloud data in step S10. In practical application scenarios, in addition to the schemes described in the following embodiments, lidar can be mounted on different carriers such as drones, the front end of harvesters, or handheld mobile devices to perform real-time, multi-view, and multi-dimensional scanning and acquisition of target crop areas. For example, when using a drone equipped with lidar, it can achieve full coverage scanning of crop fields according to a preset grid or serpentine flight path. By emitting laser pulses and receiving reflected signals, the 3D coordinate information of each laser point is calculated to generate 3D point cloud data containing details such as crop canopy height and distribution range. At the same time, the scanning frequency and point cloud resolution of lidar can be flexibly adjusted according to the crop growth cycle and planting density differences to balance the data acquisition efficiency and accuracy requirements. The acquired initial 3D point cloud data can undergo preliminary noise reduction processing, such as removing ground points or isolated noise points, to ensure the accuracy of the prediction area determination and sub-region division in the subsequent step S20.

[0048] When the system executes step S10, it can select a suitable three-dimensional point cloud data acquisition scheme based on the carrier type equipped with lidar, crop variety characteristics, and real-time environmental conditions.

[0049] In the above scheme, this disclosure does not specify the method for determining and dividing the predicted region in step S20. In practical application scenarios, in addition to the schemes described in the following embodiments, the predicted region to be detected can also be automatically identified and bounded based on the three-dimensional morphological characteristics of the crop canopy, such as the canopy height threshold and point cloud density distribution, to ensure that the region covers the main crop growth area and excludes non-crop interference areas. For sub-region division, an adaptive grid division method can be used to dynamically adjust the grid size according to the density of crop point clouds in different regions. Smaller grid units are set in densely distributed crop areas to improve detection accuracy, and larger grid units are set in sparse areas to improve processing efficiency; density-based spatial clustering algorithms can also be used to divide adjacent areas with high point cloud density into the same sub-region, thereby better reflecting the actual clustered growth distribution characteristics of crops. In addition, a row and column grid division method can be used in conjunction with the preset row and plant spacing parameters of crop planting to ensure that the boundary of the sub-region is consistent with the crop planting row direction, which facilitates subsequent density statistical analysis for each row and each crop.

[0050] When the system executes step S20, it can select an appropriate prediction area determination and division scheme according to the detection needs of different crop varieties, planting patterns and growth stages, so as to improve the adaptability and accuracy of density detection.

[0051] In the above scheme, this disclosure does not limit the scheme for constructing the multi-dimensional feature vector of the sub-region in step S30. In practical application scenarios, in addition to the scheme described in the following embodiments, diverse construction schemes can be designed by combining the multi-dimensional attributes of the LiDAR point cloud. For example, the three-dimensional spatial distribution features of the point cloud in each sub-region can be extracted, including the centroid coordinates of the point cloud, the dispersion along the XYZ axes, the local density peaks and distribution range of the point cloud; geometric features can also be introduced, such as the aspect ratio of the fitted point cloud, the distribution of the normal vector angle of the fitted plane, and the proportion of the convex hull volume of the point cloud cluster; in addition, texture features can be fused, such as the distance distribution histogram of neighboring point pairs and the entropy value of the point cloud distribution. These features can be selectively combined according to actual needs. All features can be standardized and directly spliced ​​into a multi-dimensional vector, or the high-dimensional features can be reduced in dimensionality through principal component analysis, retaining key information before integration.

[0052] When the system executes step S30, it can select a suitable multidimensional feature vector construction scheme according to the actual application scenario.

[0053] In the above scheme, this disclosure does not limit the sub-region multi-dimensional feature adaptive fusion scheme in step S40. In practical application scenarios, in addition to the schemes described in the following embodiments, a dynamic weight fusion strategy can also be adopted according to the differences in crop growth stages. For example, in the seedling stage, the local density peak features of the point cloud are given higher fusion weights to highlight the recognition accuracy of individual seedlings; in the mature stage of the crop, the weights of geometric morphological features are increased to adapt to changes in the population structure. A multi-scale feature fusion mechanism can also be introduced to adaptively associate point cloud features extracted at different resolutions, such as fine-grained neighborhood distance distribution and coarse-grained centroid coordinate distribution, through a cross-scale attention module to enhance the representation ability of key features. In addition, the fusion method can be selected based on the specificity of crop type. For crops with compact plant structure (such as wheat), a cluster-based feature fusion method can be used to reduce redundant information, while for crops with loose plant structure (such as corn), a combination strategy based on feature splicing and dimensionality reduction can be used to retain complete spatial information.

[0054] When executing step S40, the system can select an appropriate multi-dimensional feature adaptive fusion scheme based on factors such as crop variety, growth environment and detection accuracy requirements, thereby improving the adaptability and generalization ability of crop density detection.

[0055] In the above scheme, this disclosure does not limit the sample-level global feature generation scheme in step S50. In practical application scenarios, in addition to the schemes described in the following embodiments, a hierarchical feature aggregation strategy can be adopted according to the spatial scale differences of the prediction area. For example, in a small-scale prediction area, fine-grained local texture features are extracted first through a convolutional neural network and concatenated with the spatial coordinate features of the point cloud to generate global features containing detailed information; in a large-scale prediction area, a graph attention network (GAT) is introduced to weighted aggregate the local features of all sample points in the area, highlighting the contribution of key sample points to the global features. The feature generation method can also be adjusted according to the accuracy requirements of the prediction task. For high-precision detection requirements, multimodal feature fusion is used to generate global features to enrich the feature dimensions; for scenarios with high real-time requirements, the feature extraction process is simplified, and only the regional encoding of the LiDAR point cloud is retained for rapid aggregation, maintaining a certain level of accuracy while ensuring detection efficiency. In addition, the feature generation logic can be dynamically adjusted according to the growth state of different crops. For example, in abnormal growth states such as crop lodging, the weights of the normal direction features and attitude angle features of the point cloud are increased to ensure that the generated global features can accurately reflect the actual distribution of the crops.

[0056] When executing step S50, the system can flexibly select the preferred sample-level global feature generation scheme based on actual factors such as the size of the prediction area, the priority of the detection task, and the crop growth status, thereby further improving the accuracy and robustness of crop density prediction.

[0057] In the above scheme, this disclosure does not limit the crop density prediction scheme in step S60. In practical application scenarios, in addition to the schemes described in the following embodiments, a regression prediction model based on machine learning can also be used. By inputting the generated sample-level global features into a pre-trained model such as a random forest, gradient boosting tree, or deep neural network, the predicted value of crop density can be directly output. Alternatively, a time-series prediction model can be constructed by combining the spatiotemporal sequence information of crop growth, and the dynamic change trend of LiDAR point cloud features over a continuous time period can be used to achieve accurate prediction of the dynamic changes in crop density. In addition, considering the differences in growth characteristics of different crop types such as wheat, corn, or rice, crop-specific feature weight coefficients can be introduced to specifically adjust the input features of the prediction model to adapt to the density detection needs of various crops.

[0058] When the system executes step S60, it can select a suitable crop density prediction scheme based on the detection accuracy and real-time requirements in the actual scenario.

[0059] The automatic crop density detection method based on LiDAR provided in steps S10-S60 significantly improves crop density detection efficiency, real-time performance, spatial resolution, and applicability compared to related technologies. The improvements include at least four key points: First, by determining the predicted area of ​​the harvester's movement, sub-regions are divided along the forward direction, and a density representation approach is established based on the height distribution differences of the 3D point cloud data of each sub-region. Second, for each sub-region, multi-dimensional feature vectors are extracted, and an attention mechanism is used to achieve adaptive fusion processing, thereby improving the accuracy and robustness of feature representation. Third, for each sub-region, its unique region code is effectively fused with its corresponding location code, thus integrating and constructing sample-level global features to enhance the overall representation and capture richer spatial context information. Fourth, through a multi-task synchronous prediction method using discrete density levels and continuous density estimates, classification and regression problems can be handled simultaneously, integrating discrete and continuous features to improve the overall performance and generalization ability of the model, thereby achieving more accurate and efficient predictions in complex data environments. The above key points together form a complete closed loop from point cloud acquisition, feature construction, cross-feature fusion to secure output control, enabling this method to achieve real-time and high-precision density detection, which distinguishes it from related technologies. The advantages of this method include at least the following four aspects: First, compared with related technologies that require manual sampling during downtime or rely on prior remote sensing inversion, this method significantly improves the timeliness and operational continuity of crop density information acquisition. It does not rely on illumination and spectral indices, broadens the applicable scenarios, and reduces model migration costs, providing a feedforward basis for real-time adjustment of parameters such as combine harvester feed rate and travel speed. Second, by utilizing the height distribution characteristics of radar 3D point cloud data, and constructing a prediction region and dividing it into multiple sub-regions, the method models the 3D spatial density of crops at a scale close to the actual cutting width and sub-cutting width. Third, by using the multidimensional features of each sub-region as multi-source inputs, and through adaptive fusion processing based on an attention mechanism combined with position encoding, the method can fully explore the internal dependencies of features and the complementary relationships across features, greatly enhancing the modeling ability for complex point cloud distribution patterns. Fourth, by performing a series of processing steps on the multidimensional features of each sub-region to achieve fusion and adopting a multi-task structure of synchronous prediction, the method enhances the overall perception of continuous spatial changes in crop density while ensuring on-board real-time performance, enabling more reliable feed control decisions and operational safety assurance.

[0060] Some embodiments of this disclosure also provide systems, storage media, and program products corresponding to the methods described above.

[0061] The method provided by at least one embodiment of this disclosure is applicable to any existing harvester application scenario that requires harvesting crops such as wheat. For example, in large-scale mechanized wheat-producing areas on plains, this method can use real-time 3D point cloud data collected by lidar to quickly calculate crop density in zones at a scale close to the actual cutting width, dynamically adjusting the feed rate and travel speed. This effectively avoids problems such as blockage of the conveyor system or incomplete harvesting caused by sudden changes in crop density. For small wheat fields with complex terrain and unstable lighting conditions, such as hilly and mountainous areas, traditional detection methods relying on remote sensing inversion or spectral indices are easily affected by terrain shadows and cloud cover, resulting in decreased accuracy. This method does not rely on lighting and spectral information and can still continuously and stably acquire crop density data, ensuring the continuity of harvester operations in non-ideal environments. For special scenarios such as wheat lodging, where the 3D spatial distribution pattern of crops changes significantly, this method can accurately capture the density change details of lodged areas by constructing a prediction region and dividing it into multiple sub-regions, combined with adaptive feature fusion using an attention mechanism. This provides harvesters with more reliable feed control decisions, reduces the loss rate caused by lodging, and improves the safety and efficiency of harvesting operations.

[0062] Figure 4 A flowchart illustrating a three-dimensional point cloud data acquisition scheme provided in at least one embodiment of this disclosure. Figure 3 Based on the proposed solution, to ensure the accuracy of crop density detection results, during the 3D point cloud data acquisition stage, a lidar is installed at a fixed position in front of the harvester, and, as... Figure 4 As shown, step S10 further includes the following sub-steps S101-S104.

[0063] Sub-step S101: Control the lidar to perform a vertical scan of the crop area in front of the harvester.

[0064] Sub-step S102: Obtain the initial three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar through vertical scanning.

[0065] Sub-step S103: Transform the initial 3D point cloud data from the lidar coordinate system to the harvester's vehicle coordinate system.

[0066] Sub-step S104: Convert the laser ranging value of each position coordinate in the initial three-dimensional point cloud data after coordinate system transformation into the crop height value relative to the ground, and generate the final three-dimensional point cloud data. The three-dimensional point cloud data contains multiple position coordinates and the crop height value relative to the ground for each position coordinate.

[0067] Sub-steps S101-S104 enable the accurate acquisition and preprocessing of 3D point cloud data of the crop area in front of the harvester. Vertical scanning ensures comprehensive coverage of the crop canopy and plant structure; coordinate system transformation eliminates spatial discrepancies between the lidar and the harvester's coordinate system, ensuring the data closely matches the actual spatial relationships of the harvesting scenario; and the conversion from laser ranging values ​​to crop height relative to the ground effectively filters out background interference, focusing on the crop's own height information. The generated 3D point cloud data can directly support subsequent crop density analysis.

[0068] As an exemplary implementation, in the three-dimensional point cloud data acquisition stage of step S10, after the lidar acquires the initial three-dimensional point cloud data of the crop area in front through vertical scanning, the initial three-dimensional point cloud data is transformed from the lidar coordinate system to the vehicle coordinate system using the following conversion relationship:

[0069] In the formula, This represents the initial 3D point cloud data after coordinate system transformation. This represents the initial 3D point cloud data. Represents the rotation matrix. This represents the translation matrix.

[0070] Simultaneously, the laser ranging values ​​in the initial 3D point cloud data can be converted into crop height values ​​relative to the ground. Conversion schemes include, but are not limited to, using deep learning networks to establish the relationship between laser ranging values ​​and crop height values ​​relative to the ground.

[0071] Figure 5 A flowchart illustrating a prediction region determination and partitioning scheme provided for at least one embodiment of this disclosure. Figure 3 or Figure 4 Based on the proposed plan, to ensure the accuracy of crop density detection results, during the stage of determining and dividing the prediction area, such as... Figure 5 As shown, step S20 further includes the following sub-steps S201-S203.

[0072] Sub-step S201: Obtain a straight line that is a set distance from the front end of the harvester and parallel to the harvester's header as the basic boundary.

[0073] Sub-step S202: Based on the basic boundary and the installation position and scanning angle of the lidar, determine a rectangular area in the vehicle coordinate system as the prediction area.

[0074] Sub-step S203: Divide the predicted area into multiple sub-regions along the direction of the harvester's movement.

[0075] It should be noted that the predicted area in sub-step S203 can also be divided unequally along the direction of the harvester's movement.

[0076] Sub-steps S201-S203 precisely define the crop area directly related to the harvester's operation, providing a clear analytical range for subsequent accurate crop density detection. In sub-step S201, the basic boundary setting distance aligns with the actual operating width and lead time requirements of the header, effectively eliminating interference from excessively far or near areas. Sub-step S202 determines a rectangular prediction area by combining the lidar's installation parameters and scanning angle, ensuring this area completely covers the range where the lidar can effectively capture crop information, while also being highly compatible with the operating scene in the vehicle coordinate system. Sub-step S203's design of equally dividing the prediction area along the forward direction ensures that the size of each sub-area matches the lidar's scanning resolution and the local uniformity of crop growth. This facilitates independent statistical analysis of the number and distribution characteristics of crop point clouds within each sub-area and provides more refined density feedback for dynamic harvester adjustments. This phased prediction area determination and division scheme, from boundary definition and area range to unit decomposition, ensures the operability of the prediction area-based detection process and guarantees accurate real-time crop density detection results.

[0077] As an exemplary implementation, in the prediction area determination and partitioning stage of step S20, based on the long line of the harvester's header or operating components, a straight line 0.5m in front of the header and parallel to it is defined as the lower boundary (basic boundary). The harvest boundary line (which approximately coincides with the extension line of the left header) is identified by the lidar, and its intersection with the lower boundary is taken as the lower left base point. Extending a predetermined length along the forward direction, the left boundary of the ROI area is obtained. The right boundary is defined as the extension line of the right header intersecting with the lower boundary and extending a predetermined length forward. Connecting the extension lines of the left and right boundaries in front of the header forms the upper boundary, thus forming the prediction area from these four boundaries. Considering the difference in incident angle caused by the laser beam diverging from a fixed point, the prediction area is divided into four sub-regions along the harvester's forward direction. , Figure 1 Subregions of sparsely planted wheat fields, such as Figure 6 As shown, Figure 2 Subregions of medium-dense wheat fields, such as Figure 7 As shown in the figure, the prediction area is divided into 4 sub-regions, and the distance range corresponding to each sub-region is []. , ]. For the first i The first boundary of the region For the first i The second boundary of the region. It should be noted that... Figure 6 and Figure 7 Dividing the region into four sub-regions is just for feature display and ease of understanding; in reality, more sub-regions can be divided.

[0078] Figure 8 A flowchart illustrating a sub-region multidimensional feature vector construction scheme provided in at least one embodiment of this disclosure. Figure 3 , Figure 4 , Figure 5 Based on any given scheme, to ensure the accuracy of crop density detection results, during the multi-dimensional feature vector construction stage, such as... Figure 8 As shown, step S30 further includes the following sub-steps S301-S303.

[0079] Sub-step S301: Perform statistics on the 3D point cloud data in each sub-region, obtain multiple statistical features of the sub-region, and construct the statistical feature vector of the sub-region.

[0080] Sub-step S302: Based on the point cloud height value of each sub-region, divide the sub-region into multiple sub-intervals, count the number of point clouds in each sub-interval and normalize it, and construct a height histogram feature vector of the sub-region containing the proportion of the number of point clouds in each sub-interval, where the point cloud height value is the maximum value of the crop height relative to the ground in the corresponding sub-region.

[0081] Sub-step S303: Construct a multi-dimensional feature vector for each sub-region, wherein the multi-dimensional feature vector includes a statistical feature vector and a height histogram feature vector.

[0082] Among them, the multi-dimensional feature vector constructed through sub-steps S301-S303 can comprehensively integrate the statistical attributes and height distribution features of the point cloud of the sub-region, providing accurate and rich input basis for subsequent identification, clustering and density calculation of crop plants based on the feature vector, effectively improving the completeness and pertinence of feature expression, and ensuring the reliability and accuracy of subsequent crop density detection results.

[0083] In some embodiments, Figure 8Building upon this foundation, to further ensure the accuracy of crop density detection results, sub-step S301 incorporates several statistical features, including: robust height interval features, height statistical dispersion features, and height distribution shape features. The robust height interval features characterize the laser's penetration through the crop canopy in a sub-region, and are related to the difference between the 95th percentile and the 5th percentile of the point cloud height values ​​in the corresponding sub-region. The height statistical dispersion features characterize the dispersion of crop height values ​​relative to the ground in a sub-region, and include at least one of the mean, standard deviation, and coefficient of variation of the point cloud height in the corresponding sub-region. The height distribution shape features characterize the clustering pattern of crop height values ​​relative to the ground in a sub-region, and include at least one of the point cloud height skewness and kurtosis in the corresponding sub-region. A larger robust height interval feature indicates a wider laser penetration range through the crop canopy, while a smaller feature indicates weaker penetration, helping to distinguish between sub-regions with sparse and dense canopy structures. The standard deviation in height statistical discrete features reflects the dispersion of height values ​​relative to the mean, while the coefficient of variation eliminates the influence of the mean magnitude, making it more suitable for comparing the degree of dispersion across sub-regions. A large discrete feature value indicates significant differences in crop height within the sub-region, potentially suggesting uneven growth. Skewness in height distribution shape features measures the asymmetry of the height distribution; positive skewness indicates that height values ​​are concentrated in the lower range, while negative skewness has the opposite effect. Kurtosis reflects the steepness of the distribution; high kurtosis means that height values ​​are clustered near the mean, while low kurtosis indicates a more gradual distribution. These shape features can supplement the distribution details that discrete features cannot capture, improving the sensitivity to changes in crop density. By fusing these three types of statistical features, the reliability and robustness of the overall detection results can be effectively improved, meeting the high-precision requirements for monitoring crop growth status in actual agricultural production.

[0084] In some embodiments, Figure 8 Based on the scheme, in order to further ensure the accuracy of crop density detection results, in the multidimensional feature vector construction stage, step S30 also includes the following sub-steps S304-S305.

[0085] Sub-step S304: Obtain multiple operating condition parameters for the current operating condition and construct an operating condition parameter vector containing the multiple operating condition parameters.

[0086] Sub-step S305: Incorporate the operating condition parameter vector into the multidimensional feature vector of each sub-region, so that the multidimensional feature vector of each sub-region includes the statistical feature vector, the height histogram feature vector, and the operating condition parameter vector.

[0087] The operating condition parameter vector introduced through sub-steps S304-S305 effectively captures the impact of the current detection environment and equipment operating status on LiDAR data acquisition. For example, factors such as scanning angle offset, movement speed fluctuation, changes in ambient light, or differences in air humidity may cause systematic deviations in the original point cloud data. By integrating these operating condition parameters into the feature vector, the subsequent crop density detection model can maintain stable recognition performance under different operating conditions, further reducing the interference of non-crop growth factors on the detection results. This processing method complements the statistical feature vector and height histogram feature vector, comprehensively improving the ability of multi-dimensional feature vectors to represent the actual growth status of crops. This provides a more reliable feature foundation for subsequent high-precision crop density classification or regression prediction, ultimately ensuring that crop density detection results maintain high accuracy and consistency in complex and ever-changing agricultural field environments.

[0088] As an exemplary implementation, in the multi-dimensional feature vector construction stage of step S30, robust height interval features, height statistical discrete features, and height distribution shape features are extracted from the effective point cloud in each sub-region to construct a statistical feature vector. Among them, robust height range features It can be obtained using the following formula:

[0089] In the formula, Indicates the first i The 95th percentile of the point cloud height values ​​for a sub-region is defined as follows: 95% of the point cloud heights are less than or equal to this value. Indicates the first i The 5th percentile of the point cloud height values ​​for a sub-region is defined as 5% of the point cloud heights being less than or equal to that value.

[0090] By calculating robust height range features, compared with the traditional feature obtained by subtracting the minimum value from the maximum value, this method can automatically eliminate 5% of extreme outliers, significantly improving the anti-interference ability of the detection method.

[0091] Mean height of point cloud in height statistical discrete features It can be obtained using the following formula:

[0092] In the formula, Indicates the first i Number of point clouds in each sub-region Indicates the first j The crop's height relative to the ground at each location coordinate.

[0093] Standard deviation of point cloud height in statistical discrete features Based on the average height of the point cloud Obtain the standard deviation of point cloud height This reflects the uniformity of crop canopy height. Sparse crops exhibit height differences between plants, resulting in a higher standard deviation of point cloud height. Larger; dense crop canopies have uniform height, point cloud height standard deviation Smaller. Point cloud height variation coefficient in statistical discrete features. The influence of absolute height was eliminated, making this indicator more comparable across different varieties and growth stages. Point cloud height variation coefficient. Equal to the standard deviation of point cloud height and mean point cloud height The ratio.

[0094] Point cloud height skewness in height distribution shape features It can be obtained using the following formula:

[0095] Point cloud height kurtosis in height distribution shape characteristics It can be obtained using the following formula:

[0096] Constructed statistical feature vector for:

[0097] In constructing statistical feature vectors At the same time, the height range within each sub-region is divided into b For each sub-interval, the number of point clouds within each interval is counted and normalized to obtain the height histogram feature vector of each sub-region. :

[0098] Height histogram feature vector Used to characterize the distribution ratio of laser echoes in the surface canopy, intermediate layer, and near-surface layer.

[0099] If necessary, a working condition parameter vector can be further introduced, formed by working condition parameters such as force velocity and radar installation height. By concatenating the above multi-source features along the feature dimension, a multi-dimensional feature vector for each sub-region is obtained. :

[0100] Figure 9 A flowchart illustrating a sub-region multi-dimensional feature adaptive fusion scheme provided in at least one embodiment of this disclosure. Figure 3 , Figure 4 , Figure 5 or Figure 8 Based on any of the proposed methods, to ensure the accuracy of crop density detection results, such as... Figure 9 As shown, step S40 further includes the following sub-steps S401-S405.

[0101] Sub-step S401: For each sub-region, encode the statistical feature vector and the height histogram feature vector in its multidimensional feature vector respectively to generate the first encoded feature corresponding to the statistical feature vector and the second encoded feature corresponding to the height histogram feature vector.

[0102] Sub-step S402: Adaptively weight the first coding feature and the second coding feature respectively to generate the enhanced first coding feature and the enhanced second coding feature.

[0103] Sub-step S403: Construct a bidirectional cross-attention interaction network related to the enhanced first coding feature and the enhanced second coding feature, so that one of the enhanced first coding feature and the enhanced second coding feature can extract complementary information from the other.

[0104] Sub-step S404: The enhanced first coding feature, the enhanced second coding feature, and the complementary information in the bidirectional cross-attention interaction network are fused to generate the region code for the sub-region.

[0105] In particular, through sub-steps S401-S405, the inherent correlation and complementary value between multi-dimensional features of sub-regions can be fully explored, effectively integrating the global descriptive ability of statistical features with the local detailed information of height histogram features. By leveraging a bidirectional cross-attention interaction network, accurate information complementarity between the two types of features is achieved. The generated region code includes both the macroscopic distribution pattern of crop populations and the height structural characteristics of individual plants, providing more accurate and representative feature support for subsequent crop density statistics based on region coding, thereby significantly improving the overall accuracy and reliability of the detection method.

[0106] As an exemplary implementation, in the sub-region multidimensional feature adaptive fusion stage of step S40, for each sub-region's multidimensional feature vector It is compressed into region coding through a two-layer fully connected network and nonlinear mapping. :

[0107] Specifically, to achieve explicit gating fusion between statistical feature vectors and height histogram feature vectors, the two types of features can be encoded separately to obtain statistical feature vectors. The corresponding first coding feature and height histogram feature vector The corresponding second coding feature The gating coefficient is calculated using the following formula. :

[0108] In the formula, This represents the activation function. This represents the learnable weight matrix.

[0109] Then, adaptive weighting is performed according to the following formula to obtain the region code of the sub-region. :

[0110] This scheme achieves adaptive gating between statistical features and height histogram features based on different crop types and growth stages. To further enhance the expressive power of feature fusion, a hierarchical attention interaction mechanism can be introduced on the basis of the above-mentioned gating fusion. Through self-attention enhancement, bidirectional cross-attention interaction, and the cascading of multiple fusions, deep modeling of internal feature dependencies, cross-feature complementary relationships, and nonlinear interactions can be achieved. The three progressive levels of this mechanism are as follows: Figure 10 As shown.

[0111] The first layer: Internal self-attention enhancement within features, achieved through adaptive weighting. Before basic gating fusion, the first encoded features are first... Second coding features Self-attention enhancement is performed separately to capture long-range dependencies within features. A query-key-value mechanism is used to calculate self-attention weights for each feature class, enabling the model to adaptively focus on key dimensions within the features. The enhanced first encoded feature is obtained after self-attention enhancement. and enhanced second coding features In this process, residual connections and layer normalization can be introduced to stabilize the training process.

[0112] The second layer: cross-feature bidirectional cross-attention interaction. This is one of the core innovations of this publication. Building upon self-attention enhancement, it models the interdependence and complementarity between two types of features through a cross-attention mechanism. The query comes from one type of feature, while the key and value come from the other type of feature, thus achieving cross-modal information extraction. Statistical feature queries retrieve histogram features to obtain the first interaction information. This indicates complementary information extracted from histogram features by statistical features; simultaneously, histogram features query statistical features to obtain second interactive information. The core formula of this two-way interaction mechanism is: ,

[0113] In the formula, CrossAttn() represents the cross-attention operation, which is used to ensure that the two types of features can fully exchange information and discover their complementary patterns.

[0114] Third layer: Multi-adaptive gating fusion. This enhances the first encoded feature... and enhanced second coding features The first interaction information of cross-attention interaction Second interactive information By concatenating the original discriminative information and cross-modal complementary information, a first comprehensive feature representation is constructed. Second comprehensive feature representation : ,

[0115] First-channel attention for different features is calculated separately using an independent multilayer perceptron (MLP). Second channel attention This achieves adaptive weight allocation. A three-term fusion strategy is employed to simultaneously capture linear combinations and nonlinear interactions, and the region encoding of the sub-region is obtained through the following formula. : ⊙ ⊙

[0116] In the formula, the first two terms are weighted and linearly fused to capture the linear combination relationship of features. The third term... ⊙ ⊙ For element-wise non-linear interaction terms, ⊙ represents the Hadamard product. This is a learnable channel-balanced weight that captures high-order multiplicative interactions between features. LayerNorm() represents the encoding operation.

[0117] The three-element fusion design enables the model to utilize both additive and multiplicative fusion modes simultaneously, significantly enhancing its ability to model complex multimodal feature relationships.

[0118] Figure 11 A flowchart illustrating a sample-level global feature generation scheme provided for at least one embodiment of this disclosure. Figure 3 , Figure 4 , Figure 5 , Figure 8 or Figure 9 Based on any of the proposed methods, to ensure the accuracy of crop density detection results, such as... Figure 11 As shown, step S50 further includes the following sub-steps S501-S506.

[0119] Sub-step S501: Arrange the area codes of each sub-region into the first sequence according to the forward direction of the harvester.

[0120] Sub-step S502: Obtain the location code of each sub-region and arrange it into a second sequence.

[0121] Sub-step S503: Overlay the first sequence and the second sequence to obtain a feature sequence with added positional information.

[0122] Sub-step S504: Input the feature sequence with added location information into the Transformer encoder to obtain the context-enhanced feature sequence.

[0123] Sub-step S505: Weight the context-enhanced feature sequence based on the channel attention mechanism.

[0124] Sub-step S506: Perform average pooling on the different elements of the weighted context-enhanced feature sequence to obtain sample-level global features.

[0125] In this process, sub-steps S501-S506 generate sample-level global features that effectively integrate the spatial location information, contextual dependencies, and key features of each sub-region, providing a comprehensive and representative global feature representation for accurate subsequent crop density prediction. These features not only preserve the sequential correlation along the harvester's forward direction but also capture long-distance dependencies between sub-regions through a Transformer encoder. Combined with a channel attention mechanism, they highlight feature channels that contribute significantly to density detection. Finally, the global features obtained through average pooling can be directly input into subsequent density regression or classification modules to accurately estimate the crop density of the entire detection area, thereby improving the robustness and detection accuracy of the lidar-based crop density detection system.

[0126] As an exemplary implementation, in the sample-level global feature generation stage of step S50, considering that each sub-region has a natural sequential relationship along the direction of the harvester's movement, all region codes are arranged into a first sequence according to their distance. :

[0127] The first sequence Superimposed second sequence consisting of learnable positional codes P The feature sequence with added location information is obtained. :

[0128] Subsequently, feature sequences incorporating location information will be added. Input a lightweight Transformer encoder, and use a multi-head self-attention structure to weight and gate the information flow between different sub-regions, outputting a context-enhanced feature sequence. .

[0129] To further achieve channel-level gating, a channel attention mechanism (SE structure) is introduced after the output of the lightweight Transformer encoder to first enhance the contextual feature sequence. The channel description vector is obtained by global aggregation at the region dimension, and then channel gating weights are generated by compression-excitation structure. :

[0130] In the formula, This represents the Sigmoid activation function. This indicates a global aggregation operation. Represents the ReLU activation function. Represents the weight parameters. It represents the intermediate feature vector, stores global information of the sub-region, and is used to calculate the feature channels.

[0131] The following formula is used to enhance the contextual feature sequence. Each region feature Perform channel-by-channel gating to obtain weighted context-enhanced features. : = ⊙

[0132] By performing channel-wise gating to suppress redundant dimensions for each region's features and amplifying the height statistics and penetration features most critical for density discrimination, the basic gating fusion mechanism is as follows: Figure 12 As shown.

[0133] Finally, global average pooling is performed on the reweighted features of all regions to obtain sample-level global features. :

[0134] global features at the sample level As input for subsequent density prediction, it enables a holistic perception of the continuous variation pattern of height distribution across multiple regions along the forward direction. The process of sequence modeling and sample-level global feature aggregation is as follows: Figure 13 As shown.

[0135] Figure 14 A flowchart illustrating a crop density prediction scheme provided for at least one embodiment of this disclosure. Figure 3 , Figure 4 , Figure 5, Figure 8 , Figure 9 and Figure 11 Based on any of the proposed methods, to ensure the accuracy of crop density detection results, such as... Figure 14 As shown, step S60 further includes the following sub-step S601.

[0136] Sub-step S601: Input the global features at the sample level of the prediction region into a preset multi-task model to simultaneously predict the discrete density level and continuous density estimate of the crop. The multi-task model is configured to convert the input global features at the sample level of the prediction region into the output crop density level through its first fully connected layer and the Softmax function, and to convert the input global features at the sample level of the prediction region into the output continuous density estimate through its second fully connected layer.

[0137] The multi-task synchronous prediction strategy in sub-step S601 fully leverages the complementary information between discrete density levels and continuous density estimates. Discrete density levels provide prior constraints on the categories of continuous estimates, preventing anomalous results from significant deviations from the actual categories. Meanwhile, continuous density estimates provide more refined numerical basis for the classification of discrete levels, improving the accuracy of level judgment. During model training, the multi-task model is optimized using a joint loss function, which is a weighted average of the cross-entropy loss from the discrete classification task and the mean squared error loss from the continuous regression task. By balancing the loss weights of the two tasks, the model achieves optimal performance in both categories. Furthermore, since the input sample-level global features integrate the height distribution and penetration characteristics of multiple regions along the forward direction, they encompass both global statistical information for distinguishing different density levels and retain local feature details reflecting subtle density changes. Therefore, they provide comprehensive and effective feature support for multi-task prediction, further ensuring the reliability and accuracy of crop density detection results.

[0138] As an exemplary implementation, in the crop density prediction stage of step S60, this disclosure adopts a multi-task structure of classification + regression: on the one hand, through the first fully connected layer and the Softmax function, global features are... g The probability distribution mapped to each crop density level One layer is used to output discrete crop density levels; on the other hand, a continuous density estimate is obtained through another fully connected layer. It is used to characterize fine-grained variations in crop density.

[0139] During training, the classification cross-entropy loss is used. With regression mean square error loss The weighted sum is used as the overall loss function. :

[0140] Joint optimization of classification and regression is achieved within a unified feature space. During the inference phase, the Dropout layer is retained and Monte Carlo Dropout is employed, performing multiple forward propagations on the same sample. Uncertainty is predicted based on the output variance metric, and high-confidence and low-confidence thresholds are set as decision gating conditions: high-confidence samples directly enter variable operation control, medium-confidence samples prompt operator attention, and low-confidence samples trigger manual review or repeated scanning. This constitutes an adaptive density mapping mechanism integrating regional sequence modeling, multi-scale feature gating fusion (basic gating + hierarchical attention enhancement), multi-task learning, and uncertainty gating, providing a complete solution for real-time crop density monitoring and variable operation control.

[0141] In some embodiments, Figure 3 , Figure 4 , Figure 5 , Figure 8 , Figure 9 , Figure 11 and Figure 12 Based on any of the schemes, in order to further ensure the acquisition of reliable real-time density information, the method also includes the following steps S70-S90, which can be set after step S60.

[0142] Step S70: Obtain the confidence level of the crop density detection results.

[0143] Step S80: In response to a confidence level higher than a preset first threshold, generate a first instruction for using discrete density levels and continuous density estimates as control variables for harvester operation.

[0144] Step S90: In response to the confidence level being lower than a preset second threshold, a second instruction is generated to trigger manual review or lidar rescanning, wherein the first threshold is greater than the second threshold.

[0145] The process, from steps S70 to S90, achieves dynamic confidence assessment and tiered response for crop density detection results. For medium-confidence samples with confidence levels between the first and second thresholds, a third instruction is generated to prompt the operator to focus on the crop growth status and density distribution in that area, assisting in semi-automated operational adjustments. This tiered response mechanism not only covers decision-making needs across high, medium, and low confidence levels but also effectively balances detection efficiency and result reliability. It ensures that the harvester can autonomously control variables in high-confidence areas during real-time operation, while also mitigating decision-making risks in low-confidence areas through manual intervention or rescanning. Furthermore, it leverages the operator's experience to optimize operational strategies in medium-confidence areas, ultimately forming a closed-loop adaptive density detection and operational control process. This provides more robust technical support for intelligent harvesting operations in precision agriculture.

[0146] In some embodiments, Figure 3 , Figure 4 , Figure 5 , Figure 8 , Figure 9 , Figure 11 and Figure 12 In order to ensure the effectiveness of feature extraction, based on any of the schemes, the method may also include the following step S21, which can be set between steps S20 and S30.

[0147] Step S21: Preprocess the 3D point cloud data in each sub-region to remove abnormal data in the 3D point cloud data, and use the preprocessed 3D point cloud data for feature extraction.

[0148] The abnormal data includes outliers caused by LiDAR hardware noise or measurement angle deviation, as well as misidentified points caused by interference such as birds or fallen leaves in the field. In addition to the solutions provided in later embodiments, the preprocessing process can also employ statistical filtering algorithms. By calculating the mean and standard deviation of the distance between each point and its neighbors, points with a mean distance exceeding a preset standard deviation range are identified as outliers and removed. Simultaneously, radius filtering is used to retain valid points whose number of neighbors meets a preset threshold, further removing sparse noise points. Furthermore, voxel grid downsampling can be used to simplify the preprocessed point cloud data, reducing data redundancy while preserving key features of the crop canopy.

[0149] In some embodiments, step S21 further includes the following sub-steps S211-S212.

[0150] Sub-step S211: For the 3D point cloud data in each sub-region, the point cloud data in the sub-region is first-level filtered based on the preset reasonable height range of the crop canopy to filter out the first type of abnormal data that does not conform to the reasonable height range of the crop canopy.

[0151] Sub-step S212: Perform a second-level filter on the point cloud data after the first-level filtering based on the quartiles and interquartile ranges of the point cloud height values ​​to filter out the second type of outlier data that belong to the outlier points.

[0152] The two-stage filtering mechanism, through sub-steps S211 and S212, effectively improves the quality and purity of point cloud data by selectively removing different types of outlier data layer by layer. Sub-step S211 first uses the reasonable height range of the crop canopy to quickly filter out interference points that are obviously too high (such as birds or high-altitude debris) or too low (such as stones or withered leaves on the ground), significantly reducing the amount of data processed in subsequent steps. Sub-step S212 then uses statistical methods to perform a refined analysis of the height value distribution, identifying and removing extreme outliers that exceed the normal distribution range, thus preventing individual outliers from distorting the true height characteristics of the crop canopy. This layered and progressive filtering method balances the efficiency of preprocessing with the accuracy of the filtering results, providing more reliable basic data support for the extraction of key features of the crop canopy in subsequent steps.

[0153] As an exemplary implementation, in the point cloud data preprocessing stage of step S21, the 3D point cloud data of each sub-region is preprocessed, and a two-layer filtering mechanism is used to remove outliers: the first layer of filtering sets a reasonable height range based on the crop type. The first layer retains the point cloud elements that meet the conditions; the second layer filters and calculates the quartiles of the point cloud height. (25th percentile) and (75th percentile), calculate the interquartile range Retain height value Point clouds within. k This is an adjustable parameter that can be adjusted according to the actual crop height; here it is set to 1.5. A dual-layer filtering mechanism effectively removes outlier data while retaining valid point clouds, improving the accuracy of subsequent calculations.

[0154] Figure 15 An example flowchart of a LiDAR-based crop density detection method provided for at least one embodiment of this disclosure. Figure 15As shown, this method acquires the point cloud of the crop ahead from a LiDAR scanner. After coordinate transformation and height calculation, the prediction area is determined based on the geometric relationship of the header and divided into multiple sub-regions along the forward direction. For each sub-region's point cloud, a two-layer anomaly filtering (reasonable height constraint + IQR outlier removal) is applied. Then, robust height intervals, mean / standard deviation / coefficient of variation, skewness / kurtosis, and other statistical features are extracted, along with height histogram distribution features, and optionally, working conditions are added to form a multi-dimensional feature vector. Subsequently, the multi-dimensional feature vector is subjected to basic gating adaptive fusion between statistical features and histogram features. Based on this, hierarchical attention interaction (self-attention enhancement → bidirectional cross-attention → three-term gating fusion) is introduced to obtain more discriminative region codes. Next, all region codes are arranged in distance order to form a sequence and added to the positional code. A lightweight Transformer is used to model the cross-regional order and contextual relationships, and SE channel attention is used to highlight key dimensions. Finally, global average pooling is used to obtain sample-level global features. The global feature input is a multi-task head that combines classification and regression, and it outputs discrete density levels and continuous density estimates. During inference, it uses MC Dropout to assess uncertainty and performs decision gating based on confidence levels, thereby achieving real-time and robust crop density detection and variable operation control.

[0155] The improvements of this scheme are as follows: 1) A method for determining the ROI and dividing the region into sub-regions along the forward direction for harvester-following detection, and a density representation approach based on the vertical distribution differences of point clouds in sub-regions; 2) A regional multi-dimensional height feature construction scheme and a two-layer anomaly filtering preprocessing mechanism consisting of "robust height interval features + height statistical discrete / shape features + multi-scale height histogram (which may include working conditions)"; 3) Explicit gating adaptive fusion between statistical branches and histogram branches, and a hierarchical attention interaction structure composed of "self-attention enhancement, bidirectional cross-attention interaction, and three-term gating fusion"; 4) A global aggregation method that combines regional encoding into a sequence based on distance and performs position encoding, lightweight Transformer sequence modeling, and SE channel gating, and an overall framework combining "classification + regression" multi-task prediction with MC Dropout uncertainty assessment and confidence decision gating. These key points together constitute a complete closed loop from point cloud acquisition, feature construction, cross-feature fusion, regional sequence modeling to safe output control.

[0156] The advantages of this scheme are: 1) Compared to methods requiring manual sampling during downtime or relying on prior remote sensing inversion, this scheme significantly improves the timeliness and operational continuity of crop density information acquisition, does not rely on illumination and spectral indices, broadens applicable scenarios, and reduces model migration costs, providing feedforward basis for real-time adjustment of parameters such as combine harvester feed rate and travel speed; 2) This scheme utilizes the vertical distribution characteristics of radar point clouds, constructs ROI prediction regions and divides them into multiple sub-regions, and performs zonal modeling of crop three-dimensional spatial density at scales close to the actual cutting width and sub-cutting widths; 3) This scheme introduces "reasonable height constraint + IQR" in point cloud preprocessing. The dual-layer anomaly removal mechanism of "quartile filtering" effectively improves the quality of effective point clouds; at the same time, it uses 5%–95% quantiles to construct a robust height range and introduces statistical quantities such as coefficient of variation, skewness, and kurtosis to robustly represent the vertical structure of crops without relying on extreme values; 4) This scheme combines statistical features and height histogram features with working conditions to form multi-source inputs. Through a basic gating mechanism, it adaptively weights "statistical features" and "distribution features" to automatically adjust the feature emphasis. It also introduces a hierarchical attention interaction structure of "self-attention enhancement - bidirectional cross-attention - three-term gating fusion" to fully explore the internal dependencies of features and the complementary relationships across features, greatly enhancing the ability to model complex point cloud distribution patterns; 5) This scheme enhances the overall perception of the continuous spatial changes of crop density by performing sequential modeling of forward sub-region features and combining SE channel attention, while ensuring onboard real-time performance. It also adopts a "classification + regression" multi-task structure combined with MC Dropout uncertainty assessment and confidence gating to achieve more reliable feeding control decisions and operational safety assurance.

[0157] Figure 16 This is a structural block diagram of a LiDAR-based crop density detection system provided for at least one embodiment of the present disclosure. Figure 16 As shown, the crop density detection system 10 based on lidar includes an acquisition unit 11, a first-level processing unit 12, a second-level processing unit 13, and a third-level processing unit 14.

[0158] Acquisition unit 11 is configured to acquire the current three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar.

[0159] The first-level processing unit 12 is configured to determine the predicted harvesting area of ​​the harvester in the three-dimensional point cloud data based on the geometry of the harvester header, and to divide the predicted area into multiple sub-regions along the direction of the harvester's movement.

[0160] The second-level processing unit 13 is configured to extract height distribution features from the 3D point cloud data in each sub-region, construct a multi-dimensional feature vector for each sub-region, and perform adaptive fusion processing on the different features of the multi-dimensional feature vector of each sub-region based on an attention mechanism to generate a region code for each sub-region.

[0161] The third-level processing unit 14 is configured to obtain global features at the sample level of the prediction region based on the region encoding and corresponding position encoding of each sub-region, and simultaneously predict the discrete density level and continuous density estimate of the crop based on the global features at the sample level of the prediction region, and output the crop density detection result.

[0162] The specific execution methods of each unit in the above system embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0163] In some embodiments, Figure 16 Based on this, the acquisition unit 11 can be implemented by a corresponding signal receiving device, and the first-level processing unit 12, the second-level processing unit 13 and the third-level processing unit 14 can be implemented by a controller or control module with corresponding programs.

[0164] This disclosure also provides a storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method embodiments described above.

[0165] This disclosure also provides a program product, such as... Figure 17 As shown, the program product includes one or more processors 21 and memory 22. Figure 17 Take a processor 21 as an example.

[0166] The controller may also include an input device 23 and an output device 24.

[0167] The processor 21, memory 22, input device 23, and output device 24 can be connected via a bus or other means. Figure 17 Taking the example of a connection between China and Israel via a bus.

[0168] The processor 21 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The general-purpose processor can be a microprocessor or any conventional processor.

[0169] The memory 22, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 21 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 22, thereby implementing the steps of the above-described method embodiments.

[0170] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the processing device operated by the server. Furthermore, the memory 22 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 22 may optionally include memory remotely located relative to the processor 21, and these remote memories can be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. One or more modules are stored in the memory 22 and, when executed by one or more processors 21, perform functions such as... Figure 1 The method shown.

[0171] Input device 23 can receive input digital or character information, and generate key signal inputs related to driver settings and function control of the server's processing unit. Output device 24 may include display devices such as a display screen.

[0172] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory (FM), hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0173] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

[0174] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A crop density detection method based on lidar, applied to a harvester, characterized in that, include: Acquire the current 3D point cloud data of the crop area in front of the harvester, collected by lidar; Based on the geometry of the harvester header, the predicted harvesting area in the three-dimensional point cloud data is determined, and the predicted area is divided into multiple sub-regions along the direction of the harvester's movement. Height distribution features are extracted from the 3D point cloud data of each sub-region to construct a multi-dimensional feature vector for each sub-region; The different features of the multidimensional feature vector of each sub-region are adaptively fused based on an attention mechanism to generate the region code of each sub-region; Based on the region code and corresponding position code of each sub-region, global features at the sample level of the prediction region are obtained. as well as, Based on the global features at the sample level of the predicted region, the discrete density level and continuous density estimate of the crop are predicted synchronously and output as the crop density detection result.

2. The method according to claim 1, characterized in that, Also includes: Obtain the confidence level of the crop density detection results; In response to the confidence level being higher than a preset first threshold, a first instruction is generated to use the discrete density level and the continuous density estimate as harvester operation control variables; as well as, In response to the confidence level being lower than a preset second threshold, a second instruction is generated to trigger manual review or lidar rescanning, wherein the first threshold is greater than the second threshold.

3. The method according to claim 1 or 2, characterized in that, Before extracting the height distribution features of the 3D point cloud data in each sub-region, the method further includes: The 3D point cloud data in each sub-region is preprocessed to remove abnormal data, and the preprocessed 3D point cloud data is used for feature extraction.

4. The method according to claim 1 or 2, characterized in that, The lidar is positioned at a fixed location in front of the harvester, and the acquisition of the current three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar includes: The lidar is controlled to perform a vertical scan of the crop area in front of the harvester; Acquire the initial three-dimensional point cloud data of the crop area in front of the harvester, collected by the lidar through the vertical scan; The initial 3D point cloud data is transformed from the lidar coordinate system to the harvester's vehicle coordinate system; and, After coordinate system transformation, the laser ranging value of each position coordinate in the initial three-dimensional point cloud data is converted into the crop height value relative to the ground to generate the final three-dimensional point cloud data. The three-dimensional point cloud data contains multiple position coordinates and the crop height value relative to the ground for each position coordinate.

5. The method according to claim 1 or 2, characterized in that, The determination of the predicted harvesting area in the three-dimensional point cloud data based on the geometry of the harvester header includes: Obtain a straight line parallel to the harvester's header and at a predetermined distance from the front of the harvester as the basic boundary; and, Based on the basic boundary and the installation position and scanning angle of the lidar, a rectangular area is determined in the vehicle coordinate system as the prediction area.

6. The method according to claim 3, characterized in that, The preprocessing of the 3D point cloud data in each sub-region includes: For the 3D point cloud data within each sub-region, a first-level screening is performed on the 3D point cloud data of the sub-region based on a preset reasonable height range for the crop canopy, to filter out first-type abnormal data that does not conform to the reasonable height range for the crop canopy; and, A second-level filtering is performed on the point cloud data after the first-level filtering based on the quartiles and interquartile ranges of the point cloud height values, in order to filter out the second type of outlier data that are outliers.

7. The method according to claim 1 or 2, characterized in that, The process of extracting height distribution features from the 3D point cloud data within each sub-region and constructing a multi-dimensional feature vector for each sub-region includes: For each sub-region, the 3D point cloud data is statistically analyzed to obtain multiple statistical features of the sub-region, and a statistical feature vector of the sub-region is constructed. The multiple statistical features include: Robust height range characteristics, which characterize the penetration of laser light into the crop canopy in the sub-region. The height statistical discrete feature is used to characterize the degree of dispersion of crop height values ​​relative to the ground in the sub-region, and Height distribution shape features, which are used to characterize the clustering pattern of crop height values ​​relative to the ground in the sub-region; Based on the point cloud height value of each sub-region, the sub-region is divided into multiple sub-intervals. The number of point clouds in each sub-interval is counted and normalized. A height histogram feature vector is constructed for each sub-region, containing the proportion of point clouds in each sub-interval. The point cloud height value is the maximum value of the crop's height relative to the ground in the corresponding sub-region. Construct a multidimensional feature vector for each sub-region, wherein the multidimensional feature vector includes the statistical feature vector and the height histogram feature vector.

8. The method according to claim 7, characterized in that, The robust height interval feature is related to the difference between the 95th percentile and the 5th percentile of the point cloud height values ​​in the corresponding sub-region; the height statistical dispersion feature includes at least one of the mean point cloud height, standard deviation of point cloud height, and coefficient of variation of point cloud height in the corresponding sub-region; and the height distribution shape feature includes at least one of the skewness and kurtosis of point cloud height in the corresponding sub-region. The step of extracting height distribution features from the 3D point cloud data within each sub-region and constructing a multi-dimensional feature vector for each sub-region also includes: Obtain multiple operating condition parameters for the current operating condition, and construct an operating condition parameter vector containing the multiple operating condition parameters; and, The operating condition parameter vector is incorporated into the multidimensional feature vector of each sub-region, so that the multidimensional feature vector of each sub-region includes the statistical feature vector, the height histogram feature vector, and the operating condition parameter vector. The adaptive fusion process of different features of the multidimensional feature vector of each sub-region based on an attention mechanism includes: For each sub-region, the statistical feature vector and the height histogram feature vector in its multidimensional feature vector are encoded respectively to generate the first encoded feature corresponding to the statistical feature vector and the second encoded feature corresponding to the height histogram feature vector; Adaptive weighting is applied to the first coding feature and the second coding feature to generate enhanced first coding feature and enhanced second coding feature; Construct a bidirectional cross-attention interaction network related to the enhanced first coding feature and the enhanced second coding feature, such that one of the enhanced first coding feature and the enhanced second coding feature can extract complementary information from the other; and, The enhanced first coding feature, the enhanced second coding feature, and the complementary information in the bidirectional cross-attention interaction network are fused to generate the region code of the sub-region; The process of obtaining global features at the sample level for the predicted region based on the region encoding and corresponding position encoding of each sub-region includes: Arrange the area codes of each sub-region into the first sequence according to the direction of the harvester's movement; Obtain the location code of each sub-region and arrange them into a second sequence; The first sequence and the second sequence are superimposed to obtain a feature sequence with added positional information; The feature sequence with added location information is input into the Transformer encoder to obtain the context-enhanced feature sequence. The context-enhanced feature sequence is weighted based on a channel attention mechanism; and... The different elements of the weighted context-enhanced feature sequence are subjected to average pooling to obtain sample-level global features. The method of synchronously predicting the discrete density level and continuous density estimate of crops based on the global features at the sample level of the prediction region includes: The global features at the sample level of the prediction region are input into a preset multi-task model to simultaneously predict the discrete density level and continuous density estimate of the crop. The multi-task model is configured to convert the input global features at the sample level of the prediction region into the output crop density level through its first fully connected layer and Softmax function, and to convert the input global features at the sample level of the prediction region into the output continuous density estimate through its second fully connected layer.

9. A crop density detection system based on lidar, applied to a harvester, characterized in that, include: The acquisition unit is configured to acquire the current three-dimensional point cloud data of the crop area in front of the harvester collected by the lidar. The first-level processing unit is configured to determine the predicted harvesting area of ​​the harvester in the three-dimensional point cloud data based on the geometry of the harvester header, and to divide the predicted area into multiple sub-regions along the direction of the harvester's movement. The second-level processing unit is configured to extract height distribution features from the 3D point cloud data in each sub-region, construct a multi-dimensional feature vector for each sub-region, and perform adaptive fusion processing on the different features of the multi-dimensional feature vector of each sub-region based on an attention mechanism to generate a region code for each sub-region. as well as, The third-level processing unit is configured to obtain global features at the sample level of the prediction region based on the regional encoding and corresponding position encoding of each sub-region, and simultaneously predict the discrete density level and continuous density estimate of the crop based on the global features at the sample level of the prediction region, and output the crop density detection result.

10. A storage medium, characterized in that, The storage medium stores a program or instructions, wherein the program or instructions, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 8.

Images