A real-time detection method and system for the direction of a vehicle-mounted watered rice

Abstract

The present application provides a kind of real-time detection method and system for the direction of vehicle-mounted lodging rice, obtains the RGB image of the rice area in front of harvester;The region of interest of the RGB image is cut, and the image after cutting is normalized to the square image of the required size of network input;The square image is calculated, and the global dominant lodging direction angle is extracted as physical prior information;A deep neural network based on visual state space model is constructed, the square image is sequenced as input, and the physical prior information is injected into the deep network, the network is guided to pay attention to the lodging texture features by physical prior, and after extraction and aggregation by network forward propagation, global features are obtained;Global features are regressed and predicted, and continuous lodging angle values are output.The present application uses structure tensor to extract physical prior, constructs lightweight regression network combined with visual state space model, and models the lodging direction detection as a continuous regression task, so as to realize high-precision, low-delay lodging direction perception.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agricultural machinery, and in particular relates to a vehicle-mounted method and system for real-time detection of the direction of lodged rice. Background Technology

[0002] Rice is a vital global food crop, and during its later growth stages, it is frequently susceptible to lodging due to strong winds, heavy rains, or pests and diseases. Rice lodging not only affects yield and quality but also poses a significant challenge to mechanized harvesting. During combine harvester operations, the tangled stalks of lodged rice easily entangle critical components such as the reel and header, leading to poor feeding, incomplete threshing, and even machine blockages and shutdowns, severely reducing operational efficiency. To achieve efficient and low-loss harvesting of lodged rice, harvesters need to adjust operating parameters such as the reel height, speed, and spring angle in real time according to the lodging direction. For example, for unidirectional lodging (where the lodging direction is the same as the harvester's travel direction), it may be necessary to reduce the reel speed and height; while for inverse lodging, the speed needs to be increased to assist in straightening the rice. Currently, this adjustment mainly relies on manual operation based on the driver's experience, which is labor-intensive and difficult to ensure consistency. In recent years, intelligent sensing technology based on computer vision has been increasingly applied to agricultural machinery. However, existing research on lodging detection mainly focuses on the identification and segmentation of lodged areas, with relatively little research on lodging direction detection. Most existing methods treat lodging direction detection as a classification task (e.g., dividing directions into finite categories such as east, south, west, and north). This discrete classification method has significant limitations: first, large quantization errors prevent it from providing the continuous angle values ​​required for precise harvester control; second, blurred boundaries lead to abrupt changes at category boundaries; and third, traditional convolutional neural networks are limited by local receptive fields, making it difficult to capture long-distance lodging texture dependencies, while visual Transformers, although possessing global modeling capabilities, have high computational overhead, making real-time operation difficult on computationally limited vehicle-mounted edge devices.

[0003] Currently, there are reports on research on lodging direction detection in rice. Patents CN114972520B "An Online Detection Method for Lodging Direction Information of Rice", CN112287787B "A Crop Lodging Grading Method Based on Gradient Histogram Features", and CN115953690B "A Lodging Crop Recognition Method for Unmanned Harvester Travel Calibration" all adopt traditional image processing methods, which are easily affected by factors such as light and are difficult to adapt to the complex and ever-changing lodging states of crops in the field, resulting in low robustness.

[0004] Therefore, there is an urgent need for a method that can overcome the above-mentioned defects, achieve continuous, accurate, and adaptable real-time lodging direction detection of rice, and be adapted to the vehicle-mounted edge computing environment. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a real-time detection method for the direction of lodged rice in a vehicle-mounted system. This method utilizes structural tensors to extract physical priors, combines them with a visual state-space model to construct a lightweight regression network, and models lodging direction detection as a continuous regression task. This achieves high-precision, low-latency lodging direction perception, providing a reliable basis for the adaptive control of unmanned harvesters.

[0006] Another objective of this invention is to provide a vehicle-mounted real-time lodging direction detection system for rice, which achieves real-time online monitoring of lodging direction through a combination of hardware and software, providing reliable data support for intelligent control of harvesters.

[0007] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0008] A method for real-time detection of lodged rice direction on a vehicle-mounted platform includes the following steps:

[0009] Step S1: Image Acquisition: Acquire RGB images of the rice paddy area in front of the combine harvester from a camera mounted on the combine harvester;

[0010] Step S2: Image preprocessing: Crop the region of interest in the RGB image described in step S1, and normalize the cropped image into a square image of the required size for network input;

[0011] Step S3: Physical Prior Extraction: The structural tensor algorithm is used to calculate the square image described in step S2 and extract the global dominant collapse direction angle as physical prior information;

[0012] Step S4: Feature extraction and global feature aggregation: Construct a deep neural network based on a visual state space model, serialize the square image and use it as input, and inject the physical prior information from step S3 into the deep layer of the network. The physical prior guides the network to focus on the overturned texture features. After the network forward propagates, the global features are extracted and aggregated.

[0013] Step S5: Output Results: The network output layer performs regression prediction on the global features described in step S4 and outputs continuous collapse angle values.

[0014] In the above scheme, step S2, which normalizes the cropped image into a square image, specifically includes:

[0015] Step S2.1: Obtain the inscribed rectangle of the fallen area as the region of interest (ROI), and place the cropped image in the center of the square canvas;

[0016] Step S2.2: Construct a binary mask based on the blank area of ​​the canvas, the binary mask being used to identify the area that needs texture completion;

[0017] Step S2.3: Employ an image inpainting algorithm based on the fast traversal method, and use the binary mask to complete and extend the texture of the blank edges to generate a square normalized image;

[0018] Step S2.4: Scale the square normalized image to the specific resolution required for the network input.

[0019] In the above scheme, the specific steps of step S3, physical prior extraction, include:

[0020] Step S3.1: Convert the RGB channels of the square image to grayscale, and calculate the horizontal gradient I using a texture detection operator. x and vertical gradient I y ;

[0021] Step S3.2: Construct the structure tensor matrix S0, and perform Gaussian smoothing on the components of S0 to obtain the smoothed structure tensor matrix S. σ ;

[0022] Step S3.3: For S σ Perform eigenvalue decomposition to calculate the local collapse direction for each pixel. and consistency indicators ;

[0023] Step S3.4: Based on consistency metrics The global dominant fall direction angle is obtained by weighting the local orientations of all pixels and aggregating the values. .

[0024] In the above scheme, the deep neural network in step S4 includes a staged feature extraction structure, specifically including:

[0025] Step S4.1: Use the standard visual state space module to extract low-level visual features of the image and establish preliminary global context association;

[0026] Step S4.2: Using the orientation-guided visual state space module, the physical prior information extracted in step S3 is introduced to perform orientation modulation and multi-directional scanning correction on the features.

[0027] Furthermore, the direction-guided visual state space module includes a direction modulation mechanism, the specific steps of which include:

[0028] Step S5.1: Transform the scalar physical prior orientation angle using a direction encoder. Mapping to semantic embeddings generates a channel attention mask G;

[0029] Step S5.2: Perform element-wise modulation operation on the input feature X using the channel attention mask G. This enhances texture response consistent with prior directions and suppresses background noise.

[0030] Furthermore, the direction-guided visual state space module includes a multi-directional scanning mechanism, the specific steps of which include:

[0031] Step S6.1: Construct a structure including the main direction The scan set of its offset angle , where δ is the preset offset;

[0032] Step S6.2: The network performs a geometric scan of the feature map along three directions in the set, captures the texture changes in the neighborhood of the main direction, and weights and fuses the multi-view features into the main feature stream in the form of residual correction.

[0033] In the above scheme, the deep neural network uses a recurrent loss function based on the geometric properties of a unit circle during the training phase to predict angles. and the real perspective Mapping to points on the unit circle, the central angle corresponding to the shortest arc length between two points is calculated as the cycle error d, and the calculation formula is as follows:

[0034]

[0035] Among them, the prediction angle and the real perspective The domain of all values ​​is [0°, 360°); the total loss function is composed of the weighted sum of the cyclic mean square error and the cyclic mean absolute error.

[0036] In the above scheme, in step S5, the network output layer aggregates the feature sequence through a global average pooling layer, and then passes through a linear layer and an activation function to output a continuous predicted value of the overturning angle in the real number domain.

[0037] The above scheme further includes performing half-precision quantization and inference engine optimization on the trained deep neural network model for deployment on an in-vehicle edge computing platform.

[0038] A detection system for implementing the vehicle-mounted real-time detection method for lodged rice direction includes an image acquisition module, a preprocessing module, a priori extraction module, a network inference module, and a result output module.

[0039] The image acquisition module is used to acquire RGB images of the rice field in front of the harvester;

[0040] The preprocessing module is used to crop and normalize the acquired RGB image to the region of interest, and normalize the cropped image into a square image of the required size for network input.

[0041] The prior extraction module is used to calculate the square image using the structure tensor algorithm and extract the global dominant collapse direction angle as physical prior information;

[0042] The network inference module is internally deployed with a deep neural network of a pre-trained physical prior-guided visual state space model. The serialized square image is used as input, and the physical prior information is injected into the deep layers of the network. The physical prior guides the network to focus on the overturned texture features. After the network forward propagates, the features are extracted and aggregated to obtain the global features.

[0043] The result output module is used to perform regression prediction on the global features and output continuous collapse angle values.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] 1. This invention abandons the traditional technical paradigm of treating lodging direction detection as a discrete classification task, and innovatively defines it as a continuous regression problem, introducing a cyclic loss function based on the geometric characteristics of the unit circle. This loss function effectively solves the periodic boundary jump problem at the intersection of 0° and 360° in angle regression by mapping the angle to the unit circle and calculating the shortest arc length error, completely eliminating quantization error, significantly improving the accuracy and continuity of lodging direction detection, and providing high-precision angle input for the adaptive control of harvesters.

[0046] 2. This invention innovatively uses the globally dominant lodging direction angle extracted by the structural tensor algorithm as physical prior information and explicitly injects it into a deep neural network through a direction modulation mechanism. This mechanism utilizes a direction encoder to generate a channel attention mask, modulating the input features element-wise, thereby effectively enhancing the texture response consistent with the lodging direction and suppressing background noise in complex farmland environments (such as changes in lighting and weed interference). This synergistic strategy of "physical guidance + depth correction" overcomes the blindness of purely data-driven models in complex scenes and significantly improves the robustness and generalization ability of the model.

[0047] 3. This invention employs a visual state-space model with linear computational complexity as the backbone network, significantly reducing computational overhead while maintaining global receptive field modeling capabilities. Combined with FP16 half-precision quantization optimization of the TensorRT inference engine, the model achieves high frame rate real-time inference of 18 FPS on computationally limited vehicle-mounted edge computing platforms (such as NVIDIA Jetson Xavier NX) with only 7.72M parameters, fully meeting the stringent real-time requirements of combine harvester operations.

[0048] 4. To address the complex morphology of lodged rice, which often exhibits a fan-shaped or wavy spread, this invention designs a multi-directional scanning mechanism within the direction-guided visual state space module. By constructing a scan set containing the main direction and its offset angle, the network rescans the feature map along three specific geometric paths, capturing divergent texture details within the neighborhood of the main direction, and then weighted and fused into the main feature flow as residuals. This mechanism significantly enhances the model's adaptability to complex lodging morphologies and its ability to represent texture details, further improving detection accuracy.

[0049] 5. This invention achieves high-precision, robust, and low-latency real-time detection of lodged rice orientation through a technical approach of "physical prior guidance + lightweight deep network + continuous regression modeling," providing reliable technical support for the adaptive intelligent harvesting of unmanned harvesters. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for real-time detection of lodged rice direction on a vehicle according to an embodiment of the present invention.

[0051] Figure 2 This is a schematic diagram of the image preprocessing and edge-aware restoration process according to an embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram of the physical prior extraction process based on structure tensor according to an embodiment of the present invention.

[0053] Figure 4 This is an overall architecture diagram of the deep neural network constructed in this invention.

[0054] Figure 5 This is a schematic diagram of the structure of a directional guidance visual state space module according to an embodiment of the present invention.

[0055] Figure 6 This is a schematic diagram of the geometric principle of the cyclic loss function according to one embodiment of the present invention. Detailed Implementation

[0056] The embodiments of the present invention are described in detail below. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0057] Figure 1 The above describes a preferred embodiment of the vehicle-mounted real-time detection method for the direction of lodged rice, which includes the following steps:

[0058] Step S1: Image Acquisition: Acquire RGB images of the rice paddy area in front of the combine harvester from a camera mounted on the combine harvester;

[0059] Step S2: Image preprocessing: The RGB image described in step S1 is cropped into a region of interest (ROI), and an edge-aware restoration strategy is used to normalize the cropped image into a square image of the required size for network input, thereby eliminating gradient discontinuities at the image edges;

[0060] Step S3: Physical Prior Extraction: The structural tensor algorithm is used to calculate the square image described in step S2 and extract the global dominant collapse direction angle as physical prior information;

[0061] Step S4: Feature Extraction and Global Feature Aggregation: Construct a deep neural network based on a visual state space model. The serialized square image is used as input, and the physical prior information from step S3 is injected into the deep layers of the network. This physical prior guides the network to focus on lodging texture features. After forward propagation, global features are extracted and aggregated to obtain the global features. The deep neural network based on the visual state space model uses the existing open-source Vision Mamba architecture as its backbone and makes structural improvements for agricultural applications. It adopts a phased feature extraction strategy: the shallow network uses a standard visual state space module to extract general visual features, while the deep network introduces a direction-guided visual state space module. Direction modulation and multi-directional scanning mechanisms enhance the perception of lodging textures. The serialized square image obtained in step S2 is input into the network, and the physical prior information extracted in step S3 is injected into the deep improvement module to guide the network to focus on texture features related to the lodging direction.

[0062] Step S5: Output Results: The network output layer performs regression prediction on the global features described in step S4 and outputs continuous lodging angle values ​​to achieve real-time detection of the lodging direction of rice.

[0063] The specific operations of step S2, image preprocessing and dataset construction, include:

[0064] Step S2.1: Obtain the inscribed rectangle of the collapsed area as the ROI, and place the cropped image in the center of the square canvas;

[0065] Step S2.2: Construct a binary mask based on the blank areas of the canvas;

[0066] Step S2.3: The Telea image inpainting algorithm based on the fast traversal method is used to perform edge-aware texture completion and extension on the blank edges to maintain the directional continuity of the fallen texture and generate a square normalized image;

[0067] Step S2.4: Scale the square image uniformly to the specific resolution required by the network input.

[0068] The specific steps of physical prior extraction in step S3 include:

[0069] Step S3.1: Convert the entire RGB channel image of the square image generated after preprocessing in step S2 into a grayscale image, and calculate the horizontal gradient I using the texture detection operator. x and vertical gradient I y ;

[0070] Step S3.2: Construct the structure tensor matrix S0, and perform Gaussian smoothing on the components of S0 to obtain the smoothed structure tensor matrix S. σ ;

[0071] Step S3.3: For S σ Perform eigenvalue decomposition to calculate the local collapse direction for each pixel. and consistency indicators ;

[0072] Step S3.4: Based on consistency metrics The global dominant fall direction angle is obtained by weighting the local orientations of all pixels and aggregating the values. .

[0073] The deep neural network constructed in step S4 includes a staged feature extraction structure, and the specific steps include:

[0074] Step S4.1: Use the standard visual state space module to extract low-level visual features of the image and establish preliminary global context association;

[0075] Step S4.2: Using the orientation-guided visual state space module, the physical prior information extracted in step S3 is introduced to perform orientation modulation and multi-directional scanning correction on the features.

[0076] The direction-guided visual state space module includes a direction modulation mechanism, and the specific steps include:

[0077] Step S5.1: The scalar physical prior orientation angle is converted into an orientation encoder (specifically composed of a multi-layer cascaded structure of Linear-ReLU-Linear-Tanh). The mapping is a high-dimensional semantic embedding consistent with the number of input feature channels, and then a channel attention mask G is generated through the Sigmoid activation function;

[0078] Step S5.2: Perform element-wise modulation operation on the input feature X using the channel attention mask G. This enhances texture response consistent with prior orientation and suppresses background noise.

[0079] The directional guidance visual state space module includes a multi-directional scanning mechanism, with specific steps including:

[0080] Step S6.1: Construct a structure including the main direction The scan set of its offset angle Where δ is a preset offset; in a specific embodiment of the present invention, δ is 45°. ;

[0081] Step S6.2: The network performs a geometric scan of the feature map along three directions in the set, captures the texture changes in the neighborhood of the main direction, and weights and fuses the multi-view features into the main feature stream in the form of residual correction.

[0082] The deep neural network employs a cyclic loss function based on the geometric properties of a unit circle during the training phase. This function maps the predicted and true angles to points on the unit circle. The domains of both the predicted angle y* and the true angle y are set to [0°, 360°). The central angle corresponding to the shortest arc length between the two points is calculated as the cyclic error d, using the following formula:

[0083]

[0084] Specifically, the 360° modular operation effectively eliminates the equivalent numerical jump at the boundary between 0° and 360°. The total loss function is composed of the weighted sum of the cyclic mean square error (with a weight of 0.7) and the cyclic mean absolute error (with a weight of 0.3).

[0085] In step S5, the network output layer aggregates the feature sequence through a global average pooling layer, and then passes it through a linear layer and an activation function to output a continuous predicted value of the overturning angle in the real number domain.

[0086] The method further includes performing half-precision quantization and inference engine optimization on the trained deep neural network model for deployment on an in-vehicle edge computing platform. In one specific embodiment of the invention, the method further includes using the TensorRT inference engine to perform FP16 half-precision quantization optimization on the network model, while maintaining an extremely low number of model parameters (e.g., 7.72M), enabling it to achieve high frame rate (e.g., 18 FPS) real-time inference on a computationally limited in-vehicle edge computing platform (e.g., NVIDIA Jetson Xavier NX), meeting the adaptive control requirements of combine harvester operations.

[0087] A detection system for implementing the vehicle-mounted real-time detection method for lodged rice direction includes an image acquisition module, a preprocessing module, a priori extraction module, a network inference module, and a result output module.

[0088] The image acquisition module is used to acquire RGB images of the rice field in front of the harvester;

[0089] The preprocessing module is used to crop and normalize the acquired RGB image to the region of interest, and normalize the cropped image into a square image of the required size for network input.

[0090] The prior extraction module is used to calculate the square image using the structure tensor algorithm and extract the global dominant collapse direction angle as physical prior information;

[0091] The network inference module is internally deployed with a deep neural network of a pre-trained physical prior-guided visual state space model. The serialized square image is used as input, and the physical prior information is injected into the deep layers of the network. The physical prior guides the network to focus on the overturned texture features. After the network forward propagates, the features are extracted and aggregated to obtain the global features.

[0092] The result output module is used to perform regression prediction on the global features and output continuous collapse angle values.

[0093] Specifically:

[0094] The vehicle-mounted real-time detection method for the direction of lodged rice includes the following steps:

[0095] Step S1: Image Acquisition: Install a binocular or monocular industrial camera (e.g., ZED2i) on top of the combine harvester cab, with the camera's top-down angle set to approximately 30° to cover the working area in front of the harvester. Acquire RGB images containing lodged rice in real time as the harvester moves.

[0096] Step S2: Image Preprocessing: Since the collapsed area is usually an irregular shape in the image, directly stretching it to a square will cause texture distortion. This embodiment adopts an edge-aware inpainting strategy: First, the collapsed area is obtained through semantic segmentation or other means, and its inscribed rectangle is cropped as the region of interest (ROI); a square canvas is created, and the cropped ROI image is placed in the center; the Telea algorithm in OpenCV (based on fast traversal) is used to perform texture inpainting and extension on the blank areas around the ROI, such as... Figure 2 As shown, this not only preserves the original texture proportions but also eliminates the gradient abruptness caused by filling the black border, ensuring the continuity of subsequent angular feature extraction; the repaired square image is uniformly scaled to 224×224 pixels and used as network input.

[0097] Step S3: Physical Prior Extraction: Convert the RGB channels of the 224×224 square network input image generated after preprocessing in Step S2 into grayscale images, and calculate the horizontal gradient I using the Sobel operator. x and vertical gradient I y Structure tensor construction: for each pixel Construct the structure tensor matrix S0, and then apply Gaussian smoothing to its components to obtain the smoothed structure tensor matrix S. σ To enhance noise immunity; for S σ Perform eigenvalue decomposition to calculate the local collapse direction for each pixel. and consistency indicators , A larger value indicates a stronger texture directionality at that location; based on consistency metrics The global dominant fall direction angle is obtained by weighting the local orientations of all pixels and aggregating the values. As a physical prior input to the subsequent network, such as Figure 3 As shown.

[0098] Step S4: Feature Extraction and Global Feature Aggregation: Construct a lightweight network with an improved visual state space architecture, such as... Figure 4 As shown. The basic framework of this network is based on the existing general Vision Mamba model, with structural improvements made for lodging features. The network input consists of a 3×224×224 image and a scalar prior. The image is segmented into 16×16 non-overlapping blocks using an image patching module (specifically, a 2D convolutional layer with a kernel size and stride of 16). These blocks are then mapped and flattened into a feature sequence containing 196 visual markers. Learnable positional encoding is then added to preserve the spatial topology of the farmland image, and this sequence is fed into a 6-layer core backbone network. To balance general feature extraction with orientation awareness, the network is improved to a staged extraction structure: The first stage (shallow layers, i.e., the first 3 layers): uses the standard visual state space module. This module includes deep convolutions and the existing selective visual state space core (Visual SSM Core), primarily utilizing its long sequence modeling capabilities to extract general low-level features such as edges and colors. The second stage (deep layers, i.e., the last 3 layers): uses a structurally improved direction-guided visual state space module (Direction-Guided Mamba Block). Figure 5 As shown, this module introduces two key mechanisms—directional modulation and multi-directional scanning—on top of the standard architecture.

[0099] The structural improvements to the Direction-Guided Mamba Block specifically include the following modifications:

[0100] 1) A directional modulation structure is added at the module input front end: The standard module directly feeds image features into the selective scanning core, while this improved module introduces a directional encoder in parallel before that. It maps the scalar physical prior angle of the external input to a channel attention mask G, and performs element-wise multiplication modulation operation with the input feature X, thereby adaptively enhancing the texture response consistent with the prior direction and suppressing background noise before the feature enters the core layer of the visual state space;

[0101] 2) At the end of the module output, a multi-directional scanning residual correction structure is added: the standard module ends directly after output projection, while this improved module adds a multi-view geometric scanning branch at the output. This branch constructs a scan set containing the main direction and its offset angle. The network rescans the feature map along these three specific geometric paths, capturing the divergent texture details in the neighborhood of the main direction, and weights them into the main feature flow with empirical weights (e.g., 0.1) as residual terms, thereby greatly enhancing the model's adaptability and representation ability to complex overturned morphologies.

[0102] Step S5: Output Results: The network's output features, after global average pooling, are fed into the regression head to output the predicted angle. Then, the loss is calculated using a cyclic loss function. The total loss function is a weighted sum of the cyclic mean squared error and the cyclic mean absolute error, which ensures both fast convergence and improved robustness to outliers. For example... Figure 6 As shown, the angle is mapped onto the unit circle, and the shortest arc length error d is calculated:

[0103]

[0104] The training process of the deep neural network in this embodiment is as follows: The deep neural network is constructed and trained using the PyTorch deep learning framework. During the training phase, the AdamW optimizer is used to update the network parameters. The initial learning rate is set to 8e-5, and a cosine annealing strategy is used to gradually decrease the learning rate to 1e-7. The training batch size is set to 12. An early stopping mechanism is introduced, with a tolerance of 20 epochs and a minimum improvement threshold of 0.05°. During training, a preprocessed square image is serialized and input into the network. The error is calculated using the recurrent loss function and backpropagated until the model converges.

[0105] The detection system of the vehicle-mounted real-time detection method for lodged rice direction includes:

[0106] The image acquisition module includes a ZED 2i camera.

[0107] The preprocessing module is built into an in-vehicle embedded computing platform (such as the NVIDIA Jetson series) and uses OpenCV to perform image cropping and restoration.

[0108] The prior extraction module: a structure tensor algorithm implemented in C++;

[0109] The network inference module converts the trained PyTorch model into a TensorRT engine and enables FP16 half-precision quantization.

[0110] The calculated lodging angle is sent to the harvester controller via CAN bus or serial port to guide the automatic adjustment of the reel speed.

[0111] This invention acquires RGB images of the rice field in front of a combine harvester from a camera mounted on the harvester; performs ROI cropping and edge-aware restoration on the images to generate standardized square network input images; extracts the globally dominant lodging direction angle of the image as physical prior information using the structural tensor algorithm; constructs a deep neural network based on a visual state space model, which uses a selective state space mechanism to model the linear complexity of the global receptive field, injects the physical prior into the deep layers of the network through a direction modulation mechanism, and combines a multi-directional scanning mechanism to capture lodging texture features; and uses a cyclic loss function based on the geometric characteristics of a unit circle for network training and regression prediction, outputting continuous lodging angle values. This invention abandons the traditional discrete classification paradigm and solves the quantization error and periodic boundary problems in lodging direction detection through a "physical guidance + depth correction" strategy. It has advantages such as high detection accuracy, fast inference speed, and strong robustness, and can meet the real-time requirements of the harvester's onboard edge computing platform, providing a reliable basis for adaptive intelligent harvesting of lodged rice.

[0112] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0113] The detailed descriptions listed above are merely specific illustrations of feasible embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time detection of lodged rice direction on a vehicle, characterized in that, Includes the following steps: Step S1: Image Acquisition: Acquire RGB images of the rice paddy area in front of the combine harvester from a camera mounted on the combine harvester; Step S2: Image preprocessing: Crop the region of interest in the RGB image described in step S1, and normalize the cropped image into a square image of the required size for network input; Step S3: Physical Prior Extraction: The structural tensor algorithm is used to calculate the square image described in step S2 and extract the global dominant collapse direction angle as physical prior information; Step S4: Feature extraction and global feature aggregation: Construct a deep neural network based on a visual state space model, serialize the square image and use it as input, and inject the physical prior information from step S3 into the deep layer of the network. The physical prior guides the network to focus on the overturned texture features. After the network forward propagates, the global features are extracted and aggregated. Step S5: Output Results: The network output layer performs regression prediction on the global features described in step S4 and outputs continuous collapse angle values.

2. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, The step S2, which normalizes the cropped image into a square image, specifically includes: Step S2.1: Obtain the inscribed rectangle of the fallen area as the region of interest (ROI), and place the cropped image in the center of the square canvas; Step S2.2: Construct a binary mask based on the blank area of ​​the canvas, the binary mask being used to identify the area that needs texture completion; Step S2.3: Employ an image inpainting algorithm based on the fast traversal method, and use the binary mask to complete and extend the texture of the blank edges to generate a square normalized image; Step S2.4: Scale the square normalized image to the specific resolution required for the network input.

3. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, The specific steps of physical prior extraction in step S3 include: Step S3.1: Convert the RGB channels of the square image to grayscale, and calculate the horizontal gradient I using a texture detection operator. x and vertical gradient I y ; Step S3.2: Construct the structure tensor matrix S0, and perform Gaussian smoothing on the components of S0 to obtain the smoothed structure tensor matrix S. σ ; Step S3.3: For S σ Perform eigenvalue decomposition to calculate the local collapse direction for each pixel. and consistency indicators ; Step S3.4: Based on consistency metrics The global dominant fall direction angle is obtained by weighting the local orientations of all pixels and aggregating the values. .

4. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, The deep neural network in step S4 includes a staged feature extraction structure, specifically comprising: Step S4.1: Use the standard visual state space module to extract low-level visual features of the image and establish preliminary global context association; Step S4.2: Using the orientation-guided visual state space module, the physical prior information extracted in step S3 is introduced to perform orientation modulation and multi-directional scanning correction on the features.

5. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 4, characterized in that, The direction-guided visual state space module includes a direction modulation mechanism, and the specific steps include: Step S5.1: Transform the scalar physical prior orientation angle using a direction encoder. Mapping to semantic embeddings generates a channel attention mask G; Step S5.2: Perform element-wise modulation operation on the input feature X using the channel attention mask G. This enhances texture response consistent with prior orientation and suppresses background noise.

6. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 4, characterized in that, The directional guidance visual state space module includes a multi-directional scanning mechanism, with specific steps including: Step S6.1: Construct a structure including the main direction The scan set of its offset angle , where δ is the preset offset; Step S6.2: The network performs a geometric scan of the feature map along three directions in the set, captures the texture changes in the neighborhood of the main direction, and weights and fuses the multi-view features into the main feature stream in the form of residual correction.

7. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, The deep neural network employs a recurrent loss function based on the geometric properties of a unit circle during the training phase to predict angles. and the real perspective Mapping to points on the unit circle, the central angle corresponding to the shortest arc length between two points is calculated as the cycle error d, and the calculation formula is as follows: Among them, the prediction angle and the real perspective The domain of all values ​​is [0°, 360°); the total loss function is composed of the weighted sum of the cyclic mean square error and the cyclic mean absolute error.

8. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, In step S5, the network output layer aggregates the feature sequence through a global average pooling layer, and then passes it through a linear layer and an activation function to output a continuous predicted value of the overturning angle in the real number domain.

9. The vehicle-mounted real-time detection method for the direction of lodged rice as described in claim 1, characterized in that, The method also includes performing half-precision quantization and inference engine optimization on the trained deep neural network model for deployment on an in-vehicle edge computing platform.

10. A detection system for a vehicle-mounted real-time detection method for the direction of lodged rice according to any one of claims 1-9, characterized in that, It includes an image acquisition module, a preprocessing module, a priori extraction module, a network inference module, and a result output module; The image acquisition module is used to acquire RGB images of the rice field in front of the harvester; The preprocessing module is used to crop and normalize the acquired RGB image to the region of interest, and normalize the cropped image into a square image of the required size for network input. The prior extraction module is used to calculate the square image using the structure tensor algorithm and extract the global dominant collapse direction angle as physical prior information; The network inference module is internally deployed with a deep neural network of a pre-trained physical prior-guided visual state space model. The serialized square image is used as input, and the physical prior information is injected into the deep layers of the network. The physical prior guides the network to focus on the overturned texture features. After the network forward propagates, the features are extracted and aggregated to obtain the global features. The result output module is used to perform regression prediction on the global features and output continuous collapse angle values.

Images