Real-time segmentation system and method for unstructured field roads for autonomous navigation of agricultural machinery

Abstract

The present invention discloses a real-time segmentation system and method for unstructured field roads for autonomous navigation of agricultural machinery, which specifically relates to the field of autonomous navigation of agricultural machinery. The system and method are used to solve the problem of high-precision and low-latency real-time segmentation of driving areas and obstacles in unstructured field road environments. The system improves image stability under complex lighting conditions through brightness balance, and extracts texture and contour features through lightweight multi-scale coding. Texture deflection and contour focus are normalized into probability distribution, and direction consistency is analyzed based on modal competition fusion. Direction embedding is dynamically adjusted to avoid deviation. Direction-aware tensors guide the activation of BiSeNet decoder branches and output driving area probability maps and edge heat maps. Continuous frame consistency filtering eliminates instantaneous fluctuations, generates stable segmentation masks and boundary position vectors, and ensures path stability and operation continuity of navigation control.

Description

Technical Field

[0001] The present invention relates to the field of autonomous navigation of agricultural machinery, and more specifically, to a real-time segmentation system and method for unstructured field roads used for autonomous navigation of agricultural machinery. Background Art

[0002] After tilling the soil in spring, the field paths are a mixture of mud, water, and stubble, with a lack of ruts. The roadside and cultivated land have similar colors and textures. Sunlight, shadows, and dust appear alternately. The slippery surface is sometimes reflective and sometimes dark. The leaves of the seedlings occasionally extend into the paths, and crop debris rolls in the wind. The interference elements on the screen are dense and change rapidly. Autonomous driving agricultural machinery must continuously identify the drivable area and obstacles to maintain straight operation.

[0003] However, mainstream real-time segmentation models continue to use the urban road assumption. The convolutional receptive field and scale configuration make it difficult to simultaneously capture small obstacles and wide driving lanes. The inference delay increases sharply with the resolution, and the lightweight processing weakens the feature expression. In low contrast or sudden changes in illumination, mud surfaces or shadows are often misjudged as boundaries. The navigation path is shaken due to misidentification, braking is frequent, and the operation rhythm is disrupted. Relying on existing solutions cannot meet the urgent needs of agricultural machinery for high-precision and low-latency perception in field environments, thereby wasting fuel and increasing labor costs.

[0004] In order to solve the above problems, a technical solution is now provided. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a real-time segmentation system and method for unstructured field roads for autonomous navigation of agricultural machinery. The system improves image stability under complex lighting conditions through brightness balancing, and extracts texture and contour features through lightweight multi-scale coding. Texture deflection and contour focus are normalized into a probability distribution, and direction consistency is analyzed based on modal competition fusion. The direction embedding is dynamically adjusted to avoid deviation. The direction-aware tensor guides the activation of the BiSeNet decoder branches and outputs a driving area probability map and an edge heat map. Continuous frame consistency filtering eliminates instantaneous fluctuations, generates a stable segmentation mask and boundary position vector, and ensures the path stability and operation continuity of navigation control, so as to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A real-time segmentation method for unstructured field roads for autonomous navigation of agricultural machinery includes the following steps:

[0008] S1: The front camera continuously captures raw image frames of the scene, uses an adaptive brightness equalizer to unify the image radiance, and then outputs an enhanced frame stack;

[0009] S2: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuse them to generate a multi-resolution feature map.

[0010] S3: Extract the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filter the direction signal based on the multi-source perception consistency analysis, and output the direction perception tensor based on the original feature map;

[0011] S4: The direction perception tensor is input into the BiSeNet decoder. The corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and an edge heat map.

[0012] S5: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

[0013] Furthermore, step S1 includes the following contents:

[0014] Field scene images are continuously captured at a fixed frame rate by the front camera to generate a sequence of raw image frames. The raw image frames are converted from RGB color space to HSV color space, the brightness channel histogram is calculated, and the brightness channel value is adjusted according to its cumulative distribution function. The adjusted brightness channel is then recombined with the hue channel and saturation channel of the original image frame and converted back to RGB color space to generate brightness-balanced image frames. Multiple consecutive brightness-balanced image frames are stacked in chronological order to form an enhanced frame stack.

[0015] Furthermore, step S2 includes the following contents:

[0016] The enhanced frame stack is processed by a lightweight pyramid encoder, which contains multiple parallel branches. Each branch uses shared convolution kernels of different sizes. The number of shared convolution kernel parameters is fixed, and multi-scale features are extracted by adjusting the convolution kernel size. The enhanced frame stack synchronously extracts texture features and contour features within each branch.

[0017] Furthermore, step S2 also includes the following:

[0018] The texture features are generated by a convolutional layer followed by a maximum pooling layer, and the contour features are generated by an edge detection convolutional layer. The texture features and contour features are aligned at the same scale. The texture features and contour features of each branch are concatenated at the channel level to generate a fused feature map. The fused feature map is adjusted to a uniform spatial size through bilinear interpolation upsampling or average pooling downsampling, and then arranged in scale order to generate a multi-resolution feature map.

[0019] Furthermore, step S3 includes the following contents:

[0020] Each scale in the multi-resolution feature map is divided into local regions, and a fixed-size rectangular window is used to divide the feature map into non-overlapping local regions. Within each local region, the gradient direction distribution is extracted from the feature channel of the feature map, and a gradient direction distribution histogram is generated by calculating the horizontal and vertical gradients of the pixels and quantifying the direction angles. The edge pulsation frequency distribution is extracted from the edge-related channel of the feature map, and a edge pulsation frequency distribution histogram is generated by calculating the edge orientation and edge intensity change amplitude and quantifying them.

[0021] Furthermore, step S3 also includes the following contents:

[0022] The texture flow deflection factor is calculated based on the gradient direction distribution histogram, by identifying the main direction candidates and calculating the entropy of the direction distribution; the contour focus change index is calculated based on the edge pulsation frequency distribution histogram, by identifying the main direction candidates and calculating the ratio of the peak to the total.

[0023] Furthermore, step S3 also includes the following contents:

[0024] The texture flow deflection factor and contour focus change index are normalized into texture probability distribution and contour probability distribution; the main direction confidence coefficient is calculated based on the texture probability distribution and contour probability distribution, and the consistency is evaluated by symmetric divergence and mapped to the main direction confidence coefficient; the final main direction is determined according to the main direction confidence coefficient and the fused texture probability distribution and contour probability distribution, and a direction embedding vector is generated; the direction embedding vector is integrated into the multi-resolution feature map to generate a direction-aware tensor.

[0025] Furthermore, step S4 includes the following contents:

[0026] The direction-aware tensor is input into the BiSeNet decoder. The BiSeNet decoder determines the feature complexity by analyzing the distribution of local scale features in the direction-aware tensor, and activates specific branches in the decoder to process the feature complexity based on the analysis results. It ultimately generates two types of outputs, namely a pixel-level probability map indicating the drivable area and an edge heat map marking the boundary of the drivable area.

[0027] Furthermore, step S5 includes the following contents:

[0028] The continuous frame consistency filter uses the driving area probability map of the previous frame to correct the driving area probability map of the current frame. Temporal smoothing is used to reduce inter-frame fluctuations caused by illumination changes or noise. The edge heat map is combined to identify and remove misclassified areas in the current frame that overlap with the edge and were not identified as driving areas in the previous frame, generating a corrected driving area mask. A morphological closing operation is then performed on the corrected driving area mask to smooth the mask boundary and eliminate noise. Connected region analysis is then performed to retain the connected region with the largest area to generate a stable segmentation mask. Finally, the boundary position vector of the driving area is extracted from the stable segmentation mask. The boundary is identified using the Canny edge detection algorithm and boundary points are uniformly sampled along the driving direction. The coordinates of the sampling points are converted into one-dimensional vector form and transmitted to the navigation control module.

[0029] The real-time segmentation system for unstructured field roads used for autonomous navigation of agricultural machinery includes:

[0030] Image acquisition unit: The front camera continuously captures the original image frames of the scene, uses the adaptive brightness equalizer to unify the image radiometry, and then outputs an enhanced frame stack;

[0031] Feature extraction unit: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuses them to generate a multi-resolution feature map.

[0032] Direction perception unit: extracts the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filters the direction signal based on multi-source perception consistency analysis, and outputs the direction perception tensor based on the original feature map;

[0033] Semantic segmentation unit: The direction perception tensor is input into the BiSeNet decoder, and the corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and edge heat map;

[0034] Mask correction unit: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

[0035] The technical effects and advantages of the system and method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery of the present invention are as follows:

[0036] The present invention improves the stability of feature expression of input images under complex field lighting conditions by introducing brightness equalization processing in the visual front end, and constructs a highly adaptable feature extraction foundation in conjunction with a lightweight multi-scale coding structure. On this basis, the two types of heterogeneous information, texture deflection and contour focus, are normalized into a probability distribution, and the direction consistency is analyzed through a modal competition fusion mechanism, so that the direction embedding strategy is dynamically adjusted based on the main direction confidence coefficient, effectively avoiding the direction estimation deviation caused by local abnormal features. The direction perception tensor further guides the branch activation and scale selection in the semantic decoding process, so that the segmentation model has stronger adaptability when dealing with changes in target size in different regions. Finally, the continuous frame consistency filtering strategy is combined to eliminate instantaneous prediction fluctuations and stably output boundary information for navigation control. The above structure constructs a stable, efficient and fast-responding processing path in each link, achieving high-precision and low-latency recognition of the driving area in the field unstructured road scene, effectively supporting the path stability and operation continuity of agricultural autonomous driving operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the real-time segmentation method for unstructured field roads for autonomous navigation of agricultural machinery according to the present invention.

[0038] Figure 2 This is a structural diagram of the real-time segmentation system for unstructured field roads used for autonomous navigation of agricultural machinery according to the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] Example 1: Figure 1 The present invention provides a real-time segmentation method for unstructured field roads for autonomous navigation of agricultural machinery, comprising:

[0041] S1: The front camera continuously captures raw image frames of the scene, uses an adaptive brightness equalizer to unify the image radiance, and then outputs an enhanced frame stack.

[0042] S2: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuse them to generate a multi-resolution feature map.

[0043] S3: Extract the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filter the direction signal based on multi-source perception consistency analysis, and output the direction perception tensor in combination with the original feature map.

[0044] S4: The direction perception tensor is input into the BiSeNet decoder, and the corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and an edge heat map.

[0045] S5: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

[0046] Autonomous navigation technology for agricultural machinery is becoming increasingly important in modern agricultural production, especially in field operations after spring tillage. Field paths are typically unstructured, with a mixture of mud, water, and stubble, lacking distinct ruts, and with similar color and texture between the roadside and the cultivated land. Furthermore, the path is affected by environmental factors such as sunlight, shadows, and dust, resulting in complex and variable lighting conditions. Wet surfaces can be reflective or dark, seedling leaves can occasionally extend into the path, and crop debris can roll in the wind. The image is filled with dense and rapidly changing interference elements. These factors pose significant challenges for autonomous agricultural machinery in identifying drivable areas and obstacles. To ensure that agricultural machinery can continuously and stably perform straight-line operations, it is necessary to develop a method that can segment unstructured field paths in real time with high precision.

[0047] Traditional real-time segmentation models are often based on assumptions about urban roads, and their design struggles to adapt to the complexity of field environments. The convolutional receptive field and scale configuration of urban road models are unable to simultaneously capture both small obstacles and wide driving lanes. Especially in low-contrast environments or sudden changes in illumination, muddy surfaces or shadows can be easily misjudged as boundaries, leading to jittery navigation paths and frequent braking, which in turn affects operational efficiency and continuity. Therefore, a "Real-time Segmentation Method for Unstructured Field Roads for Autonomous Navigation of Agricultural Machinery" was proposed. This approach, using image processing technology, addresses the problem of high-precision, low-latency recognition of driving areas in complex field environments, providing stable and reliable perception support for autonomous agricultural driving.

[0048] Step S1, the starting point of the entire method, is responsible for collecting and preprocessing field scene images, laying the foundation for subsequent feature extraction and segmentation. The specific processing technology logic of step S1 is explained in detail below.

[0049] The goal of step S1 is to continuously capture raw image frames of the field scene through the front camera, unify the image radiance using an adaptive brightness equalizer, and finally output an enhanced frame stack to provide stable input data for subsequent steps.

[0050] Step S1 includes the following contents:

[0051] S1.1, Image Acquisition:

[0052] The front-facing camera continuously captures images of the field scene at a fixed frame rate, generating a series of raw image frames. These raw image frames are color images containing red, green, and blue channels, capturing visual information about the field path, such as dynamic scenes like mud, stubble, and shadows. Each raw image frame has a fixed height and width and is arranged in chronological order, providing initial input data for subsequent adaptive brightness equalizer processing. Continuous acquisition provides complete dynamic visual information, ensuring that subsequent processing is based on comprehensive field data.

[0053] S1.2, adaptive brightness equalizer processing:

[0054] Each frame of the original image is processed by an adaptive brightness equalizer to unify the image's radiance and enhance the stability of feature expression under complex lighting conditions. The specific processing process includes the following steps:

[0055] First, the original image frame is converted from a color space containing red, green, and blue channels to a color space containing hue, saturation, and brightness, so that the brightness information can be adjusted separately without disturbing the hue and saturation;

[0056] Then, for the brightness channel, the distribution of its pixel values ​​is analyzed and the brightness value is adaptively adjusted based on the distribution characteristics to make the details of the darker areas clearer while preventing overexposure in the brighter areas.

[0057] Finally, the adjusted brightness channel is recombined with the original hue and saturation channels to convert back into a color image containing red, green, and blue channels, generating a brightness-balanced image frame. The computational logic and processing principle is to separate brightness information through color space conversion and adjust the brightness channel based on the pixel value distribution characteristics to enhance image contrast and detail visibility. Given the complex and variable lighting conditions in the field, this processing effectively improves the overall image quality, ensuring that features remain consistent and clear under different lighting conditions, and providing optimized input data for subsequent generation of enhanced frame stacks.

[0058] S1.3, Enhanced frame stack generation:

[0059] The enhanced frame stack is formed by stacking consecutive brightness-equalized image frames in chronological order. This enhanced frame stack not only contains the spatial information of the individual frames but also incorporates temporal variations. This allows the stack to reflect dynamic features of the field scene, such as the movement of crop debris and shadow changes, providing more comprehensive data support for subsequent feature extraction and segmentation. The specific processing process involves selecting multiple consecutive brightness-equalized image frames, arranging them in chronological order, and combining them into a multidimensional data structure, the enhanced frame stack.

[0060] By stacking consecutive frames, spatial and temporal information are integrated to form multidimensional input data containing dynamic features. By stacking frames, dynamic changes in field scenes are captured, enhancing the richness and expressiveness of the data, thereby improving the accuracy and robustness of subsequent processing.

[0061] The original image frames of the field scene are acquired through the front camera. Then, the original image frames are processed using an adaptive brightness equalizer to unify the radiometric properties and generate brightness-balanced image frames. Finally, the brightness-balanced image frames are stacked into a multidimensional data structure through enhanced frame stack generation, forming an enhanced frame stack containing spatial and temporal information.

[0062] Step S1 captures raw image frames using the front-facing camera, unifies the image radiometry using an adaptive brightness equalizer, and outputs a stack of enhanced frames. However, the diverse field environments require the subsequent feature extraction stage to efficiently capture multi-scale target features while remaining lightweight enough to meet real-time requirements. Therefore, step S2 requires the design of specific processing technology logic for the enhanced frame stack to simultaneously extract texture and contour features and generate a multi-resolution feature map, providing the foundation for subsequent directional perception and semantic segmentation.

[0063] In step S2, the enhanced frame stack is processed by a lightweight pyramid encoder to simultaneously extract texture features and contour features, and fuse them to generate a multi-resolution feature map, targeting the characteristics of unstructured field road scenes, such as the mixture of mud, water and stubble, sudden changes in illumination, and the coexistence of small obstacles and wide driving strips.

[0064] Step S2 includes the following contents:

[0065] S2.1, lightweight pyramid encoder structure:

[0066] Branch design for multi-scale feature extraction:

[0067] The lightweight pyramid encoder processes an input stack of enhanced frames, which consists of multiple consecutive brightness-equalized image frames. The encoder employs a multi-branch parallel architecture, with each branch using a shared convolution kernel of varying sizes for feature extraction. Kernel sizes start at 3x3 and gradually increase to cover the extraction of small-scale features, such as small obstacles and wide driving lanes in field corridors. All branches share a set of kernel parameters, enabling feature perception across varying ranges by adjusting kernel size.

[0068] By leveraging shared parameters to reduce computational redundancy and extracting diverse features through a multi-branch architecture, the algorithm improves its adaptability to complex field environments. It also processes features at different scales in parallel and reduces the number of parameters through shared convolution kernels, ensuring efficient extraction of rich spatial and temporal information under resource-constrained conditions.

[0069] Enhanced frame stack input features:

[0070] The enhanced frame stack, which serves as input data and consists of multiple consecutive image frames that have undergone brightness equalization, provides rich temporal and spatial information. Brightness equalization ensures image frame consistency under varying lighting conditions, contributing to the stability of subsequent feature extraction. The unstructured nature of field environments requires combining multiple frames to capture dynamic changes, and brightness equalization enhances the robustness of feature extraction. By enriching input information with multiple frames and processing them separately through parallel branches, the extracted features are ensured to be comprehensive and reliable.

[0071] Feature representation of branch output:

[0072] Each branch generates a feature representation at a corresponding scale based on its convolution kernel size. Smaller kernels extract subtle local features, such as mud and water distribution within a field; larger kernels extract global features, such as the boundaries of a driving lane. The feature representations output by each branch preserve the spatial structure of the enhanced frame stack, providing a multi-scale foundation for subsequent processing. Features at different scales correspond to distinct target characteristics within the field corridor, enhancing the specificity of subsequent processing. Multi-branch parallel computation generates diverse feature representations, ensuring that feature extraction covers a wide range of target requirements.

[0073] S2.2, Synchronous extraction of texture features and contour features:

[0074] Texture feature extraction process:

[0075] Within each branch, texture feature extraction is performed by applying a convolutional layer followed by a max pooling layer to the branch's input data. The convolutional layer processes the input data to capture texture information in the image, such as the distribution of mud and water on the surface of a field path and the texture of stubble. The max pooling layer enhances the saliency of texture features by selecting the maximum value in a local region. Because the surface characteristics of the field path directly influence driving decisions, texture information provides critical clues. Using convolution and pooling operations to highlight local texture features in the image ensures that the output texture feature map accurately reflects surface details.

[0076] Contour feature extraction process:

[0077] Contour feature extraction is performed within each branch through an edge detection convolutional layer. This layer processes input data based on an optimized edge detection algorithm, capturing contour information within the image, such as roadside demarcation lines and obstacle edges. The edge detection algorithm identifies boundary features by calculating gradient changes across image pixels. The boundaries of the field driving area and the locations of obstacles are crucial for positioning and obstacle avoidance, and contour information improves recognition accuracy. Edge detection extracts boundary features, ensuring that the output contour feature map clearly depicts the object's edges.

[0078] The spatial correspondence of features ensures:

[0079] Texture and contour features are computed in parallel within each branch. The spatial size of the output feature map is determined by the convolution kernel size and pooling stride, ensuring that the two features are aligned at the same scale. Parallel extraction of the two features and controlled output size ensure spatial consistency and improve the accuracy of subsequent processing.

[0080] S2.3, feature fusion generates multi-resolution feature maps:

[0081] Channel-level splicing of texture features and contour features:

[0082] Feature fusion first performs channel-level concatenation on the texture and contour features of each branch to generate a fused feature map. Channel-level concatenation combines the two features along the channel dimension, preserving their respective information while forming a comprehensive representation. Because texture and contour features each reflect different attributes of the field channel, their combination provides a more comprehensive description of the target. Channel-level concatenation integrates the two features to generate a fused feature map containing multi-attribute information.

[0083] Spatial resizing of fused feature maps:

[0084] The fused feature maps from each branch are resized to a uniform spatial size through bilinear interpolation upsampling or average pooling downsampling. Bilinear interpolation increases the feature map size through interpolation, while average pooling reduces it through averaging, achieving scale alignment. Since the feature maps from different branches have different scales, unifying the scale facilitates subsequent integration and processing. Aligning the feature map spatial sizes through sampling ensures compatibility of multi-scale features.

[0085] Generation of multi-resolution feature maps:

[0086] The adjusted fused feature maps from all branches are arranged in scale order to generate a multi-resolution feature map. This multi-resolution feature map contains multi-scale feature information, ranging from small, fragmented obstacles to large, broad travel paths. Given the diverse targets in field corridors, multi-scale representations can meet the needs of different tasks. The adjusted feature maps are integrated to form an ordered multi-resolution representation, providing comprehensive feature support.

[0087] The lightweight pyramid encoder structure uses a multi-branch and shared convolution kernel design to perform multi-scale feature extraction on the enhanced frame stack and generate feature representations of each branch; the synchronous extraction of texture features and contour features captures the texture and contour information of the image respectively, ensuring the spatial correspondence of the two features; feature fusion generates a multi-resolution feature map to integrate the features of each branch to form a multi-scale representation.

[0088] Step S2 extracts and fuses texture and contour features from the enhanced frame stack using a lightweight pyramid encoder to generate a multi-resolution feature map. This feature map provides a multi-scale spatial and temporal information foundation for subsequent processing and is suitable for complex scenarios such as mud and water mixing, stubble interference, and sudden changes in illumination in field environments. However, field roads lack clear ruts, and the boundaries between driving areas and obstacles are blurred, making it difficult to directly distinguish directional signals using simple feature expressions. To address this issue, step S3 needs to further extract and analyze the directional information of local areas on the multi-resolution feature map, screen reliable directional signals through multi-source perception consistency, and ultimately output a directional perception tensor to provide precise directional guidance for the semantic segmentation decoding in step S4.

[0089] Step S3 addresses the need for directional identification of driving areas and obstacles in unstructured road scenes in the field. By extracting the gradient direction distribution and edge pulsation frequency on the multi-resolution feature map, calculating the texture flow deflection factor and contour focus change index, and screening reliable main directions based on modal consistency, a direction perception tensor is generated.

[0090] Step S3 includes the following contents:

[0091] S3.1, Local Area Division:

[0092] Perform local area division on each scale in the multi-resolution feature map. The specific operations are:

[0093] A fixed-size rectangular window is used to segment the multi-resolution feature map into multiple non-overlapping local regions. The width and height of the rectangular window are pre-set, and the window is moved across the feature map with a step equal to the width and height, respectively, to ensure that adjacent local regions do not overlap. Each local region contains a set of feature vectors corresponding to the position in the feature map, which are used for subsequent directional feature extraction. After segmentation, each local region acts as an independent unit, providing a standardized analysis basis for subsequent processing.

[0094] By dividing the multi-resolution feature map into local regions, we can perform detailed analysis of feature variations at different locations within a field environment, improving the accuracy and local adaptability of directional information extraction. Using fixed-size rectangular windows and non-overlapping strides ensures that each local region is independent and has no redundant information, providing a consistent input unit for directional feature extraction.

[0095] S3.2, gradient direction distribution extraction:

[0096] In each local area, the gradient direction distribution is extracted from the feature channel of the multi-resolution feature map to characterize the directional characteristics of the texture flow. The specific operation is:

[0097] First, for each pixel position in the local area, its gradient values ​​in the horizontal and vertical directions are calculated. The horizontal gradient is obtained by the difference of the pixel values ​​on the horizontal axis, and the vertical gradient is obtained by the difference of the pixel values ​​on the vertical axis. Then, the direction angle of each pixel is calculated based on the ratio of the horizontal gradient to the vertical gradient. Next, all the direction angles are quantized according to preset discrete intervals, and the number and range of discrete intervals are predefined. Finally, using the gradient amplitude of each pixel as a weight, the cumulative values ​​of the direction angles of all pixels in the local area in each discrete interval are counted to generate a gradient direction distribution histogram. Each entry in the histogram represents the weighted sum of the corresponding discrete interval.

[0098] The gradient directional distribution histogram reflects the flow characteristics of textures within a local area, providing a key basis for identifying the direction of travel in field corridors. Directional angles are calculated from pixel gradients and quantized into discrete intervals. Gradient amplitude-weighted statistical histograms capture the directional distribution characteristics of textures, providing quantitative data for subsequent texture flow analysis.

[0099] S3.3, edge pulsation frequency extraction:

[0100] In each local area, the edge pulsation frequency distribution is extracted from the edge-related channels of the multi-resolution feature map to characterize the direction and dynamic change characteristics of the edge. The specific operations are:

[0101] First, for each pixel position in the local area, the edge orientation is calculated. The edge orientation is determined by the ratio of the edge intensity components in the horizontal and vertical directions. Then, the variation of edge intensity between different channels of the multi-resolution feature map is calculated. The variation is the sum of the absolute values ​​of the differences in edge intensity between each channel. Then, all edge orientations are quantized according to preset discrete intervals. The number and range of discrete intervals are consistent with those used in the gradient direction distribution extraction. Finally, using the edge intensity variation as a weight, the cumulative value of the edge orientation of all pixels in the local area in each discrete interval is counted to generate an edge pulsation frequency distribution histogram. Each entry in the histogram represents the weighted sum of the corresponding discrete interval.

[0102] The edge pulsation frequency distribution histogram captures the directional consistency and dynamic variation of edges, helping to identify the boundaries and obstacle locations of field pathways. By calculating and quantifying edge orientation and combining it with weighted statistics of edge intensity variation, a histogram is generated to quantify the dynamic directional characteristics of edges, providing data support for subsequent contour analysis.

[0103] S3.4, Calculation of texture flow deflection factor:

[0104] Based on the gradient direction distribution histogram, the texture flow deflection factor is calculated to quantify the dispersion of the texture direction. The specific operation is:

[0105] First, the discrete interval with the highest cumulative value is identified from the gradient direction distribution histogram, and the corresponding direction is used as the candidate for the main texture direction. Then, the entropy value of the gradient direction distribution histogram is calculated. The entropy value is the sum of the ratio of the cumulative value of each discrete interval to the total value of the histogram and the logarithm of the ratio, which is the negative value. Finally, the entropy value is defined as the texture flow deflection factor. The larger the entropy value, the more dispersed the texture direction distribution.

[0106] The texture flow deflection factor quantifies the dispersion of directional distributions, assessing the reliability of texture flow information and facilitating the selection of reliable directional features. The main direction candidates are identified using histogram peaks, and the concentration of directional distributions is measured through entropy calculation. This factor is then generated to support subsequent directional consistency analysis.

[0107] S3.5, contour focus variation index calculation:

[0108] Based on the edge pulsation frequency distribution histogram, the contour focus variation index is calculated to quantify the degree of edge orientation concentration. The specific operation is:

[0109] First, the discrete interval with the highest cumulative value is identified from the edge pulsation frequency distribution histogram, and the corresponding direction is used as the candidate for the main direction of the contour. Then, the ratio of the highest cumulative value in the histogram to the sum of the cumulative values ​​of all discrete intervals is calculated, and the ratio is the highest cumulative value divided by the sum. Finally, this ratio is defined as the contour focus variation index. The larger the ratio, the more concentrated the edge orientation.

[0110] The Contour Focus Variation Index quantifies the concentration of edge directions, assessing the reliability of contour information and helping to screen for stable edge features. The peak of the histogram is used to identify candidate main directions, and the ratio of the peak to the sum is used to measure the focus of the direction distribution. This generates the Contour Focus Variation Index to support subsequent direction fusion.

[0111] S3.6, Probability distribution normalization:

[0112] The texture flow deflection factor and the contour focus change index are normalized into probability distributions to represent the directional possibilities of texture and contour, respectively. The specific operations are:

[0113] For the texture flow deflection factor, the distance of each discrete interval relative to the main texture direction candidate is first calculated. The distance is the difference between the angle of the interval center and the main direction angle. An exponential decay function is then applied to the distance of each discrete interval. The decay amplitude is controlled by the texture flow deflection factor; a larger deflection factor results in slower decay. Finally, the decay values ​​of all discrete intervals are normalized so that their sum is 1, generating a texture probability distribution. For the contour focus variation index, a weighted adjustment is performed directly on the edge pulsation frequency distribution histogram. The weighting coefficient is the contour focus variation index; a larger index indicates a higher weight for the main direction. The adjusted histogram is then normalized so that its sum is 1, generating a contour probability distribution.

[0114] Unifying texture and contour information into a probability distribution facilitates subsequent consistency analysis and directional fusion, improving processing comparability. Through exponential decay and weighted normalization, the deflection factor and focus index are converted into a probability distribution, standardizing the directional information of texture and contour, providing a unified framework for subsequent fusion.

[0115] S3.7, Calculation of principal direction confidence coefficient:

[0116] Based on the texture probability distribution and the contour probability distribution, the main direction confidence coefficient is calculated to evaluate the consistency of the two distributions. The specific operations are:

[0117] First, the symmetric divergence between the texture probability distribution and the contour probability distribution is calculated. The symmetric divergence is the weighted sum of the product of the logarithm of the probability ratio of the two distributions in each discrete interval and the probability value, and the average of the forward and reverse divergences is calculated; then, the symmetric divergence is mapped to a confidence coefficient through an exponential decay function. The smaller the divergence, the closer the confidence coefficient is to 1, indicating that the two distributions are more consistent; finally, the confidence coefficient is limited to between 0 and 1 as a reliability indicator of the main direction.

[0118] The principal direction confidence coefficient quantifies the consistency of texture and contour distributions, enabling reliable principal directions to be identified and mitigating miscalculations caused by feature anomalies. Symmetric divergence is used to measure the similarity between two probability distributions, and a confidence coefficient is generated through exponential mapping to assess the credibility of the principal direction and guide subsequent processing.

[0119] S3.8, Direction Embedding Vector Generation:

[0120] According to the main direction confidence coefficient and the fused probability distribution, the final main direction is determined and the direction embedding vector is generated. The specific operations are:

[0121] First, the probability values ​​of the texture probability distribution and the contour probability distribution in each discrete interval are averaged to generate a fused probability distribution; then, the discrete interval with the highest probability is identified from the fused probability distribution, and its corresponding direction is used as the final main direction; then, the main direction confidence coefficient is compared with a preset threshold, where the threshold is a predefined constant; if the confidence coefficient is greater than or equal to the threshold, the cosine and sine values ​​of the final main direction angle are calculated and used as the two components of the direction embedding vector respectively; if the confidence coefficient is less than the threshold, a direction embedding vector containing two zero components is generated, indicating that the direction information is unreliable.

[0122] Directional embedding vectors are generated only when the confidence coefficient meets the requirements, ensuring the accuracy of the embedded direction information and avoiding misleading subsequent processing. The final main direction is determined by averaging the fused probability distribution, and whether a non-zero vector is generated is determined based on the confidence coefficient threshold to ensure the reliability of the directional embedding.

[0123] S3.9, Direction-aware tensor output:

[0124] Integrate the directional embedding vector into the multi-resolution feature map to generate a direction-aware tensor to enhance the directional information of the feature. The specific operations are:

[0125] First, a direction tensor is generated for each local area. The size of the direction tensor is the same as the local area, and the channel values ​​of all pixel positions are set to the corresponding direction embedding vector. Then, the direction tensor is spliced ​​with the original feature vector of the corresponding local area in the multi-resolution feature map along the channel dimension to generate an enhanced feature map. The number of channels after splicing is the number of original channels plus the number of components of the direction embedding vector. Finally, the enhanced feature maps of all scales of the multi-resolution feature map are combined to form a direction-aware tensor, which is passed to subsequent processing as the output of step S3.

[0126] By embedding directional information, the directional perception of feature maps is enhanced, providing precise directional guidance for subsequent semantic segmentation and improving navigation accuracy. Channel concatenation integrates the directional embedding vector into the original feature map to generate a tensor containing directional information, enhancing the directional representation of features.

[0127] Step S3 divides the local area from the multi-resolution feature map, extracts the gradient direction distribution and edge pulsation frequency distribution, calculates the texture flow deflection factor and contour focus variation index, and normalizes them into texture probability distribution and contour probability distribution, respectively. By evaluating the consistency of the two probability distributions, the main direction confidence coefficient is calculated. Only when the confidence coefficient is above a preset threshold, a direction embedding vector is generated and integrated into the multi-resolution feature map, outputting the direction-aware tensor.

[0128] The aforementioned steps S1 to S3 have captured the original image frames through the front camera, unified the image radiance using an adaptive brightness equalizer, extracted texture and contour features through a lightweight pyramid encoder and generated a multi-resolution feature map, and extracted the gradient direction distribution and edge pulsation frequency of the local area on the multi-resolution feature map. The direction signal is filtered based on multi-source perception consistency analysis, and finally the direction perception tensor is output. However, the diversity of the field environment requires that the subsequent semantic segmentation stage can efficiently utilize multi-scale features and direction information to accurately distinguish between driving areas and obstacles. Therefore, step S4 needs to design specific processing technology logic for the direction perception tensor, use the BiSeNet decoder for semantic segmentation, and output pixel-level driving area probability map and edge heat map to provide a reliable foundation for subsequent navigation control.

[0129] The processing flow of step S4 starts with the BiSeNet decoder receiving the direction-aware tensor, extracting spatial details and semantic information through the spatial path and context path respectively, and generating a spatial path feature map and a context path feature map; then performing local scale feature distribution analysis on each scale feature map of the direction-aware tensor and calculating the scale feature complexity; calculating the activation weights of the spatial path and context path based on the scale feature complexity; then fusing the spatial path feature map and the context path feature map through a weighted sum to generate a fused feature map; finally, inputting the fused feature map into the decoder, and generating a driving area probability map and an edge heat map through upsampling and convolution operations.

[0130] Step S4 includes the following contents:

[0131] S4.1, BiSeNet decoder input and structure:

[0132] The BiSeNet decoder receives the direction-aware tensor output from step S3 as input. The direction-aware tensor contains multi-scale features and direction information. Its data structure consists of the number of scales, uniform height, uniform width, and the number of channels plus 2.

[0133] The BiSeNet decoder consists of two parts: a spatial path and a context path. The spatial path is used to preserve the spatial details of the image. It uses a shallow convolutional network to process the direction-aware tensor. The specific process is to apply a 3x3 convolution layer to the feature map at each scale to generate a spatial path feature map. Its data structure is uniform in height, uniform in width, and has the same number of spatial path channels.

[0134] The context path is used to extract high-level semantic information. It adopts a deep convolutional network, combined with global pooling and multi-scale feature extraction. The specific process is to apply a 3x3 convolution layer (stride 2), a global average pooling layer and a convolutional downsampling layer to the feature map of each scale in sequence to generate a context path feature map. Its data structure is uniform height, uniform width and number of context path channels.

[0135] The combination of the spatial and contextual paths simultaneously preserves the spatial details of the image and extracts high-level semantic information, thereby improving the accuracy and robustness of the segmentation results. The spatial path extracts spatial details of the image through a shallow convolutional network, while the contextual path extracts semantic information through a deep convolutional network combined with global pooling and downsampling, generating multi-scale feature representations for subsequent feature fusion processing.

[0136] S4.2, Local scale feature distribution analysis:

[0137] The feature map at each scale in the direction-aware tensor is partitioned into local regions. This process involves using a fixed-size rectangular window to divide the feature map into non-overlapping local regions. The rectangular window size is preset to 8x8 pixels. Within each local region, the scale feature complexity is calculated by first finding the eigenvalues ​​of all pixels in the local region for each channel. The absolute deviations of these eigenvalues ​​from the local mean of the corresponding channel are then calculated. All absolute deviations are then summed and divided by the total number of pixels in the local region to obtain the scale feature complexity.

[0138] The local mean refers to the average value of the characteristic values ​​of all pixels in the local area on the channel.

[0139] Scale feature complexity reflects the complexity of local features and helps dynamically adjust the activation weights of subsequent branches, thereby improving the adaptability of segmentation. By calculating the sum of the absolute deviations between the feature values ​​and the local mean and normalizing it, the distribution characteristics of local features are quantified, providing a basis for calculating branch activation weights.

[0140] S4.3, branch activation weight calculation:

[0141] The activation weights for the spatial and contextual paths are calculated based on the scale feature complexity. The specific process involves first calculating the difference between the scale feature complexity and a preset threshold. This difference is then input into a Sigmoid function for nonlinear mapping to obtain the spatial path activation weight. The contextual path activation weight is calculated by subtracting the spatial path activation weight from 1. The preset threshold is an empirically determined value used to adjust the sensitivity of weight changes. The output of the Sigmoid function ranges from 0 to 1 to ensure the validity of the activation weights.

[0142] Dynamically adjusting activation weights adaptively selects the contribution of spatial or contextual paths based on the complexity of local region features, improving segmentation accuracy and computational efficiency. The difference between scale feature complexity and a threshold is mapped to spatial path activation weights using a sigmoid function. This is then used to calculate contextual path activation weights, enabling dynamic allocation of branch importance.

[0143] S4.4, Feature Fusion:

[0144] For each local region, the spatial path feature map and the context path feature map are fused.

[0145] First, each element of the spatial path feature map is multiplied by the corresponding spatial path activation weight to obtain a weighted spatial path feature map; then each element of the context path feature map is multiplied by the corresponding context path activation weight to obtain a weighted context path feature map; finally, the weighted spatial path feature map and the weighted context path feature map are added element by element to generate a fusion feature map.

[0146] Feature fusion can integrate the spatial details of the spatial path and the semantic information of the contextual path, improving the comprehensiveness of the feature representation and thus the quality of the segmentation results. The spatial path feature map and the contextual path feature map are fused by weighted sum to ensure that the fused feature map contains comprehensive information at multiple scales.

[0147] S4.5, decoder processing:

[0148] The fused feature map is input into the decoder for processing.

[0149] First, bilinear upsampling is used to restore the resolution of the fused feature map to the original image height and width. A 1x1 convolutional layer is then applied to generate the driving area probability map and edge heatmap, respectively. The driving area probability map has the original height, original width, and 1, indicating the probability of each pixel belonging to a driving area. The edge heatmap has the original height, original width, and 1, indicating the probability of each pixel belonging to an edge. The output layer uses a sigmoid activation function to map all output values ​​between 0 and 1.

[0150] The decoder process restores multi-scale features to their original resolution, generating pixel-level segmentation results that improve navigation accuracy. Bilinear upsampling restores resolution and combines it with 1x1 convolution to generate pixel-level probability distributions, ultimately outputting segmentation results for driving areas and edges.

[0151] The direction-aware tensor is fed into the BiSeNet decoder, which outputs a pixel-level driving area probability map and edge heatmap. However, dynamic changes in the field environment and image noise can cause transient misclassifications in the segmentation results of a single frame, affecting the stability of the navigation path. Therefore, step S5 requires consistency filtering of the segmentation results of consecutive frames to correct transient misclassifications, generate a stable segmentation mask, and extract boundary position vectors to provide reliable input for the navigation control module.

[0152] Step S5 addresses segmentation instabilities caused by image noise and dynamic changes in unstructured field road scenes by applying a continuous frame consistency filter to the driving area probability map for temporal smoothing and correcting for misclassified regions. This generates a stable segmentation mask from which boundary position vectors are extracted. This process effectively eliminates transient prediction fluctuations caused by sudden changes in illumination or small obstacles, ensuring temporal consistency and accuracy of the segmentation results. This provides reliable boundary information for the navigation control module, enabling stable path planning and obstacle avoidance decisions for agricultural machinery in complex field environments.

[0153] Step S5 includes the following contents:

[0154] S5.1, instantaneous misclassification area correction:

[0155] Based on the smoothed driving area probability map and edge heat map, misclassified areas are identified and corrected to generate a corrected driving area mask. The specific process includes three steps: threshold segmentation, edge constraint, and misclassified area identification and correction.

[0156] In the threshold segmentation step, a preset threshold (e.g., 0.5) is applied to the smoothed driving area probability map to generate an initial driving area mask. Specifically, for each pixel in the smoothed driving area probability map, if its probability value is greater than or equal to 0.5, the corresponding position in the initial driving area mask is marked as a driving area, and the mask value is set to 1. If the probability value is less than 0.5, it is marked as a non-driving area, and the mask value is set to 0.

[0157] In the edge constraint step, a preset threshold (e.g., 0.3) is applied to the edge heatmap to generate an edge mask. Specifically, for each pixel in the edge heatmap, if its probability value is greater than or equal to 0.3, the corresponding position in the edge mask is marked as an edge, and the mask value is set to 1; if the probability value is less than 0.3, it is marked as a non-edge, and the mask value is set to 0.

[0158] In the misclassified area identification and correction step, the area in the initial driving area mask that overlaps with the edge mask is detected and corrected in combination with the driving area mask of the previous frame. The specific operation is as follows: for each pixel in the initial driving area mask, if the pixel value in the initial driving area mask is 1, the value in the edge mask is 1, and the value in the driving area mask of the previous frame is 0, then the pixel is determined to be a transient misclassification and its value in the initial driving area mask is changed from 1 to 0. If the above conditions are not met, the value of the pixel in the initial driving area mask remains unchanged.

[0159] After the above processing, a revised driving area mask is generated. By combining edge information and temporal information, it can effectively identify and correct misclassified areas caused by noise or interference, improving segmentation accuracy. The initial driving area mask of the current frame is corrected using the edge mask and the driving area mask of the previous frame to ensure the reliability of the segmentation result.

[0160] S5.2, stable segmentation mask generation:

[0161] Based on the modified driving area mask, a stable segmentation mask is generated through morphological processing and connected component analysis. The specific process includes two steps: morphological processing and connected component analysis.

[0162] In the morphological processing step, a morphological closing operation is applied to the modified driving area mask. The morphological closing operation consists of dilation followed by erosion, using a 3x3 rectangular structuring element and an iteration count of 1.

[0163] The dilation operation is as follows: for each pixel in the modified driving area mask, if at least one pixel in the surrounding 3x3 area has a mask value of 1, then the mask value of that pixel is set to 1. The erosion operation is as follows: for each pixel in the dilated mask, if all pixels in the surrounding 3x3 area have a mask value of 1, then the mask value of that pixel is retained as 1; otherwise, it is set to 0. Dilation fills small holes in the mask, and erosion removes isolated points, generating a smooth intermediate mask.

[0164] In the connected region analysis step, the connected regions of the morphologically processed intermediate mask are marked, the pixel area of ​​each connected region is calculated, and the connected region with the largest area is retained as the final driving region. Other smaller connected regions are treated as noise or small obstacles and removed. The specific operation is: all connected regions in the intermediate mask are identified, the number of pixels with a mask value of 1 in each connected region is counted, and the connected region with the largest number of pixels is selected and its mask value is retained as 1. The mask values ​​of the remaining connected regions are set to 0 to generate a stable segmentation mask.

[0165] Morphological processing and connected component analysis can smooth mask boundaries and remove noise, improving the stability and robustness of the segmentation results. The mask is smoothed through morphological closing operations, and connected component analysis is then used to retain the main driving areas to generate a stable segmentation mask.

[0166] S5.3, Boundary position vector extraction:

[0167] The boundary position vector of the driving area is extracted from the stable segmentation mask. The specific process includes three steps: boundary detection, boundary point sampling and position vector generation.

[0168] In the boundary detection step, the Canny edge detection algorithm is applied to the stable segmentation mask to generate a boundary image. Specifically, the Canny edge detection algorithm is used to identify the boundaries of the driving area in the stable segmentation mask and generate a boundary image, in which the pixel values ​​of boundary points are set to 1 and the pixel values ​​of non-boundary points are set to 0.

[0169] In the boundary point sampling step, boundary points are uniformly sampled in the boundary image along the travel direction (from bottom to top of the image), with a preset sampling interval of 10 pixels, to generate a sampling point set. The specific operation is: starting from the bottom of the boundary image, the boundary image is detected every 10 pixels along the vertical direction. If a boundary point with a pixel value of 1 is detected, the horizontal and vertical coordinates of the point are recorded. Sampling is continued until the top of the image to generate a sampling point set.

[0170] In the position vector generation step, the coordinates of the sampling points are converted into boundary position vectors. Specifically, the horizontal and vertical coordinates of each sampling point in the sampling point set are arranged alternately in sequence to form a one-dimensional boundary position vector. For example, for the first point in the sampling point set, its horizontal coordinate is the first item and its vertical coordinate is the second item. The horizontal coordinate of the second point is the third item and its vertical coordinate is the fourth item. And so on.

[0171] The boundary position vector provides the navigation control module with information about the driving area's boundaries, supporting path planning and obstacle avoidance decisions. Boundary key points are extracted through Canny edge detection and uniform sampling, and the coordinates are converted into vector form to generate the boundary position vector.

[0172] The processing flow in step S5 begins with the continuous frame consistency filter receiving the current frame's driving area probability map, the previous frame's driving area probability map, and the current frame's edge heat map. A revised driving area mask is generated through temporal smoothing and misclassification correction. Morphological processing and connected component analysis are then performed on the revised driving area mask to generate a stable segmentation mask. Finally, boundary position vectors are extracted from the stable segmentation mask. This results in a stable segmentation mask and boundary position vectors suitable for real-time segmentation of unstructured roads in fields, demonstrating high efficiency and accuracy, meeting practical application requirements.

[0173] Example 2: Figure 2 The present invention provides a real-time segmentation system for unstructured field roads used for autonomous navigation of agricultural machinery, comprising:

[0174] Image acquisition unit: The front camera continuously captures the original image frames of the scene, uses the adaptive brightness equalizer to unify the image radiometry, and then outputs the enhanced frame stack.

[0175] Feature extraction unit: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuses them to generate a multi-resolution feature map.

[0176] Direction perception unit: Extracts the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filters the direction signal based on multi-source perception consistency analysis, and outputs the direction perception tensor in combination with the original feature map.

[0177] Semantic segmentation unit: The direction perception tensor is input into the BiSeNet decoder, and the corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and an edge heat map.

[0178] Mask correction unit: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

[0179] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0180] It should be noted that the system of the present invention can be deployed on the device itself to realize embedded applications, and can also be run on a PC or other terminal with a user interface, thereby meeting a variety of hardware environments and usage requirements.

[0181] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

[0182] It should be noted that, in this document, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0183] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A real-time segmentation method for unstructured field roads for autonomous navigation of agricultural machinery, characterized by: Including steps: S1: The front camera continuously captures the original image frames of the scene, uses the adaptive brightness equalizer to unify the image radiometry, and then outputs an enhanced frame stack; S2: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuse them to generate a multi-resolution feature map. Among them, texture features are generated by a convolutional layer followed by a maximum pooling layer, and contour features are generated by an edge detection convolutional layer. Texture features and contour features are aligned at the same scale. The texture features and contour features of each branch are spliced ​​at the channel level to generate a fused feature map. The fused feature map is adjusted to a uniform spatial size through bilinear interpolation upsampling or average pooling downsampling, and then arranged in scale order to generate a multi-resolution feature map. S3: Extract the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filter the direction signal based on the multi-source perception consistency analysis, and output the direction perception tensor based on the original feature map; Specifically, each scale in the multi-resolution feature map is divided into local regions using a fixed-size rectangular window to segment the feature map into non-overlapping local regions. Within each local region, the gradient direction distribution is extracted from the feature channel of the feature map. The horizontal and vertical gradients of the pixels are calculated and the direction angles are quantified to generate a gradient direction distribution histogram. The edge pulsation frequency distribution is extracted from the edge-related channel of the feature map. The edge orientation and edge intensity variation amplitude are calculated and quantified to generate an edge pulsation frequency distribution histogram. S4: The direction perception tensor is input into the BiSeNet decoder. The corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and an edge heat map. S5: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

2. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 1, characterized in that: Step S1 includes the following contents: The front camera continuously captures field scene images at a fixed frame rate to generate a sequence of raw image frames. The raw image frames are converted from RGB color space to HSV color space, the brightness channel histogram is calculated, and the brightness channel value is adjusted according to its cumulative distribution function. The adjusted brightness channel is then recombined with the hue channel and saturation channel of the original image frame and converted back to RGB color space to generate a brightness-balanced image frame. Multiple consecutive image frames with equalized brightness are stacked in time sequence to form an enhanced frame stack.

3. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 2, characterized in that: Step S2 includes the following contents: The enhanced frame stack is processed by a lightweight pyramid encoder. The lightweight pyramid encoder consists of multiple parallel branches, each branch uses a shared convolution kernel of different sizes. The number of shared convolution kernel parameters is fixed, and multi-scale features are extracted by adjusting the convolution kernel size. Enhanced frame stacking extracts texture and contour features synchronously within each branch.

4. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 1, characterized in that: Step S3 includes the following contents: The texture flow deflection factor is calculated based on the gradient direction distribution histogram, by identifying the main direction candidates and calculating the entropy of the direction distribution; the contour focus change index is calculated based on the edge pulsation frequency distribution histogram, by identifying the main direction candidates and calculating the ratio of the peak to the total.

5. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 4, characterized in that: Step S3 also includes the following: The texture flow deflection factor and contour focus change index are normalized into texture probability distribution and contour probability distribution; the main direction confidence coefficient is calculated based on the texture probability distribution and contour probability distribution, and the consistency is evaluated by symmetric divergence and mapped to the main direction confidence coefficient; the final main direction is determined according to the main direction confidence coefficient and the fused texture probability distribution and contour probability distribution, and a direction embedding vector is generated; the direction embedding vector is integrated into the multi-resolution feature map to generate a direction-aware tensor.

6. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 5, characterized in that: Step S4 includes the following contents: The direction-aware tensor is input into the BiSeNet decoder. The BiSeNet decoder determines the feature complexity by analyzing the distribution of local scale features in the direction-aware tensor, and activates branches in the decoder to process the feature complexity based on the analysis results. It ultimately generates two types of outputs: a pixel-level probability map indicating the drivable area and an edge heat map indicating the boundary of the drivable area.

7. The method for real-time segmentation of unstructured field roads for autonomous navigation of agricultural machinery according to claim 6, characterized in that: Step S5 Includes the following: The continuous frame consistency filter uses the driving area probability map of the previous frame to correct the driving area probability map of the current frame. It uses temporal smoothing to reduce inter-frame fluctuations caused by illumination changes or noise. It also combines the edge heat map to identify and remove misclassified areas in the current frame that overlap with the edge and were not identified as driving areas in the previous frame, generating a corrected driving area mask. Then, a morphological closing operation is performed on the modified driving area mask to smooth the mask boundary and eliminate noise. Then, a connected region analysis is performed to retain the connected region with the largest area to generate a stable segmentation mask. Finally, the boundary position vector of the driving area is extracted from the stable segmentation mask. The boundary is identified by the Canny edge detection algorithm and the boundary points are evenly sampled along the driving direction. The coordinates of the sampling points are converted into a one-dimensional vector form and transmitted to the navigation control module.

8. A real-time segmentation system for unstructured field roads for autonomous navigation of agricultural machinery, used to implement the real-time segmentation method for unstructured field roads for autonomous navigation of agricultural machinery according to any one of claims 1 to 7, characterized in that: include: Image acquisition unit: The front camera continuously captures the original image frames of the scene, uses the adaptive brightness equalizer to unify the image radiometry, and then outputs an enhanced frame stack; Feature extraction unit: A lightweight pyramid encoder is used to synchronously extract texture features and contour features from the enhanced frame stack using a shared convolution kernel, and then fuses them to generate a multi-resolution feature map. Direction perception unit: extracts the gradient direction distribution and edge pulsation frequency of the local area of ​​the image on the multi-resolution feature map, filters the direction signal based on multi-source perception consistency analysis, and outputs the direction perception tensor based on the original feature map; Semantic segmentation unit: The direction perception tensor is input into the BiSeNet decoder, and the corresponding branch is activated according to the distribution of local scale features in the tensor. The decoder outputs a pixel-level driving area probability map and edge heat map; Mask correction unit: The continuous frame consistency filter corrects the instantaneous misclassified area of ​​the current frame based on the driving area probability map of the previous frame, generates a stable segmentation mask, and extracts the boundary position vector to transmit to the navigation control module.

Images